Checklist NOAI Singapore 2026 Final Assessment · Task 3
The Attention Mechanism (Building the "Brain")
Implement scaled dot-product attention, multi-head attention, a corrected positional encoding and a classification head in PyTorch.
The task
Programming Task 3 (Deep Learning – NLP/Transformers, 35 marks including a 5-mark bonus) builds the components of a Transformer encoder for a lightweight language model: scaled dot-product attention with optional masking (Part 1, 10 marks), multi-head attention (Part 2, 10 marks), debugging a sinusoidal positional encoding whose even/odd indexing is wrong (Part 3, 5 marks), a classification head (Part 4, 5 marks) and a bonus question (Part 5, 5 marks).
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- The task notebook with test cells.
- You submit
- The completed notebook.
- Scoring
- Marks per part as stated in the notebook.
- Rules
- Masked positions must be filled with a large negative number such as -1e9, not -inf.
- Format
- NOAI 2026 Final Assessment, 6 March 2026, 9:30 am–12:30 pm (3 hours, 100 marks): Section 1 (20 MCQs, 20 marks) and three programming tasks. Individual.