Discord

Checklist NEOAI 2026 Day 2 · Task 6

CutiesNLP

Answer 1,000 multiple-choice questions using a stripped Qwen3.5-2B model whose embedding table, first layer and language-modelling head have been removed.

  • NLP
  • Multiple-choice question answering from hidden states
Read the official statement

The task

Only layers 1–23 and the final RMSNorm of a Qwen3.5-2B language model are provided; embed_tokens, layers[0] and lm_head are absent. Each question has a context, a query, four answer options (A–D) and two sink options that are never correct.

Inputs arrive pre-embedded: for every question the hidden state after layer 0 of the original model is given, together with the token positions of each answer option, so the forward pass can be continued from layer 1. The training set contains 100 labelled questions; 1,000 questions must be answered.

Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.

At a glance

You get
model.pt and model_config/ (Qwen3_5TextConfig); student_train.pkl (100 records: idx, embeds_after_l0 float16 array of shape (seq_len, 2048), eol_positions, correct_idx); student_val.pkl (1,000 records without correct_idx); sample_submission.csv.
You submit
CSV with columns id, target, where target ∈ {0, 1, 2, 3}.
Scoring
Accuracy.
Rules
  • General NEOAI 2026 rules apply.
Format
NEOAI 2026 main round, Day 2 (4 May 2026). Individual.

Notebooks

Competition

Source files

Details

Year
2026, SberUniversity, Russia; online and national venues for other countries
Round
Day 2 · Task 6
Language
English
License
Per-competition Kaggle data licence: MIT (Task 4); Subject to Competition Rules (Tasks 5 and 6), as stated by the source