Checklist ROAI 2026 Selection Camp – CPU Practical Round · Task 1
Polyglot
Recover the row permutations that align three embedding matrices of the same 4,000 documents produced by three different encoders.
The task
Different transformer encoders place the same sentences in unrelated coordinate systems, although the relative geometry of the corpus is largely preserved. The contestant receives three L2-normalised embedding matrices M1, M2 and M3 of the same N = 4000 documents, without the underlying text.
The rows of M2 and M3 are shuffled by unknown permutations. The first 400 rows of M2 and only the first 20 rows of M3 are aligned with M1 and serve as anchors. For every row of M1 the contestant must output the index of the matching row in M2 (subtask 1) and in M3 (subtask 2).
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- Three L2-normalised embedding matrices (M1, M2, M3) of 4,000 documents from three encoders; K1 = 400 aligned anchors for M2 and K2 = 20 for M3.
- You submit
- One CSV with subtaskID (1 or 2), datapointID (row index of M1, 0–3999) and answer (predicted row index in M2 or M3).
- Scoring
- Raw accuracy of the predicted mapping. Subtask 1 (30 points): below 0.20 gives 3 points, 0.20–0.95 is scaled linearly from 3 to 30, 0.95 or above gives 30. Subtask 2 (70 points): below 0.20 gives 2 points, 0.20–0.95 scaled linearly from 2 to 70, 0.95 or above gives 70.
- Rules
- CPU round
- On-site at the National College of Informatics "Tudor Vianu", Bucharest
- Internet access and packages restricted to the allowed list in the round rules
- Final score computed on hidden test data; only selected submissions count
- Format
- ROAI 2026 selection camp, CPU practical round, 28 May 2026 (06:00–12:14 UTC on the judge).