Checklist AICC 2026 Round 8 · Task 3
Archivist's Cards
Retrieve, for each index card of region-level annotations, the matching photo from a gallery of 1500 images using only CLIP ViT-B/16.
The task
A cataloguer described each photo not with a caption but with an index card of dozens of short observations, one per small image region. No card describes the whole scene, and the archive contains many photos of similar scenes. For each card, the matching photo must be found.
Each card has 48 to 97 annotation lines, most of the form 'Label: observation', some a bare label; lines may be near or exact duplicates, and every line is true of the photo. The training set has 500 cards with their photos. The test set has 700 cards and a gallery of 1500 photos (the 700 answers plus 800 distractors, many showing similar scenes); exactly one gallery photo is correct per card.
For each test card the contestant returns the 10 best-matching gallery ids, best first.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
train.csv(500 rows:image_id, description),train_imgs/ (500 photos, trn_*),test.csv(700 rows: datapointID, description),test_imgs/ (1500 gallery photos, img_*); starter kit withbaseline.ipynb.- You submit
submission.csvwith columns subtaskID (always 1), datapointID, answer (10 space-separated gallery ids); 700 rows. Only the first 10 distinct ids count.- Scoring
- Mean reciprocal-rank-squared M: a card scores 1/k² if the correct photo is at rank k within the 10, otherwise 0; M is averaged over the 700 cards. Points = 100 × (M − 0.23) / (0.94 − 0.23), clamped to 0–100 (0.23 is the provided baseline, 0.94 the reference).
- Rules
- The only model allowed is CLIP ViT-B/16 (openai/clip-vit-base-patch16), image and text encoders; no other pretrained weights and no external data.
- Each submission must include the source code as a .py file (maximum 35 KB) and the predictions CSV (maximum 50 MB); organiser-provided seed-setting code must be used.
- Round rules (honour system): no LLMs or AI assistants for code or ideas; no internet beyond official library documentation; no communication or sharing of approaches during the contest; individual participation only.
- Format
- AICC Round 8 (special edition following the CEOAI syllabus), online on Nitro AI Judge; open from 12 Jun 2026 17:00 to 14 Jun 2026 17:00 (time zone not stated on the platform); contestants are asked to work within a single self-chosen 6-hour session.