Checklist NEOAI 2025 Day 1 · Task 2
Underfitting CV
Fine-tune an image classifier on ten examples per poorly recognised class without degrading its accuracy on the well-recognised classes.
The task
A pretrained network (tiny_vit_5m_224 with 102 output classes in the baseline) achieves about 90% accuracy on one part of the classes ("good classes") but only about 30% on the other ("bad classes"). A small training set with 10 images per class is provided, but it contains only the bad classes.
The contestant must fine-tune the network so that accuracy on the bad classes improves while accuracy on the good classes is preserved.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
model.pt(initial weights),train_images/ andtrain.csv(path, class; bad classes only),test_images/,sample_submission.csv(id, class).- You submit
- CSV with columns id (test image filename) and class, generated by the fixed
make_predictfunction. - Scoring
- SCORE = sqrt(
acc_good·acc_bad), the geometric mean of the accuracies on the good and bad classes (the English text calls it the "harmonic mean"; the formula and the Russian text give the geometric mean). - Rules
- The submission must be generated with the supplied
make_predictfunction, which must not be changed. - Only data from
train_imagesmay be used for fine-tuning. - General NEOAI 2025 rules apply: by default no pretrained models or additional data unless the statement allows them; no external human help; on-site participants get one NVIDIA V100, an LLM chat (GPT-4o or similar) and a whitelist of websites.
- The submission must be generated with the supplied
- Format
- NEOAI 2025 main round, Task 2, Day 1 (7 May 2025, 07:00–13:00 UTC). Individual; 20 submissions per day.