Checklist ROAI 2025 Selection Camp – Test 1 · Task 2
Angry Birds
Train an unbiased Earth-bird versus Water-bird classifier on ResNet50 features despite background and red-square spurious correlations.
The task
Training images show mostly Earth birds on land and Water birds on water, and every training image of the Earth class carries a red square (side 10–20 pixels, red channel set to 255) along a random edge; validation and test images have no square.
The contestant must build a classifier on top of the provided ImageNet-pretrained ResNet50 that separates Earth (0) and Water (1) birds. Performance is measured separately for the four bird/background combinations and the lowest of the four accuracies counts.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- Training images (biased), a group-balanced validation set and test images, shared through a Google Drive folder; a starting kit that extracts ResNet50 features.
- You submit
- CSV with subtaskID (always 1), datapointID (from
test.csv) and answer (0 = Earth, 1 = Water). - Scoring
- Worst-group accuracy over the four bird/background groups; the interval from 50% to 80% accuracy is mapped linearly onto 0–100 points.
- Rules
- Features only from the provided ResNet50, whose architecture may not be changed; the backbone may only be fine-tuned from the given weights
- No manual labelling; only automated labelling of training images
- Validation and test images may not be used for training, labelling or statistics
- Submitted code must reproduce the submitted results
- On-site during the national training and selection camp (17–24 June 2025, POLITEHNICA Bucharest)
- Internet limited to the contest platform and the documentation of the available libraries
- Only submissions ticked during the round count; anti-fraud session locking on the judge
- Format
- 2025 national selection camp, test 1, 19 June 2025.