Checklist USA-NA-AIO 2025 Round 2 · Task 3
CLIP Image–Text Matching
Build and train a CLIP-style model from frozen ViT and BERT encoders on 1,000 Flickr30k samples and analyse the contrastive loss.
The task
Problem 3 (100 points, 20 parts) studies Contrastive Language-Image Pre-Training using the Hugging Face dataset USAAIO/2025-Round2-Problem3 (Flickr30k). The contestant samples 1,000 examples with seed 2025, converts images to a (1000, 3, 224, 224) tensor in [−1, 1], tokenises captions with bert-base-uncased, writes a Dataset and a padding collate function, and answers questions about google/vit-base-patch16-224 and BERT.
A model MyCLIP with a learnable log-temperature and 512-dimensional output projections is trained for 100 epochs on GPU with Adam (learning rate 1e-3) and the symmetric contrastive loss, with the ViT and BERT weights frozen. Final parts prove lemmas on random projections that justify cosine similarity in high-dimensional spaces and analyse the limits of the softmax as the temperature tends to 0 and infinity.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
- Hugging Face dataset USAAIO/2025-Round2-Problem3; pretrained google/vit-base-patch16-224 and bert-base-uncased.
- You submit
- Written answers and code in a notebook.
- Rules
- Only the starter imports; ViT and BERT parameters must stay frozen (only output projections and temperature are trained).
- Format
- 2025 USA-NA-AIO Round 2, 27 April 2025 at MIT (IOAI news article). Statements and official solutions were posted on the USAAIO forum on 14 May 2025.