Checklist INAIO 2025 Stage 2 · Task 1
Analogy Oracle: Cracking the Code of Word Relationships
Predict the missing word D in analogies A:B :: C:? first zero-shot with pre-trained embeddings, then with a trained model.
The task
Q1 (zero-shot): implement predict_analogy using pre-trained embeddings (BERT, Word2Vec or GloVe) without training. Q2: implement learn_analogy_model and predict_analogy. Q3: evaluate on the public test set. Q4 is a private evaluation by INAIO (14 points in total).
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
analogy_train.csvandanalogy_test_public.csv(columns A, B, C, D).- You submit
- Completed notebook functions and reported metrics.
- Scoring
- Precision@1 and Precision@K.
- Rules
- Make a copy of the notebook, share it with the organisers and report LLM chats and external resources used in the main test form.
- Format
- INAIO 2025 Stage 2, 2 March 2025, 10:00 AM – 1:00 PM IST (from the test form). Theory problems in a Google Form plus ML coding problems in Colab notebooks shared with the organisers.