Checklist Slovak AI Olympiad 2026 National Round · Task 2
Parrot
English title: Papagáj
Analyse a corpus of Slovak human and LLM-generated texts: compute statistics, detect machine text, attribute it to one of four models, and detect paraphrased machine text.
The task
Slovak texts from parliamentary transcripts (NRSR), news articles and Wikipedia are mixed with texts generated by four LLMs (GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Pro, Gemma 4 31B). Operation 'Parrot' (after 'stochastic parrots') asks the contestant to act as an analyst.
Subtask 1 (10 pts) computes 14 statistics of test_s2.jsonl (share of machine texts, counts, word/character/token lengths with the tiktoken o200k_base tokenizer, type-token ratio, top bigram/trigram frequency, punctuation, quartiles). Subtask 2 (15 pts) classifies human vs machine text. Subtask 3 (45 pts) attributes machine texts to ModelA–ModelD. Subtask 4 (30 pts) detects machine texts rewritten to look human, with only a paired validation set (human and paraphrased text on the same topic) and no training set.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
train.jsonl,valid.jsonl,test_s2.jsonl;train_s3.jsonl,valid_s3.jsonl,test_s3.jsonl;valid_s4.jsonl,test_s4.jsonl;starter.pyandsample_output.csv.- You submit
- One CSV with subtask number (1–4), answer index (from 0) and answer; not all subtasks need to be answered.
- Scoring
- Subtask 1: each statistic earns a share of 10 points, full points up to 5% relative error, decreasing linearly to 0. Subtasks 2–4: weighted F1 with linear normalisation between lower and upper thresholds (thresholds not stated).
- Rules
- AI-assisted coding tools (ChatGPT, Gemini, Copilot or similar) are prohibited.
- No collaboration, plagiarism, pre-written solutions or external code repositories.
- Only the packages installed in the competition environment (a long list of standard ML/DL/NLP/CV libraries is given in the rules).
- Submission limit 50, two final submissions (platform settings).
- Format
- National round (celoštátne kolo), 16–18 April 2026, Faculty of Science, Pavol Jozef Šafárik University (UPJŠ), Košice; contest on Nitro AI Judge on 17 April 2026, 06:30–12:00 UTC; 24 contestants; individual. Statements on Nitro are bilingual (English first, then Slovak).