Checklist AI Olymp 2025 Final · Day 2 task
Which Paper Is Newer?
Given the opening sentences of two arXiv papers, predict which one was published later.
The task
The task uses real arXiv data. For each pair of papers, the contestant receives the first two sentences of each abstract and must decide which paper was published later; the model has to recognise temporal signals such as trends, fashionable terms and references to older or newer methods.
The training sentences cover 140 arXiv categories with at least 10 papers per category per year (a paper may belong to several categories); a time series of daily paper counts per category from 2000 to April 2025 (the homework dataset) is also provided. For the test set, 400 papers per category were sampled uniformly over 2000–2025 and randomly paired so that the papers in each pair are at least two years apart.
The overview links this task to the qualification (predicting a paper's topic from two sentences) and the homework (forecasting how many papers will be published).
Abridged and translated by SOTA from the official Russian materials. The official statement has the exact rules, and it wins wherever this summary differs.
In English
This task was published in Russian. SOTA translated its 2 files into English on 17 September 2026.
- Competition overview Russian original of Competition overview
- Data description Russian original of Data description
Read the competition overview in English
Which Paper Is Newer?
English translation by SOTA – AI Community of the Russian and Kazakh original. Licensed CC BY-SA 3.0, like the original. Organisers who would like this translation removed can email [email protected].
Day 2 of the final of AI Olymp, the Republican Olympiad in Artificial Intelligence of Kazakhstan (Federation of Sports Programming of Kazakhstan and DSML.KZ), 2025. This is the Overview tab of the Kaggle competition "Kazakhstan Respa Final - Day 2 (afterparty)" (subtitle: "Find the most recent papers"), the public late-submission copy of the Day 2 contest, open from 7 May to 1 June 2025. Original: kaggle.com/competitions/kazakhstan-respa-final-day-2-late-competition. The Overview and Description are bilingual on Kaggle, with Russian and Kazakh versions side by side; both columns are translated below. The Evaluation section is in Russian only. The Data tab is translated in a separate file.
Overview
| Russian version | Kazakh version |
|---|---|
| Welcome to the final round of AI Olymp! | Welcome to the final stage of the AI Olymp competition! |
| In the qualification round you determined the topic of a scientific paper from a pair of sentences. | In the qualifying stage you determined the topic of a paper from two sentences in it. |
| In the homework assignment you predicted how many papers would be published in the future. | In the homework assignment you had to predict how many papers will be published in the future. |
| Today it is something in between! | Today's task is halfway between these two! |
| We give you two papers, and you need to determine which of them is newer. | You are given two papers, and you must determine which of them came out later. |
| This is no longer just classification, and not quite a time series: it is a step towards teaching AI to understand progress in science through language. | This is neither plain classification nor pure time-based forecasting. It is a step towards understanding progress in science through language. |
| Good luck! | Good luck! |
Description
| Russian version | Kazakh version |
|---|---|
| In this task you work with real data from arXiv, the world's largest open-access repository of scientific papers. | In this task you work with arXiv data, the world's largest open database of scientific papers. |
| You have the text of the first sentences of scientific papers, and the task is to work out which of two papers was published later. | You have the first sentences of scientific papers, and the task is to determine which of the two papers came out later. |
| The model must learn to recognise signs of time: trends, fashionable terms, references to old or new methods. | The model must be able to distinguish signs of time: trends, references to new or old methods, terms. |
| The training set is made up of the first two sentences of papers selected across 140 topics, with no fewer than 10 papers per year for each topic. | The training set consists of the first two sentences of papers, with at least 10 papers selected every year for each of the 140 topics. |
| Moreover, one paper may belong to several topics at once, which makes the task more realistic. | Each paper may relate to several topics; this makes the model more complex and more realistic. |
| In the test set, 400 papers were chosen at random for each topic, uniformly over the entire period from 2000 to 2025. | In the test set, 400 papers were selected from each topic, at random and evenly distributed (over the years 2000–2025). |
| From this pool the papers were randomly combined into pairs so that the publication dates within each pair are at least 2 years apart. | Random pairs were formed from these papers, and within each pair the publication times differ by at least 2 years. |
Evaluation
Accuracy - the accuracy of binary classification
Final points
Benchmarks (on the public leaderboard) 55 per cent - 30 points 60 per cent - 60 points 65 per cent - 90 points 70 per cent - 120 points
Positional points
(on the private leaderboard)
- 1st place - 120 points
- 2nd place - 100 points
- 3rd place - 90 points
- 4th place - 85 points
- places 5 - 10 - 60 - 80 points (100 - 4x)
- places 11 - 20 - 20 - 57 points (80 - 3x)
- places 21 - 40 - 0 - 19 points (40 - x)
To be eligible for places 1-9 you must beat at least one benchmark In the event of a tie in points, your place is taken to be the lowest position of the group
Example:
- Participant 1 - 1st place - 56 per cent - receives 30 points for the benchmark and 120 points for first position
- Participant 2 - 2nd place - 54 per cent - receives 0 points for the benchmark and 80 points for tenth position
Translated by SOTA. The Russian original is the official version and wins wherever the two differ. The Overview and Description on Kaggle are bilingual, with Russian and Kazakh versions side by side; both columns are translated, and the Kazakh wording differs slightly in places. The Evaluation section and the Data tab are in Russian only, and the Rules tab says only "Нельзя махлевать" (No cheating). As published, one points range does not match its formula (80 − 3x gives 47 points for 11th place, not 57), and the second worked example awards 80 points for tenth position, where 100 − 4x gives 60. The submission file's columns are not described. The competition data on Kaggle are licensed CC BY-SA 3.0. This translation is shared under CC BY-SA 3.0, the licence of the original. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.
At a glance
- You get
train_timeseries.csv,train_sentences.csv,test_sentences.csv.- You submit
- For each test pair, 0 if the first paper is newer and 1 if the second paper is newer (submission column layout not described on the pages opened).
- Scoring
- Accuracy. Final points combine benchmark points on the public leaderboard (55% → 30, 60% → 60, 65% → 90, 70% → 120) and position points on the private leaderboard (1st 120, 2nd 100, 3rd 90, 4th 85, 5th–10th 100 − 4x, 11th–20th 80 − 3x, 21st–40th 40 − x); places 1–9 require beating at least one benchmark.
- Rules
- Individual participation; up to 5 submissions per day on the public re-run.
- Rules page: 'Нельзя махлевать' (no cheating).
- Format
- Day 2 of the AI Olymp 2025 final; the public 'afterparty' Kaggle competition was open 7 May – 1 June 2025 for late submissions. The original Day 2 competition (kazakhstan-respa-final-day-2) is private.