Checklist IOAI TST 2025 IOAI Team Selection Test · Task 3
LeetCode Difficulty Prediction from Solution Descriptions
Russian title: Прогнозирование сложности задач LeetCode по текстовому описанию решения
Predict whether a LeetCode problem is easy, medium or hard from a step-by-step natural-language description of its Python solution.
The task
Each example is a human-readable, step-by-step description of the algorithm of a Python solution to a LeetCode problem, written in English and containing no code, function names or variable names.
From this description the contestant must predict the difficulty of the original problem: easy, medium or hard. Any model may be used, from logistic regression to transformers.
Abridged and translated by SOTA from the official Russian materials. The official statement has the exact rules, and it wins wherever this summary differs.
In English
This task was published in Russian. SOTA translated its 2 files into English on 17 September 2026.
- Competition overview Russian original of Competition overview
- Data description Russian original of Data description
Read the competition overview in English
LeetCode Difficulty Prediction from Solution Descriptions
English translation by SOTA – AI Community of the Russian original. Licensed CC BY-NC-SA 4.0, like the original. Organisers who would like this translation removed can email [email protected].
Contest 3 (28 June 2025) of the Kazakhstan IOAI Team Selection Test (Отборочные на IOAI), 2025. This is the Overview tab of the Kaggle competition "TST Day3 Upsolving", the host's public upsolving copy of the contest, open from 28 June to 15 July 2025. Its subtitle reads: "Classifying the difficulty of algorithmic problems from a step-by-step text description". Original: kaggle.com/competitions/tst-day-3-upsolving. The Data tab is translated in a separate file.
Overview
🧠 Predicting the difficulty of LeetCode problems from a text description of the solution
In this competition you are to build a model that, from a step-by-step text description of a Python solution, predicts how difficult the original LeetCode problem was:
easy, medium or hard.
Each solution is presented as a human-readable step-by-step description of the algorithm, without code or names of functions and variables.
Description
📌 What is the input?
Each solution is presented as a step-by-step text description of the algorithm, written in natural language. Instead of the source code, you receive a short explanation of how the solution works.
Example:
- Initialize m and n as the lengths of the input string s and pattern p.
- Use a while loop with condition i < m.
- If the characters match or pattern has '?', move both pointers.
- If '*', record positions and try to match the remainder.
- Return true if pattern ends correctly.
🧠 Your task is to predict from such a description how difficult the problem to which this solution belongs was:
easymediumhard
Evaluation
🏆 Evaluation
The quality of solutions is evaluated by F1 on a hidden test set:
- 50% of the examples are used for the public leaderboard
- 50% are used for the private leaderboard, which determines the final placings
📌 Rules
- Any models may be used: from logistic regression to transformers
Translated by SOTA. The Russian original is the official version and wins wherever the two differ. This translates the Overview and Data tabs of the host's public upsolving copy on Kaggle; the Rules tab (code of conduct, allowed websites and libraries) is not included. The solution descriptions in the data are in English, and the column that holds them is named code. Kaggle scores this copy with its F1 Score metric; the pages do not say which averaging is used. The competition data on Kaggle are licensed CC BY-NC-SA 4.0. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.
At a glance
- You get
train.csv(id, difficulty, code — the text description),test.csv(id, code) andsample_submission.csv.- You submit
submission.csvwith columns id and difficulty (easy, medium or hard).- Scoring
- F1 score on a hidden test set (Kaggle 'F1 Score'; the averaging is not stated), 50% public and 50% private leaderboard. Contest scoring (hub page): each task is worth at most 100 points; Norm_Score = (Submission_Score - Min_Score) / (Max_Score - Min_Score) x 100, where Min_Score is the lower of 0.9 x baseline score and the lowest participant score, and Max_Score is the higher of 0.9 x the Scientific Committee solution score and the best participant score.
- Rules
- Read-only access to stackoverflow.com, scikit-learn.org, pytorch.org, huggingface.co, numpy.org, github.com, python.org, pypi.org, a search engine restricted to these sites and a translation site.
- GPT-4o (or a newer version announced before the contest) is allowed through the platform integration; other LLMs, chats, Copilot tools and APIs are forbidden unless the task says otherwise.
- Allowed libraries include torch, scikit-learn, xgboost, catboost, lightgbm, transformers, spacy, nltk, gensim, fasttext, pandas, numpy, scipy, opencv-python, Pillow, torchvision, scikit-image, matplotlib, seaborn, plotly and common utilities; TensorFlow and Keras are not available.
- Pre-trained models and external data are forbidden unless the task allows them; no extra training time after the contest ends; contestants may be asked to upload code, models and/or predictions.
- Format
- Kazakhstan IOAI Team Selection Test 2025, contest 3 of 4 (28 June 2025), run as a private Kaggle competition; the public Kaggle copy is an upsolving clone of it.