Predicting the Rating of the Next Book Read

Predicting the Rating of the Next Book Read

English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email [email protected].

3. Second qualifying stage

3.1. Engineering tour

3.1.1. General description of the tasks

At the second qualifying stage, the Olympiad participants are asked to solve practice-oriented machine learning tasks on data from a large online book platform.

The tasks test the ability to:

• work with tabular and text data,
• build features, train and validate models,
• produce correct solution files for automatic checking.

The stage consists of two tracks:

• individual track: a regression task — predict a user's rating for the next book read;
• team track: a ranking task — order a list of candidate books by three-level relevance.

Educational programme

Within the second qualifying stage, participants are asked to complete tasks covering the following key competencies:

1. recommender systems,
2. machine learning on tabular data and working with sparse features,
3. text processing (e.g. TF-IDF),
4. regression and regression quality metrics (RMSE, MAE),
5. ranking (Learning-to-Rank) and ranking quality metrics (NDCG@20),
6. reproducibility and self-containedness of ML solutions.

To master these areas and raise their level of preparation, participants have access to the following educational materials:

• Competency 1: general training in machine learning — a machine learning course, basic level https://ai-academy.ru/training/courses/kurs-po-mashinnomu-obucheniyu-osnovnoj-uroven/.
• Competency 2: advanced machine learning methods and neural network technologies — a machine learning course, advanced level (advanced machine learning algorithms: gradient boosting, SVM, clustering and dimensionality reduction methods; natural language processing with neural networks; development and deployment of neural network solutions, including Telegram bots) https://ai-academy.ru/training/courses/kurs-po-mashinnomu-obucheniyu-prodvinutij-uroven/.
• Competency 3: practical preparation for competitive tasks in machine learning and data analysis — the "AI Vector" (Vektor II) bootcamp (preparation for the "Artificial Intelligence" profile of the National Technology Olympiad; going through the full competition cycle: dataset analysis, feature construction, model training, validation and uploading a solution to the leaderboard) https://ai-academy.ru/events/vector-ii-autumn/.

3.1.2. Individual tasks

Introduction

In the individual part of the second qualifying stage, the participant builds a model that predicts the rating a user will give to the next book they read. The data are split chronologically:

• training is done on the interaction history up to a point in time,
• the test contains the next interaction in the future for each user.

Task 1 (100 points)

Predicting the rating of the next book read

Topics: recommender systems, regression, working with sparse tabular data, feature engineering, text processing.

Statement

A large online book platform collects data on users' interactions with books. The data are split chronologically into several parts. This stage models a real-world situation: there is a history of user interactions up to a certain point in time (train.csv), and the rating for their next interaction in the future (test.csv) must be predicted.

You need to develop a machine learning model that, based on a user's interaction history (train.csv), predicts the rating (rating) that this user will give to the next book they read.

Features of the setting:

• time gap: training and test belong to different chronological periods;
• only "warm" users: the test set includes only users who have a sufficient history in the training set (criterion: in train.csv, at least one record with has_read=1 or at least three records with has_read=0);
• one prediction per user: test.csv contains exactly one pair (user_id, book_id) for each user.

The data and the file formats are provided on the page: https://ods.ai/competitions/nto25-26-2-individ/dataset/.

Note on formats:

• the source data are provided in CSV, where the field separator may differ (see the accompanying data reference);
• the solution file (submission) is a CSV with a comma (,) separator, see "Solution format (submission)" below.

Constraints, submission format and scoring (from the collection's Solution/Answer sections)

Constraints:

• using external data not provided by the organisers is forbidden;
• the solution must be reproducible and self-contained; it must not depend on internet access when the pipeline is reproduced and the final submission is prepared.

Solution format (submission):

• a CSV file with a comma (,) separator;
• columns: user_id, book_id, rating_predict;
• rating_predict is a real number in the range from 0 to 10 (rounding is not required; the checking system will clip the values to the range $[0, 10]$);
• the file must contain predictions for all pairs from test.csv (the order of the rows does not matter; a header is allowed).

Evaluation metric: the final Score is computed from two regression metrics (RMSE and MAE) and normalisation by the width of the rating range ($R = 10$).

The answer is the submission file submission.csv uploaded to the competition page, in CSV format with the columns user_id, book_id, rating_predict.

Scoring details:

1. The public leaderboard (Public) is computed on the visible part of the test data.
2. The private leaderboard (Private) is computed on the hidden part of the test data; the final results are determined by it alone.
3. The limits on the number of submissions per day and for the whole stage are given on the competition page.
4. For the result to be taken into account in the final selection of submissions, the participant must fill in the confirmation form.

The collection's solution for this task is in the separate solution file.