Lost Interactions

Lost Interactions

English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email [email protected].

4.3. Engineering tour

4.3.1. General information

The task of the final stage of the "Artificial Intelligence" profile: for each user in targets.csv, build a ranked list of 20 editions (edition_id) that are most likely to be lost positive interactions. The solution consists in building an ML pipeline: candidate generation, ranking and assembly of the final list.

4.3.2. Task backstory

The overnight release of a book service went wrong. During the migration of the event schema, some of the production workers began writing logs in the new schema, while the rest stayed on the old one. Because of a bug in the consumer and in deduplication, some of the positive events never reached the final storage.

As a result, the users' interaction history has become incomplete. This degrades personalisation, recommendations and product analytics.

Your role is the recommender-system team, which has to restore the most likely lost positive interactions from the observed data.

4.3.3. Requirements for the team and the participants' competencies

Number of participants in a team: two people.

Competencies that the team members must have:

• data analysis;
• machine learning;
• deep learning.

4.3.4. Equipment and software

Item	Description
Name and link (if any)	What it is used for in the task
Sberbook <Insert the Sberbook specifications>	Laptop on which the solution is developed, run and tested
PyCharm/ VSCode	IDE for development and for working with the solution code.

4.3.5. Task description

For each user in targets.csv, you need to build a ranked list of 20 editions (edition_id) that are most likely to be lost positive interactions.

The solution consists in building an ML pipeline.

1. Candidate generation — searching for potentially relevant editions across the entire available catalogue.
2. Ranking — selecting the top 20 most likely lost items.
3. Assembly of the final list — outputting the top 20 for each user.

A detailed description of the file structure and the fields is given in the "Data" section and in data_description.md in the baseline repository.

Positive events in the logs:

1. event_type = 1 — adding to the wishlist (wishlist).
2. event_type = 2 — reading (read).

Scoring is done at the level of a unique pair (user_id, edition_id), not at the level of event rows.

A lost item is a hidden (lost) positive pair (user_id, edition_id) from the incident window. The solution does not need to predict the type of the positive event (wishlist/ read).

The loss of positive interactions is partial: in the incident window, about 20% of the positive pairs (user_id, edition_id) are hidden. The losses are uneven across users and over time. The event types (wishlist and read) may be affected to different degrees. The exact protocol by which the hidden lost items were formed is not disclosed. The case is built on a period of 214 days that contains an incident window: from 2025–10–01 00:00:00 to 2025–11–01 00:00:00. Participants receive the observed log for the whole period, in which artificial losses have been applied within the incident window.

Incomplete logs are a realistic problem of production systems. Event losses can arise from migration errors, ETL failures, schema desynchronisation and problems in the message-delivery pipelines.

This case models the emergency recovery of signals for a recommender system. The goal is to build a reproducible ML pipeline that restores the most likely lost positive interactions from the observed data.

Important! In real systems, recovered events are usually treated as a model layer rather than as a replacement for the primary logs.

Data description

Link to the data: https://disk.360.yandex.ru/d/9rEAVmnI_Ic56Q/data.

File interactions.csv

The observed log of user interactions after the partial loss of events. Each row is one event.

Field	Description	Comment
user_id	user identifier (FK → users.user_id)	-
edition_id	edition identifier (FK → editions.edition_id)	-
event_type	event type: 1 — wishlist, 2 — read	-
rating	rating (only for read, otherwise NULL)	-
event_ts	date and time of the event	-

File targets.csv

The list of users for whom the lost positive interactions must be restored.

Field	Description	Comment
user_id	user identifier	a single column

File editions.csv

Reference table of editions (edition-level). Each edition is linked to one book.

Field	Description	Comment
edition_id	edition identifier (PK)	-
book_id	book identifier	-
author_id	author identifier (FK → authors.author_id)	-
publication_year	year of publication of the edition, not of the book	of the edition, not of the book
age_restriction	age restriction	e.g. 18+
language_id	language identifier	no reference table
publisher_id	publisher identifier	no reference table
title	title	-
description	description	text

File authors.csv

Reference table of authors.

Field	Description	Comment
author_id	author identifier (PK)	-
author_name	author name	full name or pseudonym

File genres.csv

Reference table of genres.

Field	Description	Comment
genre_id	genre identifier (PK)	-
genre_name	genre name	-

File book_genres.csv

Many-to-many link between books and genres.

Field	Description	Comment
book_id	book identifier	-
genre_id	genre identifier (FK → genres.genre_id)	-

File users.csv

Reference table of users with demographic features.

Field	Description	Comment
user_id	user identifier (PK)	-
gender	gender: 1, 2 or NULL	-
age	age (may be NULL)	-

Solution format and requirements

You must submit the file submission.csv of the following form:

user_id,edition_id,rank.

Technical requirements:

1. For each user_id in targets.csv there must be exactly 20 rows.
2. Rank must be an integer from 1 to 20.
3. Within one user_id, the rank values must not repeat.
4. Within one user_id, the edition_id values must not repeat.
5. The file must contain predictions for all users in targets.csv.
6. The order of the rows in the file does not matter.

Example of a valid solution file (sample_submission.csv): https://disk.360.yandex.ru/d/9rEAVmnI_Ic56Q.

4.3.6. Scoring system

The final score is computed as the mean NDCG@20 over all test users. The metric rewards separately for hitting a lost item and for correct ranking.

For each user_id in targets.csv, the participant predicts 20 rows with the fields edition_id and rank (1–20). The organisers hold a hidden labelling $is\_lost \in \{0, 1\}$ for the pairs (user_id, edition_id), where 1 means that the pair is a lost item and 0 means that it is not.

Definitions: $rel(rank) = 1$ if the edition at position rank is a lost item for this user, otherwise $rel(rank) = 0$.

$discount(rank) = 1/\log_2(rank + 1)$.

$\text{DCG\_user@20} = \sum rel(rank) \cdot discount(rank)$ over $rank$ from 1 to 20.

$\text{IDCG\_user@20} = \sum discount(rank)$ over the ideal ranking up to $\min(|\text{relevant\_items}|, 20)$.

$\text{NDCG\_user@20} = \text{DCG\_user@20}/\text{IDCG\_user@20}$ if $\text{IDCG\_user@20} > 0$, otherwise 0.

Averaging is done over users. The metric evaluates only the hidden lost positive interactions. Scoring is done over the pairs (user_id, edition_id). The Public leaderboard and the Private leaderboard use the same scoring principle on different hidden subsets of users. The split into Public and Private is made by users in a 50/50 proportion.

The collection's solution for this task is in the separate solution file.

4.3.8. Preparation materials

• Baseline: https://github.com/Orange-Hack/nto-ai-25-26-final-baseline.
• A detailed description of the file structure and the fields is given in the "Data" section and in data_description.md in the baseline repository.
• Examples of computing the metric are given in metric_examples.md in the baseline repository.