Checklist OAI 2026 Final (Stage III) · Task 4
Hidden Categories
Polish title: Ukryte Kategorie
Predict the set of hierarchical e-commerce category labels of each product from its 256-dimensional collaborative-filtering embedding.
The task
An employee of an e-commerce company must label product categories consistently with a predecessor who left no instructions. Recommender systems based on collaborative filtering learn product embeddings only from user behaviour (clicks, purchases), so products appearing in similar contexts receive similar vectors.
The contestant implements and trains a model that, from product embeddings produced by a recommender system, predicts the set of categories assigned to each product; the predicted set should contain all original labels (in any order) and no extra ones. The data consist of 256-dimensional embeddings and a list of label lists; a product may belong to more than one category, and the labels have a structure that should be considered.
The training set has 12,145 products, the validation set 2,603 and the hidden test set 2,603. The embeddings are optimised so that a higher dot product between a user embedding and a product embedding places the product higher in the user's personalised ranking.
Abridged and translated by SOTA from the official Polish materials. The official statement has the exact rules, and it wins wherever this summary differs.
In English
This task was published in Polish. SOTA translated it into English on 16 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.
Read the task notebook in English
Hidden Categories
Source: Image generated with ChatGPT.
Introduction
Kuba has just started working at a company that runs an e-commerce shop. One of his tasks is to describe product categories. Some of the products were described by his predecessor, and the company cares a great deal about a consistent choice of labels. Unfortunately, the predecessor did not leave any instructions. Help Kuba fill in the missing labels.
Modern recommender systems often use a collaborative filtering approach, which learns solely from user behaviour (e.g. clicks or purchases) -- without using any information about the products. Products that appear in a similar context receive similar numerical representations (embeddings), which reflect the connections between them.
Task
Your task is to implement and train a model which, on the basis of the product embeddings coming from the recommender system, predicts the set of categories assigned to each product. As far as possible, this set should contain all the original labels (not necessarily in the original order), with no additional elements.
Data
The data consist of embeddings (256-dimensional vectors representing the products in a latent space) and a collection of product labels (a list of lists of labels). They have been split into:
-
a training set (12145 products),
-
a validation set (2603 products),
-
a secret test set (2603 products).
Successive rows of the embedding matrix correspond to the same products as successive elements of the list of label sets.
Scoring criterion
Your solution will be scored using Intersection over Union (IoU), computed for each product:
The metric will be computed without taking the structure and order of the labels into account.
The final result is the mean IoU value over all products, converted into points:
-
if the result is lower than 36%, you will receive 0 points,
-
if the result is higher than 46%, you will receive the maximum number of points, i.e. 100.
Points for values between these thresholds will be awarded proportionally.
Constraints
- You may use only the training set to train the model.
- Your solution will be tested on the Contest Platform without internet access and without a GPU.
- Training and evaluation of your final solution on the Contest Platform must not take longer than 5 minutes.
- Permitted libraries:
numpy,pandas,sklearn.
Submission files
The solution to the task is this notebook, completed with your solution in the form of a model definition, implemented as a special class YourSolution (see the section: Your Solution).
Hints
- Pay attention to the structure of the labels. Each product may belong to more than one category.
- The product embeddings are optimised in such a way that the higher the dot product between a user embedding and a product embedding, the higher the product is placed in the personalised ranking.
Evaluation
Remember that during grading the FINAL_EVALUATION_MODE flag will be set to True.
You can score between 0 and 100 points for this task. The number of points you receive will be calculated on the (secret) test set on the Contest Platform according to the formula given above, rounded to an integer. If your solution does not meet the criteria above or does not run correctly, you will receive 0 points for the task.
Starter Code
######################### DO NOT CHANGE THIS CELL ##########################
import json
import numpy as np
import pandas as pd
import sklearn
import mathFINAL_EVALUATION_MODE = FalseLoading the Data
######################### DO NOT CHANGE THIS CELL ##########################
with open('data/train_categories.json', 'r') as f:
train_categories = json.load(f)
train_embeddings = np.load('data/train_embeddings.npy', allow_pickle=True)
with open('data/val_categories.json', 'r') as f:
val_categories = json.load(f)
val_embeddings = np.load('data/val_embeddings.npy', allow_pickle=True)Code with the Scoring Criterion
######################### DO NOT CHANGE THIS CELL ##########################
def intersection_over_union(x, y):
set_x = set(x)
set_y = set(y)
return len(set_x.intersection(set_y)) / len(set_x.union(set_y))def loss(true_sets, predicted_sets):
return np.mean([intersection_over_union(x, y) for x, y in zip(true_sets, predicted_sets)])######################### DO NOT CHANGE THIS CELL ##########################
def round_half_up(number: float) -> int:
return int(math.floor(number + 0.5))
def compute_score(loss_val: float):
upper_limit = 0.46
lower_limit = 0.36
if loss_val > upper_limit:
return 100
elif loss_val < lower_limit:
return 0
else:
return round_half_up((loss_val - lower_limit) / (upper_limit - lower_limit) * 100)Your Solution
As your solution, you must implement the class YourSolution, which satisfies certain formal requirements described below.
The definition of the class YourSolution must implement the following methods:
-
A method
fitwhich takes as arguments annp.ndarrayarray whose rows are 256-dimensional product embeddings, and a list whose elements are the lists of labels for the successive products. -
A method
predictwhich takes as its argument annp.ndarrayarray whose rows are 256-dimensional product embeddings. It must return a list whose elements are the lists of labels for the successive products.
An example solution is included in this section. It is a very simple baseline based on the most frequent label. Thanks to it, the notebook runs correctly even without editing this cell.
You may modify the definition of this class freely, as long as the conditions above are met. You may add the attributes and methods you need and modify the existing ones. Your solution will be evaluated solely on the basis of the class YourSolution.
######################### THIS IS THE PLACE FOR YOUR FINAL SOLUTION ##########################
class YourSolution:
def __init__(self):
"""
init should use hard-coded or default hyperparameter values
"""
pass
def fit(self, X, y):
# Here you can include the preparation of your data
# Train the model
pass
def predict(self, X):
return [['Sports & Outdoors'] for _ in X]Evaluation
Running the cell below lets you check how many points your solution would score on the training data. Before submitting, make sure that the whole notebook runs from start to finish without errors and without any user intervention after selecting the "Run All" option.
######################### DO NOT CHANGE THIS CELL ##########################
if not FINAL_EVALUATION_MODE:
# Training the model
yourSolutionInstance = YourSolution()
yourSolutionInstance.fit(train_embeddings, train_categories)
# Preparing the predictions
predictions = yourSolutionInstance.predict(val_embeddings)
# Scoring the solution
loss_val = loss(val_categories, predictions)
print(compute_score(loss_val))Translated by SOTA. The Polish original is the official version and wins wherever the two differ. The data are in the data/ folder next to the original notebook; the category labels in the data are already in English. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.
At a glance
- You get
data/train_embeddings.npy,data/train_categories.json,data/val_embeddings.npyanddata/val_categories.jsonin the task folder.- You submit
- This notebook with YourSolution implementing fit(embeddings,
label_lists) and predict(embeddings) → list of label lists. - Scoring
- IoU between predicted and true label sets per product, ignoring order and structure, averaged over products. 0 points below 36%, 100 points above 46%, linear in between; rounded.
- Rules
- Only the training set may be used for training.
- Tested without Internet access and without a GPU; training and evaluation must take at most 5 minutes.
- Allowed libraries: numpy, pandas, sklearn.
- Format
- Final (Stage III), 17–20 April 2026, Faculty of Mathematics and Computer Science, Adam Mickiewicz University in Poznań; two 5-hour contest sessions (Saturday and Sunday, i.e. 18 and 19 April 2026); 45 finalists; maximum 400 points. Evaluated automatically on the Competition Platform (Platforma Konkursowa) on a hidden test set; points are rounded to an integer, and a notebook that fails the requirements or does not run scores 0.