Checklist OAI 2026 Stage II · Task 3
Decision Trees
Polish title: Drzewa decyzyjne
Automatically preprocess two-dimensional datasets and choose decision-tree hyperparameters so that the tree beats the organisers' baseline accuracy on as many of 100 hidden datasets as possible.
The task
A decision tree splits feature space into rectangular regions using thresholds on individual features; its effectiveness depends on whether the classes are separable, on the noise, and on hyperparameters such as depth or the minimum number of samples in a leaf.
The contestant writes an algorithm that, from a training set alone, automatically preprocesses the data and selects decision-tree hyperparameters: preprocess_data(X_train, X_test) returns transformed matrices of the same shapes, and get_decision_tree_hyperparameters(X_train, y_train) returns a dictionary of hyperparameters (for example max_depth, min_samples_split, min_samples_leaf, max_features).
Collection A (100 datasets) is provided: each has a training matrix n_train × 2 with 0/1 labels and a test matrix n_test × 2, with test labels and the organisers' baseline accuracies in a separate file. The datasets differ in grid size, overlapping cells, noise, rotation, probabilistic labels and size. Solutions are tested on a hidden collection B with the same format and number of datasets.
Abridged and translated by SOTA from the official Polish materials. The official statement has the exact rules, and it wins wherever this summary differs.
In English
This task was published in Polish. SOTA translated it into English on 16 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.
Read the task notebook in English
Decision Trees
Introduction
In machine learning we like to train models. The work, however, begins much earlier: with the choice of hyperparameters and data transformations. A decision tree divides the feature space into rectangular regions using thresholds on individual features. Its effectiveness depends on, among other things, whether it is able to separate points from different classes in the data and how much noise there is — as well as on its own hyperparameters, such as the depth or the minimum number of samples in a leaf.
Task
Create an algorithm which automatically, on the basis of the training dataset, performs data preprocessing and
selects the hyperparameters of a decision tree so that the final tree achieves the best possible score on the preprocessed test set. Provide the functions preprocess_data(X_train, X_test), which returns the transformed X_train and X_test with the same dimensions as the corresponding input matrices, and get_decision_tree_hyperparameters(X_train, y_train), which returns a dictionary of hyperparameters.
Data
A collection of 100 datasets named A is available, in the files:
-
A_input.npz— contains, for each dataseti(from 0 to 99):X_i— the training matrix (n_train_i × 2)y_i— the vector of training labels (n_train_i,), values 0 or 1X_test_i— the test matrix (n_test_i × 2)
-
A_output.npz— contains:y_test_i— the vector of test labels (n_test_i,), 0/1baseline_acc— the vector of baseline results of our method (n_datasets,), floating-point values.
Both files contain the field n_datasets = 100.
The datasets differ from one another: they may have different grid sizes, overlapping cells, noise, rotated data, probabilistic labels, etc. They may also have different numbers of examples.
Your solution will be tested on a hidden collection B, with an identical format and number of datasets. You do not have access to it during the contest.
Scoring Criterion
For each dataset in the collection you can score 0 or 1.2 points, depending on whether the decision tree trained using your hyperparameters and data transformations achieves a higher accuracy than our solution. There are 100 datasets in total, so the maximum number of points available is 120. The final score is the number of points capped at 100.
Constraints
- Your solution will be tested on the Contest Platform without internet access.
- The evaluation of the final solution must not take longer than 3 minutes (without a GPU).
- List of permitted libraries:
scikit-learn,numpy,pandas.
Submission Files
This notebook, completed with your solution (see the functions get_decision_tree_hyperparameters and preprocess_data).
Evaluation
During grading, the FINAL_EVALUATION_MODE flag will be set to True. The number of points will be calculated on the secret test set on the Contest Platform. If the solution does not meet the criteria or does not run correctly, you will receive 0 points for it.
Starter Code
In this section we initialise the environment by importing the required libraries and setting the random seed.
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
FINAL_EVALUATION_MODE = False # During grading on the grading system, the flag is automatically changed to True.# ######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
import random
import numpy as np
from sklearn.tree import DecisionTreeClassifier
if not FINAL_EVALUATION_MODE:
import matplotlib.pyplot as plt
seed = 42
random.seed(seed)
np.random.seed(seed)Loading the Data
File format:
A_input.npz:X_0,y_0,X_test_0, ...,X_99,y_99,X_test_99,n_datasetsA_output.npz:y_test_0, ...,y_test_99,n_datasets,baseline_acc
Where:
X_i— the matrix of training features (n_train × 2, float32),y_i— the training labels (n_train, int8, values 0/1),X_test_i— the matrix of test features (n_test × 2, float32),y_test_i— the test labels (n_test, int8).
# ######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
def load_data(prefix="A"):
input_data = np.load(f'data/{prefix}_input.npz')
output_data = np.load(f'data/{prefix}_output.npz')
n = int(input_data['n_datasets'])
print(f"\n{prefix}: {n} datasets")
for i in range(min(n, 3)):
X_tr = input_data[f'X_{i}']
y_tr = input_data[f'y_{i}']
X_te = input_data[f'X_test_{i}']
y_te = output_data[f'y_test_{i}']
print(f" Dataset {i}: train={X_tr.shape}, test={X_te.shape}, "
f"num_1s={y_tr.sum()}, num_0s={(y_tr == 0).sum()}")
print(" ...")
return n, input_data, output_data
n, input_data, output_data = load_data()Data inspection
A visualisation of selected datasets from collection A. For each selected dataset we train a default decision tree and plot the training points together with the decision regions.
# ######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
def extract_regions(tree, feature_names=None, node=0, bounds=None):
"""
Recursively extracts rectangular regions from a decision tree.
Returns a list of (bounds, predicted_class) for each leaf.
"""
if bounds is None:
n_features = tree.n_features
bounds = [(-np.inf, np.inf) for _ in range(n_features)]
if tree.children_left[node] == tree.children_right[node]:
return [(bounds, np.argmax(tree.value[node]))]
feature = tree.feature[node]
threshold = tree.threshold[node]
regions = []
left_bounds = bounds.copy()
left_bounds[feature] = (left_bounds[feature][0], min(left_bounds[feature][1], threshold))
regions.extend(extract_regions(tree, feature_names, tree.children_left[node], left_bounds))
right_bounds = bounds.copy()
right_bounds[feature] = (max(right_bounds[feature][0], threshold), right_bounds[feature][1])
regions.extend(extract_regions(tree, feature_names, tree.children_right[node], right_bounds))
return regions
def plot_dataset_with_tree(X_train, y_train, X_test, y_test, tree_params=None, ax=None, title=""):
"""Plots a dataset with the decision regions of a trained tree."""
if tree_params is None:
tree_params = {"max_depth": 10, "ccp_alpha": 0.01}
if ax is None:
_, ax = plt.subplots(1, 1, figsize=(5, 5))
tree = DecisionTreeClassifier(**tree_params)
tree.fit(X_train, y_train)
acc = tree.score(X_test, y_test)
regions = extract_regions(tree.tree_)
colors = ['lightcoral', 'lightblue']
for bounds, label in regions:
(x0, x1), (y0, y1) = bounds
if np.isinf(x0): x0 = X_train[:, 0].min() - 0.5
if np.isinf(x1): x1 = X_train[:, 0].max() + 0.5
if np.isinf(y0): y0 = X_train[:, 1].min() - 0.5
if np.isinf(y1): y1 = X_train[:, 1].max() + 0.5
ax.fill_between([x0, x1], y0, y1, color=colors[label % 2], alpha=0.25)
ax.scatter(X_train[:, 0], X_train[:, 1], c=1-y_train, cmap='bwr', s=3, alpha=0.6)
ax.set_title(f"{title}\nacc={acc:.3f}, nodes={tree.tree_.node_count}", fontsize=9)
ax.set_aspect('equal')
ax.tick_params(labelsize=6)
return acc
if not FINAL_EVALUATION_MODE:
# Visualisation of 10 selected datasets from collection A
sample_ids = [0, 10, 20, 30, 40, 50, 60, 70, 80, 90]
fig, axes = plt.subplots(2, 5, figsize=(25, 10))
for j, i in enumerate(sample_ids):
X_tr = input_data[f'X_{i}']
y_tr = input_data[f'y_{i}']
X_te = input_data[f'X_test_{i}']
y_te = output_data[f'y_test_{i}']
ax = axes[j // 5, j % 5]
plot_dataset_with_tree(X_tr, y_tr, X_te, y_te, ax=ax, title=f"Dataset {i}")
plt.tight_layout()
plt.show()Your Solution
Place your solution in this section. Implement the functions
get_decision_tree_hyperparameters(X_train, X_test), which returns a dictionary with the hyperparameters of the decision tree,preprocess_data(X_train, X_test), which returns the processed data. The resulting matrices must have the same dimensions as the corresponding input matrices.
Example tree hyperparameters:
max_depth— the maximum depth of the tree,min_samples_split— the minimum number of samples required to split a node,min_samples_leaf— the minimum number of samples in a leaf,max_features— the number of features considered at each split.
def preprocess_data(X_train, X_test):
"""
Data preprocessing.
"""
# TODO: write your solution here!
return X_train, X_test
def get_decision_tree_hyperparameters(X_train, y_train):
"""
Selection of the decision tree hyperparameters.
"""
# TODO: write your solution here!
return {"max_depth": 3}Evaluation
Running the cell below lets you check how many points your solution would score on the datasets from group A, which are available during the contest. Before submitting, make sure that the whole notebook runs from start to finish without errors and without any user intervention after selecting the "Run All" option.
During grading, the model will be scored on the hidden group B of datasets, using an evaluation function that works on a similar principle.
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
if not FINAL_EVALUATION_MODE:
baseline_acc = output_data['baseline_acc']
beats_baseline = 0
accuracies = []
for i in range(n):
X_train = input_data[f'X_{i}']
y_train = input_data[f'y_{i}']
X_test = input_data[f'X_test_{i}']
y_test = output_data[f'y_test_{i}']
X_train_shape = X_train.shape
X_test_shape = X_test.shape
X_train_proc, X_test_proc = preprocess_data(X_train, X_test)
assert X_test_shape == X_test_proc.shape
assert X_train_shape == X_train_proc.shape
hyperparameters = get_decision_tree_hyperparameters(X_train_proc, y_train)
model = DecisionTreeClassifier(**hyperparameters)
model.fit(X_train_proc, y_train)
accuracy = model.score(X_test_proc, y_test)
accuracies.append(accuracy)
if accuracy >= baseline_acc[i]:
beats_baseline += 1
mean_acc = np.mean(accuracies)
print(f"\nFinal result:")
print(f" Analysed: {n} datasets")
print(f" Mean accuracy: {mean_acc:.4f}")
print(f" Beats baseline: {beats_baseline} / {n}")
print(f" Number of points: {min(beats_baseline * 1.2, 100)}")Translated by SOTA. The Polish original is the official version and wins wherever the two differ. The data (collection A) are in the data/ folder next to the original notebook. As in the original, the 'Your Solution' section writes get_decision_tree_hyperparameters(X_train, X_test), but the function receives (X_train, y_train); the evaluation code counts a dataset as beaten when your accuracy is greater than or equal to the baseline. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.
At a glance
- You get
data/A_input.npz(X_i,y_i, X_test_i,n_datasets) anddata/A_output.npz(y_test_i,n_datasets,baseline_acc) in the task folder.- You submit
- This notebook with
get_decision_tree_hyperparametersandpreprocess_data. - Scoring
- For each of the 100 datasets, 1.2 points if the tree trained with the submitted preprocessing and hyperparameters achieves higher test accuracy than the organisers' solution, otherwise 0; the total (maximum 120) is capped at 100.
- Rules
- Tested without Internet access; evaluation must take at most 3 minutes (without a GPU).
- Allowed libraries: scikit-learn, numpy, pandas.
- Preprocessed matrices must keep the input shapes.
- Format
- Stage II (regional, on site in Kraków, Poznań, Warsaw and Wrocław), 13–15 March 2026; two tasks per day in 5-hour sessions. Evaluated automatically on the Competition Platform (Platforma Konkursowa) on a hidden test set; points are rounded to an integer, and a notebook that fails the requirements or does not run scores 0.