Checklist OAI 2024 Stage II – Final · Task 2
Anomaly Detection
Polish title: Detekcja anomalii
Train a neural anomaly detector only on normal images and classify test images as normal (0) or anomalous (1).
The task
Anomaly detection means finding the rare samples that come from outside the distribution of the typical data, as in quality control on a production line or network intrusion detection. Using the supplied images, the contestant proposes a self-supervised or unsupervised algorithm that uses a neural network; anomalies carry label 1 and normal observations label 0.
The model may be trained only on the training set, which contains normal observations exclusively. The solution must set two global variables: BATCH_SIZE, used for the test data loader, and model, a trained instance of the Model class with forward() and predict(); predict() must assign each image in a batch to class 0 or 1 without changing its signature.
The test set, available only to the organisers, is balanced and has the same characteristics as the supplied images.
Abridged and translated by SOTA from the official Polish materials. The official statement has the exact rules, and it wins wherever this summary differs.
In English
This task was published in Polish. SOTA translated it into English on 16 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.
Read the task notebook in English
Anomaly detection

Anomaly detection is a problem that is frequently tackled in business applications. One example is quality analysis of the products manufactured on a production line, where most products are manufactured correctly (they meet the standards) and a small number of samples are defective. Another example of an anomaly detection problem is systems that analyse network traffic, where the goal is to detect outlying events, e.g. DDoS attacks.
The anomaly detection problem can therefore be formalised as finding outlying samples/events (outliers), which usually occur rarely in the dataset. One can say that the dataset contains many "typical" samples coming from some data distribution, and the goal is to find the few samples that come from outside this distribution.
Task
On the basis of the provided dataset, propose a self-supervised or unsupervised algorithm for anomaly detection. Anomalies are marked with the label 1. Samples containing normal observations are marked with the label 0. Use a neural network in your solution. In the end, your solution should assign values to two global variables:
BATCH_SIZE- the size of the data batches that the dataloaders will operate on. During grading we will read this value and use it for the dataloader with the test data.model- the prepared model, which will consist of aforward()function, traditionally used to propagate data through the network, and apredict()function, which is to make the final assignment of a given sample to one of the classes:0or1. Themodelvariable must be a trained instance of theModelclass.modelwill be used to evaluate the solution on the test set.
NOTE: The model may be trained only on the training set, which contains only normal observations.
NOTE: The variable names and the signature of the predict function must remain unchanged - you can use the checker to verify these requirements.
Scoring
After the model you prepared has been trained, the predict() method will be called to score the solution. It is to contain the mechanism that assigns the input data sample on the basis of the model's output and any additional criteria you have prepared for judging the model's predictions.
The quality of the proposed solution will be determined by computing the classification accuracy on the test set, which is available to the organisers. The test set is balanced, and the images in it have the same characteristics as those provided to participants.
A model achieving a classification accuracy below 60% will receive 0 points, one above 90% will receive 1 point, and intermediate values will be scaled linearly within this range. The participant's final score will be computed using the formula below:
where is the classification accuracy on the test set.
Submission files
- Only this notebook
Constraints
- The evaluation of your solution (including the training of the neural network, with the
FINAL_EVALUATION_MODEflag set toTrue) on 10 000 test examples should take no longer than 15 minutes on Google Colab with a GPU.
Evaluation
Remember that during grading the FINAL_EVALUATION_MODE flag will be set to True.
For this task you can score between 0 and 1 point. The number of points you score will be equal to the value of score, computed on the test set.
Starter code
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
FINAL_EVALUATION_MODE = False
# While grading your solution, we will change this value to True
# The value of this flag M U S T be set to False in the solution you send us!
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
import random
import warnings
from pathlib import Path
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
import torch.optim as optim
import torchvision.transforms as transforms
from torch.utils.data import DataLoader, Dataset
warnings.filterwarnings("ignore", category=UserWarning)
np.random.seed(0)
random.seed(0)
torch.manual_seed(0)
if not FINAL_EVALUATION_MODE:
! gdown https://drive.google.com/uc?id=1QoTW4eZctWvzmjrNrwmai9zhL8RWnhBU
# ! gdown https://drive.google.com/uc?id=1bD38bZf8pcUyinuvYDXO56RPlXUluzxn
# ! gdown https://drive.google.com/uc?id=108d2ERztZzq5_ZXyizxeWDOCctG5xhIy
! unzip anomaly.zip
! unzip train.zip
! unzip valid.zip
Loading the data
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
TRAIN_DIR: Path = Path("./train")
VALID_DIR: Path = Path("./valid")
TRAIN_CSV: Path = Path("./train.csv")
VALID_CSV: Path = Path("./valid.csv")
# Set your batch size -- we will require this variable to be present during grading and will use it during testing
BATCH_SIZE: int = 32
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
class ImageDataset(Dataset):
def __init__(self, dir: Path, csv: Path):
self.dir: Path = dir
self.csv = pd.read_csv(csv)
def __len__(self):
return len(self.csv)
def __getitem__(self, idx) -> tuple[torch.Tensor, int]:
if torch.is_tensor(idx):
idx = idx.tolist()
path, label = self.csv.iloc[idx]
img = plt.imread(self.dir / path)
to_tensor = transforms.ToTensor()
return to_tensor(img), label
def train_dataloader() -> DataLoader:
"""Create a Dataloader with the training data."""
return DataLoader(ImageDataset(TRAIN_DIR, TRAIN_CSV), batch_size=BATCH_SIZE, shuffle=True)
def valid_dataloader() -> DataLoader:
"""Create a Dataloader with the validation data."""
return DataLoader(ImageDataset(VALID_DIR, VALID_CSV), batch_size=BATCH_SIZE, shuffle=False)
Example solution skeleton + naive solution
Below we present a simple solution which is obviously not optimal. Its purpose is to show how the whole notebook is meant to work.
class Model(torch.nn.Module):
def __init__(self):
super().__init__()
def forward(self, x: torch.Tensor) -> torch.Tensor:
return torch.rand(x.shape[0])
def predict(self, batch: torch.Tensor) -> torch.Tensor:
"""
Assign 0 or 1 to each image in the batch.
Do not change or add arguments in this method!
"""
predictions = torch.ones(batch.shape[0])
return predictions
def train_model() -> Model:
"""Create and train the model"""
return Model()
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
model = train_model()
Scoring criterion code
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
def grade_solution(model):
dataloader = valid_dataloader() # During grading we will replace this with test_dataloader
predictions = np.concatenate([
model.predict(images).cpu().to(dtype=torch.int32).numpy() for images, _ in dataloader
], axis=0)
labels = np.concatenate([label for _, label in dataloader], axis=0, dtype=np.int32)
accuracy = sum(labels == predictions) / len(labels)
score = min(max(accuracy - 0.6, 0.0), 0.3) / 0.3
print(f"Accuracy: {accuracy}")
print(f'Your score is {score} pts')
return score
Evaluation run on the provided dataset
(ultimately it will be run on the test set)
######################### DO NOT CHANGE THIS CELL WHEN SUBMITTING ##########################
if not FINAL_EVALUATION_MODE:
grade_solution(model)
Translated by SOTA. The Polish original is the official version and wins wherever the two differ. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.
At a glance
- You get
- Image folders train and valid with
train.csvandvalid.csv(path, label), fromanomaly.zipon Google Drive. - You submit
- This notebook only, defining BATCH_SIZE and the trained model.
- Scoring
- Accuracy on the balanced test set; score = min(max(accuracy − 0.6, 0), 0.3) / 0.3, i.e. 0 points below 60%, 1 point above 90%, linear in between.
- Rules
- Evaluation including training (FINAL_EVALUATION_MODE = True) on 10,000 test examples must take at most 15 minutes on Google Colab with a GPU.
- Training on the normal-only training set only; a neural network must be used.
- Format
- Stage II final contest (five hours) held during the final scientific camp in Krzyżowa, 15–21 June 2024; about 30 top Stage I participants took part.