Checklist Bulgaria selection 2026 National Competition in AI, final (in-person) round · Task 3

Training-Free Statistical Defect Localisation in Latent Space

Bulgarian title: Статистическо локализиране на дефекти в латентното пространство без обучение

Detect and localise product defects from precomputed patch embeddings, without training, robustly across two data regimes.

Vision
Anomaly detection and localisation
Bulgarian original · English translation

Download the English notebook Open the task notebook (Bulgarian)

The task

For each product only precomputed patch embeddings are available (1,369 patches, i.e. a 37x37 grid). Some examples contain a real local defect; the data come in two regimes (shift_id 0 or 1) that change the embedding distribution without indicating a defect. For each category (atlas_id) an 'atlas' of normal embeddings is given, together with calibration data containing only normal examples.

Contestants implement compute_heatmap(X, atlas) returning a 37x37 anomaly map, smooth_heatmap(heatmap), an optional compute_alarm(heatmap) returning an image-level score, and calibrate_thresholds(calibration_data) returning per-category, per-regime image-level and pixel-level thresholds, and must describe the method briefly. A small public validation set is provided for checking; the final evaluation uses hidden data.

Abridged and translated by SOTA from the official Bulgarian materials. The official statement has the exact rules, and it wins wherever this summary differs.

In English

This task was published in Bulgarian. SOTA translated it into English on 17 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.

Task notebookEnglish · .ipynb · 16 KB Bulgarian original of Task notebook

Read the task notebook in English 575 words and 8 code cells

Training-Free Statistical Defect Localisation in Latent Space

You work at the "Atlas" factory. For each product you have only pre-computed embeddings (with no access to the original images). Some examples contain a real defect (a local problem in the product), while others are normal. The data comes in two modes (shift_id = 0 or 1), which change the distribution of the embeddings without this meaning a defect. Your task is to detect defects and to be robust to these differences between the modes.

You have an "atlas" of normal embeddings for each category (atlas_id), stored in the files data/atlas_{atlas_id}.pt (a tensor (M, D) which was collected from many images, not just from 1, so you cannot build a grid of patches from it), as well as calibration data data/calibration.pt, which contains only normal examples and includes atlas_id and shift_id. The evaluation examples available to you locally are in val_data/: this is a small public set, intended only for checking whether your method works and roughly what results it gives. You must give a short description of your solution.

Important: your final solution will not be evaluated on val_data/; it will be tested on other (hidden) data. Therefore, do not "tune" your method specifically for val_data/—the goal is an approach that generalises.

For each example you must produce:

a heatmap of size 37×37, where a larger value means a higher probability that the corresponding patch contains an anomaly;
an alarm (scalar), a score of whether the example as a whole is defective.

Constraints: only the libraries torch and numpy are allowed. In each of the functions, saving and loading any data whatsoever to or from the disk is strictly forbidden. Training or fine-tuning is not allowed. Do NOT change the cell tags!

Scoring:

separation of normal and defective examples at image-level (via alarm)
localisation of the defect at pixel/patch-level (via heatmap).

Both continuous metrics (AUC) and discrete metrics (F1) are used, and the discrete ones require choosing a threshold/calibration. The final score on the test set is computed as $0.5(\text{AUC}_\text{img} + \text{AUC}_\text{pix} + 0.2(\text{F1}_\text{img} + \text{F1}_\text{pix}))$ . On the validation set you will be able to evaluate only the $\text{AUC}$ metrics.

Submitting the solution:

the executed Jupyter notebook (the description of the task must be HERE), and the file must be named as follows: Task_3_USER_ID.ipynb, where USER_ID is your identification number.

# Downloading the data from Google Drive
import os
from pathlib import Path
import zipfile
import gdown

VAL_ID = "1z-aeaEwY9yp47x2UFrCzrKD7leVM5OrW"  # val_data.zip
DATA_ID = "1HyyI-0oB72E8ak1X3gQZrMgdExGlQ7oB"  # data.zip

def download_and_extract(file_id, out_zip, out_dir):
    if not Path(out_zip).exists():
        gdown.download(id=file_id, output=out_zip, quiet=False)
    Path(out_dir).mkdir(parents=True, exist_ok=True)
    with zipfile.ZipFile(out_zip, 'r') as zf:
        zf.extractall(out_dir)

download_and_extract(VAL_ID, 'val_data.zip', './')
download_and_extract(DATA_ID, 'data.zip', './')

import json
from pathlib import Path

import numpy as np
import torch
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec

DATA_DIR = Path("data")
VAL_DIR  = Path("val_data")

# We load the validation data
val_manifest = VAL_DIR / "val_manifest.json"
with open(val_manifest) as f:
    manifest = json.load(f)

# We load the atlases
atlases = {}
for p in sorted(DATA_DIR.glob("atlas_*.pt")):
    atlas_id = int(p.stem.split("_")[-1])
    atlases[atlas_id] = torch.load(p, map_location="cpu").float()

if not atlases:
    raise FileNotFoundError("There is no atlas_*.pt in data/.")


# We load the calibration data
calib_list = torch.load(DATA_DIR / "calibration.pt", map_location="cpu")
calibration_data = []
for item in calib_list:
    atlas_id = int(item["atlas_id"])
    shift_id = int(item["shift_id"])
    X = item["X"]
    atlas = atlases[atlas_id]
    calibration_data.append(
        {"X": X, "atlas": atlas, "atlas_id": atlas_id, "shift_id": shift_id}
    )

Visual intuition

The following plot compares the distributions of the L2 norms of the embeddings for:

A defect-free example (category 0)
A defective example (category 0)

X_good = next(torch.load(m["file"])["X"].float() for m in manifest
              if m["atlas_id"] == 0 and not bool(torch.load(m["file"]).get("is_anomaly", False)))

bad = next(torch.load(m["file"]) for m in manifest
           if m["atlas_id"] == 0 and bool(torch.load(m["file"]).get("is_anomaly", False)))
X_bad, mask_bad = bad["X"].float(), bad.get("mask", None)

map_good = torch.norm(X_good, p=2, dim=1).view(37,37).cpu().numpy()
map_bad  = torch.norm(X_bad,  p=2, dim=1).view(37,37).cpu().numpy()
vmin, vmax = min(map_good.min(), map_bad.min()), max(map_good.max(), map_bad.max())
fig = plt.figure(figsize=(10, 4))
gs = GridSpec(1, 4, figure=fig, width_ratios=[1, 1, 0.05, 1])
ax0 = fig.add_subplot(gs[0, 0])
ax1 = fig.add_subplot(gs[0, 1])
axc = fig.add_subplot(gs[0, 2])
axm = fig.add_subplot(gs[0, 3])
im0 = ax0.imshow(map_good, cmap="viridis", vmin=vmin, vmax=vmax)
ax0.set_title("Defect-free (cat 0)")
ax0.axis("off")
im1 = ax1.imshow(map_bad, cmap="viridis", vmin=vmin, vmax=vmax)
ax1.set_title("Defective (cat 0)")
ax1.axis("off")
fig.colorbar(im1, cax=axc, label="L2 norm")
axm.imshow(mask_bad.cpu().numpy(), cmap="gray")
axm.set_title("Mask")
axm.axis("off")
plt.tight_layout()
plt.show()

Computing the heatmap

Implement the function compute_heatmap(X, atlas).

Input:

X: a tensor of size (1369, D)
atlas: a tensor of size (M, D)

Output:

a tensor of size (37, 37) - the patches that contain an anomaly must have a higher value

@torch.no_grad()
def compute_heatmap(X: torch.Tensor, atlas: torch.Tensor) -> torch.Tensor:
    # TODO: Implement the function here.
    return torch.zeros(37, 37)

Smoothing (smoothing)

Implement the function smooth_heatmap(heatmap).

@torch.no_grad()
def smooth_heatmap(heatmap: torch.Tensor) -> torch.Tensor:
    # TODO: Implement the function here.
    return heatmap

Image score (alarm) - optional

Implement the function compute_alarm(heatmap).

@torch.no_grad()
def compute_alarm(heatmap: torch.Tensor) -> float:
    """
    This is the default function, which returns the maximum.
    """
    return float(heatmap.max().item())

Threshold calibration

Implement the function calibrate_thresholds(calibration_data).

Input:

calibration_data: a list of records, each containing X, atlas, atlas_id, shift_id

Output:

a dictionary of the form thresholds[atlas_id][shift_id] = {"tau_img": ..., "tau_pix": ...} - calibration thresholds for image-level and for pixel_level

@torch.no_grad()
def calibrate_thresholds(calibration_data):
    # calibration_data: a list of records (X, atlas, atlas_id, shift_id)
    img_scores = {}   # (atlas_id, shift_id) -> list[float]
    pix_scores = {}   # (atlas_id, shift_id) -> list[np.ndarray]

    for item in calibration_data:
        X = item["X"]
        atlas = item["atlas"]
        atlas_id = int(item["atlas_id"])
        shift_id = int(item["shift_id"])
        key = (atlas_id, shift_id)

        # 1) heatmap for a normal example
        heat = compute_heatmap(X, atlas)

        # 2) smoothing
        heat = smooth_heatmap(heat)

        # 3) image-level score
        alarm = compute_alarm(heat)

        # 4) we collect the data (for computing the thresholds later)
        img_scores.setdefault(key, []).append(float(alarm))
        pix_scores.setdefault(key, []).append(heat.reshape(-1).detach().cpu().numpy())

    # TODO: This part is yours
    thresholds = {}
    for (atlas_id, shift_id) in img_scores.keys():
        thresholds.setdefault(atlas_id, {})[shift_id] = {
            "tau_img": None,
            "tau_pix": None,
        }

    return thresholds

Testing

Use the provided validation data to check your solution.

# --- Evaluation on val_data ---

def auc_safe(y_true, y_score):
    """ROC AUC, but if both classes are not present -> NaN."""
    y_true = np.asarray(y_true, dtype=np.uint8)
    y_score = np.asarray(y_score, dtype=np.float32)
    if len(np.unique(y_true)) < 2:
        return float("nan")
    return float(roc_auc_score(y_true, y_score))

thresholds = calibrate_thresholds(calibration_data)  # may be empty/None — OK for AUC

img_true = []
img_score = []

pix_true_all = []
pix_score_all = []

for m in manifest:
    data = torch.load(m["file"], map_location="cpu")

    atlas_id = int(m["atlas_id"])
    shift_id = int(m.get("shift_id", 0))
    X = data["X"]
    atlas = atlases[atlas_id]

    # heatmap + smoothing
    heat = compute_heatmap(X, atlas)
    heat = smooth_heatmap(heat)

    # Raw scores
    raw_alarm = float(compute_alarm(heat))
    raw_heat = heat.detach().cpu().view(-1).numpy().astype(np.float32)

    # Normalize if thresholds exist
    t = thresholds.get(atlas_id, {}).get(shift_id, None)
    tau_img = t.get("tau_img", 1.0) if t else 1.0
    tau_pix = t.get("tau_pix", 1.0) if t else 1.0

    if tau_img is None or tau_img <= 1e-12: tau_img = 1.0
    if tau_pix is None or tau_pix <= 1e-12: tau_pix = 1.0

    # image-level ground truth and score
    y_img = 1 if bool(data.get("is_anomaly", False)) else 0
    img_true.append(y_img)
    img_score.append(raw_alarm / tau_img)

    # heatmap-level ground truth and score
    gt = data["mask"].to(torch.uint8).view(-1).numpy()  # (1369,)
    pix_true_all.append(gt)
    pix_score_all.append(raw_heat / tau_pix)

# Metrics: AUC and final score
img_auc = auc_safe(img_true, img_score)

pix_true = np.concatenate(pix_true_all, axis=0).astype(np.uint8)
pix_score = np.concatenate(pix_score_all, axis=0).astype(np.float32)
heat_auc = auc_safe(pix_true, pix_score)

final_score = 0.5 * img_auc + 0.5 * heat_auc

print(f"img_auc      : {img_auc:.4f}")
print(f"heatmap_auc  : {heat_auc:.4f}")
print(f"final_score  : {final_score:.4f}")
print(f"val_images   : {len(img_true)} (anom={sum(img_true)}, normal={len(img_true)-sum(img_true)})")

Translated by SOTA. The Bulgarian original is the official version and wins wherever the two differ. The notebook downloads data.zip and val_data.zip from Google Drive with gdown. The statement asks not to change the cell tags, but the published copy has no cell tags. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.

At a glance

You get: data/atlas_{atlas_id}.pt (M x D tensors), data/calibration.pt, and a small val_data/ set.
You submit: The executed notebook Task_3_USER_ID.ipynb containing the functions and the description.
Scoring: 0.5 x (AUC_img + AUC_pix + 0.2 x (F1_img + F1_pix)) on hidden test data; only the AUC metrics can be computed on the validation set.
Rules: Only torch and numpy.
No training or fine-tuning.
Functions must not save or load any data from disk.
Cell tags must not be changed.
Format: Final (in-person) round of Bulgaria's first National Competition in AI (school year 2025/2026), held on 28 February 2026 in Plovdiv (the organisers announced 28 Feb - 1 Mar 2026); the top 120 of the online first round (31 Jan 2026) were invited. Task 3.

Details

Olympiad: Bulgarian AI olympiad selection: National Competition in Artificial Intelligence (Национално състезание по изкуствен интелект) and IOAI team selection
Year: 2026, Mathematics High School 'Akad. Kiril Popov', Plovdiv, Bulgaria
Round: National Competition in AI, final (in-person) round · Task 3
Language: Bulgarian; English translation by SOTA
License: Not stated by the source

Training-Free Statistical Defect Localisation in Latent Space

The task

In English

Training-Free Statistical Defect Localisation in Latent Space

Visual intuition

Computing the heatmap

Smoothing (smoothing)

Image score (alarm) - optional

Threshold calibration

Testing

At a glance

Links

In English (SOTA)

Notebook

Source files

Details