Checklist OAI 2025 Final (Stage III) · Task 4
SOTA-AI Editorial
Data Prototypes
Select or construct 150 labelled prototype embeddings so that one-nearest-neighbour classification of test image embeddings is as accurate as possible.
Hints
Hint 1
The starter wastes accuracy by picking prototypes at random.
Hint 2
Always allocate at least one center per class so no CIFAR-100 label disappears.
Hint 3
Use cluster representatives rather than raw random points to reduce within-class distance.
Hint 4
Allocate extra centers to broad or confusable classes using train-only statistics, then validate against the released held-out embeddings.
The solution in one line
the notebook's random baseline scores 0/100 (18.34%); a train-only classwise k-means candidate is saved, but the absent CIFAR embeddings prevent local scoring and hidden-test accuracy is unknown.
The editorial
Data Prototypes
Problem, domain, and metric
- Domain: dataset compression and nearest-neighbor classification in an embedding space.
- Task: produce exactly 150 labeled embeddings from the released training embeddings. Each hidden sample receives the label of its closest prototype under Euclidean distance.
- Metric: validation/test accuracy. Scores are 0 at accuracy ≤0.40, 100 at accuracy ≥0.68, and linear between those thresholds. Evaluation must fit within five minutes on GPU.
- Abridged statement: reduce a 40,000-example CIFAR-100 training representation to a 150-vector set that retains as much classification accuracy as possible.
Released data and EDA
The official task notebook describes 40,000 training images and 10,000 validation images (about 400/100 examples per class) from CIFAR-100. Images are resized to 224×224 and normalized with ImageNet mean/std. The provided MobileNetV3 Small returns a 1,000-dimensional output embedding. Required output is 150 float32 vectors and 150 labels in [0,100).
The notebook downloads train_val.gz, a 10.3 MB MobileNet checkpoint, and validation embeddings from Google Drive. None of those data/checkpoint assets are present in the available workspace, so embedding-level EDA and a local candidate score cannot be computed. The notebook's saved baseline output is available and gives useful evidence about the task's difficulty.
Experiments and score evidence
| Method | Validation accuracy | Official score | Evidence source |
|---|---|---|---|
| Starter: random 1–2 training embeddings per class | 18.34% | 0/100 | Saved output in the official/mirrored notebook |
| Different random starter draw | 17.97% | 0/100 | Saved output in the mirrored notebook |
| Classwise centers with one extra cluster for 50 high-spread classes | Not measured | Unknown | Candidate below; required embeddings are missing |
The two random scores are the only measured validation results. They are far below the 40% score floor. There is no local evidence that the candidate reaches the 68% full-credit threshold, and the hidden-test score is unknown.
Selected candidate
data_prototypes.py assigns every class one prototype at its training centroid, then allocates one additional center to each of the 50 classes with the largest ratio of within-class radius to nearest competing centroid distance. Those classes receive a deterministic farthest-first, two-center k-means fit; the other 50 retain one center. It reads only train_embeddings and train_labels and returns the required shapes and dtypes.
This is a train-only candidate, not a validation-verified solution. Its rationale is to spend the limited extra 50 prototypes on classes whose examples are broad relative to their separation from other class centers. The missing embedding archive blocks both validation comparison and runtime measurement in this workspace.
Alternatives and caveats
The starter's random sample should be replaced by classwise cluster representatives. Other train-only choices include medoids, allocating extras by an internal train split's class-confusion matrix, or directly optimizing prototype locations against a train-only nearest-prototype loss. Those alternatives need the missing embedding assets before they can be compared. The candidate's radius/separation heuristic may misallocate capacity, and its accuracy should not be reported as a score until it is run in the official evaluator.
Code
Solution code
"""Train-only 150-prototype construction for the 2025 OAI task."""
from __future__ import annotations
import torch
class YourSolution:
"""Allocate prototypes to broad/confusable classes, then run classwise k-means."""
@staticmethod
def _two_means(x: torch.Tensor, mean: torch.Tensor) -> torch.Tensor:
"""Small deterministic farthest-first k-means with two centers."""
squared_from_mean = (x - mean).square().sum(dim=1)
center0 = mean
center1 = x[squared_from_mean.argmax()]
centers = torch.stack((center0, center1))
for _ in range(12):
distances = torch.cdist(x, centers).square()
assignment = distances.argmin(dim=1)
updated = []
for cluster_id in range(2):
members = x[assignment == cluster_id]
if members.shape[0] == 0:
other = centers[1 - cluster_id]
updated.append(x[((x - other) ** 2).sum(dim=1).argmax()])
else:
updated.append(members.mean(dim=0))
new_centers = torch.stack(updated)
if torch.max(torch.abs(new_centers - centers)).item() < 1e-5:
centers = new_centers
break
centers = new_centers
return centers
def get_prototypes(self, train_embeddings, train_labels):
"""Return 150 float32 centers and their long class labels.
Only the released training embeddings/labels are read. Every class
receives one center; one extra center is assigned to the 50 classes
with the largest within-class spread relative to centroid separation.
"""
device = train_embeddings.device
x = train_embeddings.detach().to(device=device, dtype=torch.float32)
y = train_labels.detach().to(device=device, dtype=torch.long)
classes = torch.unique(y, sorted=True)
if classes.numel() != 100:
raise ValueError(f"expected 100 classes, got {classes.numel()}")
class_rows = []
means = []
radii = []
for class_id in classes:
class_x = x[y == class_id]
if class_x.shape[0] == 0:
raise ValueError(f"training embeddings are missing class {int(class_id)}")
mean = class_x.mean(dim=0)
means.append(mean)
radii.append((class_x - mean).square().sum(dim=1).mean().sqrt())
class_rows.append(class_x)
means = torch.stack(means)
center_distances = torch.cdist(means, means)
center_distances.fill_diagonal_(float("inf"))
separation = center_distances.min(dim=1).values.clamp_min(1e-6)
spread_score = torch.stack(radii) / separation
extra_class_indices = torch.topk(spread_score, k=50, largest=True).indices
use_two = torch.zeros(100, dtype=torch.bool, device=device)
use_two[extra_class_indices] = True
prototypes = []
prototype_labels = []
for class_index, class_id in enumerate(classes):
class_x = class_rows[class_index]
if bool(use_two[class_index]):
class_centers = self._two_means(class_x, means[class_index])
else:
class_centers = means[class_index].unsqueeze(0)
prototypes.append(class_centers)
prototype_labels.extend([int(class_id)] * class_centers.shape[0])
prototypes = torch.cat(prototypes, dim=0).to(device=device, dtype=torch.float32)
prototype_labels = torch.tensor(
prototype_labels, device=device, dtype=torch.long
)
if prototypes.shape[0] != 150 or prototype_labels.shape[0] != 150:
raise RuntimeError("prototype allocation did not produce exactly 150 rows")
return prototypes, prototype_labelsEditorials are written by members of the SOTA community, not by the olympiad. The official statement and scoring win wherever an editorial differs.