Checklist OAI 2026 Stage I · Task 3

Whisper or Scream?

Polish title: Szept czy krzyk?

Classify 16 kHz recordings of single Polish words as spoken normally, screamed or whispered.

Audio
Audio classification (3 classes)
Polish original · English translation

Download the English notebook Open the task notebook (Polish)

The task

A fictional acoustics institute has built a "Speech Loudness Assessment Machine" that lacks a model recognising whether a word was spoken normally, whispered or screamed. The contestant implements a model that classifies recordings into three classes: 0 normal, 1 scream and 2 whisper.

The training set has 2,400 samples and the validation set 200; the hidden test set has 400. All recordings are sampled at 16 kHz and contain single Polish words whose meaning is irrelevant; recordings are normalised to the same loudness level.

The model must inherit from nn.Module, be stored in the variable your_model and provide predict(), which returns a list of predicted labels; your_model.predict is what is evaluated.

Abridged and translated by SOTA from the official Polish materials. The official statement has the exact rules, and it wins wherever this summary differs.

In English

This task was published in Polish. SOTA translated its 2 files into English on 16 September 2026. Only the words changed in the notebooks: markdown, code comments, messages and printed output. The code, file names and paths are the original's, so a translated notebook runs with the original data.

Read the task notebook in English 54091 words and 14 code cells

Whisper or Scream?

Image generated with ChatGPT's image generation tool.

Introduction

There is exceptional excitement at the Institute of Advanced Experimental Acoustics. After many years, a prototype of the Speech Loudness Assessment Machine has been launched: an advanced device that can assess the loudness of speech.

Or at least it should be able to.

Unfortunately, the machine lacks one key feature - a good model that recognises whether a word was spoken normally, in a whisper, or in a scream. Without it, the device flashes its lights completely at random, which, according to the research team, somewhat lowers users' trust.

Your task is to prepare a model that will allow this machine to work as described in the documentation.

Task

Implement a model that classifies audio recordings into three classes:

0 (normal) - a word spoken in a normal tone,
1 (scream) - a word spoken in a scream,
2 (whisper) - a word spoken in a whisper.

Data

In this task, two datasets are available to you:

training - $2\,400$ samples,
validation - $200$ samples.

The hidden test set, on which your solution will be evaluated, has $400$ samples.

All recordings have a sampling rate of 16 kHz and contain single words in Polish, whose meaning has no bearing on the task.

Scoring Criterion

Your solution will be evaluated on the basis of balanced classification accuracy (balanced accuracy):

\text{BalancedAccuracy} = \frac{1}{3} \sum_{k} \frac{\text{TP}_k}{\text{TP}_k + \text{FN}_k}

where $k \in \{0, 1, 2\}$ , $\text{TP}$ is the number of samples for which the model correctly detected the given class, and $\text{FN}$ is the number of samples for which the model did not detect the given class.

You can score between 0 and 100 points for this task. The score will be scaled linearly according to the value of Balanced Accuracy:

Balanced Accuracy $\leq$ 0.6: 0 points.
Balanced Accuracy $\geq$ 0.99: 100 points.
Values between 0.6 and 0.99: scaled linearly.

Score formula:

\text{Points} = \begin{cases} 0 & \text{for } \text{Balanced Accuracy} \leq 0.6 \\ 100 \times \frac{\text{Balanced Accuracy} - 0.6}{0.99 - 0.6} & \text{for } 0.6 < \text{Balanced Accuracy} < 0.99 \\ 100 & \text{for } \text{Balanced Accuracy} \geq 0.99 \end{cases}

Constraints

You may use a GPU in this task.
Evaluation of your final solution on the Contest Platform may not take longer than 5 minutes with a GPU.
List of permitted libraries: sklearn, tqdm, seaborn, matplotlib, numpy, torch, torchaudio, librosa.
The model must inherit from the nn.Module class.

Notes and hints

The recordings are normalised to the same loudness level.
You may validate your solution on the training and validation sets, but the score for the task will be awarded solely on the basis of the result on the test set.

Submission files

This notebook, completed with your solution (see the YourClassifier class).

Evaluation

Remember that during checking, the FINAL_EVALUATION_MODE flag will be set to True.

You can score between 0 and 100 points for this task. The number of points you receive will be calculated on the (secret) test set on the Contest Platform using the formula above, rounded to the nearest integer. If your solution does not meet the above criteria or does not run correctly, you will receive 0 points for the task.

Starter Code

In this section, we initialise the environment by importing the required libraries and functions. The prepared code will make it easier for you to work with the data efficiently and to build the actual solution.

######################### DO NOT CHANGE THIS CELL ###########################

FINAL_EVALUATION_MODE = False  # We will set this flag to True during checking.

######################### DO NOT CHANGE THIS CELL ###########################

import os
import torch
import numpy as np
import torch.nn as nn
import matplotlib.pyplot as plt
import seaborn as sns
import torchaudio
import librosa
from typing import Callable
from torch.utils.data import Dataset
from tqdm import tqdm
from torch.utils.data import DataLoader
from sklearn.metrics import balanced_accuracy_score, confusion_matrix
from IPython.display import Audio

if not FINAL_EVALUATION_MODE:
    import gdown

DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")

assert torch.cuda.is_available(), "CUDA unavailable!"

######################### DO NOT CHANGE THIS CELL ###########################

seed = 12345

os.environ["PYTHONHASHSEED"] = str(seed)
torch.manual_seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

######################### DO NOT CHANGE THIS CELL ###########################
# Cell containing helper functions for preparing the data.

class WhisperScreamDataset(Dataset):
    """
    Dataset loaded from an npz file.

    Args:
        path (str): Path to the npz file containing the data.
    """
    def __init__(self, path: str) -> None:
        data = np.load(path, allow_pickle=True)

        self.signals = data["signals"]
        self.labels = data.get("labels", None)
        self.sr = int(data["sr"])

        if self.labels is None:
            self.labels = [-1] * len(self.signals)

    def __len__(self) -> int:
        """Returns the number of samples in the dataset."""
        return len(self.signals)

    def __getitem__(self, index: int) -> dict:
        sig = self.signals[index]
        label = self.labels[index]
        sig = torch.tensor(sig, dtype=torch.float32)
        return {"audio": sig, "label": label}


def setup_data(root: str = "data/") -> tuple:
    """
    Prepares the datasets for training and validation, downloading them if necessary.

    Args:
        root (str, optional): Base directory for the data files.

    Returns:
        tuple: The datasets (train_ds, val_ds).
    """
    train_file = root + "train.npz"
    val_file = root + "val.npz"

    if not os.path.exists(root):
        os.makedirs(root)

    if not os.path.exists(train_file):
        url = "https://drive.google.com/uc?id=1WpPqcKTh_jzCsBm7k32IRtqlsxgpV8u4"
        gdown.download(url, train_file, quiet=True)

    if not os.path.exists(val_file):
        url = "https://drive.google.com/uc?id=1PwVhwFAGX3cS9gA7SYbJORjpQroPVQEN"
        gdown.download(url, val_file, quiet=True)

    train_ds = WhisperScreamDataset(train_file)
    val_ds = WhisperScreamDataset(val_file)

    return train_ds, val_ds

######################### DO NOT CHANGE THIS CELL ###########################
# Cell containing helper functions for computing the metrics that assess model quality.

def predict(model_predict: Callable, dataset: Dataset) -> list:
    """
    Function that predicts all labels for a dataset using the prediction function.

    Args:
        model_predict (Callable): Prediction function returning a list of labels.
        dataset (Dataset): The dataset.

    Returns:
        list: A list containing all predicted labels, one for each sample in the dataset.
    """
    predicted = []

    for sample in dataset:
        x = sample["audio"].to(DEVICE)
        pred = model_predict(x)
        predicted.append(pred)

    return predicted


def calculate_balanced_accuracy(predicted: list, dataset: Dataset) -> float:
    """
    Function that computes the balanced accuracy of the predicted labels for the whole dataset.

    Args:
        predicted (list): A list containing the predicted labels for each sample in the
            dataset.
        dataset (Dataset): The dataset containing the expected labels.
    """
    expected = [sample["label"] for sample in dataset]
    return balanced_accuracy_score(y_true=expected, y_pred=predicted)


def plot_confusion_matrix(predicted: list, dataset: Dataset) -> None:
    """
    Function that displays the confusion matrix for the classification of recordings.

    Args:
        predicted (list): A list of predicted labels.
        dataset (Dataset): The dataset.
    """
    expected = [sample["label"] for sample in dataset]
    conf_matrix = confusion_matrix(expected, predicted)

    labels = ["normal", "scream", "whisper"]
    plt.figure(figsize=(6, 4))
    sns.heatmap(
        conf_matrix,
        annot=True,
        fmt="d",
        cmap="Blues",
        xticklabels=labels,
        yticklabels=labels,
        cbar=False,
    )

    plt.xlabel("Predicted labels", labelpad=15)
    plt.ylabel("Expected labels", labelpad=15)
    plt.xticks(rotation=45)

    plt.title("Confusion matrix for the classification of recordings")
    plt.show()


def evaluate_algorithm(your_model_predict: Callable, dataset: Dataset):
    """
    Function used to evaluate the solution.

    Args:
        your_model_predict (Callable): Prediction function returning a list of labels.
        dataset (Dataset): The dataset on which the solution will be evaluated.
    """
    your_out = predict(your_model_predict, dataset)

    your_balanced_accuracy = calculate_balanced_accuracy(your_out, dataset)
    score = (np.clip(your_balanced_accuracy, 0.6, 0.99) - 0.6) / (0.99 - 0.6) * 100
    score = int(round(score))

    print(f"Balanced accuracy on the validation set: {your_balanced_accuracy:.4f}")
    print(f"Estimated number of points for the task: {score}")

    plot_confusion_matrix(your_out, dataset)

Loading the data

The code below will load the data and prepare it appropriately.

######################### DO NOT CHANGE THIS CELL ###########################

train_ds, val_ds = setup_data(root="data/")

if not FINAL_EVALUATION_MODE:
    print(f"Number of recordings in the training set: {len(train_ds)}")
    print(f"Number of recordings in the validation set: {len(val_ds)}")
    print(f"Each sample contains: {list(train_ds[0].keys())}")

To listen to an example sample, you can use the code below.

######################### DO NOT CHANGE THIS CELL ###########################

if not FINAL_EVALUATION_MODE:
    sample = next(iter(train_ds))
    waveform = sample["audio"].cpu().numpy()

    if waveform.ndim == 2:
        waveform = waveform.T

    Audio(waveform, rate=16000)

Example solution

Below we present a simplified solution that demonstrates the basic functionality of the notebook. It can serve as a starting point for developing your solution.

######################### DO NOT CHANGE THIS CELL ###########################

class DummyClassifier(nn.Module):
    def __init__(self, in_features: int):
        super(DummyClassifier, self).__init__()
        self.max_len = in_features
        self.linear = nn.Linear(in_features=in_features, out_features=3)

    def forward(self, audio: torch.Tensor) -> torch.Tensor:
        """
        Function that classifies the recording x into one of 3 classes: 0 - normal, 1 - scream, 2 - whisper.

        Args:
            x (torch.Tensor): The recording to classify.

        Returns:
            torch.Tensor: The model's predictions.
        """
        x = self.linear(audio)
        return nn.functional.softmax(x, dim=0)

    @torch.no_grad()
    def predict(self, audio: torch.Tensor) -> list[int]:
        """
        Function that predicts the label for the recording `x`.

        Args:
            x (torch.Tensor): The recording to classify.

        Returns:
            list[int]: A list of predicted labels.
        """
        if audio.size(-1) < self.max_len:
            audio = nn.functional.pad(audio, (0, self.max_len - audio.size(-1)))
        elif audio.size(-1) > self.max_len:
            audio = audio[:self.max_len]
        if audio.dim() < 2:
            audio = audio.view(1, -1)
        self.eval()
        logits = self.forward(audio)
        preds_idx = torch.argmax(logits, dim=1)
        return preds_idx.tolist()

######################### DO NOT CHANGE THIS CELL ###########################

class DataPreprocessor:
    def __init__(self, max_len: int) -> None:
        self.max_len = max_len

    def __call__(self, batch: list) -> dict:
        audios, labels = [], []

        for sample in batch:
            audio = sample["audio"]

            if audio.size(-1) < self.max_len:
                audio = nn.functional.pad(audio, (0, self.max_len - audio.size(-1)))
            elif audio.size(-1) > self.max_len:
                audio = audio[:self.max_len]

            label = sample["label"]

            audios.append(audio)
            labels.append(label)

        return {"audios": torch.stack(audios, dim=0), "labels": torch.tensor(labels)}

######################### DO NOT CHANGE THIS CELL ###########################

def train_eval_dummy_classifier(train_ds: Dataset, val_ds: Dataset):
    max_len = 0

    for sample in train_ds:
        audio = sample["audio"]
        if audio.size(-1) > max_len:
            max_len = audio.size(-1)

    print(f"The longest recording in the training set has length: {max_len}") # This should come out as 80000.

    preprocess = DataPreprocessor(max_len=max_len)

    model = DummyClassifier(in_features=max_len).to(DEVICE)
    optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
    criterion = nn.CrossEntropyLoss()

    train_dl = DataLoader(train_ds, batch_size=2, shuffle=True, collate_fn=preprocess)

    epochs = 1

    pbar = tqdm(range(epochs), desc="Training", total=epochs)
    for _ in pbar:
        epoch_losses = []

        for batch in train_dl:
            x = batch["audios"].to(DEVICE)
            y = batch["labels"].to(DEVICE)

            optimizer.zero_grad()
            output = model(x)
            loss = criterion(output, y)
            loss.backward()
            optimizer.step()

            epoch_losses.append(loss.detach().cpu().item())

        avg_loss = np.mean(epoch_losses)
        pbar.set_postfix({"train loss": avg_loss})

    model.eval()

    evaluate_algorithm(model.predict, val_ds)

######################### DO NOT CHANGE THIS CELL ###########################

if not FINAL_EVALUATION_MODE:
    train_eval_dummy_classifier(train_ds, val_ds)

Your solution

Place your solution in this section. Make changes only here!

Your model must be stored in the variable your_model. In addition, it must have a .predict() function that returns a list of predicted labels. It is the your_model.predict function that constitutes the final solution and will be evaluated.

# Here you can implement your classifier, which will return a list of predicted labels.

class YourClassifier(nn.Module):
    def __init__(self):
        # Here you can initialise your model
        super(YourClassifier, self).__init__()
        pass

    def forward(self, audio: torch.Tensor) -> torch.Tensor:
        # Implement the logic of your classifier here.
        pass

    def predict(self, audio: torch.Tensor) -> list:
        # Implement the prediction here.
        # An example model that always returns the same label:
        return [0]

# Train your model here using train_ds and val_ds
# ...

your_model = YourClassifier()

Evaluation

The code below will be used to evaluate the solution. After you send the solution to us, the function evaluate_algorithm(your_model.predict, test_ds) will be executed, i.e. code almost identical to the code below will be run on the test set, which is available only to the task graders.

Before submitting, make sure that the whole notebook runs from start to finish without errors and without user intervention after executing the Run All command.

######################### DO NOT CHANGE THIS CELL ###########################

if not FINAL_EVALUATION_MODE:
    evaluate_algorithm(your_model.predict, val_ds)

Translated by SOTA. The Polish original is the official version and wins wherever the two differ. The plots saved in the solution notebook are images from the authors' own run, so their titles and axis labels are still in Polish; when you run the notebook, the code draws them in English. If you organise this olympiad and would like the translation removed, email [email protected] and we will take it down.

At a glance

You get: train.npz and val.npz, downloaded from Google Drive.
You submit: This notebook with YourClassifier and your_model.
Scoring: Balanced accuracy over the three classes. Points = 0 at ≤ 0.6, 100 at ≥ 0.99, 100 × (BA − 0.6)/(0.99 − 0.6) in between; rounded.
Rules: A GPU may be used; evaluation must take at most 5 minutes with a GPU.
Allowed libraries: sklearn, tqdm, seaborn, matplotlib, numpy, torch, torchaudio, librosa.
The model must inherit from nn.Module.
Format: Stage I (online), 1 December 2025 – 25 January 2026; up to 100 points per task (500 in total; the qualification threshold for Stage II was 350 points). Tasks are ordered by intended increasing difficulty. Evaluated automatically on the Competition Platform (Platforma Konkursowa) on a hidden test set; points are rounded to an integer, and a notebook that fails the requirements or does not run scores 0.

Details

Olympiad: Polish Olympiad in Artificial Intelligence (Olimpiada Sztucznej Inteligencji)
Year: 2026, Online
Round: Stage I · Task 3
Language: Polish; English translation by SOTA
License: Not stated by the source

Whisper or Scream?

The task

In English

Whisper or Scream?

Introduction

Task

Data

Scoring Criterion

Constraints

Notes and hints

Submission files

Evaluation

Starter Code

Loading the data

Example solution

Your solution

Evaluation

At a glance

Links

In English (SOTA)

Notebook

Data

Solution

Results

Source files

Details