# Pink Lemurs: Classification Metrics

*English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

*Task 3 of the school stage (Moscow) of the All-Russian School Olympiad (VsOSh) 2025/26 in artificial intelligence, grades 9–11. Original: [tasks-ai-9-11-sch-msk-25-26.pdf](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/school/ai/tasks-ai-9-11-sch-msk-25-26.pdf).*

A nature reserve plans to introduce automatic monitoring of rare *pink lemurs*. To test the system, a group of 100 animals was selected, in which 10% are pink lemurs (class 1) and the remaining 90% are ordinary lemurs (class 0). We will call this group the *test set*.

For each individual in the test set, AI models made a prediction of which class (0 or 1) that lemur belongs to. To assess the quality of recognition, we introduce the following standard notation:

- $TP$. Predicted class (1), actual class (1).
- $FP$. Predicted class (1), actual class (0).
- $TN$. Predicted class (0), actual class (0).
- $FN$. Predicted class (0), actual class (1).

In total, the test set contains $P = TP + FN$ objects (lemurs) of class 1 and $N = TN + FP$ objects of class 0.

Four AI models (A, B, C and D) recognised the lemurs from the test set. Model A assigned all lemurs to class 0, and model B assigned all lemurs to class 1. Data on the performance of models C and D are given in the table below.

| Model | TP | FP | TN | FN |
|:---:|:---:|:---:|:---:|:---:|
| C | 7 | 8 | 82 | 3 |
| D | 8 | 14 | 76 | 2 |

For models A, B, C, D, we compute the following five *metrics* $M_1, M_2, M_3, M_4, M_5$ (if a division by zero occurs for some model, the corresponding metric is not computed for it).

$$
\begin{aligned}
M_1 &= \frac{TP + TN}{P + N} && \text{(overall accuracy)},\\
M_2 &= \frac{TP}{TP + FN} && \text{(correct on class 1)},\\
M_3 &= \frac{TN}{TN + FP} && \text{(correct on class 0)},\\
M_4 &= \tfrac{1}{2}\,(M_2 + M_3) && \text{(balanced accuracy)},\\
M_5 &= \frac{TP}{TP + FP} && \text{(precision on predictions of 1)}.
\end{aligned}
$$

For each of the metrics $M_1, M_2, \ldots, M_5$, determine for which of the models its maximum value is attained.

*Translator's note: the statement paper gives only the total for the olympiad (112 points); the official solutions give this task a maximum of 12 points.*