# Final Stage 2025–2026, Tour 2: Official Solutions

*English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

All-Russian School Olympiad in Informatics 2025–2026, Final Stage<br>
“Artificial Intelligence” profile, Tour 2, Moscow, 25 March 2026

*Translator's note: the official solution is the authors' Jupyter notebook exported to PDF. Code cells are marked “In [n]”, their results “Out [n]”, and printed output follows the cell. Lines that the PDF wrapped are joined again. Comments, docstrings and messages are translated; the code itself is unchanged. Figures are described in words; see the original PDF for the images.*

---

## A. What to Watch?

### 1.1 Summary

This notebook **reads the data from files in the same folder**, solves 3 subtasks and saves the answers to CSV:

- **A1** `favorite_genre` → `answer_a1.csv`
- **A2** `actor_match` → `answer_a2.csv`
- **A4** `next_in_session` → `answer_a3.csv`

### 1.2 Files that the notebook must see

Input (in the same folder as the notebook): - `queries_A.csv`: queries (5 candidates each, `c1..c5`) - `items_A.csv`: film metadata (`genre`, `duration`) - `events_A.csv`: user events (`open/finish/like`) - `item_meta_A.json`: film metadata (lists of actors `actors`) - `sessions_A.json`: sessions (inside `sessions[*].path`)

Output (created next to the notebook):

| File | Relative path | Purpose |
|---|---|---|
| `answer_a1.csv` | `./answer_a1.csv` | answers for `favorite_genre` |
| `answer_a2.csv` | `./answer_a2.csv` | answers for `actor_match` |
| `answer_a4.csv` | `./answer_a3.csv` | answers for `next_in_session` |

### 1.3 A universal solution pattern

The same skeleton is used in all three subtasks:

1) `melt` converts the candidates `c1..c5` from wide to long format (one row per candidate).

2) `merge(..., how='left')` attaches the features/score **without losing candidates**.

3) If there are lists (actors, transition pairs), we use `explode`. Important: **empty lists give** a `NaN` row (this is convenient for keeping the candidate and getting a contribution of 0).

4) The winner is chosen by sorting with `sort_values` + `drop_duplicates('query_id')` (the first in sorted order).

5) Saving: `to_csv(index=False)`.

Links to the pandas documentation (official):

- `pandas.melt`:
  https://pandas.pydata.org/docs/reference/api/pandas.melt.html
- `DataFrame.explode`:
  https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.explode.html
- `pandas.merge`:
  https://pandas.pydata.org/docs/reference/api/pandas.merge.html
- `DataFrame.sort_values`:
  https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_values.html
- `DataFrame.drop_duplicates`:
  https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.drop_duplicates.html
- `DataFrame.to_csv`:
  https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html

In [1]:

```python
import pandas as pd
import matplotlib.pyplot as plt
```

#### 1.3.1 Quick check of data loading

Below, we read all the files once and look at the table sizes and the distribution of `query_type`.

In [2]:

```python
Q = pd.read_csv('queries_A.csv')
I = pd.read_csv('items_A.csv')
E = pd.read_csv('events_A.csv')
M = pd.read_json('item_meta_A.json')[['item_id','actors']]
S = pd.read_json('sessions_A.json')

print('Q', Q.shape, 'I', I.shape, 'E', E.shape, 'M', M.shape, 'S', S.shape)
Q.query_type.value_counts()
```

```text
Q (800, 8) I (260, 3) E (5334, 3) M (260, 2) S (180, 2)
```

Out [2]:

```text
query_type
favorite_genre     400
actor_match        200
next_in_session    200
Name: count, dtype: int64
```

In [3]:

```python
vc = Q.query_type.value_counts().sort_index()
ax = vc.plot(kind='bar', title='Number of queries by type')
ax.set_xlabel('query_type')
ax.set_ylabel('count')
plt.tight_layout()
plt.show()
```

*[Figure: see page 3 of the [original solutions PDF](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/final/ai/sol-ai-9-11-tur2-final-25-26.pdf). A bar chart titled "Number of queries by type": x axis `query_type`, y axis `count`; actor_match = 200, favorite_genre = 400, next_in_session = 200.]*

#### 1.3.2 **A1.** `favorite_genre`

In [4]:

```python
q = Q[Q.query_type == 'favorite_genre'][['query_id','user_id','c1','c2','c3','c4','c5']]

e = E[['user_id','item_id','event_type']].copy()
e['w'] = e.event_type.map({'open': 1, 'finish': 2, 'like': 3})
e = e.merge(I[['item_id','genre']], on='item_id', how='left')
g = e.groupby(['user_id','genre'], as_index=False)['w'].sum()

c = q.melt(['query_id','user_id'], ['c1','c2','c3','c4','c5'], value_name='item_id')[['query_id','user_id','item_id']]
c = c.merge(I[['item_id','genre','duration']], on='item_id', how='left')
c = c.merge(g, on=['user_id','genre'], how='left')
c['w'] = c.w.fillna(0)

c = c.sort_values(['query_id','w','duration','item_id'], ascending=[1,0,1,1])
a1 = c.drop_duplicates('query_id')[['query_id','item_id']]

a1.to_csv('answer_a1.csv', index=False)
a1.head()
```

Out [4]:

```text
      query_id  item_id
400          1     1199
1201         2     1065
1602         3     1211
403          4     1086
4            5     1163
```

#### 1.3.3 **A2.** `actor_match`

In [5]:

```python
q = Q[Q.query_type == 'actor_match'][['query_id','user_id','c1','c2','c3','c4','c5']]
e = E[E.event_type.isin(['finish','like'])][['user_id','item_id']]

m = M[['item_id','actors']].copy()
m['actors'] = m.actors.apply(lambda x: x if isinstance(x, list) else [])

ua = e.merge(m, on='item_id', how='left')
ua = ua.explode('actors').dropna(subset=['actors'])
ua = ua.drop_duplicates(['user_id','actors'])
ua['hit'] = 1
ua = ua[['user_id','actors','hit']]

c = q.melt(['query_id','user_id'], ['c1','c2','c3','c4','c5'], value_name='item_id')[['query_id','user_id','item_id']]
c = c.merge(m, on='item_id', how='left')
c = c.explode('actors')
c = c.drop_duplicates(['query_id','item_id','actors'])

c = c.merge(ua, on=['user_id','actors'], how='left')
c['hit'] = c.hit.fillna(0)

s = c.groupby(['query_id','item_id'], as_index=False)['hit'].sum()
s = s.sort_values(['query_id','hit','item_id'], ascending=[1,0,1])
a2 = s.drop_duplicates('query_id')[['query_id','item_id']]

a2.to_csv('answer_a2.csv', index=False)
a2.head()
```

Out [5]:

```text
    query_id  item_id
4        401     1252
9        402     1255
14       403     1227
19       404     1216
24       405     1232
```

#### 1.3.4 **A3.** `next_in_session`

In [6]:

```python
q = Q[Q.query_type == 'next_in_session'][['query_id','user_id','c1','c2','c3','c4','c5']]

fin = E[E.event_type == 'finish'][['user_id','item_id']].drop_duplicates()
fin = fin.rename(columns={'item_id': 'from_item'})

t = S.explode('sessions').dropna(subset=['sessions'])
t['path'] = t.sessions.apply(lambda x: x.get('path', []))
t = t[['user_id','path']]
t['pair'] = t.path.apply(lambda p: list(zip(p[:-1], p[1:])))
t = t.explode('pair').dropna(subset=['pair'])
t['from_item'] = t.pair.str[0]
t['item_id'] = t.pair.str[1]
t['hit'] = 1
tr = t.groupby(['user_id','from_item','item_id'], as_index=False)['hit'].sum()

c = q.melt(['query_id','user_id'], ['c1','c2','c3','c4','c5'], value_name='item_id')[['query_id','user_id','item_id']]
c = c.merge(fin, on='user_id', how='left')
c = c.merge(tr, on=['user_id','from_item','item_id'], how='left')
c['hit'] = c.hit.fillna(0)

s = c.groupby(['query_id','item_id'], as_index=False)['hit'].sum()
s = s.sort_values(['query_id','hit','item_id'], ascending=[1,0,1])
a3 = s.drop_duplicates('query_id')[['query_id','item_id']]

a3.to_csv('answer_a3.csv', index=False)
a3.head()
```

Out [6]:

```text
    query_id  item_id
2        601     1028
5        602     1071
11       603     1049
17       604     1101
20       605     1004
```

---

## B. Seismically Active Island

In [ ]:

```python
import numpy as np
import pandas as pd

data = np.load('data_B.npy')
N = data.shape[0]
m = np.nanmean(data, axis=1)

# Pairwise correlation
sim = np.zeros((N, N))
for i in range(N):
    for j in range(i, N):
        mask = ~np.isnan(m[i]) & ~np.isnan(m[j])
        a, b = m[i, mask] - m[i, mask].mean(), m[j, mask] - m[j, mask].mean()
        d = np.sqrt((a**2).sum() * (b**2).sum())
        if d > 0:
            sim[i, j] = sim[j, i] = (a * b).sum() / d

# Each sample goes to the cluster with the highest mean correlation
# Initialisation: greedy
labels = np.full(N, -1)
reps = []
for i in range(N):
    best_sim, best_c = -1, -1
    for c, r in enumerate(reps):
        if sim[i, r] > best_sim:
            best_sim = sim[i, r]
            best_c = c
    if best_sim > 0.35:
        labels[i] = best_c
    else:
        labels[i] = len(reps)
        reps.append(i)

# Refinement: reassign by mean correlation with the cluster
for _ in range(10):
    clusters = [np.where(labels == c)[0] for c in range(max(labels) + 1)]
    changed = False
    for i in range(N):
        best_score, best_c = -2, labels[i]
        for c, members in enumerate(clusters):
            others = members[members != i]
            score = sim[i, others].mean() if len(others) > 0 else sim[i, members[0]]
            if score > best_score:
                best_score = score
                best_c = c
        if best_c != labels[i]:
            labels[i] = best_c
            changed = True
    if not changed:
        break
    # Update the clusters
    clusters = [np.where(labels == c)[0] for c in range(max(labels) + 1)]

y = pd.read_csv('y.csv')['cluster'].values
from sklearn.metrics import adjusted_rand_score
print(f"Clusters: {len(set(labels))}, ARI = {adjusted_rand_score(y, labels):.6f}")

pd.DataFrame({'ID': range(N), 'target': labels}).to_csv('submission_B.csv', index=False)
```

*(Translator's note: in the original, this cell has not been run and shows no output.)*

---

## C. A Beginner's Mistake

In [1]:

```python
import json
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from catboost import CatBoostClassifier
import warnings
from IPython.display import display
warnings.filterwarnings('ignore')
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

# Fix the seed for reproducibility of all steps
SEED = 42
np.random.seed(SEED)

# Load the available files
train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')
solution = pd.read_csv('ytest.csv')
with open('map.json', 'r') as f:
    map_data = json.load(f)

display(train.head())
display(test.head())
```

```text
   longitude   latitude  target  habitat_quality_score  biodiversity_index  \
0 -68.624132 -32.868160       0              38.746018            0.869221
1 -51.964445   1.978971       0              17.456343            3.414865
2 -50.585866  -5.300425       0              43.883046            1.930266
3 -64.666741 -32.100087       0              29.736058            1.724559
4 -72.512342  -5.718456       1              66.547123            9.257867

   canopy_density  soil_moisture_level  forest_age_years  \
0        0.584679             6.031727         97.911830
1        0.327984             4.694215        169.739698
2        0.750885             7.864357        214.602112
3        0.322355             4.448480        275.153342
4        0.428812             3.822563        179.540444

   tree_density_per_hectare  annual_rainfall_mm  habitat_fragmentation_index  \
0                396.493164         1293.066100                     0.507221
1                350.579890         2353.814818                     0.247614
2                237.872776         1147.090078                     0.574802
3                458.725205         1252.207036                     0.648601
4                346.391230          604.782917                     0.732562

   predator_pressure_score  vegetation_complexity          forest_type  \
0                 1.040142                    3.0  tropical_rainforest
1                 7.522185                    9.0           dry_forest
2                 3.255146                    4.0           dry_forest
3                 6.404152                    9.0  tropical_rainforest
4                 4.880406                    2.0           dry_forest

  climate_zone soil_type disturbance_level conservation_status  \
0      montane      peat               low         unprotected
1    temperate      loam           minimal         unprotected
2  subtropical      loam          moderate         unprotected
3    temperate      loam               low         buffer_zone
4     tropical      clay               low         unprotected

  dominant_tree_species water_source_type  temperature_variability  \
0                  palm            stream                 4.037666
1                  pine              none                 5.882945
2                  palm          seasonal                 3.366330
3              mahogany            stream                 8.984897
4                bamboo          seasonal                 3.966467

   elevation_range  human_activity_index  canopy_height_avg  \
0       730.654928              0.075256           6.710190
1       316.780429              3.614124          30.770752
2       474.802210              2.251489           5.000000
3       187.292371              3.063365          16.284970
4       191.639490              3.888626          36.796504

   seasonal_variation_score
0                  6.874974
1                  6.384739
2                  5.690438
3                  6.328448
4                  5.780874
```

```text
   longitude   latitude  habitat_quality_score  biodiversity_index  \
0 -72.102869  19.626859              45.511580            3.743753
1 -50.441167  -8.396785              34.751298            4.712511
2 -58.372430   0.312577              73.162533            2.579567
3 -76.362920 -14.279388              86.832262            4.097908
4 -67.162159   6.450007              63.631579            4.895632

   canopy_density  soil_moisture_level  forest_age_years  \
0        0.648016             3.260149        172.921350
1        0.226638             5.191137        357.568190
2        0.747227             7.533181        300.220431
3        0.602197             6.429846        325.612904
4        1.000000             4.502920        308.714905

   tree_density_per_hectare  annual_rainfall_mm  habitat_fragmentation_index  \
0                226.157888         1177.791799                     0.558007
1                429.731562         1922.536202                     0.332591
2                144.840477         1844.822680                     0.654825
3                323.532762         1571.667498                     0.383862
4                693.209698         1688.758358                     0.344897

   predator_pressure_score  vegetation_complexity          forest_type  \
0                 7.795448                    3.0     secondary_forest
1                 5.712367                    7.0           dry_forest
2                 2.073700                    1.0         cloud_forest
3                 0.798348                    5.0           dry_forest
4                 1.378641                    1.0  tropical_rainforest

  climate_zone soil_type disturbance_level conservation_status  \
0      montane  laterite              high         unprotected
1    temperate     sandy          moderate           protected
2  subtropical      loam              high           protected
3     tropical  alluvial           minimal         buffer_zone
4  subtropical     sandy           minimal           protected

  dominant_tree_species water_source_type  temperature_variability  \
0                 kapok            stream                 5.205539
1                  palm          seasonal                 8.634001
2                  pine              none                 5.382953
3                  pine            stream                 8.282785
4                  pine            stream                10.690082

   elevation_range  human_activity_index  canopy_height_avg  \
0       318.967595              3.626041          23.686628
1       184.900374              1.830241          28.697284
2       536.571256              0.074063          26.366329
3       649.036995              0.000000          34.803212
4       388.522453              4.500977          45.697464

   seasonal_variation_score  ID
0                  6.492307   0
1                  4.313709   1
2                  3.480140   2
3                  3.417066   3
4                  6.823274   4
```

### 1.4 Visualisation function

Let us create a helper function for visualising points on a map with the coastline contour (`map.json`).

In [2]:

```python
def plot_map(points, coastline, color_by=None, figsize=(5, 5)):
    """
    Visualises points on a map with the coastline contour.

    Parameters:
    ----------
    points : pd.DataFrame
        DataFrame with the points; must contain the columns 'longitude' and 'latitude'
    coastline : list
        List of coastline coordinates [[lon1, lat1], [lon2, lat2], ...]
    color_by : str, optional
        Name of the column used to colour the points. If None, all points have the same colour
    figsize : tuple, optional
        Figure size (width, height)

    Example of use:
    --------------------
    # Simple visualisation without colouring
    plot_map(train, coastline)

    # Coloured by the target
    plot_map(train, coastline, color_by='target')

    # Changing the size
    plot_map(train, coastline, color_by='target', figsize=(16, 10))
    """
    fig, ax = plt.subplots(figsize=figsize)

    # Draw the coastline
    if coastline:
        coastline_array = np.array(coastline)

        # Draw all coastline points as a dense scatter with square markers
        ax.scatter(coastline_array[:, 0], coastline_array[:, 1],
                  s=1.5, marker='s', color='#CCCCCC', alpha=0.7, label='Coastline')

    # Draw the points
    if color_by is None:
        # No colouring: all points have the same colour
        ax.scatter(points['longitude'], points['latitude'],
                   s=50, alpha=0.6, edgecolors='black', linewidths=0.5,
                   label='Points')
    else:
        # Coloured by the given column
        if points[color_by].dtype in ['object', 'category']:
            # Categorical variable
            unique_vals = points[color_by].unique()
            colors = plt.cm.tab10(np.linspace(0, 1, len(unique_vals)))

            for i, val in enumerate(unique_vals):
                mask = points[color_by] == val
                ax.scatter(points[mask]['longitude'], points[mask]['latitude'],
                          s=50, alpha=0.6, edgecolors='black', linewidths=0.5,
                          label=f'{color_by}={val}', color=colors[i])
        else:
            # Numerical variable
            scatter = ax.scatter(points['longitude'], points['latitude'],
                               c=points[color_by], cmap='RdYlGn',
                               s=50, alpha=0.6, edgecolors='black', linewidths=0.5)
            plt.colorbar(scatter, ax=ax, label=color_by)

    ax.set_xlabel('Longitude')
    ax.set_ylabel('Latitude')
    ax.set_title(f'Map of points (n={len(points)})')
    ax.legend()
    ax.grid(True, alpha=0.3)
    plt.tight_layout()
    plt.show()

print("The plot_map() function is ready to use")
```

```text
The plot_map() function is ready to use
```

### 1.5 Demonstration of the visualisation function

Let us try the function on the loaded data.

In [3]:

```python
# Get the coastline from map.json
coastline = map_data.get('coastline', [])

# Example 1: Train without colouring
plot_map(train, coastline)

# Example 2: Train coloured by the target
plot_map(train, coastline, color_by='target')
```

*[Figure: see page 13 of the [original solutions PDF](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/final/ai/sol-ai-9-11-tur2-final-25-26.pdf). A map titled "Map of points (n=502)": the grey coastline of South America (longitude from about −95 to −35, latitude from about −55 to 25) with the 502 training points drawn as blue dots; legend "Coastline", "Points"; axes Longitude and Latitude.]*

*[Figure: see page 14 of the [original solutions PDF](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/final/ai/sol-ai-9-11-tur2-final-25-26.pdf). The same map, "Map of points (n=502)", with the points coloured by `target` on a red-yellow-green colour bar from 0.0 (red) to 1.0 (green); legend "Coastline".]*

### 1.6 Step 1: A first baseline approach

According to the statement, we have `train.csv` (direct observations) and `test.csv`, in which some of the points were added from satellite data.

First, we build an honest baseline on all features and look at the `F1 score` on the labels from `ytest.csv`. This is needed as a checkpoint, to understand how prone the model is to overfitting in the original setting.

In [4]:

```python
# Function for evaluating quality
def evaluate(y_pred, solution):
    """F1-score evaluation: overall, public, private"""
    from sklearn.metrics import f1_score

    merged = solution.copy()
    merged['pred'] = np.asarray(y_pred).reshape(-1)

    # Compute the metric on the already rounded predictions (0/1)
    merged['pred_label'] = np.clip(np.rint(merged['pred']), 0, 1).astype(int)

    public_mask = merged['Usage'] == 'public'
    private_mask = merged['Usage'] == 'private'
    overall_f1 = f1_score(merged['target'], merged['pred_label'])
    public_f1 = f1_score(merged.loc[public_mask, 'target'], merged.loc[public_mask, 'pred_label'])
    private_f1 = f1_score(merged.loc[private_mask, 'target'], merged.loc[private_mask, 'pred_label'])

    print(f"F1 overall: {overall_f1:.6f}")
    print(f"F1 public:  {public_f1:.6f}")
    print(f"F1 private: {private_f1:.6f}")

    return public_f1, private_f1, overall_f1

# Select the features (all except the service columns)
feature_cols_numeric = [col for col in train.columns
                        if col not in ['ID', 'target', 'cluster', 'is_train']
                        and train[col].dtype in ['int64', 'float64']]

categorical_cols = train.select_dtypes(include=['object']).columns.tolist()
categorical_cols = [col for col in categorical_cols if col not in ['ID']]

all_features = feature_cols_numeric + categorical_cols

X_train = train[all_features]
y_train = train['target']
X_test = test[all_features]

from sklearn.model_selection import train_test_split
X_train_tr, X_train_val, y_train_tr, y_train_val = train_test_split(
    X_train, y_train, test_size=0.2, random_state=SEED, stratify=y_train
)

model = CatBoostClassifier(
    iterations=500,
    learning_rate=0.1,
    depth=12,
    random_seed=SEED,
    verbose=0,
 )

model.fit(X_train_tr, y_train_tr, cat_features=categorical_cols)
y_pred_test_baseline = np.asarray(model.predict(X_test)).reshape(-1).astype(int)

baseline_public, baseline_private, baseline_overall = evaluate(y_pred_test_baseline, solution)
```

```text
F1 overall: 0.646440
F1 public:  0.645707
F1 private: 0.647208
```

In [5]:

```python
from sklearn.metrics import f1_score
f1_score(solution.target.values, (solution.target.values * 0 + 0.5 >= 0.5).astype(int))
```

Out [5]:

```text
0.5270917803169922
```

### 1.7 Step 2: Feature importance analysis

We relate the result of the baseline to the task statement. If the coordinates (`longitude`, `latitude`) turn out to be at the top of the importance ranking, the model may be using a geographic "hint" instead of robust ecological patterns. This is exactly the first typical beginner's mistake: implicit leakage/spurious correlation through features that encode the split too directly.

In [6]:

```python
# Feature importance
importance_df = pd.DataFrame({
    'feature': all_features,
    'importance': model.get_feature_importance()
}).sort_values('importance', ascending=False)

print("\nTop 10 features:")
print(importance_df.head(10).to_string(index=False))

print("This may mean that the model is overfitting to the coordinates.")
print("\nWARNING: longitude and latitude are at the top of the importance ranking!")
```

```text

Top 10 features:
                 feature  importance
                latitude   25.842854
   habitat_quality_score   18.788004
      biodiversity_index   12.708726
               longitude    9.644228
             forest_type    6.298411
       disturbance_level    5.888199
            climate_zone    2.864203
     conservation_status    1.980591
seasonal_variation_score    1.594778
    human_activity_index    1.516413
This may mean that the model is overfitting to the coordinates.

WARNING: longitude and latitude are at the top of the importance ranking!
```

### 1.8 Step 3: Visualising the coordinates

Let us look at how the classes are distributed in coordinate space.

In [7]:

```python
# Use our function for visualisation
print("Train data coloured by the target:")
plot_map(train, coastline, color_by='target', figsize=(10, 8))

# Analysis of the class distribution
print("\n" + "="*70)
print("COORDINATE ANALYSIS")
print("="*70)

# Statistics by longitude
print("\nDistribution of the classes by longitude:")
for target_val in [0, 1]:
    subset = train[train['target'] == target_val]
    print(f"\nClass {target_val}:")
    print(f"  Longitude: min={subset['longitude'].min():.2f}, max={subset['longitude'].max():.2f}, mean={subset['longitude'].mean():.2f}")
    print(f"  Latitude:  min={subset['latitude'].min():.2f}, max={subset['latitude'].max():.2f}, mean={subset['latitude'].mean():.2f}")

print("\n" + "="*70)
print("CONCLUSION: The classes are clearly separated by the coordinates!")
print("   The model has memorised the geographic boundary between the classes.")
print("   In the real test, this pattern does not hold.")
print("   SOLUTION: longitude and latitude must be removed from the features.")
print("="*70)
```

```text
Train data coloured by the target:
```

*[Figure: see page 17 of the [original solutions PDF](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/final/ai/sol-ai-9-11-tur2-final-25-26.pdf). A larger version of the map "Map of points (n=502)" with the 502 training points coloured by `target` (red = 0, green = 1; colour bar "target" from 0.0 to 1.0); legend "Coastline". Most green points lie in a band between latitudes of about 0 and −30; red points appear both inside this band and in the north and south of the continent.]*

```text

======================================================================
COORDINATE ANALYSIS
======================================================================

Distribution of the classes by longitude:

Class 0:
  Longitude: min=-91.65, max=-39.05, mean=-64.11
  Latitude:  min=-55.12, max=22.89, mean=-10.67

Class 1:
  Longitude: min=-90.20, max=-38.22, mean=-62.39
  Latitude:  min=-29.95, max=-0.10, mean=-15.78

======================================================================
CONCLUSION: The classes are clearly separated by the coordinates!
   The model has memorised the geographic boundary between the classes.
   In the real test, this pattern does not hold.
   SOLUTION: longitude and latitude must be removed from the features.
======================================================================
```

### 1.9 Step 4: Removing the coordinates and retraining the model

We fix the first mistake: we remove `longitude` and `latitude` so that the model relies on meaningful environmental features. We compute the `F1 score` again and compare it with the baseline. If the quality improves, the coordinates really were hindering generalisation.

In [8]:

```python
# Remove the coordinates
features_no_coords = [f for f in all_features if f not in ['longitude', 'latitude']]
categorical_cols_filtered = [c for c in categorical_cols if c in features_no_coords]

X_train_no_coords = train[features_no_coords]
X_test_no_coords = test[features_no_coords]

X_train_tr_no_coords, X_train_val_no_coords, y_train_tr_no_coords, y_train_val_no_coords = train_test_split(
    X_train_no_coords, y_train, test_size=0.2, random_state=SEED, stratify=y_train
)

model.fit(X_train_tr_no_coords, y_train_tr_no_coords, cat_features=categorical_cols_filtered)
y_pred_test_no_coords = np.asarray(model.predict(X_test_no_coords)).reshape(-1).astype(int)

nocoords_public, nocoords_private, nocoords_overall = evaluate(y_pred_test_no_coords, solution)
```

```text
F1 overall: 0.783972
F1 public:  0.798155
F1 private: 0.769591
```

### 1.10 Step 5: A train vs test detector (searching for distribution shift)

We check the second beginner's mistake from the statement: ignoring the fact that `train` and part of `test` were collected in different ways. If a model easily distinguishes `train` from `test`, this is a sign of domain shift (the distributions differ). Then, for some of the test points, a standard model may be overconfident and make systematic errors.

In [9]:

```python
from catboost import cv, Pool

# Create a dataset for classifying train vs test
X_combined = pd.concat([X_train_no_coords, X_test_no_coords], axis=0)
y_combined = np.concatenate([np.ones(len(X_train_no_coords)), np.zeros(len(X_test_no_coords))])

model_ood = CatBoostClassifier(
    iterations=50,
    learning_rate=0.1,
    depth=6,
    random_seed=SEED,
    verbose=0
)

cv_data = Pool(data=X_combined, label=y_combined, cat_features=categorical_cols_filtered)
_ = cv(
    pool=cv_data,
    params={
        'iterations': 50,
        'learning_rate': 0.1,
        'depth': 6,
        'loss_function': 'Logloss',
        'eval_metric': 'AUC',
        'random_seed': SEED,
        'verbose': False
    },
    fold_count=5,
    shuffle=True,
    stratified=True,
    partition_random_seed=SEED
)

model_ood.fit(X_combined, y_combined, cat_features=categorical_cols_filtered)
test_similarity_scores = model_ood.predict(X_test_no_coords, prediction_type='Probability')[:, 1]

plt.figure(figsize=(10, 5))
plt.hist(test_similarity_scores, bins=100, alpha=0.7, edgecolor='black')
plt.xlabel('Probability of belonging to train')
plt.ylabel('Number of objects')
plt.title('Train/Test Classifier: distribution of the probabilities on test')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
```

```text
Training on fold [0/5]

bestTest = 0.6328465347
bestIteration = 9

Training on fold [1/5]

bestTest = 0.6250990099
bestIteration = 4

Training on fold [2/5]

bestTest = 0.6888
bestIteration = 6

Training on fold [3/5]

bestTest = 0.6909022556
bestIteration = 11

Training on fold [4/5]

bestTest = 0.6967669173
bestIteration = 16
```

*[Figure: see page 20 of the [original solutions PDF](https://vos.olimpiada.ru/upload/files/Arhive_tasks/2025-26/final/ai/sol-ai-9-11-tur2-final-25-26.pdf). A histogram (100 bins) titled "Train/Test Classifier: distribution of the probabilities on test"; x axis "Probability of belonging to train" (from 0.0 to about 0.43), y axis "Number of objects". There is a tall, narrow peak just above 0 (up to about 80 objects per bin at around 0.01–0.02), a second, broad mode between about 0.12 and 0.32 (roughly 30–50 objects per bin), and only a few objects above 0.35.]*

### 1.11 Step 6: OOD correction of the predictions

We use the train/test detector as an indicator of OOD points (how similar a test point is to train). For points with very low similarity, we lower the probability of class 1 and again compute only the `F1 score`.

In this way we address the second mistake: we take the distribution shift into account and make the predictions more robust on the "satellite" part of the test.

In [10]:

```python
print("\nCorrecting the predictions for OOD objects...")
print("="*70)

# OOD threshold: points with low similarity to train are considered shifted in distribution
ood_threshold = 0.07

y_pred_corrected = y_pred_test_no_coords.copy()
ood_mask = test_similarity_scores < ood_threshold
y_pred_corrected[ood_mask] = 0

# Print the F1-score for each approach: overall/public/private
_, _, _ = evaluate(y_pred_test_baseline, solution)
_, _, _ = evaluate(y_pred_test_no_coords, solution)
_, _, corrected_overall = evaluate(y_pred_corrected, solution)
```

```text

Correcting the predictions for OOD objects...
======================================================================
F1 overall: 0.646440
F1 public:  0.645707
F1 private: 0.647208
F1 overall: 0.783972
F1 public:  0.798155
F1 private: 0.769591
F1 overall: 0.918549
F1 public:  0.918340
F1 private: 0.918768
```

In [11]:

```python
# Save the submission with the best available predictions
final_pred = y_pred_corrected

submission = pd.DataFrame({
    'ID': test['ID'].values,
    'target': (final_pred > 0.5) * 1
})
submission.to_csv('author_submission.csv', index=False)
```

---

## D. Humanity's Heritage

In [1]:

```python
# Step 0. Imports
import csv
import json
from pathlib import Path

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.preprocessing import StandardScaler

# Fix the random number generator for reproducibility
rng = np.random.default_rng(42)
```

### 1.12 Step-by-step walkthrough of the solution

The solutions go in order and become more elaborate step by step:

1. Solution 0: IOU/Jaccard over words.
2. Solution 1: unigram TF-IDF.
3. Solution 2: word+char TF-IDF.
4. Solution 3: solution 2 + greedy one-to-one.
5. Solution 4: solution 3 + logreg-reranker.
6. Solution 5: solution 4 + greedy one-to-one.

#### 1.12.1 Step 1. Loading train/test/ytest

We read `train.json` and `test.json` and restore the train pairs, while `true_test` and the `Public/Private` split are built from `ytest.csv` via the correspondences `left_id` $\leftrightarrow$ `right_id` and the column `Usage`.

In [2]:

```python
# Step 1. Loading train/test/ytest (even more compact)
base_dir = Path("/Users/aguschin/Git/uni/vsosh/zakl/nlp")
train_json_path, test_json_path, ytest_csv_path = (
    base_dir / "train.json",
    base_dir / "test.json",
    base_dir / "ytest.csv",
)

def split_poem(text):
    lines = [line.strip() for line in text.split("\n") if line.strip()]
    cut = min(8, len(lines) - 1)
    if cut < 1:
        return None, None
    left, right = "\n".join(lines[:cut]).strip(), "\n".join(lines[cut:]).strip()
    return (left, right) if left and right else (None, None)

# train: from full texts -> pairs of halves -> shuffle the right halves
train_pairs = [split_poem(text) for text in json.loads(train_json_path.read_text(encoding="utf-8"))]
train_pairs = [(left, right) for left, right in train_pairs if left and right]
left_parts = [left for left, _ in train_pairs]
right_parts = [right for _, right in train_pairs]
train_idx = np.arange(len(left_parts), dtype=int)
perm_train = np.random.default_rng(123).permutation(len(left_parts))
left_train = left_parts
right_train_shuffled = [right_parts[i] for i in perm_train]
true_train = np.empty(len(left_parts), dtype=int)
true_train[perm_train] = np.arange(len(left_parts))
n_train = len(left_train)

# test: ready-made left/right
test_data = json.loads(test_json_path.read_text(encoding="utf-8"))
left_items, right_items = test_data["left"], test_data["right"]
left_test = [item["left_text"] for item in left_items]
left_ids = [item["left_id"] for item in left_items]
right_test_shuffled = [item["right_text"] for item in right_items]
right_ids = [item["right_id"] for item in right_items]
n_test = len(left_test)

# ytest: left_id -> right_id and Usage
with open(ytest_csv_path, "r", encoding="utf-8") as f:
    ytest_rows = [row for row in csv.DictReader(f)]

left_to_right = {}
left_to_usage = {}
for row in ytest_rows:
    left_id = (row.get("left_id") or "").strip()
    right_id = (row.get("right_id") or "").strip()
    usage = (row.get("Usage") or "Private").strip().title()
    if not left_id or not right_id:
        continue
    if usage not in {"Public", "Private"}:
        usage = "Private"
    left_to_right[left_id] = right_id
    left_to_usage[left_id] = usage

right_id_to_pos = {rid: j for j, rid in enumerate(right_ids)}
true_test = np.array([right_id_to_pos[left_to_right[lid]] for lid in left_ids], dtype=int)
usage_test = np.array([left_to_usage.get(lid, "Private") for lid in left_ids], dtype=object)
public_mask = usage_test == "Public"
private_mask = usage_test == "Private"

# checks and service variables
assert all(lid in left_to_right for lid in left_ids), "Not all left_id values from test are present in ytest.csv"
assert all(left_to_right[lid] in right_id_to_pos for lid in left_ids), "Some right_id values from ytest.csv are missing from test.json"
vowels = set("аеёиоуыэюя")  # the Russian vowel letters
test_idx = np.arange(n_test, dtype=int) + 10_000_000
overlap = len(set(train_idx.tolist()) & set(test_idx.tolist()))
assert overlap == 0, "train/test overlap detected"

print(f"Source train: {train_json_path}")
print(f"Source test : {test_json_path}")
print(f"Source gt   : {ytest_csv_path}")
print(f"Total pairs: Train: {n_train} | Test: {n_test}")
print(f"Usage split: Public={int(public_mask.sum())} | Private={int(private_mask.sum())}")
print(f"Split fairness: overlap(train,test)={overlap}\n")
```

```text
Source train: /Users/aguschin/Git/uni/vsosh/zakl/nlp/train.json
Source test : /Users/aguschin/Git/uni/vsosh/zakl/nlp/test.json
Source gt   : /Users/aguschin/Git/uni/vsosh/zakl/nlp/ytest.csv
Total pairs: Train: 2000 | Test: 640
Usage split: Public=320 | Private=320
Split fairness: overlap(train,test)=0
```

#### 1.12.2 Solution 0: IOU/Jaccard over words

Base score: for each pair of halves, we compute the intersection of the words and divide it by the union. Scoring/choice of the pair: top1 via `argmax` (greedy matching is not used).

In [3]:

```python
# Step 2. Solution 0: IOU/Jaccard via split()

# Solution 0: a base score from the intersection of words

def _masked_acc(pred_idx: np.ndarray, true_idx: np.ndarray, mask: np.ndarray):
    return float((pred_idx[mask] == true_idx[mask]).mean()) if mask.any() else float("nan")

# Function: computes a single score via argmax (Overall/Public/Private)
def evaluate_similarity_argmax(sim: np.ndarray, true_idx: np.ndarray):
    pred = sim.argmax(axis=1)
    score = float((pred == true_idx).mean())
    score_public = _masked_acc(pred, true_idx, public_mask)
    score_private = _masked_acc(pred, true_idx, private_mask)
    return {
        "score": score,
        "score_public": score_public,
        "score_private": score_private,
    }

# Function: turns each text into a set of tokens
def _token_sets(texts):
    return [set(text.split()) for text in texts]

# Function: builds the Jaccard/IOU matrix for all left-right pairs
def _jaccard_matrix(left_sets, right_sets):
    n_left = len(left_sets)
    n_right = len(right_sets)
    sim = np.zeros((n_left, n_right), dtype=np.float32)
    for i, lset in enumerate(left_sets):
        for j, rset in enumerate(right_sets):
            union = lset | rset
            if union:
                sim[i, j] = len(lset & rset) / len(union)
    return sim

word_train_l = _token_sets(left_train)
word_train_r = _token_sets(right_train_shuffled)
word_test_l = _token_sets(left_test)
word_test_r = _token_sets(right_test_shuffled)

sim0_test = _jaccard_matrix(word_test_l, word_test_r)
metrics0 = evaluate_similarity_argmax(sim0_test, true_test)
print(
    f"0_iou_argmax: "
    f"score={metrics0['score']:.4f} (pub={metrics0['score_public']:.4f}, priv={metrics0['score_private']:.4f})"
)
```

```text
0_iou_argmax: score=0.1734 (pub=0.1594, priv=0.1875)
```

#### 1.12.3 Solution 1: unigram TF-IDF

We improve solution 0: we replace the plain intersection of words with a TF-IDF representation of the words and cosine similarity.

In [4]:

```python
# Step 2. Solution 1: unigram TF-IDF

# Solution 1: improve solution 0 with TF-IDF features
vec1 = TfidfVectorizer(ngram_range=(1, 1), min_df=5, max_df=0.95, sublinear_tf=False)
vec1.fit(left_train + right_train_shuffled)
sim1_test = cosine_similarity(vec1.transform(left_test), vec1.transform(right_test_shuffled))
metrics1 = evaluate_similarity_argmax(sim1_test, true_test)
print(
    f"1_unigram_tfidf: "
    f"score={metrics1['score']:.4f} (pub={metrics1['score_public']:.4f}, priv={metrics1['score_private']:.4f})"
)
```

```text
1_unigram_tfidf: score=0.2734 (pub=0.2562, priv=0.2906)
```

#### 1.12.4 Solution 2: word+char TF-IDF

We improve solution 1: we blend the similarity from word TF-IDF and char TF-IDF and then compute the `argmax` metrics.

In [5]:

```python
# Step 3. Solution 2: word+char TF-IDF

# Function: builds a blended similarity from word and char TF-IDF
def _build_tfidf_blend(left_train, right_train, left_eval, right_eval, cfg, alpha):
    vec_word = TfidfVectorizer(analyzer="word", **cfg["word"] )
    vec_char = TfidfVectorizer(analyzer="char_wb", **cfg["char"] )
    vec_word.fit(left_train + right_train)
    vec_char.fit(left_train + right_train)
    sim_word = cosine_similarity(vec_word.transform(left_eval), vec_word.transform(right_eval))
    sim_char = cosine_similarity(vec_char.transform(left_eval), vec_char.transform(right_eval))
    return alpha * sim_word + (1.0 - alpha) * sim_char

cfg2b = {
    "word": dict(ngram_range=(1, 2), min_df=2, max_df=0.95, sublinear_tf=True),
    "char": dict(ngram_range=(2, 5), min_df=1, max_df=0.99, sublinear_tf=True),
}
alpha2b = 0.25

# Solution 2: improve solution 1 by blending word/char features
sim2b_train = _build_tfidf_blend(left_train, right_train_shuffled, left_train, right_train_shuffled, cfg2b, alpha2b)
sim2b_test = _build_tfidf_blend(left_train, right_train_shuffled, left_test, right_test_shuffled, cfg2b, alpha2b)
metrics2 = evaluate_similarity_argmax(sim2b_test, true_test)
print(
    f"2_word_char_tfidf: "
    f"score={metrics2['score']:.4f} (pub={metrics2['score_public']:.4f}, priv={metrics2['score_private']:.4f})"
)
```

```text
2_word_char_tfidf: score=0.4938 (pub=0.4969, priv=0.4906)
```

#### 1.12.5 Solution 3: TF-IDF + greedy one-to-one

We take the score matrix of solution 2 and add a greedy one-to-one matching to increase `one2one`.

In [6]:

```python
# Function: builds a greedy one-to-one correspondence
def greedy_one2one_matching(sim: np.ndarray):
    n = sim.shape[0]
    flat_order = np.argsort(sim, axis=None)[::-1]
    pred = np.full(n, -1, dtype=int)
    used_left = np.zeros(n, dtype=bool)
    used_right = np.zeros(n, dtype=bool)
    matched = 0
    for idx in flat_order:
        i, j = divmod(idx, n)
        if (not used_left[i]) and (not used_right[j]):
            pred[i] = j
            used_left[i] = True
            used_right[j] = True
            matched += 1
            if matched == n:
                break
    return pred

# Function: computes a single score via greedy
def evaluate_similarity_greedy(sim: np.ndarray, true_idx: np.ndarray):
    pred = greedy_one2one_matching(sim)
    score = float((pred == true_idx).mean())
    score_public = _masked_acc(pred, true_idx, public_mask)
    score_private = _masked_acc(pred, true_idx, private_mask)
    return {
        "score": score,
        "score_public": score_public,
        "score_private": score_private,
    }

# Solution 3: add greedy to solution 2
metrics3 = evaluate_similarity_greedy(sim2b_test, true_test)
print(
    f"3_tfidf_greedy: "
    f"score={metrics3['score']:.4f} (pub={metrics3['score_public']:.4f}, priv={metrics3['score_private']:.4f})"
)
```

```text
3_tfidf_greedy: score=0.5062 (pub=0.5094, priv=0.5031)
```

#### 1.12.6 Solution 4: logreg reranker

We improve solution 3: we add pairwise features and a logistic regression that re-scores the candidates before `argmax`.

In [7]:

```python
# In short: compute statistics of the halves and assemble a pairwise-feature cube
def _half_stats(texts):
    stats = []
    for text in texts:
        lines = [line.strip() for line in text.split("\n") if line.strip()] or [text.strip()]
        words = ["".join(ch for ch in w if ch.isalpha()) for w in text.replace("\n", " ").split()]
        words = [w for w in words if w]
        letters = [ch for ch in text if ch.isalpha()]
        v_all = [sum(1 for ch in line.lower() if ch in vowels) for line in lines]
        v_edge = [sum(1 for ch in line.lower() if ch in vowels) for line in lines[-4:]]
        cap_ratio = (sum(1 for w in words if w[0].isupper()) / len(words)) if words else 0.0
        latin_ratio = (sum(1 for ch in letters if 'a' <= ch.lower() <= 'z') / len(letters)) if letters else 0.0
        stats.append([len(lines), np.mean([len(line) for line in lines]), np.mean(v_all), np.mean(v_edge), cap_ratio, latin_ratio])
    return np.asarray(stats, dtype=np.float32)

# In short: turn the difference of a scalar feature into a similarity
def _pair_sim(left_vals, right_vals):
    diff = np.abs(left_vals[:, None] - right_vals[None, :])
    norm = np.maximum(left_vals[:, None], right_vals[None, :])
    return 1.0 - diff / (norm + 1e-12)

# In short: form the training pairs (true + hard negatives + random negatives)
def _sample_pairs(cube, true_idx, hard_source_sim, hard_k=8, rand_k=8, random_state=42):
    rng_local = np.random.default_rng(random_state)
    n = cube.shape[0]
    rows, labels = [], []
    for i in range(n):
        j_true = true_idx[i]
        rows.append(cube[i, j_true]); labels.append(1)
        hard_neg = [j for j in np.argsort(-hard_source_sim[i]) if j != j_true][:hard_k]
        forbidden = set(hard_neg + [j_true])
        pool = np.array([j for j in range(n) if j not in forbidden], dtype=int)
        rand_neg = rng_local.choice(pool, size=min(rand_k, len(pool)), replace=False).tolist() if len(pool) else []
        for j in hard_neg + rand_neg:
            rows.append(cube[i, j]); labels.append(0)
    return np.asarray(rows, dtype=np.float32), np.asarray(labels, dtype=np.int32)

left_train_feat = _half_stats(left_train)
right_train_feat = _half_stats(right_train_shuffled)
left_test_feat = _half_stats(left_test)
right_test_feat = _half_stats(right_test_shuffled)

train_maps = [sim2b_train] + [_pair_sim(left_train_feat[:, k], right_train_feat[:, k]) for k in range(left_train_feat.shape[1])] + [_jaccard_matrix(word_train_l, word_train_r).astype(np.float32)]
test_maps = [sim2b_test] + [_pair_sim(left_test_feat[:, k], right_test_feat[:, k]) for k in range(left_test_feat.shape[1])] + [_jaccard_matrix(word_test_l, word_test_r).astype(np.float32)]
cube_train = np.stack(train_maps, axis=2).astype(np.float32)
cube_test = np.stack(test_maps, axis=2).astype(np.float32)

x_train, y_train = _sample_pairs(cube_train, true_train, sim2b_train, hard_k=8, rand_k=8, random_state=42)
scaler = StandardScaler()
model = LogisticRegression(C=0.03, max_iter=2000, class_weight="balanced", solver="lbfgs", random_state=42)
model.fit(scaler.fit_transform(x_train), y_train)
flat_scores = model.predict_proba(scaler.transform(cube_test.reshape(-1, cube_test.shape[-1])))[:, 1]
test_scores_4 = flat_scores.reshape(cube_test.shape[0], cube_test.shape[1])

alpha4 = 0.82
sim4_test = alpha4 * test_scores_4 + (1.0 - alpha4) * sim2b_test
metrics4 = evaluate_similarity_argmax(sim4_test, true_test)
print(
    f"4_logreg_rerank: "
    f"score={metrics4['score']:.4f} (pub={metrics4['score_public']:.4f}, priv={metrics4['score_private']:.4f})"
)
```

```text
4_logreg_rerank: score=0.5203 (pub=0.5031, priv=0.5375)
```

#### 1.12.7 Solution 5: logreg reranker + greedy

The final improvement: to the score matrix of solution 4 we again apply greedy one-to-one matching for the maximum `one2one`.

In [8]:

```python
# Solution 5: add greedy to the scores of solution 4
metrics5 = evaluate_similarity_greedy(sim4_test, true_test)
print(
    f"5_logreg_rerank_greedy: "
    f"score={metrics5['score']:.4f} (pub={metrics5['score_public']:.4f}, priv={metrics5['score_private']:.4f})"
)
```

```text
5_logreg_rerank_greedy: score=0.5656 (pub=0.5687, priv=0.5625)
```

In [10]:

```python
# Save the author's submission from the best solution
sim_test = sim4_test
pred_idx = sim_test.argmax(axis=1)

author_submission_path = base_dir / "author_submission.csv"
with open(author_submission_path, "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["left_id", "right_id"])
    for i, lid in enumerate(left_ids):
        writer.writerow([lid, right_ids[int(pred_idx[i])]])

print(f"Saved {author_submission_path}")
```

```text
Saved /Users/aguschin/Git/uni/vsosh/zakl/nlp/author_submission.csv
```

#### 1.12.8 Step 5. Matching results

We show only examples of matching for the best current solution: the best and the worst pairs.

In [12]:

```python
# Step 5. Matching results by quantiles for the best solution

def _preview_text(text, max_chars=220, max_lines=8):
    lines = [line.strip() for line in text.split("\n") if line.strip()][:max_lines]
    preview = "\n".join(lines)
    return preview[:max_chars] + ("..." if len(preview) > max_chars else "")

sim_test = sim4_test
pred_idx = sim_test.argmax(axis=1)

quantiles_to_show = [0.99, 0.75, 0.5, 0.25, 0.0]  # can be changed by hand
if not quantiles_to_show or any(q < 0 or q > 1 for q in quantiles_to_show):
    raise ValueError("Quantiles must lie in [0, 1], and the list must not be empty")

match_scores = sim_test[np.arange(len(left_test)), pred_idx]
used = set()

def pick_idx_for_quantile(q):
    target = float(np.quantile(match_scores, q))
    order = np.argsort(np.abs(match_scores - target))
    idx = next((int(i) for i in order if int(i) not in used), int(order[0]))
    used.add(idx)
    return idx, target

print(f"Matching for the best solution")
print(f"Quantiles: {quantiles_to_show}")
print()

for q in quantiles_to_show:
    i, target = pick_idx_for_quantile(q)
    j = int(pred_idx[i])
    print(f"=== q={q:.2f} | target={target:.4f} | sim={match_scores[i]:.4f} ===")
    print(f"left={i} -> pred_right={j} | true={true_test[i]} | correct={j == true_test[i]}")
    print("LEFT :", _preview_text(left_test[i]))
    print("RIGHT:", _preview_text(right_test_shuffled[j]))
    print()
```

*(Translator's note: the poem fragments below are the Russian data of the task and are left untranslated.)*

```text
Matching for the best solution
Quantiles: [0.99, 0.75, 0.5, 0.25, 0.0]

=== q=0.99 | target=0.9055 | sim=0.9076 ===
left=383 -> pred_right=628 | true=628 | correct=True
LEFT : Человек живет на белом свете.
Где - не знаю. Суть совсем не в том.
Я - лежу в пристрелянном кювете,
Он - с мороза входит в теплый дом.
Человек живет на белом свете,
Он - в квартиру поднялся уже.
Я - лежу в пристрелянном ...
RIGHT: Человек живет на белом свете
Он - в квартире зажигает свет
Я - лежу в пристрелянном кювете,
Я - вмерзаю в ледяной кювет.
Снег не тает. Губы, щеки, веки
Он засыпал. И велит дрожать...
С думой о далеком человеке
Легче до а...

=== q=0.75 | target=0.7871 | sim=0.7870 ===
left=208 -> pred_right=508 | true=508 | correct=True
LEFT : Бронепоезда взвывают вдруг,
Стылый ветер грудью разрывая.
Бронепоезда идут на юг
Вдоль твоих перронов,
Лозовая!
Звезды первую звезду зовут.
Дым заката холоден и розов.
Над бронеплощадками плывут
RIGHT: Бескозырки черные матросов.
Говорит, гремит, вздыхает бронь
Отдаленно
и громоподобно.
И горит на станции огонь,
Керосиновый огонь бездомный.
Лист осенний, запоздавший лист,
Братьев в путь-дорогу созывает.

=== q=0.50 | target=0.7115 | sim=0.7113 ===
left=342 -> pred_right=350 | true=350 | correct=True
LEFT : Славянка тихая, сколь ток приятен твой,
Когда, в осенний день, в твои глядятся воды
Холмы, одетые последнею красой
Полуотцветшия природы.
Спешу к твоим брегам... свод неба тих и чист;
При свете солнечном прохлада повевае...
RIGHT: Иду под рощею излучистой тропой;
Что шаг, то новая в глазах моих картина;
То вдруг сквозь чащу древ мелькает предо мной,
Как в дыме, светлая долина;
То вдруг исчезло все... окрест сгустился лес;
Все дико вкруг меня, и су...

=== q=0.25 | target=0.6497 | sim=0.6498 ===
left=532 -> pred_right=415 | true=415 | correct=True
LEFT : Посвящается Феллини
Мертвец играл на дудочке,
По городу гулял,
И незнакомой дурочке
Он руку предлагал.
А дурочка, как Золушка,
Ему в глаза глядит,—
Он говорит о золоте,
RIGHT: О славе говорит.
Мертвец, певец и умница,
Его слова просты —
Пусты ночные улицы,
И площади пусты.
«Мне больно, мне невесело,
Мне холодно зимой,
Возьми меня невестою,

=== q=0.00 | target=0.1137 | sim=0.1137 ===
left=533 -> pred_right=373 | true=329 | correct=False
LEFT : Нет.
Это неправда.
Нет!
И ты?
Любимая,
за что,
за что же?!
Хорошо -
RIGHT: Что нашу грусть —
В листы,
И груз — в цветы
Всего за только всхруст
Руки
В руке:
Игру.
Индус, а может Златоуст
```

---

## E. Suspicious Cakes

The solution consists of the following stages:

1. We load the pre-trained CNN and obtain embeddings for the images.
2. For each image, we compute the outlier metrics `robust_mahalanobis`, `global_knn`, `class_knn`.
3. We aggregate the metrics with the weights (2.0, 1.5, 0.5).
4. We sort the images by the value of the metric, select the top-K and save them to `submission_author.csv`.

In [1]:

```python
import json
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset

ARTIFACTS_DIR = "."
TEST_PACKAGE = f"{ARTIFACTS_DIR}/public_test_package.npz"
TEST_META = f"{ARTIFACTS_DIR}/public_test_meta.json"
WEIGHTS_PATH = f"{ARTIFACTS_DIR}/model_weights.pt"
SUBMISSION_PATH = "submission.csv"

DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Device:", DEVICE)
```

```text
Device: cpu
```

The code of the network and of obtaining the image embeddings:

In [2]:

```python
class SmallCNN(nn.Module):
    def __init__(self, n_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
            nn.Conv2d(16, 32, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
        )
        self.fc1 = nn.Linear(64 * 8 * 8, 64)
        self.act = nn.ReLU(inplace=True)
        self.fc2 = nn.Linear(64, n_classes)

    def forward(self, x):
        z = self.features(x)
        z = z.flatten(1)
        h = self.act(self.fc1(z))
        logits = self.fc2(h)
        return logits, h

def infer_embeddings_and_preds(model, x_np, batch_size=256):
    ds = TensorDataset(torch.from_numpy(x_np).float())
    loader = DataLoader(ds, batch_size=batch_size, shuffle=False)
    all_emb, all_pred = [], []
    model.eval()
    with torch.no_grad():
        for (xb,) in loader:
            xb = xb.to(DEVICE)
            logits, emb = model(xb)
            all_emb.append(emb.cpu().numpy())
            all_pred.append(logits.argmax(dim=1).cpu().numpy())
    emb = np.concatenate(all_emb, axis=0).astype(np.float32)
    pred = np.concatenate(all_pred, axis=0).astype(np.int64)
    return emb, pred

state = torch.load(WEIGHTS_PATH, map_location=DEVICE)
n_classes = int(state["fc2.weight"].shape[0]) if "fc2.weight" in state else 10
model = SmallCNN(n_classes=n_classes)
model.load_state_dict(state)
model = model.to(DEVICE).eval()
print("Model loaded")
```

```text
Model loaded
```

In [3]:

```python
test = np.load(TEST_PACKAGE)
x_test = test["images"].astype(np.float32)

K = 1000

emb_test, pred_test = infer_embeddings_and_preds(model, x_test)

n_test = len(x_test)
print(f"n_test={n_test}, K={K}, emb_dim={emb_test.shape[1]}")
```

```text
n_test=10000, K=1000, emb_dim=64
```

#### 1.12.9 The idea of the solution

We will look for outliers in the embedding space of the neural network.

We assume that, after passing through the trained model, objects of the same class form compact clusters in the feature space. Then outliers can be sought as points that are located "atypically" relative to the other points.

For this, several types of metrics are used:

1. **Global kNN score.**
   For each object, the mean distance to its nearest neighbours among all objects of the dataset is computed.
   If an object is isolated, its score will be large.
2. **Class kNN score.**
   Similar to the previous item, but the neighbours are sought only among objects of the same predicted class.
   This makes it possible to find points that look unusual specifically within their own class, even if globally they are not very far from the others.
3. **Robust Mahalanobis score within the class.**
   For each predicted class, the mean $\mu$ and the covariance matrix $\Sigma$ of the embeddings are estimated, after which, for each object, the Mahalanobis distance to the centre of its class is computed. The Mahalanobis distance is computed as follows:

   $$(x_i - \mu)^T \Sigma^{-1} (x_i - \mu)$$

   The idea of using the covariance is that Mahalanobis looks at a point relative to the shape of the class distribution. Sometimes an outlier point can be caught by the fact that it deviates not in length but in a "wrong direction" in the space.
   So that outliers do not distort the initial sample estimates of $\mu$ and $\Sigma$, we use a robust scheme: at each iteration we temporarily discard the farthest points and estimate the parameters only from the remaining "core" of the class.

Thus, the solution relies on the following hypothesis:

**outliers are objects that, in the feature space, are either far from the typical distribution of their class or fit poorly into the local structure of their neighbours.**

**Note**: without using the Mahalanobis distance, one can get up to 80% of the points for the task.

The metric functions are implemented below:

In [4]:

```python
import numpy as np

def score_robust_mahalanobis(emb, pred, lam=0.15, trim_frac=0.10, iters=2):
    """
    Computes a robust Mahalanobis score within each predicted class.

    Idea:
    - for each class, we estimate the "normal" distribution of the embeddings;
    - then, for each object, we compute the Mahalanobis distance to the centre of the class;
    - so that outliers do not spoil the estimates of the mean and the covariance,
      we discard the farthest points several times and recompute the statistics.

    Parameters:
    - emb: matrix of embeddings, shape [N, d]
    - pred: predicted classes, shape [N]
    - lam: covariance regularisation coefficient
    - trim_frac: fraction of the farthest points temporarily excluded during trimming
    - iters: number of iterations of the robust recomputation
    """
    d = emb.shape[1]  # dimension of the feature space
    out = np.zeros(len(emb), dtype=np.float64)  # final score for all objects
    eye = np.eye(d, dtype=np.float64)  # identity matrix for regularisation

    # Process each class separately
    for c in np.unique(pred):
        m = pred == c          # mask of the objects of class c
        h = emb[m].astype(np.float64)  # embeddings of this class only
        n = len(h)

        # If there are too few objects, a robust estimate is unreliable.
        # Then we simply compute the ordinary Mahalanobis distance.
        if n < 8:
            mu = h.mean(axis=0, keepdims=True)   # class centre
            cen = h - mu                         # centred embeddings

            # Covariance + diagonal regularisation for stability
            cov = (cen.T @ cen) / max(1, n - 1) + lam * eye

            # Pseudo-inverse instead of the ordinary inverse:
            # more stable if the covariance is ill-conditioned
            inv = np.linalg.pinv(cov)

            # Squared Mahalanobis distance for each point:
            # (x - mu)^T inv (x - mu)
            out[m] = np.einsum("bi,ij,bj->b", cen, inv, cen)
            continue

        # Initially, we assume that all points of the class are kept
        keep = np.ones(n, dtype=bool)

        # Initial rough estimate of the mean and of the inverse covariance
        mu = h.mean(axis=0, keepdims=True)
        inv = np.linalg.pinv(np.cov(h.T) + lam * eye)

        # Several iterations of robust trimming
        for _ in range(max(1, iters)):
            # Take only the current "reliable" points
            hh = h[keep]

            # Recompute the centre from the remaining points
            mu = hh.mean(axis=0, keepdims=True)

            # Centre the remaining points
            cen_hh = hh - mu

            # Recompute the covariance on the cleaned subset
            cov = (cen_hh.T @ cen_hh) / max(1, len(hh) - 1) + lam * eye
            inv = np.linalg.pinv(cov)

            # Compute the Mahalanobis distances, now for all points of the class
            cen_all = h - mu
            dist_all = np.einsum("bi,ij,bj->b", cen_all, inv, cen_all)

            # Threshold: keep the (1 - trim_frac) fraction of the nearest points
            thr = float(np.quantile(dist_all, 1.0 - trim_frac))
            keep = dist_all <= thr

            # Protection against too few points remaining:
            # in that case, relax the trimming
            if keep.sum() < max(5, int(0.5 * n)):
                keep = dist_all <= float(np.quantile(dist_all, 0.7))

        # After the final estimate, compute the final Mahalanobis score
        # for all points of this class
        cen = h - mu
        out[m] = np.einsum("bi,ij,bj->b", cen, inv, cen)

    return out


def score_global_knn(emb, k=10):
    """
    Computes the global kNN score:
    the mean distance to the k nearest neighbours over the whole dataset.

    If a point is isolated from the others,
    its score will be large.
    """
    n = emb.shape[0]

    # If there are too few points, there is no meaningful score
    if n <= 2:
        return np.zeros(n, dtype=np.float64)

    # We cannot take more neighbours than there are other points
    kk = min(k, n - 1)

    x = emb.astype(np.float64)

    # Squared norms of all points
    x2 = np.sum(x * x, axis=1, keepdims=True)

    # Matrix of squared pairwise Euclidean distances:
    # ||xi - xj||^2 = ||xi||^2 + ||xj||^2 - 2 <xi, xj>
    d2 = x2 + x2.T - 2.0 * (x @ x.T)

    # Remove the distance from each point to itself
    np.fill_diagonal(d2, np.inf)

    # Take the kk smallest distances in each row
    knn = np.partition(d2, kk - 1, axis=1)[:, :kk]

    # Return the mean ordinary distance to the nearest neighbours
    return np.mean(np.sqrt(np.maximum(knn, 0.0)), axis=1)


def score_class_knn(emb, pred, k=8):
    """
    Computes the class-wise kNN score:
    the mean distance to the k nearest neighbours within the predicted class.

    This helps to find objects that are atypical
    specifically for their own class.
    """
    out = np.zeros(len(emb), dtype=np.float64)

    # Compute the score separately within each class
    for c in np.unique(pred):
        m = pred == c
        h = emb[m].astype(np.float64)
        n = len(h)

        # If the class has too few points, the score is taken to be zero
        if n <= 2:
            out[m] = 0.0
            continue

        kk = min(k, n - 1)

        # Squared norms of the points within the class
        h2 = np.sum(h * h, axis=1, keepdims=True)

        # Matrix of squared pairwise distances within the class
        d2 = h2 + h2.T - 2.0 * (h @ h.T)

        # Exclude the distance to the point itself
        np.fill_diagonal(d2, np.inf)

        # Find the kk nearest neighbours
        knn = np.partition(d2, kk - 1, axis=1)[:, :kk]

        # Mean distance to the nearest neighbours within the class
        out[m] = np.mean(np.sqrt(np.maximum(knn, 0.0)), axis=1)

    return out

def rank_score(x):
    """
    Converts an array of values into ranks.

    The smallest element gets rank 0,
    the next one gets 1, and so on.
    This is convenient if several scores of different scales
    have to be combined later.
    """
    # Indices of the elements in ascending order of the values of x
    order = np.argsort(x)

    # Array for the ranks
    r = np.empty_like(order, dtype=np.float64)

    # For each element, write its position in the sorted order
    r[order] = np.arange(len(x), dtype=np.float64)
    return r
```

In [5]:

```python
# Build the outlier score on the test embeddings from an ensemble of the ranks
# of three metrics: robust Mahalanobis + global kNN + class kNN.

# Robust Mahalanobis distance within the predicted class:
# shows how atypical an object is relative to the distribution of its class.
s_robust = score_robust_mahalanobis(emb_test, pred_test, lam=0.15, trim_frac=0.10, iters=2)

# Global kNN score:
# the mean distance to the nearest neighbours in the whole set.
# Large values correspond to more isolated points.
s_gknn = score_global_knn(emb_test, k=10)

# Class-wise kNN score:
# the mean distance to the nearest neighbours only within the object's own predicted class.
# Helps to find points that look strange specifically within the class.
s_cknn = score_class_knn(emb_test, pred_test, k=8)

# Combine the three scores into one final suspiciousness indicator.
# Before combining, each score is converted into ranks,
# so that the different value scales do not interfere with each other.
# Here robust Mahalanobis has the largest weight,
# global kNN a medium one, and class kNN a smaller additional contribution.
final_score = (
    2.0 * rank_score(s_robust)
    + 1.5 * rank_score(s_gknn)
    + 0.5 * rank_score(s_cknn)
)

# Select the K objects with the largest final score
# as the most likely outliers.
topk = np.argsort(-final_score)[:K]

# Form the binary answer vector:
# 1 means the object is considered an outlier, 0 an ordinary object.
is_outlier = np.zeros(n_test, dtype=np.int64)
is_outlier[topk] = 1

# Assemble the file for submission:
# for each id, give the prediction is_outlier.
submission = pd.DataFrame({
    "id": np.arange(n_test, dtype=np.int64),
    "is_outlier": is_outlier,
})

# Save the submission to CSV without the DataFrame index.
submission.to_csv(SUBMISSION_PATH, index=False)

# Print service information:
# where the file was saved and how many objects are marked as outliers.
print("Saved:", SUBMISSION_PATH)
print("Predicted outliers:", int(is_outlier.sum()))

# Show the first rows of the submission table.
submission.head()
```

```text
Saved: submission.csv
Predicted outliers: 1000
```

Out [5]:

```text
   id  is_outlier
0   0           0
1   1           0
2   2           0
3   3           0
4   4           0
```

Let us check the solution: the cell below computes the score on the public and private parts of the dataset.

In [6]:

```python
#!/usr/bin/env python3
"""
Validate submission.csv against y_test.csv and compute hits@k metrics.

Expected columns:
  y_test.csv: id, is_outlier_true, Usage
  submission.csv: id, is_outlier
"""

from __future__ import annotations

import argparse
import json
from pathlib import Path
from typing import Dict, List

import pandas as pd

BASELINE_SOLUTION_SCORE_PUBLIC=113
BASELINE_SOLUTION_SCORE_PRIVATE=95
AUTHOR_SOLUTION_SCORE_PUBLIC=338
AUTHOR_SOLUTION_SCORE_PRIVATE=308


def _strip_jupyter_kernel_args(unknown: List[str]) -> List[str]:
    cleaned: List[str] = []
    i = 0
    while i < len(unknown):
        tok = unknown[i]
        if tok == "-f" and i + 1 < len(unknown) and unknown[i + 1].endswith(".json"):
            i += 2
            continue
        cleaned.append(tok)
        i += 1
    return cleaned


def _validate_binary_column(col: pd.Series, name: str) -> pd.Series:
    numeric = pd.to_numeric(col, errors="coerce")
    bad = numeric.isna() | (~numeric.isin([0, 1]))
    if bad.any():
        first_bad_idx = int(col.index[bad][0])
        raise ValueError(
            f"Column '{name}' must contain only 0/1. "
            f"First invalid row: {first_bad_idx}"
        )
    return numeric.astype(int)


def _clip_score_0_100(value: float) -> float:
    return max(0.0, min(100.0, float(value)))


def _compute_hits_metrics(
    df: pd.DataFrame, include_score_0_100: bool = True
) -> Dict[str, float | int | bool]:
    k_true = int(df["is_outlier_true"].sum())
    k_pred = int(df["is_outlier"].sum())
    hits_at_k = int(((df["is_outlier_true"] == 1) & (df["is_outlier"] == 1)).sum())
    if AUTHOR_SOLUTION_SCORE <= BASELINE_SOLUTION_SCORE:
        normalized_score_0_100 = 0.0
    else:
        normalized_score_0_100 = 100.0 * (
            (hits_at_k - BASELINE_SOLUTION_SCORE)
            / (AUTHOR_SOLUTION_SCORE - BASELINE_SOLUTION_SCORE)
        )
        normalized_score_0_100 = _clip_score_0_100(normalized_score_0_100)

    metrics: Dict[str, float | int | bool] = {
        "n_samples": int(len(df)),
        "k_true": k_true,
        "k_pred": k_pred,
        "k_match": bool(k_true == k_pred),
        "hits_at_k": hits_at_k,
    }
    if include_score_0_100:
        metrics["score_0_100"] = normalized_score_0_100
    return metrics


def validate_and_score_submission(ytest_path: Path, submission_path: Path) -> Dict[str, object]:
    try:
        ytest = pd.read_csv(ytest_path)
    except Exception as exc:  # pragma: no cover
        raise ValueError(f"Error reading ytest.csv: {exc}") from exc

    try:
        submission = pd.read_csv(submission_path)
    except Exception as exc:  # pragma: no cover
        raise ValueError(f"Error reading submission.csv: {exc}") from exc

    if ytest.empty:
        raise ValueError("ytest.csv is empty")
    if submission.empty:
        raise ValueError("submission.csv is empty")

    required_ytest_cols = {"id", "is_outlier_true", "Usage"}
    required_submission_cols = {"id", "is_outlier"}

    missing_ytest_cols = required_ytest_cols - set(ytest.columns)
    if missing_ytest_cols:
        raise ValueError(f"Columns missing from ytest.csv: {sorted(missing_ytest_cols)}")

    missing_submission_cols = required_submission_cols - set(submission.columns)
    if missing_submission_cols:
        raise ValueError(
            f"Columns missing from submission.csv: {sorted(missing_submission_cols)}"
        )

    ytest = ytest[["id", "is_outlier_true", "Usage"]].copy()
    submission = submission[["id", "is_outlier"]].copy()

    ytest["id"] = ytest["id"].astype(str).str.strip()
    ytest["Usage"] = ytest["Usage"].astype(str).str.strip().str.title()

    submission["id"] = submission["id"].astype(str).str.strip()

    if (ytest["id"] == "").any():
        raise ValueError("ytest.csv contains empty ids")
    if (submission["id"] == "").any():
        raise ValueError("submission.csv contains empty ids")

    invalid_usage = sorted(set(ytest["Usage"]) - {"Public", "Private"})
    if invalid_usage:
        raise ValueError(f"Invalid Usage values in ytest.csv: {invalid_usage}")

    if ytest["id"].duplicated().any():
        duplicates = (
            ytest.loc[ytest["id"].duplicated(keep=False), "id"]
            .unique()
            .tolist()
        )
        suffix = " ..." if len(duplicates) > 10 else ""
        raise ValueError(
            f"ytest.csv contains duplicate ids: {duplicates[:10]}{suffix} "
            f"({len(duplicates)} in total)"
        )

    if submission["id"].duplicated().any():
        duplicates = (
            submission.loc[submission["id"].duplicated(keep=False), "id"]
            .unique()
            .tolist()
        )
        suffix = " ..." if len(duplicates) > 10 else ""
        raise ValueError(
            f"submission.csv contains duplicate ids: {duplicates[:10]}{suffix} "
            f"({len(duplicates)} in total)"
        )

    y_ids = set(ytest["id"])
    s_ids = set(submission["id"])

    missing_ids = y_ids - s_ids
    if missing_ids:
        miss = sorted(list(missing_ids))
        suffix = " ..." if len(miss) > 10 else ""
        raise ValueError(
            f"ids missing from submission.csv: {miss[:10]}{suffix} "
            f"({len(miss)} in total)"
        )

    extra_ids = s_ids - y_ids
    if extra_ids:
        extra = sorted(list(extra_ids))
        suffix = " ..." if len(extra) > 10 else ""
        raise ValueError(
            f"submission.csv contains extra ids: {extra[:10]}{suffix} "
            f"({len(extra)} in total)"
        )

    ytest["is_outlier_true"] = _validate_binary_column(ytest["is_outlier_true"], "is_outlier_true")
    submission["is_outlier"] = _validate_binary_column(submission["is_outlier"], "is_outlier")

    merged = ytest.merge(submission, on="id", how="left", validate="one_to_one")

    if merged["is_outlier"].isna().any():
        raise ValueError("NaN in the predictions after the merge")

    k_true_total = int(merged["is_outlier_true"].sum())
    k_pred_total = int(merged["is_outlier"].sum())
    if k_pred_total != k_true_total:
        raise ValueError(
            "submission.csv must contain exactly K ones in the is_outlier column, "
            f"where K={k_true_total}. Currently: {k_pred_total}"
        )

    overall_metrics = _compute_hits_metrics(merged, include_score_0_100=True)
    public_metrics = _compute_hits_metrics(
        merged.loc[merged["Usage"] == "Public"], include_score_0_100=False
    )
    private_metrics = _compute_hits_metrics(
        merged.loc[merged["Usage"] == "Private"], include_score_0_100=False
    )
    overall_metrics["score_0_100"] = _clip_score_0_100(overall_metrics["score_0_100"])

    return {
        **overall_metrics,
        "n_total": int(len(merged)),
        "n_public": int((merged["Usage"] == "Public").sum()),
        "n_private": int((merged["Usage"] == "Private").sum()),
        "public": public_metrics,
        "private": private_metrics,
    }


def main(argv: List[str] | None = None) -> None:
    parser = argparse.ArgumentParser(description="Check the submission and compute hits@k")
    parser.add_argument("--ytest", type=str, default="y_test.csv", help="Path to y_test.csv")
    parser.add_argument(
        "--submission",
        type=str,
        default="submission.csv",
        help="Path to submission.csv",
    )
    parser.add_argument(
        "--save-json",
        type=str,
        default=None,
        help="Optional path for saving the metrics as JSON",
    )
    args, unknown = parser.parse_known_args(argv)
    unknown = _strip_jupyter_kernel_args(unknown)
    if unknown:
        parser.error(f"unrecognized arguments: {' '.join(unknown)}")

    ytest_path = Path(args.ytest)
    submission_path = Path(args.submission)

    if not ytest_path.exists():
        raise FileNotFoundError(f"ytest file not found: {ytest_path}")
    if not submission_path.exists():
        raise FileNotFoundError(f"submission file not found: {submission_path}")

    metrics = validate_and_score_submission(ytest_path, submission_path)
    print(json.dumps(metrics, indent=2))

    if args.save_json is not None:
        out_path = Path(args.save_json)
        with out_path.open("w", encoding="utf-8") as file:
            json.dump(metrics, file, indent=2)
        print(f"Metrics saved to JSON: {out_path}")


if __name__ == "__main__":
    main()
```

*(Translator's note: the original shows no output for this cell. As printed, `_compute_hits_metrics` refers to `AUTHOR_SOLUTION_SCORE` and `BASELINE_SOLUTION_SCORE`, which the cell does not define; only the `_PUBLIC` and `_PRIVATE` variants are defined.)*