# Examples of computing NDCG@20

*English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

These examples follow the definition of the metric in [`task.md`](task.md).

Useful discount values:

- `discount(1) = 1 / log2(2) = 1`
- `discount(2) = 1 / log2(3) ≈ 0.63093`
- `discount(3) = 1 / log2(4) = 0.5`
- `discount(4) = 1 / log2(5) ≈ 0.43068`

In all the examples, `rel(rank) = 1` if the edition at position `rank` is a lost item for the user, otherwise `rel(rank) = 0`.

## Example 1: all lost items at the top (NDCG = 1.0)

The user's hidden lost items (ground truth): `{101, 202}`.

The participant's prediction (only the positions with `rel = 1` are shown; for the remaining positions 3..20, `rel = 0`):

| rank | edition_id | rel | contribution to DCG |
|-----:|-----------:|----:|------------:|
| 1 | 101 | 1 | \(1·1=1\) |
| 2 | 202 | 1 | \(1·0.63093≈0.63093\) |

In total:

- `DCG ≈ 1 + 0.63093 = 1.63093`
- `IDCG` is the same (ideal ranking: both lost items in the first positions) ⇒ `IDCG ≈ 1.63093`
- `NDCG = DCG / IDCG = 1.0`

## Example 2: lost items ranked lower (NDCG < 1.0)

The same hidden labelling `{101, 202}`, but both lost items are placed lower:

| rank | edition_id | rel | contribution to DCG |
|-----:|-----------:|----:|------------:|
| 2 | 101 | 1 | \(1·0.63093≈0.63093\) |
| 3 | 202 | 1 | \(1·0.5=0.5\) |

In total:

- `DCG ≈ 0.63093 + 0.5 = 1.13093`
- `IDCG` is the same as in Example 1: `IDCG ≈ 1.63093`
- `NDCG ≈ 1.13093 / 1.63093 ≈ 0.6937`

## Example 3: misses and imperfect ordering

The user's hidden lost items (ground truth): `{11, 22, 33}`.

The participant's prediction (only the positions with `rel = 1` are shown):

| rank | edition_id | rel | contribution to DCG |
|-----:|-----------:|----:|------------:|
| 1 | 11 | 1 | \(1·1=1\) |
| 4 | 22 | 1 | \(1·0.43068≈0.43068\) |

In total:

- `DCG ≈ 1 + 0.43068 = 1.43068`

Now `IDCG`: the ideal ranking has the three lost items in the first positions:

- `IDCG ≈ discount(1) + discount(2) + discount(3) = 1 + 0.63093 + 0.5 = 2.13093`
- `NDCG ≈ 1.43068 / 2.13093 ≈ 0.6713`