# Catacomb Adventure

*English translation by SOTA – AI Community of the Hungarian original. Licensed CC BY-NC-SA 4.0, like the original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

**Magyar MI Diákolimpia** (Hungarian AI Olympiad)\
**Catacomb Adventure** (Katakomba Kaland)\
Task description\
24 May 2025

## 1. Catacomb Kata

Kátai Kata (better known as **Catacomb Kata** – *Katakomba Kata*) decided one Monday afternoon to go looking for diamonds. Not because she is a romantic type, but because she had heard that diamonds make the best pickaxe, one that can break even obsidian. And with a tool like that you can mine anything: for example, meaning in life after the school-leaving exams, or at least a way out of a catacomb.

Things, however (not very surprisingly), did not go according to plan. Kata got lost. Instead, she found herself in a damp labyrinth of angular walls, where her only hope lies not in maps but in algorithms.

**Your task:** help Kata get out by trying to steer her towards the exit with various reinforcement learning methods. For this, a custom-built `gymnasium` environment is at your disposal, which can be found here:

> 🔗 [Creepy Catacombs - PyPI](https://pypi.org/project/creepy-catacombs-s1/)

During the adventure we will guide you through eight algorithmic challenges, large and small, which you can solve in a Jupyter Notebook (Google Colab or locally). You will need neither a spade nor a torch. It will help, though, if you are not scared of a value function.

## 2. Scoring

1. Exploring the `gymnasium` environment – **[5 points]**
2. Building a search algorithm – **[20 points]**
3. Running the algorithm and visualising the path – **[10 points]**
4. Modifying the algorithm – **[15 points]**
5. Implementing Monte Carlo Control – **[20 points]**
6. Running Monte Carlo and visualisation – **[5 points]**
7. Completing the SARSA algorithm – **[20 points]**
8. Running SARSA and visualisation – **[5 points]**
9. Visualising the aggregated results and learning curves – **[15 points, optional]**

**Maximum score:** 100 points\
**Optional bonus points available:** 15 points

## 3. Technical information

To solve the task, installing the `gymnasium` environment and the attached Notebook file are sufficient. The list of required libraries and the installation instructions can be found at the beginning of the notebook.

After solving the task, regardless of whether you worked in Google Colab or locally, you must upload the `.ipynb` file to the CMS system for the corresponding task. No other file needs to be uploaded.

## 4. Useful resources

- 🔗 [Gymnasium documentation](https://pypi.org/project/gymnasium/)
- 🔗 [Monte Carlo Control](https://towardsdatascience.com/reinforcement-learning-part-4-monte-carlo-control-ae0a7f29920b/)
- 🔗 [SARSA](https://medium.com/@heyamit10/sarsa-algorithms-explained-b94078fd658b)