# Theory Questions

*English translation by SOTA – AI Community of the Hungarian original. Licensed CC BY-NC-SA 4.0, like the original. Organisers who would like this translation removed can email sota.ai.community@gmail.com.*

**Magyar MI Diákolimpia** (Hungarian AI Olympiad)<br>
**National Selection**<br>
Round 1

May 21, 2025

Name: ______________________________

Contestant ID: ____________________

You have exactly **60 minutes** to complete the paper. The round **consists of 28 questions**, and the whole paper is **worth a maximum of 200 points**.

This paper is the first round of the **Magyar Mesterséges Intelligencia Diákolimpia** (Hungarian Artificial Intelligence Olympiad), which must be completed on paper, without the use of any aids.

The questions test your knowledge in four main topic areas: **machine learning** (ML), **computer vision** (CV), **natural language processing** (NLP) and **reinforcement learning** (RL).

A total of **600 points** can be earned during the olympiad. The result of the first round counts towards the final score.

We wish you the best of luck!

---

## Multiple-choice questions (5 points per question)

**1.** How many independent datasets are needed during training and evaluation when L1 regularisation and early stopping are used together?

- ○ One
- ○ Two
- ○ Three
- ○ Four
- ○ It depends on which loss function we use

**2.** Which of the following is **not** biologically inspired?

- ○ Hebbian learning
- ○ ADAM
- ○ Self-organising map
- ○ Convolutional network
- ○ Spike-time dependent plasticity

**3.** Select the supervised learning algorithms from the following.

- ○ K-means
- ○ K-nearest neighbours (K-NN)
- ○ Self-organising map
- ○ Perceptron learning
- ○ PCA

**4.** Which of the following methods use **only the first-order derivative**?

- ○ SGD with Nesterov momentum
- ○ ADAM
- ○ Newton's method
- ○ Coordinate descent method
- ○ L-BFGS

**5.** Consider the following two statements:

* The sigmoid is actually a transformed hyperbolic tangent.
* In general, softplus does not perform worse than ReLU.

- ○ Both are true
- ○ The first is true, the second is false
- ○ The first is false, the second is true
- ○ Both are false
- ○ The second depends on the specific task

**6.** What is the correct course of action if a gradient with a NaN value arises during training?

- ○ Nothing, this does not affect training
- ○ Taking a small random step after the update
- ○ Skip the current step
- ○ Skip the step, and take a small random step instead
- ○ The whole training must be restarted

**7.** Which method is the Pointer Network related to?

- ○ Seq2Seq architecture
- ○ Bahdanau attention
- ○ Nesterov momentum
- ○ Copy mechanism
- ○ Transformers

**8.** Which statement is **true** about the dropout technique?

- ○ It sets the same weights to zero in every step
- ○ It permanently sets some of the network's weights to zero
- ○ It permanently sets certain parts of the output to zero
- ○ It temporarily switches off some parts of the network during training
- ○ It reduces the learning rate locally

**9.** Which statements are true regarding datasets?

- ○ The validation set and the test set are always identical
- ○ The validation set is used during training
- ○ The test set serves only for the final evaluation
- ○ The test set and the training set must be the same size
- ○ A validation set is not always needed

**10.** Which of the following are **hyperparameters**?

- ○ Learning rate
- ○ Batch size
- ○ Model accuracy
- ○ Current values of the weights
- ○ Number of layers in the network

**11.** Which statements are true about the loss function in artificial neural networks?

- ○ Its value is always between 0 and 1
- ○ It must increase during training
- ○ It always decreases monotonically
- ○ It must be differentiable for training (e.g. with SGD)
- ○ Log loss is better than MSE

**12.** Which statements are true about weight initialisation?

- ○ The weights must always be set to zero
- ○ The weights are usually initialised randomly
- ○ The initial value does not matter
- ○ The weights can also be derived from retraining (e.g. transfer learning)
- ○ The bias values are always set to 1

**13.** Which statements are true about the K-means clustering algorithm?

- ○ The number of clusters (k) is a parameter of the method
- ○ Every point belongs to the centre nearest to it
- ○ The loss function strongly influences convergence
- ○ It does not require initialisation
- ○ It can only handle spherical clusters

**14.** Which statements are true about normalisation and scaling?

- ○ If it is not applied, variables with large values can dominate
- ○ Normalisation brings the mean to zero
- ○ Normalisation and scaling are the same
- ○ Scaling always means dividing by the mean
- ○ Scaling often also includes dividing by the standard deviation

**15.** Which metrics are frequently used to evaluate binary classification tasks?

- ○ F1-score
- ○ ROC-AUC
- ○ Precision and Recall
- ○ Mean Squared Error
- ○ BLEU score

**16.** Which variables influence learning in reinforcement learning?

- ○ Learning rate ($\alpha$)
- ○ Discount factor ($\gamma$)
- ○ Exploration rate ($\epsilon$)
- ○ Error tolerance
- ○ Dropout rate

**17.** Which characteristics describe the Q-learning algorithm?

- ○ Off-policy
- ○ It can use tabular value updates
- ○ It does not require knowledge of a model of the environment
- ○ It is based on rule-based inference
- ○ It does not take future rewards into account

**18.** Which statement is true about the attention mechanism?

- ○ It can create context-dependent representations
- ○ It weights the influence of a token relative to the other tokens
- ○ It is used exclusively for processing images
- ○ It serves to optimise the learning rate
- ○ It can only be used in recurrent networks

**19.** Which statements are true about convolutional layers in computer vision?

- ○ They are capable of recognising local patterns
- ○ They massively reduce the number of parameters to be learnt
- ○ They apply the same filter at different positions
- ○ Every neuron receives a separate weight for every pixel
- ○ They can only be used on black-and-white images

**20.** Which factors can lead to overfitting in a machine learning model?

- ○ An overly complex model with little data
- ○ Missing regularisation
- ○ Use of dropout
- ○ Training for too long without validation
- ○ Low learning rate

---

## Essay questions (20 points per question)

**1. What limitations does the use of linear and convolutional layers on sequential inputs entail?**

Describe what problems can arise from using these layers directly when the input is temporal or sequential in nature.

Also discuss how these limitations can be overcome with the help of modern architectures or learning strategies.

**2. What is meant by batch optimisation?**

Describe the concept, explain how it works in practice, and compare it with other optimisation strategies:

- Full-dataset (batch)
- Mini-batch
- Stochastic (SGD)

Explain what effect increasing or decreasing the batch size has on:

- the stability of learning
- the gradient estimate
- the speed of convergence
- the ability to generalise

**3. Describe the early stopping technique. In solving what problem can it be useful?**

Describe how the early stopping method works, what types of metrics we monitor, and under what condition we stop the training.

Explain what phenomenon it serves to prevent, and under what circumstances it is advisable to use it.

**4. Binary classification: the significance of FP and FN**

In a binary classification task, the answers can be true positives (TP), true negatives (TN), false positives (FP) or false negatives (FN).

Give an example of an application where:

- it is more important to reduce FP, even at the expense of FN
- it is more important to minimise FN, even if more FPs arise

Justify both examples. In what types of applications do these differences matter?

**5. How does the value of the discount factor ($\gamma$) influence learning in reinforcement learning?**

Describe what the discount factor ($\gamma$) means and how it appears in the calculation of the return or of the value function.

Address the following points:

- What does it mean if $\gamma$ is close to 0? And if it is close to 1?
- For what types of environments is it advisable to choose a lower or a higher $\gamma$?
- What differences in behaviour can a discount factor that is too small or too large lead to?