Machine (Un)learning

Machine (Un)learning

Image generated using the DALL-E model.

Introduction

Motivation

Machine unlearning is a fascinating and increasingly popular topic related to learning algorithms, above all to deep learning networks. As the number of parameters and the sophistication of models grow, their ability to remove outdated, erroneous or sensitive information (e.g. information related to user privacy) becomes crucial. Unlearning makes it possible to remove selected information from the model's memory in a way that minimises the resulting harm with respect to the other learned data. This allows us to obtain more precise and safer models, without the constant need to train models from scratch.

Benefits

Introducing unlearning into the world of deep learning networks can bring many benefits. Among other things, it makes it possible to remove a model's ability to generate harmful content. For example, if a generative model has been trained on a huge set of images, but some of them contain inappropriate content, such as nudity, unlearning can be applied to remove the concept of nudity from the model's memory, which will reduce the likelihood of similar images being generated in the future.

Secondly, under personal data protection legislation (such as the GDPR, i.e. the General Data Protection Regulation in the European Union), users have the right to withdraw their consent to the processing of their data by a given company. However, if a model has been trained using that data, it is not trivial to get rid of this knowledge without having to retrain the model.

Finally, unlearning can help to eliminate biases and errors that may appear in models, which leads to fairer and more accurate results. Machine learning models can sometimes absorb biases and inequalities contained in the training data (for instance, related to sex or skin colour), which results in biased predictions. Unlearning makes it possible to remove these biases, which improves the quality and fairness of the results.

Note: In the classic machine unlearning problem, we do not so much want the model to be unable to predict a given class, but rather want it to behave as if that data had not been in the training set (which means that the model is resistant to a membership inference attack). Hence, the desired property is often not to minimise performance on a certain part of the data directly, but to aim for the model to behave randomly towards that data, or in the same way as towards data that really was not in the training set. For the purposes of our task, we simplify this goal: we want the model simply to predict the given class with the lowest possible accuracy.

Problem description

For our adventure with unlearning, we will analyse a classification problem on a standard computer vision dataset, Fashion MNIST. To solve this problem, we will use the classic LeNet convolutional architecture (visualisation and description below). We are provided with a baseline model that has been trained on the whole dataset, i.e. on all ten classes of the Fashion MNIST dataset (T-shirt/top, trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, ankle boot). The code for loading this model is given below.

Your task

Your task is to make the baseline model unlearn a given, selected class of the Fashion MNIST dataset. We require the last layer (the classification layer) to remain untouched; in other words, your intervention must take place at the level of the feature extractor, and not through a mechanical modification of the signal responsible for generating the logits for the individual classes. The intervention should be confined exclusively to the earlier layers of the model, excluding any direct change to the model's classification function itself.

Evaluation

To check how well you have coped with this task, we have prepared a set of metrics that will allow us to assess the quality of your solution.

We will assess you on four aspects:

• Unlearning the classification of the selected class - after all, that is what unlearning is about! However, you do not know which class you will ultimately be assessed on unlearning, within the test set. For validation, in this notebook we have chosen one of the ten classes of the Fashion MNIST dataset, but remember that in the end you will be assessed on a different class!
• Maintaining high performance on the remaining classes - the model must stop working only on the selected class, but it must not lose performance on the remaining classes - here the accuracy must be as high as possible!
• Intervention in the baseline model - try to keep the modification of the baseline model's weights as small as possible!
• Diversity of predictions for the forgotten class - the point is for the model to be resistant to the previously mentioned membership inference attack, i.e. it must not be possible to tell that the model has ever seen the data it is supposed to unlearn!

Detailed description:

Unlearning the i-th class, $i \in \lbrace 1, ..., 10 \rbrace$.

The process of unlearning the knowledge of the i-th class (one of the ten possible classes) consists in removing the model's ability to recognise and classify data belonging to that particular class. This is a key aspect of unlearning, which makes it possible to eliminate undesirable or inappropriate data from the model. Specifically, we can measure the accuracy on the data belonging to the specific class number i as $\Psi_{i}$:

$$\Psi_{i} = \frac{TP_i}{TP_i + FN_i},$$

where $TP_i$ denotes the number of examples correctly classified into class $i$ (true positive rate), and $FN_i$ denotes the number of examples from class $i$ that were incorrectly classified as belonging to another class (false negative rate). In our problem we want to minimise the value of $\Psi_{i}$.

Maintaining high performance on the remaining classes

An important aspect of unlearning is ensuring that removing the i-th class does not adversely affect the model's performance on the remaining classes. The model must still be able to achieve high classification accuracy on all the remaining classes. This is a challenge, because the unlearning process can disturb the balance of the model and adversely affect its ability to extract features, including for the remaining classes. The set of the data that should still be classified well by the model is simply called the remain dataset. We can assess the model's classification accuracy on this data by measuring it for all classes except class number $i$, and define this as a function $\Phi$ that depends on $i$.

Specifically, we can use the accuracy on the remain set as $\Phi_{i}$:

$$\Phi_{i} = \frac{\sum_{k \neq i} TP_k}{\sum_{k \neq i} \left( TP_k + FN_k \right)},$$

where: $TP_k$ is the number of correctly classified examples belonging to class $k$, excluding class $i$ (true positive rate), and $FN_k$ is the number of examples incorrectly not classified as class $k$, excluding class $i$ (false negative rate). In other words, the numerator contains the number of correctly classified examples from all classes except class number $i$, and the denominator the number of examples belonging to all classes, excluding the $i$-th class.

Intervention in the baseline model

The unlearning process involves an intervention in the baseline model, which may have various consequences. It is important that this intervention is minimal and does not lead to destabilisation of the model. In practice, this means that the changes made to the model should be limited to the modifications necessary to remove class i, without affecting the structure and functionality of the model, and therefore with minimal modification of the model's parameters. We require that the last layer of the LeNet model remains untouched, but we will also measure the distance between the original model and the unlearned model that you propose. This is also a way to better understand how the model works. If the distance from the baseline model is not too large, and we trust the baseline model, then the model you obtain is close enough to earn our trust.

In our problem we will use the traditional $\mathbf{L_2}$ distance, which can be expressed by the formula:

$$\mathcal{d}_{dist} = \mathcal{L_2} (\theta; \theta_0) = \sqrt{\sum_j (\theta_j - \theta_{0,j})^2},$$

where

• $\theta$ denotes the vector of the model's parameters after the unlearning process,
• $\theta_0$ denotes the vector of the baseline model's parameters,
• $\theta_j$ and $\theta_{0,j}$ are the corresponding parameter values for the j-th layer of the models.

The $\mathcal{L}_2$ distance sums the squared differences between the successive layers of the baseline model and the corresponding layers of the modified model, and finally takes the square root of this sum. The smaller the value of $\mathcal{d}_{dist}$, the smaller the intervention in the baseline model, which is desirable in the context of minimising changes to the structure of the model.

Diversity of predictions for the forgotten class

The last aspect of the assessment is the diversity of predictions for the forgotten class. After the unlearning process, the model should generate diverse predictions for data that belong to class $i$, which the model was supposed to forget. This means that the model should simply assign them to other classes in a varied way, so that it cannot easily be determined whether this data was originally in the training set of the baseline model.

The model should also not assign all predictions to one particular class, because this could be interpreted as merging the selected classes, which we do not want.

To measure the diversity of predictions, we will use the Kullback-Leibler (KL) divergence. It is a standard measure of the discrepancy between two probability distributions. In our case, we examine the distance of the distribution of the model's predictions from the uniform distribution, which means that none of the remaining classes should be favoured.

We define the KL divergence as:

$$\mathcal{D}_{KL}(p \parallel u) = \sum_{c=1}^{C} p(c) \log \left( \frac{p(c)}{u(c)} \right),$$

where:

• $p(c)$ is the probability, assigned by the model, that the sample belongs to class $c$,
• $u(c)$ is the probability according to the uniform distribution, i.e. $u(c) = \frac{1}{C}$,
• $C$ is the total number of classes.

In the case of ideal diversity of predictions for the forgotten class, the distribution $p(c)$ should be as close as possible to the uniform distribution $u(c)$, which means a minimal value of $\mathcal{D}_{KL}$.

If the KL divergence still seems rather complicated, do not worry - in the Supplementary Information section we have provided some helpful hints. In addition, take a look at the code of the function definition - we have left a few comments in it.

We will evaluate all four of the above objectives on the Fashion MNIST data, but you do not know which class we will ultimately want to unlearn. Moreover, you do not know which subset of the data from the individual classes will be used for unlearning. To prepare your solution you may choose any class, but the solution must be ready to unlearn any class of this dataset. Although the data distribution in the test set will be similar to the distribution in this notebook, try to avoid overfitting to the selected data.

So check whether your solution is universal. Make sure that it works for different labels designated to be forgotten and for different datasets (the architecture is fixed, adapted to processing visual data of size $28 \times 28$).

Final score

Overall, your score can be written mathematically as a weighted sum:

$$\Sigma_{score} = \frac{1}{4} \cdot \Sigma_{target} + \frac{1}{4} \cdot \Sigma_{remain} + \frac{1}{4} \cdot \Sigma_{dist} + \frac{1}{4} \cdot \Sigma_{kl}$$

Note: We would like to maximise the metric $\Phi$ and to minimise the others, i.e. $\Psi$, $d_{dist}$ and $\mathcal{D}_{KL}$. Hence, pay attention to the order of subtraction when computing the individual $\Sigma$ components for the final evaluation!

$\Sigma_{target}$ – the score for the effectiveness of unlearning on the selected class (for simplicity, in this task we assume that the lower the accuracy on the target class, the better; scaled to the range [0, 100] according to the thresholds $[0.09, 0.3]$):

$$\Sigma_{target}(\Psi_{i}) = \begin{cases} 100 & \text{if } \Psi_{i} \leq 0.09 \\ 0 & \text{if } \Psi_{i} \geq 0.3 \\ 100 \cdot \dfrac{0.3 - \Psi_{i}}{0.3 - 0.09} & \text{otherwise} \end{cases}$$

$\Sigma_{other}$ – the score for maintaining accuracy on the remaining classes (the higher the accuracy, the higher the score for this metric; scaled to the range [0, 100] according to the thresholds $[0.87, 0.90]$):

$$\Sigma_{other}(\Phi_{i}) = \begin{cases} 100 & \text{if } \Phi_{i} \geq 0.90 \\ 0 & \text{if } \Phi_{i} \leq 0.87 \\ 100 \cdot \dfrac{\Phi_{i} - 0.87}{0.90 - 0.87} & \text{otherwise} \end{cases}$$

$\Sigma_{dist}$ – the score for the degree of intervention in the model (the smaller the $L_2$ distance, the higher the score for this metric; scaled to the range [0, 100] according to the thresholds $[1.3, 3.0]$):

$$\Sigma_{dist}(\mathcal{d}_{dist}) = \begin{cases} 100 & \text{if } \mathcal{d}_{dist} \leq 1.3 \\ 0 & \text{if } \mathcal{d}_{dist} \geq 3.0 \\ 100 \cdot \dfrac{3.0 - \mathcal{d}_{dist}}{3.0 - 1.3} & \text{otherwise} \end{cases}$$

$\Sigma_{kl}$ – the score for the diversity of predictions (the lower the KL divergence, the higher the score for this metric; scaled to the range [0, 100] according to the thresholds $[0.2, 0.5]$):

$$\Sigma_{kl}(\mathcal{D}_{KL}) = \begin{cases} 100 & \text{if } \mathcal{D}_{KL} \leq 0.2 \\ 0 & \text{if } \mathcal{D}_{KL} \geq 0.5 \\ 100 \cdot \dfrac{0.5 - \mathcal{D}_{KL}}{0.5 - 0.2} & \text{otherwise} \end{cases}$$

However, the task is scored outright as 0 points in all categories if your solution:

• does not comply with the guidelines, for example:
  • a change is made to the network architecture;
  • the last layer of the network is modified;
  • any attempt at cheating is made, e.g. by modifying the evaluation function;
• is unsatisfactory:
  • the classification accuracy on the set of the class to be forgotten stays above 0.5;
  • the classification accuracy on the set of the remaining classes falls below 0.75;
  • the $L_2$ distance is greater than 8.0;
  • the value of the Kullback-Leibler divergence exceeds 1.75.

Furthermore, the model unlearning you propose, i.e. the execution of the unlearn() function, may take no longer than 5 minutes using a GPU.

You can receive a maximum of 100 points for this task.

Remember that during checking, the FINAL_EVALUATION_MODE flag will be set to True.

Good luck!

Starter code

Data

Model

Source: GeeksForGeeks

Loading the model

Metrics

Compliance

Trivial solution

Trivial solution - evaluation

Supplementary information

Explanation of the Kullback-Leibler divergence

The Kullback-Leibler divergence (Kullback-Leibler divergence, KL) is a measure of the discrepancy between two probability distributions. In the context of machine learning and statistics, the KL divergence makes it possible to assess how much one distribution differs from another. Formally, for two discrete distributions $P$ and $Q$, it is defined as:

$$D_{KL}(P \parallel Q) = \sum_{x \in X} P(x) \log \frac{P(x)}{Q(x)}$$

where:

• $P(x)$ is the true distribution (e.g. real data),
• $Q(x)$ is the approximating distribution (e.g. the model's prediction),
• $X$ is the event space.

The KL divergence is not a symmetric measure, which means that $D_{KL}(P \parallel Q) \neq D_{KL}(Q \parallel P)$. The value of the divergence is 0 when the two distributions are identical.

Examples of comparisons of discrete distributions:

1. Example 1: Identical distributions

   • $P = [0.4, 0.6]$
   • $Q = [0.4, 0.6]$
   • $D_{KL}(P \parallel Q) = 0$

2. Example 2: Slightly different distributions

   • $P = [0.4, 0.6]$
   • $Q = [0.5, 0.5]$
   • $D_{KL}(P \parallel Q) \approx 0.02$

3. Example 3: Very different distributions

   • $P = [0.9, 0.1]$
   • $Q = [0.1, 0.9]$
   • $D_{KL}(P \parallel Q) \approx 0.75$.

Visualisation of the KL divergence

Below is Python code that visualises the KL divergence for two discrete distributions.

import numpy as np
import matplotlib.pyplot as plt

def kl_divergence(p, q):
     """Computes the KL divergence between two distributions."""
     p = np.array(p)
     q = np.array(q)
     return np.sum(p * np.log(p / q))

# Example distributions
P = [0.4, 0.6]
Q_list = [
     [0.4, 0.6],  # Identical distribution
     [0.5, 0.5],  # Slightly different
     [0.9, 0.1]   # Very different
]

# Computing the KL divergence
kl_values = [kl_divergence(P, Q) for Q in Q_list]

# Visualisation
labels = ['Q1 (identyczny)', 'Q2 (nieznacznie różny)', 'Q3 (bardzo różny)']  # "identical", "slightly different", "very different"
x = np.arange(len(Q_list))

plt.bar(x, kl_values, color='skyblue')
plt.xticks(x, labels, rotation=15)
plt.ylabel('KL Dywergencja')  # "KL divergence"
plt.title('KL Dywergencja dla różnych rozkładów Q względem P')  # "KL divergence for different distributions Q relative to P"
plt.show()

Interpretation

• When $P$ and $Q$ are identical, the KL divergence is 0.
• When $Q$ differs from $P$, the KL divergence increases.
• The KL divergence is not symmetric, so swapping the order of $P$ and $Q$ changes the result. This is worth bearing in mind when interpreting the results.
• You may notice a difference between the formula in this task and the formula in the measure_target_uniformity() function. This is because we compare our probability distribution with the uniform distribution, in which case the formula transforms into that form.

Submission files

All we need from you is this notebook - together with the key definition of the unlearn method, which will find the final weights of the model on which the evaluation will be carried out for a class of the Fashion MNIST dataset chosen by us.

Constraints

We have proposed guidelines that enforce low performance of the model on the target class. Remember that, in the general case, the goal of our unlearning algorithm should be to reach a state in which the model behaves as if the data from the target class had never been part of the training set.

In addition, we prohibit modifying the hyperparameters of the architecture, as well as any parameters of the classification layer.

We expect your solution to be based on, and to use, only standard libraries used in machine learning, such as torch, numpy, matplotlib/seaborn and scikit-learn.

Your solution