ECG Signal Disorder Detection

ECG Signal Disorder Detection

Image generated using the DALL-E model.

Introduction

The development of artificial intelligence opens up new possibilities in medical diagnostics, especially in the analysis of complex data such as electrocardiographic (ECG) signals. The ECG is one of the most commonly used diagnostic tools in medicine, making it possible to assess the functioning of the heart and detect abnormalities in it.

Traditionally, an ECG signal is obtained from twelve leads; in this task, however, we will focus on a single-lead signal, i.e. we have one variable representing the electrical voltage generated by the heart over time. These data are recorded as a curve that depends on time, so we can speak of a time series here. Characteristic parts can be distinguished in an ECG signal, namely the P, Q, R, S and T waves, and the intervals between two events in the ECG, among which the R-R interval (the time between the occurrence of two consecutive R waves) plays an important role. In addition, we speak of segments, i.e. the length between two specific waves in the ECG, between which the signal should be at its baseline amplitude. A complex, in turn, is made up of several grouped waves. Here we mainly distinguish the QRS complex. A schematic drawing of an ECG with its parts labelled is presented below.

In the ECG of a healthy person, the PQRST sequence can be observed. First, we distinguish the P wave, which represents the contraction of the atria and is a small vertical deflection before the QRS complex. Next, the QRS complex indicates the contraction of the ventricles and is formed by three deflections: the Q wave, the R wave and the S wave. After that, the ST segment can be seen, i.e. the flat segment between the QRS complex and the T wave, which corresponds to the early phase of ventricular repolarisation. Finally, the T wave, which is a rounded, vertical deflection, relates to the repolarisation of the ventricles and their return to the initial state. The PQRST sequence resembles a sinusoid whose maximum is reached at the R wave. In the case of cardiac disorders, the ECG may show various anomalies, such as additional minima or maxima, or a considerably increased standard deviation over the whole measurement. The characteristics of these anomalies depend on the type and cause of the disorder.

In the task below, you have to deal with samples containing single PQRST sequences and their surroundings. Most samples will correspond to data without anomalies, an example of which is shown in the image below:

There will also be measurements corresponding to four types of disorders: AFib, i.e. atrial fibrillation, PAC, i.e. premature atrial contraction, PVC, i.e. premature ventricular contractions, and STEMI, i.e. ST-elevation myocardial infarction.

NOTE: The data below are synthetic and are only an approximation of real ECG data!

The ECG is a typical example of a time series that can be analysed using dedicated machine learning methods, including neural networks, e.g. recurrent networks. However, using neural networks is not always necessary, or even advisable. For some problems, satisfactory results can be obtained with simpler methods, in which appropriate preparation of the data is key. Skilful analysis of the data makes it possible to select a few meta-features – features that concisely describe the samples in the dataset, e.g. the mean, minimum, maximum, standard deviation, etc. They can be used for classification instead of the original features. As a result, we work with low-dimensional input data; for example, we reduce a 150-dimensional vector containing the information from the original time steps to a 4-dimensional vector containing specially prepared features.

Examples of the use of a small number of meta-features are machine learning models that are meant to run on embedded devices or small mobile devices, where constraints such as the requirement of low power consumption, a small amount of available RAM or limited computing power are crucial. In such cases, simpler models must be used that are able to provide adequate classification accuracy while satisfying the required constraints.

Task

Prepare a solution (including training a random forest model) that meets the requirements of our embedded device. Analyse the data and prepare a set of 4 meta-features that give the best balanced accuracy for the ECG signal classification problem. The dataset consists of a training set and a validation set (with labels), on which you can verify your approach. Your solution will be evaluated on a separate (secret) test set, in which the number of observations will differ from the number of observations in the training and validation sets. Each sample is described by 150 values corresponding to consecutive time steps and is assigned to one of the following five classes:

The new features should contain the key diagnostic information that differentiates the classes above and that will allow the listed anomalies to be classified effectively.

The classifier for this task is a random forest with no more than 10 decision trees and a maximum depth of 10. Solutions that do not meet these conditions will be disqualified! There are no restrictions on the other parameters of the forest. Preprocessing, i.e. initial processing of the input data (e.g. applying data normalisation), is also allowed.

Scoring Criterion

Your solution will be scored on the secret test set on the basis of balanced classification accuracy (balanced accuracy):

$$\text{score}(balanced\_accuracy) = \begin{cases} 0 &\quad \text{if } balanced\_accuracy \leq 75 \% \\ 100 &\quad \text{if } balanced\_accuracy \geq 98 \% \\ \dfrac{balanced\_accuracy - 75 \%}{98 \% - 75 \%} &\quad \text{otherwise} \end{cases}$$

This means that all solutions that achieve up to $75\%$ balanced classification accuracy on the test set will receive $0$ points, while those that achieve at least $98\%$ balanced classification accuracy will receive the maximum number of points for the task. All values in the range $75-98\%$ will be converted into a number of points (between $0$ and $100$) according to the formula above.

Hint: The score on the validation set should be your measure of the quality of the proposed solution.

In problems concerning the detection of diseases, we quite often deal with an imbalanced (unbalanced) dataset. This means that the data are usually dominated by normal examples, corresponding to healthy people, while the samples representing sick people usually form a minority. Imagine a situation in which, out of 100 samples, only 10 concern sick people and the remaining 90 concern healthy people. A model that assigned the class healthy to every sample would then achieve 90% classification accuracy, but only 50% balanced classification accuracy! Of course, such a model would be useless. In such cases, we need a measure that better corresponds to the needs arising from the problem at hand and that reports the effectiveness of the model in a way that is useful from the point of view of its eventual user.

In this task, you must therefore focus on making sure that each of the classes is assigned correctly.

Constraints

• Your solution will be tested on the Contest Platform without internet access and in an environment without a GPU.
• The evaluation of your final solution on the Contest Platform must not take longer than 1 minute without a GPU.
• When preparing the data, remember that:
  • the use of machine learning methods other than random forests, whether supervised or unsupervised (e.g. autoencoders, multilayer perceptrons and other neural networks, support vector machines (SVM), and others), is forbidden; dimensionality reduction methods such as principal component analysis (PCA) are, however, allowed;
  • when constructing the meta-features, you may only use functions available as standard in Python (v3.11), as well as Numpy (v2.0.2) and Scipy (v1.14.1);
  • at most 4 meta-features may be computed,
• Only the random forest (RandomForestClassifier) from the scikit-learn library (v1.5.2) may be used for classification:
  • consisting of at most 10 decision trees (n_estimators $\leq 10$);
  • each tree is to have a maximum depth of 10 (max_depth $\leq 10$);
  • the other hyperparameters may be modified without restriction;

Submission Files

This notebook, completed with your solution (see the YourSolution class), in which you prepare a set of 4 meta-features describing the dataset and a set of random forest hyperparameters.

Evaluation

Remember that during checking, the FINAL_EVALUATION_MODE flag will be set to True.

You can score between 0 and 100 points for this task. The number of points you receive will be calculated on the (secret) test set on the Contest Platform on the basis of the formula given above, rounded to an integer. If your solution does not meet the above criteria or does not run correctly, you will receive 0 points for the task.

Supplementary Information

Balanced Classification Accuracy

Let $C$ be the number of classes, and let $N_j$ be the number of samples belonging to the $j$-th class, where $j \in \lbrace 1, ..., C \rbrace$. Furthermore, let $\hat{y}_{i,j}$ be the class predicted by the model for the $j$-th sample that actually belongs to the $i$-th class. The balanced classification accuracy can then be computed as follows:

$$balanced\_accuracy = \dfrac{1}{C} \sum\limits_{i=1}^{C} \sum\limits_{j=1}^{|N_c|} \dfrac{1}{|N_c|} \cdot \mathbf{1} \left( \hat{y}_{i, j} = i \right),$$

where $\mathbf{1} \left( \hat{y}_{i, j} = i \right)$ is an indicator function that takes the value 1 if $\hat{y}_{i, j} = i$, i.e. when the class predicted for the $j$-th sample is the same as the true class of that sample, and 0 otherwise. The outer sum runs over the successive classes, and the inner sum over the successive samples belonging to the given class.

Example: Let

$$\mathbf{y} = [0, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3]$$

be the vector representing the true classes of the successive samples, and let

$$\mathbf{\hat{y}} = [0, 0, 0, 0, 0, 2, 2, 2, 2, 3, 3, 3]$$

be the vector representing the model's predictions for these samples. We therefore have four classes, and the model had a problem with class number $1$. All the other examples were assigned without error. In total, 10 out of 12 samples were classified correctly, which means that if we were to measure "ordinary" classification accuracy, we would obtain approx. $83.3\%$ . However, when we look at the balanced classification accuracy, we obtain a score of $75\%$.

Now suppose that

$$\mathbf{\hat{y}} = [0, 0, 0, 1, 0, 2, 2, 2, 2, 3, 3, 3]$$

i.e. the model correctly classifies $50\%$ of the samples from class 1 and $100\%$ of the samples from the other classes. The "ordinary" classification accuracy here is just under $92\%$, whereas the balanced classification accuracy is $87.5\%$.

Anomalies Occurring in the Dataset Under Consideration

AFib

Atrial Fibrillation (AFib) occurs when action potentials are triggered very rapidly and chaotically, and as a result the heart rhythm is irregular. In this disorder, the P waves may not be visible on the ECG, and the QRS complex becomes irregular.

PAC

Premature Atrial Contractions (PACs) are associated with an abnormal P wave followed by a normal QRS complex. Note! The samples in this task include examples in which only the premature P wave is visible within a single sample of the dataset.

PVC

Premature Ventricular Contractions (PVCs) are extra heartbeats that begin in one of the two ventricles of the heart and disrupt its regular rhythm. They are one of the common types of arrhythmia. These contractions occur earlier than would be expected from the preceding R-R intervals.

STEMI

ST-elevation myocardial infarction (STEMI) blocks the flow of blood to the heart muscle and causes the muscle to die. The ST segment occurs immediately after the QRS complex. Normally there is no electrical activity there, which is why it is flat. If, however, the ST segment is elevated, this indicates a blockage of one of the main arteries supplying blood to the heart.

Sources for the medical description of the ECG: 1, 2, 3, 4, 5, 6, 7; the image of the PQRST waves is based on 8.

Starter Code

In this section, we initialise the environment by importing the required libraries and functions. The prepared code will help you to work with the data efficiently and to build the actual solution.

Loading the Data

The code below loads the data.

Public Solution Interface

This is all we require of your class. In your solution, you may modify your class as you wish by adding attributes and new methods that compute meta-features – anything you need to solve the task.

Code with the Scoring Criterion

Code similar to the code below will be used to score the solution on the test set.

Example Solution

Below we present a simplified solution that serves as an example demonstrating the basic functionality of the notebook. It can serve as a starting point for developing your own solution.

Your Solution

Place your solution in this section. Make changes only here!

Evaluation

Running the cell below lets you check how many points your solution would score on the validation data. Before submitting, make sure that the whole notebook runs from start to finish without errors and without any user intervention after selecting the "Run All" option.

During checking, the model will be saved as your_model.pkl and scored on the test set.