Multi-class Animal Species Classification in Camera-trap Photos

Multi-class Animal Species Classification in Camera-trap Photos

English translation by SOTA – AI Community of the Russian original. Organisers who would like this translation removed can email [email protected].

Source: the 2024/25 task collection of the National Technology Olympiad (NTO), "Artificial Intelligence" profile, Stage 2, engineering tour, pp. 109–112: original PDF.

3.2.2. Team task

The task is devoted to the classification of animals.

For this, participants receive access to unique data: images from camera traps provided by partner research organisations. The solution to the task should sort all images with animals into the given animal species and make it possible to organise prompt monitoring of injured animals and control over wild animals entering populated areas.

All materials for the task are available at: https://disk.yandex.ru/d/Jx8fZJTSsk_LeQ.

Task 3.2.2.1. Multi-class Animal Species Classification in Camera-trap Photos (100 points)

Topics: neural networks, computer science.

Problem statement

Task description

Monitoring wild animal populations is an important task for the worldwide community of wildlife researchers. It helps to find out which animal species are endangered, how they behave in different periods of their lives, where they live, and much more.

Various tools are used to monitor animals, including camera traps: special cameras that are installed in the forest and react to movement in the frame. Every year these cameras deliver hundreds of thousands of photographs, in which the animals have to be found and categorised. This is very difficult and painstaking work, because the total number of species can reach several hundred, and there may be some that are visually hard to tell apart. In addition, only part of an animal may be in the frame at the moment the camera is triggered, and because many species are active at night, some photographs may be blurred or overexposed owing to the way the cameras work at night.

Within the NTO Olympiad, participants are invited to help scientists automate the routine work of processing camera-trap data by training machine learning models for this purpose.

To monitor the numbers of each animal species, participants need to develop a classifier that sorts photographs of animals by species.

Metric

The model's quality is assessed with the macro F1 score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html (PDF version of the page: https://disk.yandex.ru/i/tRtB4WGtGfSFlQ).

Its distinctive feature is that it computes the arithmetic mean of the F1 score over the classes. The value of the metric ranges from 0 (in the worst case) to 1, if all predictions were correct.

The metric is computed automatically on the platform when a solution is submitted.

Data description

The dataset is a set of images and a file train.csv that maps each image to a particular animal species.

The correspondence between the species index and its name is given below:

• 0 — brown bear,
• 1 — Himalayan bear (Asiatic black bear),
• 2 — wild boar,
• 3 — Manchurian wapiti (izubr),
• 4 — sika deer,
• 5 — maral,
• 6 — Siberian roe deer,
• 7 — Asian badger,
• 8 — sable,
• 9 — Amur leopard cat,
• 10 — Pallas's cat (manul),
• 11 — lynx,
• 12 — tiger,
• 13 — snow leopard (irbis),
• 14 — argali,
• 15 — ibex,
• 16 — wolf,
• 17 — fox,
• 18 — raccoon dog,
• 19 — hare,
• 20 — marmot.

Note that the dataset contains difficult examples: some animal species are similar to one another (for example, Siberian roe deer, sika deer, Manchurian wapiti and maral), photos in which only a small part of an animal is in the frame, as well as night-time and blurred shots.

The data were labelled by species by zoologists. In disputed cases, the labels were confirmed by several experts.

The data are split into a training set and a test set in a ratio of approximately 70% / 30%.

The test part is split into a public part and a private part in a ratio of approximately 40% / 60%.

Data: https://disk.yandex.ru/d/Jx8fZJTSsk_LeQ

• train.zip — archive with the training data;
• train.csv — file with the labels for the training data;
• test.zip — the test data on which predictions must be made;
• baseline.ipynb — a jupyter-notebook with a simple starting solution;
• sample_submission.csv — an example prediction file (all class labels are filled with zeros).

Baseline

A Jupyter-notebook that takes you all the way from installing the libraries and training the model to obtaining a file with predictions, which can be uploaded to the platform to see the metrics.

Input format

The competition involves submitting a file with the model's predictions to the platform for the metric to be computed. The file itself is a csv document with two columns, image_name and predicted_class. The rows are pairs of an image file name and the corresponding class index predicted by your model.

An example of producing a correct prediction file.

For the structure of the prediction file, see Table 3.2.2.

Table 3.2.2

Use of third-party datasets

The use of open (available online) datasets with a licence permitting free non-commercial use is allowed.

Solution

An algorithm is to be developed for classifying the group of animals in camera-trap photos, and the result of its work is to be presented as a csv file with the columns image_name, predicted_class.

The solution to the task is a csv file with predictions, uploaded to the competition platform to evaluate the quality of the solution. Example of a submission file: submission.csv.

Notes

A walkthrough of the reference solution is available at: https://gist.github.com/ntomaterials/b44ef19161f1dcb4259667d967abfbd2.