Discord

Checklist NEOAI 2025 Day 3 · Task 8

Intent Detection and Slot Filling

Build a joint intent-classification and slot-filling model for Russian utterances when the labelled training data are only in English.

  • NLP
  • Cross-lingual intent classification and sequence labelling
Read the official statement

The task

Contestants must build a model that, for each user utterance, predicts the intent class and the BIO slot tag of every token. The labelled training set is English; the validation and test sets are Russian. Russian–English sentence pairs and unlabelled Russian texts about Russian cities are provided as additional material.

The baseline trains a fixed joint BERT-style model (Ilseyar-kfu/base_model with two classification heads) whose architecture and training set-up are marked "do not change".

Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.

At a glance

You get
train.conll (English, labelled), validation.conll (Russian, labelled), test.conll (Russian, unlabelled), ru_en_pairs.jsonl, unlabeled_texts.txt. CoNLL records carry id, text, intent, character-level slot spans and per-token intent and BIO slot tags.
You submit
CSV with columns id (1-based), intent (predicted intent label) and slots (space-separated BIO slot tags, one per token), written by the fixed prepare_submission function.
Scoring
Mean of the weighted F1 for intents and the weighted F1 for slot tags (padding excluded): avg_weighted_f1 = (intent_weighted_f1 + slot_weighted_f1) / 2.
Rules
  • Do not translate the dataset.
  • No data other than those provided (augmentation only from provided data).
  • Do not change the base model or architecture, or cells marked as not to be modified.
  • Validation and test data may be used only for evaluation, not for training.
  • No other transformer-based pretrained models or LLMs.
  • General NEOAI 2025 rules apply: by default no pretrained models or additional data unless the statement allows them; no external human help; on-site participants get one NVIDIA V100, an LLM chat (GPT-4o or similar) and a whitelist of websites.
Format
NEOAI 2025 main round, Task 8, Day 3 (9 May 2025, 07:00–13:00 UTC). Individual; 20 submissions per day.

Notebooks

Competition

Solution

Source files

Details

Year
2025, SberUniversity, Russia and online
Round
Day 3 · Task 8
Language
English
License
Per-competition Kaggle data licence: MIT (Tasks 7 and 8), as stated by the source