Checklist AICC 2026 Round 7 · Task 3
Scientific Facts
Decide whether evidence from scientific abstracts supports, contradicts or is insufficient for a claim, using only SciBERT.
The task
Automated scientific fact-checking verifies claims against the published literature. The task uses SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts from the biomedical literature.
Given a claim and text from the cited abstracts, the model predicts whether the evidence SUPPORTs the claim, CONTRADICTs it, or gives NOT_ENOUGH_INFO. For NOT_ENOUGH_INFO training claims the full abstract is provided as evidence.
The training set provides claims, evidence and labels; the test set provides claims and evidence only.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
train.csv(887 rows: id, claim, evidence, label) andtest.csv(222 rows: id, claim, evidence).- You submit
submission.csvwith header datapointID,answer; answer is SUPPORT, CONTRADICT or NOT_ENOUGH_INFO; one row per test example.- Scoring
- Macro F1 over the three classes.
- Rules
- The only pretrained model allowed is allenai/
scibert_scivocab_uncased. - No external datasets; no internet access.
- Individual participation (maximum team size 1); at most 50 submissions per day.
- AICC contest rules (stated on each Kaggle rules page, not enforceable): no use of LLMs for writing code or getting task ideas; no internet use other than official library documentation and the contest platform; no communication with anyone during the contest; clarifications only via the #clarification-requests channel on the AICC Discord server.
- The only pretrained model allowed is allenai/
- Format
- AICC Round 7, online on Kaggle, 22 May 2026 20:00 UTC – 24 May 2026 20:00 UTC.