Checklist IOAI PH 2026 Team Selection · Task 3
Katotohanan: Filipino Fake News Classifier
Classify 22,458 Philippine news articles for veracity, manipulation type, target group, harm level and fact-check need with free-tier LLMs, including calibrated confidence.
The task
Contributed by Dr Charibeth K. Cheng (De La Salle University). The corpus is the Philippine Fake News Corpus (Fernandez, 2018): 22,458 articles from 1 January 2016 to 31 October 2018 in Filipino, Taglish and regional languages, 14,802 from mainstream broadsheets and 7,656 from sources classed as unreliable or satire; ground-truth labels derive from the source website.
For each article the pipeline outputs veracity (Fake/Real), manipulation_type (Fabricated, Satire-as-fact, Opinion-as-fact, Misleading-headline, Politically-motivated fabrication, Authentic), target_group, confidence_score (0–1), harm_level and needs_fact_check, using few-shot prompting, strict JSON output and rate-limit back-off. Regional-language performance (Cebuano, Ilocano, Waray) is weighted heavily.
Abridged by SOTA from the official materials. The official statement has the exact rules, and it wins wherever this summary differs.
At a glance
- You get
test.csv(index and raw article content),sample_submission.csv.- You submit
- A CSV of exactly 134,748 rows (six per article): RowId '[index]_[
label_name]' and Prediction. - Scoring
- Task worth 25% of the total grade: Kaggle performance 10% (10 × candidate macro F1 / top macro F1 on veracity and manipulation type), calibration 5% (ECE bands: <0.1 → 5, 0.1–0.2 → 4, 0.2–0.4 → 3, >0.4 → 2, uniform confidence → 1), ethical and linguistic logic 5% and engineering 5% (manual review). Deductions: −5 for source-name lookup tables, −2 if more than 90% of confidence scores are exactly 1.0, −1 for ignoring regional languages.
- Rules
- Individual; one Kaggle account; 2 submissions per day; one final submission.
- Free-tier LLM APIs only (e.g. Gemini Flash, Groq, OpenRouter free models).
- All documentation in Markdown cells of the Kaggle notebook, shared with organisers by the deadline.
- Format
- IOAI Philippines 2026 Team Selection, Task 3: 6 April 2026 00:00 – 13 April 2026 23:59 PHT.