# Human vs AI

*English translation by SOTA – AI Community of the Romanian original, published with the permission of Asociația Nitro. Organisers who would like this translation removed can email team@sota-ai.org.*

## Statement

In a world in which almost everyone uses artificial intelligence models such as ChatGPT to generate content, it is becoming increasingly difficult to distinguish between a text written by a human and one generated automatically. At the same time, identifying the main topic of a text remains essential for organising and interpreting information.

The aim of this exercise is twofold:

* **Subtask 1:** Detecting the source of a text – is it written by a human or generated by AI?
* **Subtask 2:** Classifying the text into one of four major topics: science, business, religion or crime.


## **Dataset**

The input file contains a table with the following columns:

* **ID** – a unique identifier for each sentence.
* **text** – the sentence itself.
* **label** – the source of the text (`0` if it is written by a human, `1` if it is generated by AI).


## **Subtask 1 – Detecting the source of the text (AI vs. Human)**

You will train a binary classification model, using the data in the file **train_data.csv**, that takes a text as input and returns:

* `0` if the text is written by a **human**.
* `1` if the text is generated by **AI**.

Using this model, you will predict the source of all the texts in **test_data.csv** that have the value **1** in the **subtaskID** column.



## **Subtask 2 – Topic classification of the texts**

For each text, the main topic is to be determined by classifying it into one of four categories:
*  "SCIENCE"
*  "BUSINESS"
*  "CRIME"
* "RELIGION"


**The train_data.csv dataset does not directly contain these topic labels.**

The methods you design will be applied only to the texts in **test_data.csv** that have the value **2** in the **subtaskID** column.

## **Output file format**

The model will generate a `.csv` file named `output.csv`, with the following 3 columns:

* `subtaskID` – `1` for the source of the text (AI vs. Human), `2` for the topic classification.
* `datapointID` – the value from the `ID` column of the text in the input file.
* `answer` – the label predicted by the model (in numeric format: `0/1` for subtask 1, or a string for subtask 2).

## **Example output.csv:**

```
subtaskID,datapointID,answer
1,101,0
1,102,1
2,101,SCIENCE
2,102,CRIME
...
```

For the complete format, see the file `sample_output.csv`.

##  **Training file – `train_data.csv`**

### **Column format:**

| Column | Type      | Description                                                                |
| ------- | --------- | -------------------------------------------------------------------------- |
| `ID`    | `int`     | Unique identifier for each text instance.                                  |
| `text`  | `str`     | The sentence or paragraph to be analysed.                                  |
| `label` | `int` | The label associated with the text (`0/1`). |

### **Examples:**



```csv
ID,text,label
1,Acest text este generat de AI.,1
2,Aceasta este o propoziție scrisă de un om.,0
```

(The two example texts are in Romanian: "This text is generated by AI." and "This is a sentence written by a human.")


## **Test file – `test_data.csv`**

### **Column format:**

| Column        |     Type    |     Description                                                                 |
| ------------- | ---------- | -------------------------------------------------------------------------------- |
| `ID`          |   `int`    |   Unique identifier for each text instance.                                     |
| `subtaskID`   |   `int`    |   Value `1` or `2`, indicating which subtask the prediction is for.             |
| `text`        |   `str`    |   The sentence or paragraph to be classified.                                   |



## **Summary:**

| File        | Purpose                  | Columns                   |
| ----------- | ------------------------ | ------------------------- |
| `train.csv` | Training the models      | `ID`, `text`, `label`     |
| `test.csv`  | Prediction + classification | `ID`, `subtaskID`, `text` |


## **Scoring**

### **Subtask 1 – AI vs Human classification (F1-score)**

| F1-score (%)   | Points awarded   |
| -------------- | ---------------- |
| **≥ 95**       | 60 points        |
| **70 – 94.99** | between 10 and 50 |
| **< 70**       | 0 points         |

The score is computed from the macro-averaged F1 and converted into points by linear interpolation within the corresponding interval.


### **Subtask 2 – Topic classification (Accuracy)**

| Accuracy (%)   | Points awarded   |
| -------------- | ---------------- |
| **< 25**       | 5 points         |
| **25 – 49.99** | between 5 and 10  |
| **50 – 79.99** | between 10 and 20 |
| **80 – 96.99** | between 20 and 35 |
| **≥ 97**       | 40 points        |

The accuracy (`accuracy_score`) is converted into a score by linear interpolation within the interval into which it falls.