Hallucination Detection

Hallucination Detection

Image generated using the DALL-E model.

Introduction

Language models help us with everyday tasks such as proofreading texts, writing code or answering questions. They are also increasingly used in fields such as medicine and education.

But how can we know whether the answers they generate are correct? Language models do not always have complete knowledge of a given topic, and yet they can formulate answers that sound credible but are in fact misleading. We call such incorrect answers hallucinations.

Task

In this task you will tackle the detection of hallucinations in answers to factual questions generated by large language models (LLMs). You will analyse a dataset that will help to assess whether the answers generated by a language model are actually correct or contain hallucinations.

Each example in the dataset contains:

• A question, e.g. "What is the main responsibility of the US Department of Defense?"
• The language model's answer, e.g. "The main responsibility is the defence of the country."
• Tokens associated with generating the answer.
• Four alternative answers generated by the same model at a higher temperature.
• Tokens of the alternative answers generated by the same model at a higher temperature.
• Probabilities of the alternative answers generated by the same model at a higher temperature.
• A label (is_correct) indicating whether the main answer is correct according to a trusted source.

Example:

[
    {
        "question_id": 34,
        "question": "What is the name of the low-cost carrier that operates as a wholly owned subsidiary of Singapore Airlines?",
        "answer": "Scoot is the low-cost carrier that operates as a wholly owned subsidiary of Singapore Airlines.",
        "tokens": [" Sco", "ot", " is", ..., " Airlines", ".", "\n"],
        "supporting_answers": [
            "As a wholly owned subsidiary of Singapore Airlines, <answer> Scoot </answer> stands as a low-cost carrier that revolutionized air travel in the region.",
            "Scoot, a subsidiary of <answer> Singapore Airlines </answer> , is the low-cost carrier that operates under the same brand.",
            "<answer> Scoot </answer> is the low-cost carrier that operates as a wholly owned subsidiary of Singapore Airlines.",
            "Singapore Airlines operates a low-cost subsidiary named <answer> Scoot </answer> , offering affordable and efficient air travel options to passengers."
        ],
        "supporting_tokens": [
            [" As", " a", ..., ".", "<answer>"],
            [" Sco", "ot", ..., " brand", ".", "\n"],
            ["<answer>", " Sco", ..., ".", "\n"],
            [" Singapore", " Airlines", ..., ".", "\n"]
        ],
        "supporting_probabilities": [
            [0.0029233775567263365, 0.8621460795402527, ..., 0.018515007570385933],
            [0.42073577642440796, 0.9999748468399048, ..., 0.9166142344474792],
            [0.3258324861526489, 0.9969879984855652, ..., 0.921079695224762],
            [0.11142394691705704, 0.960810661315918, ..., 0.9557166695594788]
        ],
        "is_correct": true
    },
    .
    .
    .
]

Data

The data available to you in this task are:

• train.json - a dataset containing 2967 questions and answers.
• valid.json - 990 additional questions.

Scoring Criterion

ROC AUC (Receiver Operating Characteristic Area Under Curve) is a measure of the quality of a binary classifier. It shows the model's ability to distinguish between two classes - here, a hallucination (false) and a correct answer (true).

• ROC (Receiver Operating Characteristic): A plot showing the relationship between the True Positive Rate (sensitivity) and the False Positive Rate (1 - specificity) at various decision thresholds.
• AUC (Area Under Curve): The area under the ROC curve, which takes values from 0 to 1:
  • 1.0: A perfect model.
  • 0.5: A random model (no ability to distinguish between the classes).

The higher the AUC value, the better the model performs at classification.

You can score between 0 and 100 points for this task. The score will be scaled linearly depending on the ROC AUC value:

• ROC AUC ≤ 0.7: 0 points.
• ROC AUC ≥ 0.82: 100 points.
• Values between 0.7 and 0.82: scaled linearly.

Scoring formula:

$$\text{Points} = \begin{cases} 0 & \text{for } \text{ROC AUC} \leq 0.7 \\ 100 \times \frac{\text{ROC AUC} - 0.7}{0.82 - 0.7} & \text{for } 0.7 < \text{ROC AUC} < 0.82 \\ 100 & \text{for } \text{ROC AUC} \geq 0.82 \end{cases}$$

Constraints

• Your solution will be tested on the Contest Platform without internet access and in an environment without a GPU.
• The evaluation of your final solution on the Contest Platform must not take longer than 5 minutes without a GPU.
• List of permitted libraries: xgboost, scikit-learn, numpy, pandas, matplotlib.

Submission Files

This notebook, completed with your solution (see the predict_hallucinations function).

Evaluation

Remember that during checking the FINAL_EVALUATION_MODE flag will be set to True.

You can score between 0 and 100 points for this task. The number of points you receive will be calculated on the (secret) test set on the Contest Platform using the formula given above, rounded to an integer. If your solution does not meet the above criteria or does not run correctly, you will receive 0 points for the task.

Starter Code

In this section we initialise the environment by importing the required libraries and functions. The prepared code will make it easier for you to work with the data efficiently and to build a proper solution.

Loading the Data

The code below will load the data and prepare it appropriately.

Scoring Criterion Code

Code similar to the code below will be used to evaluate the solution on the test set.

Your Solution

Place your solution in this section. Make changes only here!

Evaluation

Running the cell below will let you check how many points your solution would score on the validation data. Before submitting, make sure that the whole notebook runs from start to finish without errors and without any user intervention after choosing the "Run All" option.

During checking, the model will be saved as your_model.pkl and evaluated on the test set.