Extrinsic vs. In-Context: Two Types of LLM Hallucination
LLM hallucinations split into two types: in-context, where output contradicts provided sources, and extrinsic, where it conflicts with world knowledge. This distinction is critical for engineers debugging AI systems, as RAG pipelines fight in-context errors while open-ended generation faces extrinsic ones. Mitigating extrinsic hallucinations requires models to not only be factual but also to admit when they don't know an answer, a major challenge given the impracticality of verifying against tra
### Why it matters For engineers building with LLMs, distinguishing between in-context and extrinsic hallucinations is crucial for debugging. This framework helps diagnose *why* a model is wrong. If an output contradicts a document in a RAG system, the problem is likely in-context hallucination, pointing to issues in retrieval or the model's ability to follow instructions. If a chatbot generates incorrect facts on its own, it's an extrinsic hallucination, a harder problem rooted in the model's core knowledge.
This distinction helps teams focus their efforts: improve the RAG pipeline for in-context errors, or re-evaluate the base model and its fine-tuning for extrinsic ones. The core challenge with extrinsic hallucination is that it's computationally impossible to fact-check every generation against the petabytes of data the model was trained on.
### What changed Lilian Weng's analysis provides a clear vocabulary for two distinct failure modes:
* **Extrinsic Hallucination:** The model generates output that is unfaithful to its pre-training data (i.e., world knowledge). It fabricates facts or misremembers them. * **In-context Hallucination:** The model generates output that contradicts the source content provided within the prompt. This is a primary failure mode for RAG applications. * **The Mitigation Goal:** To solve extrinsic hallucination, a model must be both factual and capable of acknowledging when it does not know the answer.
### What to watch * **Uncertainty Training:** Watch for new fine-tuning and RLHF techniques that explicitly reward a model for expressing uncertainty or stating "I don't know" instead of guessing. This is a key research frontier for improving model trustworthiness. * **Evaluation Benchmarks:** Look for new benchmarks that measure not just factual accuracy but also a model's ability to identify gaps in its own knowledge. This is critical for building systems that can be trusted in high-stakes domains.
Read the original → lilianweng.github.io
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.