Extrinsic vs. In-Context: Two Types of LLM Hallucination
LLM hallucinations split into two types: in-context, where output contradicts provided sources, and extrinsic, where it conflicts with world knowledge. This distinction is critical for engineers debugging AI systems, as RAG pipelines fight in-context errors while open-ended generation faces extrinsic ones. Mitigating extrinsic hallucinations requires models to not only be factual but also to admit when they don't know an answer, a major challenge given the impracticality of verifying against tra
LLM hallucinations are broadly defined as fabricated content, but a key distinction separates them into two types: in-context and extrinsic. In-context hallucination occurs when a model's output contradicts the specific source material provided in the prompt, a common failure mode in Retrieval-Augmented Generation (RAG) systems. Extrinsic hallucination is when the output conflicts with verifiable world knowledge learned during pre-training. This matters for engineers because mitigation strategies differ; fighting in-context errors involves improving retrieval, while tackling extrinsic ones is harder because verifying against a massive training corpus is computationally infeasible. The core challenge is making models both factual and, just as importantly, training them to acknowledge when they don't have an answer.
Read the original → Lilian Weng's Log
- #llm
- #ai
- #hallucination
- #rag
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.