Hallucination Detection in LLMs

WHY IT EXISTS A language model optimizes for likely-sounding continuations, not truth, so it can assert false facts with the same fluency as correct ones. In high-stakes uses, medical, legal, factual question answering, an undetected fabrication is dangerous, and users cannot tell confident truth from confident fiction by tone alone. Detection exists to attach a trust signal to outputs.

THE MENTAL MODEL Treat a hallucination as a claim that is not grounded, either unsupported by any external source or inconsistent with the model's own reliable knowledge. If the model genuinely knows something, it should answer consistently when asked repeatedly and should align with trustworthy references; instability and lack of support are red flags for fabrication.

HOW IT WORKS Self-consistency methods sample multiple answers and measure agreement; high variance suggests the model is guessing. Uncertainty methods examine token probabilities or predictive entropy, where flat, low-confidence distributions over factual claims signal risk. Grounding and verification methods retrieve documents or use a second model or tool to check whether each claim is supported, an approach related to retrieval-augmented generation and natural-language inference between claim and evidence. Some methods probe internal activations for signatures of unsupported generation.

WHEN IT MATTERS It matters wherever factual accuracy is required and errors carry cost: enterprise assistants, search, summarization of documents, and any agent taking actions based on generated facts. It is also central to evaluating and comparing models on faithfulness.

ONE CONCRETE EXAMPLE A question-answering system asks the model the same question several times with sampling. For a well-known fact the answers agree, raising confidence. For an obscure entity the answers diverge wildly across runs, signaling likely hallucination. The system then retrieves supporting passages and runs an entailment check: if no retrieved passage supports the claim, the answer is flagged as unverified and either withheld, hedged, or routed to a human, rather than presented as fact.