Context Stuffing: Giving LLMs Short-Term Memory

WHY IT EXISTS: LLMs only know what's in their training data, which can be outdated or lack specific private information. Context stuffing is the most direct way to provide this missing information for a single query without permanently retraining or fine-tuning the model.

THE MENTAL MODEL: Think of an LLM's context window as its short-term memory. Context stuffing is like handing a brilliant but amnesiac expert a specific briefing document right before they answer a question. They can only use the information you just gave them, plus their general knowledge. They forget the briefing as soon as they move to the next question.

HOW IT WORKS: A user's query is combined with relevant external text (e.g., a PDF, a web page, a company document) into a single, large prompt that is sent to the LLM. The LLM then generates a response based on both the user's original query and the supplementary text provided in the context. The "stuffing" is the act of prepending or appending this external data to the query.

WHEN TO USE IT: Use this for simple, one-off tasks where you need an LLM to reason about information not in its training set. Examples include summarizing a recent article or answering questions about an internal company policy document. It's the foundational technique within more complex Retrieval-Augmented Generation (RAG) systems, which automate the selection of which documents to stuff.

WHEN NOT TO USE IT: Avoid this method when dealing with very large documents or entire knowledge bases that won't fit into the model's context window. It is also inefficient for repeated queries about the same domain, where fine-tuning the model might be a better long-term solution. The quality of the output depends entirely on the quality of the stuffed text.

ONE CANONICAL EXAMPLE: A support engineer pastes a customer's technical support ticket and the full text of a relevant internal troubleshooting guide into an LLM prompt. They then ask the LLM: "Based on the guide, what are the next three steps to resolve the customer's issue?" The LLM uses the "stuffed" guide to generate a specific, actionable response.