Chunking and embedding a RAG corpus

WHAT THIS TESTS This evaluates whether you understand that ingestion choices, especially chunk size and embedding model, directly determine retrieval quality, which bounds the whole RAG system.

A GOOD ANSWER COVERS First clean and normalize the documents, stripping boilerplate and extracting text from PDFs or HTML. Chunking is the pivotal decision: chunk size trades context against precision. Large chunks keep more surrounding context but dilute the embedding's focus, retrieve less precisely, and consume more of the prompt budget; small chunks are precise but can lose the context needed to answer. A common approach is moderate chunks with some overlap so a fact near a boundary is not severed, and respecting semantic boundaries such as paragraphs, sections, or sentences rather than cutting blindly at a fixed token count mid-sentence. For embedding, choose a model matched to your domain and language, since a general model may underperform on specialized text. The same embedding model must encode both documents and queries so they share a space. Consider the embedding dimension, which affects storage and search cost, and the model's max input length relative to chunk size. Attach metadata such as source, title, and timestamp to each vector to enable filtering and citation. Finally upsert vectors into the index and validate retrieval quality on sample queries.

COMMON WRONG ANSWERS Splitting at a fixed token count regardless of sentence boundaries; using no overlap; embedding documents and queries with different models; ignoring domain fit of the embedding model; forgetting metadata for filtering and citations; treating chunk size as unimportant.

LIKELY FOLLOW-UPS How do you pick chunk size empirically? Why does overlap help? Why must query and document embeddings match? How does metadata enable hybrid or filtered search?

ONE CONCRETE EXAMPLE For a corpus of technical manuals you split on section and paragraph boundaries into moderate chunks with a small overlap, embed each with a domain-appropriate model, attach the manual name and section as metadata, and store the vectors. Queries are embedded with the same model, and metadata lets you filter to the relevant product before semantic search, sharpening retrieval.