Design dynamic few-shot example retrieval from a vector database
Tests RAG-style prompt engineering with semantic retrieval and latency. Use shared embeddings, approximate nearest neighbors with metadata filters, diversity reranking, and token-bounded prompt templates.
What's really being asked
Whether you can architect a production retrieval pipeline that balances relevance, latency, and token economy. Interviewers want to see you understand that few-shot examples are not static assets but dynamic context windows that compete for the same token budget as the user query and model response. They are probing for experience with embedding alignment, approximate search, and prompt assembly at scale.
The full answer
First, embedding strategy: use a shared bi-encoder or sentence transformer to embed both the user query and the example inputs into the same dense vector space so cosine similarity is meaningful. Second, indexing: store examples in an approximate nearest neighbor index like FAISS, HNSW, or a managed vector database with metadata filters for domain, task, or user tier to narrow the search space before vector comparison. Third, retrieval logic: fetch a larger candidate set than needed, then apply a secondary reranking step for diversity or task-specific relevance to avoid redundant examples. Fourth, prompt assembly: format retrieved input-output pairs with clear delimiters, reserve a token budget for examples using the tokenizer, and truncate or drop the lowest-similarity example if the prompt exceeds the limit. Fifth, latency management: run embedding and retrieval asynchronously or cache frequent query clusters to keep inference time predictable.
The mistakes people make
Proposing exact k-nearest neighbor search on millions of vectors, which is O of N and too slow for online serving. Using different embedding models for examples and queries without projection or alignment. Treating retrieval and LLM calls as synchronous blocking steps without latency budgeting. Ignoring token limits by stuffing retrieved examples without counting tokens, causing prompt truncation or API errors. Selecting examples based only on embedding similarity without considering diversity, which can yield redundant demonstrations.
What usually comes next
How would you handle a cold-start scenario with no similar historical examples? How do you update the index when new examples arrive without full reindexing? What would you do if the retrieved examples conflict with each other? How would you measure whether dynamic selection actually improves accuracy over static examples?
A concrete example
In a customer support system, you store thousands of resolved ticket pairs as embeddings using all-MiniLM-L6-v2. When a user asks why their bill is higher, the system retrieves the top three most similar past billing questions from the vector store, reranks to ensure one covers refunds and one covers proration, then injects them into a FewShotPromptTemplate with XML tags. This improved response relevance by 25 percent and cut token usage by 30 percent compared to a static three-example prompt.
Interview question
Which design best avoids redundant demonstrations while keeping retrieval latency acceptable at scale?
- a.Fetch a larger candidate set using ANN, apply a diversity reranker, then assemble a token-bounded prompt asynchronously.Correct
- b.Cache static few-shot prompts for all frequent queries and disable vector search to eliminate retrieval variance.
- c.Use different embedding models for queries and examples, aligning them with a learned projection layer after retrieval.
- d.Retrieve the exact top-k nearest neighbors from a flat index and inject them synchronously before the LLM call.
Why? this is the answer
Fetching a larger ANN candidate set and reranking for diversity prevents redundant examples while keeping latency low, and asynchronous assembly protects inference time. Option D is tempting but exact flat search is O(N) and too slow for online serving, and it lacks any diversity mechanism.
Just read this? Test yourself on what you have been reading.
Read the original → app.readytensor.ai
- #llms
- #vector-databases
- #prompt-engineering
- #retrieval
- #few-shot-learning
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llms — each one lists the topics its interview covers.
See open roles