Kent C. Dodds Adds SQLite FTS5 to Vector Search
Kent C. Dodds added SQLite FTS5 to Vectorize embeddings after semantic search missed exact matches like "React Testing Library." The hybrid pipeline uses Reciprocal Rank Fusion. If your search fails on API names, add BM25 backup, not bigger embedding models.
Why it matters
Pure vector search is now table stakes, but it fails on exact identifiers. Kent C. Dodds discovered this when his own semantic search could not surface the canonical "React Testing Library" introduction post despite an exact title match. Embedding models optimize for conceptual meaning, so they routinely miss library names, API identifiers like useState or loader, error messages, and version numbers. If your product includes technical documentation, code snippets, or structured content, a search pipeline that relies only on embeddings will frustrate users who know exactly what they are looking for.
What changed
Dodds kept his existing Cloudflare Vectorize semantic pipeline and added lexical retrieval via SQLite FTS5, which provides BM25-ranked full-text search with minimal infrastructure because his site already uses SQLite replicated through LiteFS. The architecture works as follows: indexers emit lexical artifacts as JSON files to R2 alongside existing vector manifests; the application syncs those artifacts into a local FTS5 index; at query time, lexical and semantic retrieval run in parallel; and results merge through Reciprocal Rank Fusion. The entire first implementation took roughly twenty minutes using Cursor with GPT-5.4 through an iterative workflow that included investigating the existing system, diagnosing failure modes, exploring alternatives, generating an implementation plan, and verifying data landed correctly before deployment.
What to watch
Dodds notes that production revealed issues the development build did not, suggesting that hybrid search tuning is an ongoing process rather than a one-time setup. Engineers adopting similar pipelines should monitor RRF weighting between lexical and semantic signals, measure recall on exact identifier queries separately from natural-language questions, and consider whether their own SQLite or Postgres full-text extensions can avoid adding a new managed service. The broader pattern is that hybrid retrieval is becoming the default for site search, not an advanced optimization.
Interview question
According to Dodds, what is the best response when vector search misses exact identifiers like 'React Testing Library'?
- a.Fine-tune the embedding model exclusively on technical documentation
- b.Switch to a more powerful embedding model with a larger context window
- c.Add a BM25 lexical layer and merge results with Reciprocal Rank FusionCorrect
- d.Abandon vector search and rely solely on a SQLite FTS5 index
Why? this is the answer
Dodds kept his existing vector pipeline and augmented it with SQLite FTS5 BM25 search, merging both result sets via Reciprocal Rank Fusion, because embedding models inherently optimize for conceptual meaning rather than exact string matches. Simply scaling up the embedding model or switching entirely to lexical search would not solve the hybrid retrieval problem.
Just read this? Test yourself on what you have been reading.
Read the original → kentcdodds.com
- #search
- #sqlite
- #vector-search
- #full-text-search
- #nextjs
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles