NLP
50 bites tagged NLP — interview questions with model answers, and 60-second explainers.
Byte Pair Encoding: Compressing Text for LLMs
Think of Byte Pair Encoding (BPE) as creating custom abbreviations for common letter pairs to compress text. It repeatedly finds the most frequent pair, like 'th', and merges it into a new token. LLMs use this to build vocabularies of common sub-word units, helping them understand rare words. The main footgun is that the final vocabulary size is fixed; choosing the wrong size can hurt model performance and efficiency.
Large Language Models (LLMs)
A large language model is a sophisticated pattern-matching engine trained on a massive library of text. They power modern chatbots and can generate, summarize, or translate text by predicting the most probable next word based on the patterns they've learned. The key footgun is that their output reflects the biases and inaccuracies of their training data, making them confident but potentially unreliable.
Get NLP bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.