NLP
50 bites tagged NLP — interview questions with model answers, and 60-second explainers.
BERTScore: Judging AI Text on Meaning, Not Just Words
BERTScore evaluates AI-generated text by comparing its meaning to a reference, not just matching words. It's used to score machine translation or summarization where phrasing can vary.
HellaSwag: A Benchmark Designed to Fool LLMs
HellaSwag is a commonsense benchmark designed to fool language models. It asks an AI to pick the most logical sentence ending, but the wrong answers are specifically generated to trick machines, not humans. It's used to test for true contextual understanding.
Speech-to-Text (ASR): Turning Spoken Words into Data
Speech-to-Text (ASR) is a digital stenographer, turning spoken language into machine-readable text. It's the engine behind voice assistants, automated call routing, and video captioning.
Embedding Models: The 'Retrieval' in RAG
An embedding model acts like a librarian for your data, converting text into numerical vectors so similar concepts are grouped together. In RAG, it finds relevant documents to feed an LLM, but using the wrong model type will yield poor retrieval results.
What is a Large Language Model (LLM)?
An LLM is a neural network trained on vast text data to process and generate human-like language. It's the core technology behind chatbots, enabling them to generate, summarize, and translate text. Its reliability depends entirely on its training data.
Causal Language Modeling: The Autocomplete Engine
Causal Language Modeling is like a powerful autocomplete, predicting the next word based only on what came before. It's the engine for text generation in chatbots, creative writing tools, and coding assistants. The footgun: it can't see future words.
Transformer Preprocessing: From Text to Tensors
Transformers don't read text; they read numbers. A tokenizer is the translator, converting sentences into numerical tensors the model understands. This is the mandatory first step for any NLP task. The footgun is using a tokenizer that doesn't match the model.
Seq2Seq: Turning One Sequence Into Another
A Seq2Seq model acts like a universal translator, reading one sequence to generate another. It's foundational for machine translation and text summarization. The main footgun is its fixed-size context vector, which can forget details from long inputs.
Word2Vec: Word Meaning as a Point in Space
Word2Vec turns words into numerical vectors, where semantic similarity becomes spatial proximity. It powers synonym detection and analogy tasks by learning from a word's context in a large text corpus.
Vector Embeddings: Turning Meaning into Math
Vector embeddings turn complex data like words or images into lists of numbers (vectors). This lets computers measure "similarity" by calculating the distance between these vectors, powering search and recommendations.
Large Language Models (LLMs)
An LLM is a massive neural network trained on vast text datasets to perform language tasks. It powers modern chatbots by generating, summarizing, and translating text. The key footgun: biased or inaccurate training data makes its output unreliable.
Named Entity Recognition: Finding the 'Who, What, Where' in Text
Named Entity Recognition (NER) is a smart highlighter for text, automatically finding and tagging nouns like people, places, and organizations. It powers search and extracts structured data from news or support tickets.
Topic Modeling: Finding Themes in Unstructured Text
Topic modeling automatically finds themes in text by grouping words that often appear together. It's used to analyze customer feedback or organize large document sets.
Word Embeddings: Turning Words into Math
Word embeddings turn words into vectors, where distance equals a difference in meaning. They power features like search relevance and text classification by letting algorithms 'understand' context.
Sentiment Analysis: Reading the Room at Scale
Sentiment analysis is a digital mood ring for text, automatically classifying content as positive, negative, or neutral. It's used to sift through customer reviews and social media to gauge opinion, but often fails on sarcasm or complex context.
Self-Attention: How Models Weigh Word Importance
Self-attention lets a model weigh the importance of all words in a sequence simultaneously, asking "which other words are most relevant?" It's the core of Transformers, enabling parallel processing for tasks like translation, unlike sequential RNNs.
Visual Question Answering (VQA): Teaching AI to See and Reason
VQA models combine vision and language to answer questions about an image, like a visual Turing test. It's used in assistive tech and advanced image search. The footgun is models learning to answer based on language patterns, not visual content.
Image Captioning: Teaching Machines to Describe What They See
Image captioning teaches a computer to generate a human-readable sentence describing an image, translating pixels into words. This powers accessibility features and better image search.
Multi-Head Attention: Seeing Data From Multiple Angles
Multi-head attention lets a model analyze a sequence from multiple perspectives at once. It runs several "attention heads" in parallel, each focusing on different relationships, like syntax vs. semantics.
Cloud NLP Services: Pre-trained Language Models as an API
Cloud NLP services are like having a team of linguists on-demand via an API. Use them to instantly analyze text for sentiment, entities (people, places), or topics without building your own models.
Encoder-Only vs. Decoder-Only vs. Encoder-Decoder Transformers?
This tests your ability to connect transformer architecture to specific NLP tasks. A great answer explains how each model's attention mechanism dictates its use: encoder-only (bidirectional attention) for understanding content, decoder-only (causal attention) for text generation, and encoder-decoder for sequence-to-sequence tasks like translation. The key red flag is failing to explain the *why* behind the task suitability—the attention mechanism.
What is the trade-off between top-k and top-p sampling?
This tests your practical knowledge of tuning LLM output for the creativity vs. coherence trade-off. A strong answer defines top-k (static token count) and top-p (dynamic probability mass), then explains that top-p's adaptive window is generally more robust than top-k's fixed window. A red flag is failing to contrast the static nature of top-k with the dynamic nature of top-p, which is the core of the trade-off.
Attention: Weighing Input by Relative Importance
The attention mechanism lets a model decide which parts of a sequence are most important relative to others. In natural language processing, it assigns 'soft' weights to words, allowing the model to focus on what's most relevant for a given task. It's used to encode sequences of token embeddings, from short phrases to massive documents. The main pitfall is forgetting that these weights are contextual and relative, not absolute measures of a word's importance.
Word Embeddings: Turning Words into Vectors
Word embeddings turn words into numerical vectors, like coordinates on a map of meaning. Words with similar meanings, like "king" and "queen," are placed close together in this vector space. This is fundamental for text analysis in machine learning, allowing models to grasp semantic relationships instead of just matching text. The footgun is assuming the vector's individual numbers are human-interpretable; they are abstract features learned from data.
Get NLP bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.