Top 30 NLP Interview Questions and Answers
30 multiple-choice questions on NLP, drawn from 30 bites out of the 50 tagged NLP on Tezvyn. Answer them here or read straight down. Every question carries the correct option, why it is correct, and a link to the bite it came from.
30 questions. Pick an answer, or open “Show the answer” to read it.
Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.
Question 1 of 30
What is the fundamental principle behind how Word2Vec represents word meaning?
Show the answer
Answer: c · Words with similar meanings are mapped to points that are spatially close in a multi-dimensional vector space.
The core idea of Word2Vec is that words with similar meanings are represented by vectors that are close to each other in a multi-dimensional space, capturing semantic similarity through spatial proximity. Option A describes a method Word2Vec aims to improve upon, as it fails to capture semantic relationships.
Read the full bite: Word2Vec: Word Meaning as a Point in Space
Question 2 of 30
What core limitation of a vanilla RNN encoder-decoder does the attention mechanism specifically address?
Show the answer
Answer: a · Compressing the entire input into one fixed-size vector loses detail on long sequences
Attention removes the single fixed-context-vector bottleneck by letting the decoder weight all encoder states per step. Parallelism and positional encodings are Transformer concerns, not what classic seq2seq attention was introduced to fix.
Read the full bite: Attention in Sequence-to-Sequence Models
Question 3 of 30
Beyond modeling long-range dependencies, what key training advantage does self-attention have over an LSTM?
Show the answer
Answer: c · It can process all sequence positions in parallel rather than step by step
Self-attention computes all token interactions simultaneously, enabling parallel training that sequential LSTMs cannot match. It actually needs added positional encodings and is quadratic, not cheaper, in sequence length.
Read the full bite: Self-Attention versus Recurrent Architectures
Question 4 of 30
What is the primary limitation of a basic Seq2Seq model when dealing with very long input sequences?
Show the answer
Answer: a · The fixed-size context vector acts as an information bottleneck, leading to the forgetting of early input details.
The card states that the "primary weakness of basic Seq2Seq is its reliance on a single, fixed-size context vector" which "becomes an information bottleneck" for "very long inputs," causing the model to "forget details from the beginning of the input." Option D describes the problem that Seq2Seq was designed to solve, not its limitation.
Read the full bite: Seq2Seq: Turning One Sequence Into Another
Question 5 of 30
Why does multi-head attention generally not cost much more than a single full-width attention head?
Show the answer
Answer: c · The model dimension is split across heads, so each head operates in a smaller subspace
The total dimension is partitioned among heads, keeping aggregate compute comparable to one full head while gaining diverse attention patterns. Heads run in parallel with their own projections, not shared weights, and each still uses softmax.
Question 6 of 30
When designing a pipeline to discover unknown pain-point categories from thousands of unstructured reviews, which sequence best ensures valid grouping and reliable severity ranking?
Show the answer
Answer: c · Deduplicate and normalize the corpus, cluster to discover themes, then apply sentiment analysis within each cluster to rank by severity.
The correct sequence matches the card's recommended lifecycle: preprocess to remove noise and duplicates, use unsupervised clustering to discover emergent themes since categories are unknown, and score sentiment within each cluster to rank pain points by frequency and severity. Option A is a tempting distractor because LLMs are popular, but the card flags jumping straight to summarization without cleaning as a red flag that yields unreliable, unvalidated output.
Read the full bite: Outline an NLP pipeline to categorize reviews and identify pain points
Question 7 of 30
Why is layer normalization preferred over batch normalization inside Transformer blocks for NLP?
Show the answer
Answer: c · Layer norm normalizes per token over features, independent of batch size and sequence length
Layer norm uses per-token feature statistics, so variable sequence lengths and small or padded batches do not destabilize it, and it matches at train and inference. Batch norm's batch-wise statistics are exactly what makes it unreliable here.
Read the full bite: Layer Norm and Residuals in Transformer Blocks
Question 8 of 30
What fundamental limitation necessitates the tokenization step for feeding text into a Transformer model?
Show the answer
Answer: b · Transformer models are mathematical functions that exclusively process numerical tensors.
The card explicitly states that "Neural networks, including Transformers, are mathematical functions that operate on numbers, not raw text strings." Thus, text must be converted into a numerical tensor format. While tokenization helps manage vocabulary and prepares for embeddings, the core reason is the model's numerical input requirement.
Read the full bite: Transformer Preprocessing: From Text to Tensors
Question 9 of 30
For which task would a Causal Language Model be the most suitable choice?
Show the answer
Answer: c · Extending a partial sentence into a complete, coherent paragraph
Causal Language Models are designed for open-ended text generation, predicting the next word based only on preceding words, making them ideal for continuing or extending text. Tasks like sentiment analysis, information extraction, or summarization typically require understanding the entire input, which is a limitation for CLMs.
Read the full bite: Causal Language Modeling: The Autocomplete Engine
Question 10 of 30
Which combination of NLP features and dashboard design best operationalizes a simple and empowering brand voice for executive stakeholders?
Show the answer
Answer: d · Readability scores and agency verb detection rendered as time-series alignment KPIs with team-level drill-downs
The card states that strong answers translate simplicity into readability metrics and empowerment into agency verb detection, then visualize them as business-friendly time-series KPIs with faceted breakdowns. Option A reflects the common error of using generic sentiment tools and technical model metrics, while Option B represents the non-scalable manual review red flag.
Read the full bite: Propose an NLP approach to audit brand voice and build a dashboard
Question 11 of 30
Despite advancements, what is a fundamental limitation of sentiment analysis?
Show the answer
Answer: d · Its struggle with sarcasm, implicit meaning, and context-dependent nuance, hindering deep individual understanding.
The card explicitly states sentiment analysis "often fails on sarcasm or complex context," "is not about deep comprehension," and "struggles with domain-specific jargon, comparative sentences, and text where the sentiment is not explicitly stated," concluding it's "not deep individual understanding." This directly supports option D. While machine learning models require labeled data, the card highlights sentiment analysis's purpose as automating large-scale analysis, implying it reduces, rather than constantly requires, extensive human labeling for every new specific instance.
Read the full bite: Sentiment Analysis: Reading the Room at Scale
Question 12 of 30
According to the card, what is the most accurate mental model for understanding how a Large Language Model (LLM) processes information?
Show the answer
Answer: b · A sophisticated pattern-matching engine that predicts word sequences based on learned statistical relationships.
The card describes an LLM as a "sophisticated pattern-matching engine for words and ideas" that learns "statistical relationships between words" to predict text. Option C is incorrect because the card explicitly states an LLM is "not a database of facts".
Question 13 of 30
Which architecture best serves related articles for both archived content and stories published minutes ago?
Show the answer
Answer: a · Pre-compute related lists for popular archived articles and run on-the-fly embedding queries with tag fallbacks for breaking news
This separates offline batch jobs for scale and latency from online serving for freshness, while tag and embedding fallbacks solve the cold-start problem for new articles. Option D is tempting because pre-computation is a valid best practice, yet serving solely from cache cannot handle breaking news published after the last batch run.
Read the full bite: How would you technically approach building a related articles feature?
Question 14 of 30
When using TF-IDF over the top-ranking pages for a query, what does a consistently high-scoring term most likely indicate?
Show the answer
Answer: a · A distinctive topical term competitors cover that your draft may be missing
High TF-IDF means a term is frequent within documents yet rare across the broader corpus, marking distinctive topical vocabulary worth covering. Ubiquitous words like 'the' score low, and TF-IDF is about coverage, not stuffing to a density target.
Read the full bite: Explain TF-IDF and its use in SEO analysis
Question 15 of 30
In an SEO NLP pipeline, what is the primary purpose of using sentence embeddings like SBERT on crawled articles?
Show the answer
Answer: d · To compare query intent against article vectors and identify subtopic gaps
SBERT embeddings encode semantic meaning to compare query intent against article vectors and reveal subtopic gaps, whereas word-frequency clustering only captures term co-occurrence without contextual nuance.
Read the full bite: How would you use NLP to analyze top articles for semantic themes?
Question 16 of 30
When is it usually best to apply neither stemming nor lemmatization to your text?
Show the answer
Answer: c · When feeding a subword-tokenized transformer like BERT
Transformers use subword tokenization and pretraining that already handle morphology and casing, so normalizing can mismatch their expected input. Bag-of-words models, by contrast, benefit from vocabulary reduction via stemming or lemmatization.
Read the full bite: Stemming versus lemmatization in text preprocessing
Question 17 of 30
What is the primary advantage of using an embedding model for document retrieval in a RAG system?
Show the answer
Answer: b · It efficiently identifies documents that are semantically similar to the user's query.
The card explains that embedding models convert text into vectors where similar meanings are close, allowing the system to find documents that are semantically similar to a query. Option D describes keyword matching, which the card states embedding models are not ideal for. Options C and D describe functions of the LLM or data storage, not the embedding model's primary role.
Read the full bite: Embedding Models: The 'Retrieval' in RAG
Question 18 of 30
A word appears frequently in one article but also appears in almost every other document in the corpus. What happens to its TF-IDF weight?
Show the answer
Answer: b · It is pulled down because inverse document frequency is near zero
A term appearing in nearly all documents has an IDF near zero, which drags the product down regardless of high term frequency. TF-IDF deliberately penalizes ubiquitous words rather than letting term frequency win out.
Question 19 of 30
What is the fundamental purpose of Speech-to-Text (ASR) technology?
Show the answer
Answer: c · To bridge the gap between human speech and machine-readable text.
The card explicitly states that ASR "was created to bridge this gap" between natural human speech and the text/data computers operate on. While ASR provides the input for understanding, it does not inherently interpret the meaning of conversations (Option D), nor is it error-free (Option A). Option B describes Text-to-Speech, a different technology.
Read the full bite: Speech-to-Text (ASR): Turning Spoken Words into Data
Question 20 of 30
Why does ROUGE poorly evaluate open-ended dialogue responses?
Show the answer
Answer: a · Many valid responses exist, so overlap with one reference unfairly penalizes correct paraphrases
Dialogue is one-to-many, so lexical overlap with a single gold reply penalizes valid alternatives. ROUGE is precisely an n-gram overlap metric, not a fluency or judge-based one, so the other options misdescribe it.
Read the full bite: Reference-free evaluation for open-ended dialogue
Question 21 of 30
An NLP pipeline clusters and tags research transcripts into themes. Which risk does the card call out as the deepest trap when interpreting the results?
Show the answer
Answer: a · Mistaking the most frequently mentioned theme for the most important one, even in a small sample where counts are not real statistics.
The card explicitly calls conflating theme frequency with importance the deepest trap, since mention counts in a small qualitative sample are not real statistics. The other options are real challenges it lists but are not the one singled out as deepest.
Read the full bite: NLP pipeline to theme and tag research transcripts
Question 22 of 30
What is the main reason word embeddings capture semantic relationships that one-hot encoding cannot?
Show the answer
Answer: d · Embeddings learn dense vectors where contextually similar words are close together
Embeddings are trained on co-occurrence patterns so words with shared contexts occupy nearby points in dense vector space, whereas one-hot vectors are orthogonal and equidistant. Distractor A is tempting but wrong because embeddings are not simply hashed or compressed one-hot vectors; their values are learned to create meaningful geometric relationships.
Read the full bite: What is a word embedding and how does it beat one-hot encoding?
Question 23 of 30
What is the key characteristic of the incorrect answer options in the HellaSwag benchmark that makes it challenging for language models?
Show the answer
Answer: a · They are engineered to appear statistically probable to models while being nonsensical to human understanding.
The card explains that HellaSwag's wrong answers are "specifically generated to trick machines" and are "plausible-sounding nonsense that LLMs tend to generate," which are "ridiculous to humans but often misclassified." This directly describes option A. Option B, while a general LLM challenge, does not capture the unique adversarial design of HellaSwag's distractors, which focuses on commonsense meaning rather than just syntax.
Read the full bite: HellaSwag: A Benchmark Designed to Fool LLMs
Question 24 of 30
What is the primary advantage of BERTScore over traditional metrics like BLEU or ROUGE?
Show the answer
Answer: c · It evaluates text quality by understanding contextual meaning, even with different phrasing.
BERTScore's core advantage is its ability to assess semantic similarity by understanding contextual meaning, allowing it to score valid paraphrases highly where traditional metrics would fail. Option A is incorrect because BERTScore prioritizes meaning over exact keyword matching, making it less suitable for tasks where precise keyword extraction is critical.
Read the full bite: BERTScore: Judging AI Text on Meaning, Not Just Words
Question 25 of 30
When fine-tuning BERT on a niche dataset of only 800 labeled examples, which approach best preserves pretrained knowledge while avoiding overfitting?
Show the answer
Answer: d · Start from a domain-adjacent checkpoint, freeze the first eight layers, and fine-tune at 2e-5 with early stopping
Starting from a domain-adjacent checkpoint and freezing early layers preserves low-level features while a tiny learning rate and early stopping prevent overfitting on scarce labels. Training from scratch is infeasible with only 800 examples, a large learning rate destroys pretrained weights, and raw text without tokenization or attention masks is incompatible with the model's input requirements.
Read the full bite: How do you leverage and fine-tune BERT for niche classification?
Question 26 of 30
What is the main representational advantage of multi-head attention over a single attention head?
Show the answer
Answer: b · It lets the model attend to different positions and relation types in parallel subspaces
Splitting into heads over distinct subspaces lets each head specialize in different relationships, which a single averaged head cannot represent at once. It is about representational diversity, not adding layers or removing positional information.
Read the full bite: Why Transformers use multi-head attention
Question 27 of 30
What is the primary way word embeddings capture the semantic meaning of words?
Show the answer
Answer: a · By mapping words to points in a multi-dimensional space where their proximity reflects their semantic similarity.
Word embeddings represent meaning by placing words as vectors in a high-dimensional space, where the distance between these points directly corresponds to their semantic similarity. While co-occurrence is used to learn these embeddings, it is not the representation itself.
Read the full bite: Word Embeddings: Turning Words into Math
Question 28 of 30
What is the main limitation of static embeddings like Word2Vec that contextual embeddings overcome?
Show the answer
Answer: c · Static embeddings assign one fixed vector per word, failing to disambiguate multiple meanings
A static embedding is one vector per word regardless of context, so polysemous words collapse their senses. Contextual models give a different vector per occurrence. The forward-pass cost actually describes contextual models, not static ones.
Question 29 of 30
For which application would topic modeling be the most appropriate technique?
Show the answer
Answer: a · Discovering the underlying, unstated themes present in thousands of customer feedback comments.
Topic modeling is designed for unsupervised discovery, making it ideal for finding unstated themes in large text collections like customer feedback. It is not suitable for supervised classification tasks with predefined categories, such as spam detection, or for simple keyword extraction or sentiment analysis.
Read the full bite: Topic Modeling: Finding Themes in Unstructured Text
Question 30 of 30
What is a task that a standard Named Entity Recognition (NER) system is NOT designed to perform?
Show the answer
Answer: b · Determining the emotional tone or sentiment expressed in a piece of text
NER's purpose is to identify and categorize specific entities like people, places, and organizations. The card explicitly states that NER 'is not the right tool for understanding... the overall sentiment of a text,' which is a task for sentiment analysis, not entity recognition.
Read the full bite: Named Entity Recognition: Finding the 'Who, What, Where' in Text
Could you explain these out loud?
That is what an interview actually tests. Tezvyn gives you questions like these with what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.