Classic image captioning architecture
the encoder-decoder captioning pattern.
a CNN encoder extracts image features, a recurrent or transformer decoder generates the caption word by word, and attention lets the decoder focus on image regions per word.
What's really being asked
The interviewer wants the classic encoder-decoder mental model and the role of attention, confirming you see captioning as conditional sequence generation, not classification.
The full answer
The two core components are a visual encoder and a language decoder. The encoder is typically a convolutional network pretrained on image classification, with its final layers used to produce either a single global feature vector or, more powerfully, a grid of spatial feature maps. The decoder is a sequence model, classically an LSTM and in modern systems a transformer, that generates the caption autoregressively, predicting one word at a time conditioned on the image features and the words generated so far, starting from a start token until an end token. The bridge between them is attention: at each decoding step the model computes weights over the encoder's spatial features so it can focus on the relevant region, attending to a dog when emitting the word dog and to grass when emitting grass. Training minimizes cross-entropy against reference captions, often with teacher forcing.
The mistakes people make
Describing captioning as a single multi-class prediction, which cannot produce variable-length fluent sentences. Another error is omitting attention and assuming a single fixed image vector suffices; that works but produces vaguer captions because the decoder cannot localize.
What usually comes next
Expect questions on beam search versus greedy decoding, on evaluation with BLEU and CIDEr, on teacher forcing and exposure bias, and on how modern vision-language models replace this with a unified transformer.
A concrete example
Given a photo of a dog catching a frisbee, the CNN encodes spatial features, the decoder emits a, then dog, attending to the animal, then catching and frisbee while attending to the disc, producing a dog catching a frisbee, a fluent variable-length sentence rather than one class label.
Interview question
Why is a classic image captioning model built as an encoder-decoder rather than a single image classifier?
- a.A classifier would be slower than autoregressive decoding
- b.Classifiers cannot process images, only the decoder can
- c.It must generate a variable-length word sequence conditioned on the image, not pick one labelCorrect
- d.Encoder-decoder models avoid needing any training data
Why? this is the answer
Captioning produces fluent, variable-length sentences generated word by word, which requires a sequence decoder conditioned on encoded image features. A single classifier can only emit one fixed label.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #image-captioning
- #encoder-decoder
- #attention
- #cnn
- #sequence-generation
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles