Adapting a classification CNN for segmentation
turning a classifier into a dense predictor.
replace the dense head with conv layers, upsample via transposed convolutions, and fuse encoder skip connections to recover spatial detail lost to downsampling.
What's really being asked
Whether you understand the tension between the downsampling that builds semantic features and the full-resolution output segmentation demands.
The full answer
A classification ResNet repeatedly downsamples through strided stages, then applies global average pooling and a fully connected layer to emit a single class vector. Segmentation needs a class label per pixel at the input resolution. The first change is making the network fully convolutional, removing the dense head so it can accept any input size and produce a spatial map. The encoder still downsamples to extract rich semantics, producing a coarse, low-resolution feature map. A decoder then upsamples this back to full resolution using transposed convolutions, which learn the upsampling, or bilinear upsampling. The core challenge is that downsampling discards precise location information, so naive upsampling yields blurry, imprecise boundaries. Skip connections solve this by fusing high-resolution feature maps from early encoder stages into the decoder, restoring fine spatial detail while keeping the deep semantics.
The mistakes people make
Saying you just enlarge the output with interpolation; that alone gives coarse boundaries. Keeping the global pooling and dense layer. Ignoring that low-level skip features carry the localization the deep layers lost. Confusing transposed convolution with simple unpooling.
What usually comes next
What is a transposed convolution and what causes its checkerboard artifacts. How does FCN's skip fusion differ from U-Net's. Why combine deep semantic and shallow spatial features. How do dilated convolutions offer an alternative to heavy upsampling.
A concrete example
Take a ResNet that downsamples a two-twenty-four by two-twenty-four image to seven by seven. Replace the classifier head with a one by one conv producing per-class scores at seven by seven, then upsample by thirty-two times. Predicting boundaries from a seven by seven grid is hopelessly coarse, so you add skip connections that add in the sixteen-fold and eight-fold resolution feature maps before upsampling, sharpening object edges dramatically, which is exactly the progression from FCN-32s to FCN-8s.
Interview question
Why are skip connections added when converting a classification CNN into a segmentation network?
- a.To restore fine spatial detail lost during encoder downsamplingCorrect
- b.To reduce the number of parameters in the decoder
- c.To prevent overfitting on the segmentation labels
- d.To increase the receptive field of the deepest layer
Why? this is the answer
Downsampling builds semantics but discards precise localization, so skip connections inject high-resolution encoder features into the decoder for sharper boundaries. They are not primarily about parameters, receptive field, or regularization.
Just read this? Test yourself on what you have been reading.
Read the original → arxiv.org
- #computer-vision
- #semantic-segmentation
- #encoder-decoder
- #transposed-convolution
- #skip-connections
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on computer-vision — each one lists the topics its interview covers.
See open roles