Computer Vision

Design a tracking-by-detection tracker

Detect per frame, then associate boxes across frames by IoU or appearance using Hungarian matching, maintaining track ids.

Sparse vs dense optical flow and Lucas-Kanade

Sparse flow tracks selected feature points, dense flow computes a vector per pixel; Lucas-Kanade solves brightness constancy in a local window assuming constant motion.

Adapting ViT for dense semantic segmentation

Reassemble patch tokens into a 2D feature map, add a decoder, and handle low resolution plus quadratic attention cost.

Explain panoptic segmentation and Panoptic Quality

Panoptic assigns every pixel a class and instance id over things and stuff; PQ factors into SQ, average IoU of matches, times RQ, an F1 over matched segments.

How to improve coarse segmentation boundaries?

Skip connections and higher-resolution features, boundary-aware losses, and point-based or CRF refinement.

How does Mask R-CNN do instance segmentation?

Faster R-CNN backbone plus RPN, then RoIAlign and a parallel mask head predicting per-class binary masks.

U-Net architecture and its skip connections

U-Net has a contracting encoder, an expanding decoder, and skip connections that concatenate matching-resolution encoder features into the decoder to recover spatial detail lost in downsampling.

Semantic versus instance segmentation

Semantic segmentation labels each pixel by class but merges objects of the same class; instance segmentation also separates individual objects.

Detector head losses: regression versus classification

The head splits into a classification branch using cross-entropy over classes and a regression branch using a robust Smooth L1 or IoU loss on box offsets, combined as a weighted sum.

Deploying real-time detection on edge devices

Pick an efficient one-stage detector, train with augmentation, then quantize, prune, and compile to a hardware-accelerated runtime, measuring latency and accuracy tradeoffs.

Focal Loss and class imbalance in detectors

Focal loss multiplies cross-entropy by a (1-p)^gamma factor that down-weights easy, well-classified examples so the vast easy background does not swamp the loss.

easy2 min read

Mean Average Precision in object detection

AP is the area under the precision-recall curve per class; mAP averages AP over classes, and COCO also averages over IoU thresholds.

Intersection over Union for detection

IoU is the area of overlap divided by the area of union of predicted and ground-truth boxes; a threshold decides true positives.

Image classification versus object detection

Classification assigns one label to the whole image; detection localizes and labels multiple objects with bounding boxes and class scores.

Translation equivariance versus invariance in CNNs

Convolution is equivariant, shifting input shifts feature maps; invariance comes only from pooling and global aggregation. Strict invariance is partial and broken by strided sampling.

Depthwise separable convolution cost savings

Separable conv splits standard conv into per-channel spatial filtering plus a 1x1 pointwise mix, cutting cost by roughly 1/N plus 1/k².

Adapting a classification CNN for segmentation

Replace the dense head with conv layers, upsample via transposed convolutions, and fuse encoder skip connections to recover spatial detail lost to downsampling.

Uses of the 1x1 convolution

A 1x1 conv is a per-pixel linear combination across channels; it reshapes channel depth cheaply and adds nonlinearity. Uses: dimensionality reduction in bottlenecks and channel mixing.

Receptive fields in convolutional networks

Receptive field is the input region affecting a neuron; it grows with depth, larger kernels, and stride. It matters for capturing context in detection and segmentation.