More in Computer Vision — page 4
Why U-Net skip connections matter for denoising
WHAT IT TESTS: role of skip connections in denoising. OUTLINE: skips carry high-resolution spatial detail from encoder to decoder, preserving fine structure lost in downsampling and easing gradient flow, which lets the model restore detail while removing…
Evaluating generative models with FID versus IS
WHAT IT TESTS: rigorous generative evaluation. OUTLINE: FID compares Inception feature distributions of real and fake images via Frechet distance between two Gaussians; it uses real data as reference and detects diversity issues, unlike IS which uses no real…
Mode collapse in GAN training
WHAT IT TESTS: understanding GAN diversity failure. OUTLINE: generator produces few outputs ignoring data diversity, caused by chasing whatever fools the current discriminator; mitigate with minibatch discrimination, unrolled GANs, or Wasserstein loss.
Diffusion forward and reverse processes
WHAT IT TESTS: the noise-and-denoise framing. OUTLINE: forward process gradually adds Gaussian noise until data is pure noise; reverse process learns to denoise step by step; the network predicts the noise added at each timestep.
GAN architecture: generator and discriminator roles
WHAT IT TESTS: the adversarial minimax setup. OUTLINE: generator maps noise to fake samples, discriminator classifies real versus fake, they train as a two-player game until samples fool the discriminator.
Attention in diffusion U-Nets for text conditioning
WHAT IT TESTS: how text conditioning enters the denoiser. OUTLINE: self-attention mixes spatial features at low-res blocks; cross-attention has image queries attend to text-token keys/values; placed inside transformer blocks.
Pure ViT vs hybrid CNN-Transformer for medical segmentation
WHAT IT TESTS: matching architecture to data and task constraints. OUTLINE: pure ViT captures global context but is data hungry and weak on local detail; hybrid CNN-Transformer gets local features cheaply plus global attention, ideal for scarce…
Core principles of a Neural Radiance Field
WHAT IT TESTS: implicit 3D representation and differentiable rendering. OUTLINE: an MLP maps a 3D point plus view direction to color and density; novel views render by casting rays, sampling points, querying the MLP, and volume-integrating along each ray.
Why ViTs need positional embeddings
WHAT IT TESTS: why order matters for attention but not convolution. OUTLINE: self-attention is permutation invariant so patch order is lost; positional embeddings restore spatial location. CNNs encode position implicitly via the fixed convolution grid.
Cross-attention for visual question answering
WHAT IT TESTS: fusing two modalities with attention. OUTLINE: in cross-attention queries come from one modality and keys/values from the other, e.g. text queries attend over image features so the question selects relevant regions.
How Swin Transformer achieves linear attention
WHAT IT TESTS: making attention scale to high resolution. OUTLINE: Swin computes attention within local non-overlapping windows of fixed size, making cost linear in patches, then shifts windows between layers so information crosses boundaries.
Inductive biases of ViT versus CNN
WHAT IT TESTS: how built-in priors affect data needs. OUTLINE: CNNs bake in locality and translation equivariance; a plain ViT has almost none beyond patch structure, so it must learn spatial relations from data, needing large datasets or strong pretraining.
Self-attention over image patches explained
WHAT IT TESTS: the QKV mechanism on patch tokens. OUTLINE: each patch projects to query, key, value; a patch's query is scored against all keys, softmax-normalized into weights, used to combine all values.
How ViT and CNN process an image differently
WHAT IT TESTS: the input pipelines of two paradigms. OUTLINE: a CNN slides local filters over the raw pixel grid; a ViT splits the image into patches, flattens and linearly embeds each into a token, adds positional embeddings, and feeds the sequence to…
Scene flow versus optical flow
WHAT IT TESTS: 2D versus 3D motion estimation. OUTLINE: optical flow is 2D pixel motion in the image plane; scene flow is the 3D motion field of points in space, needing depth via stereo, RGB-D, or LiDAR.
Self-supervised pretraining for video understanding
WHAT IT TESTS: learning from unlabeled video via pretext tasks. OUTLINE: define a label-free task like temporal order prediction or contrastive clip matching that forces temporal reasoning, then fine-tune on labeled action data.
Re-identification in multi-object tracking
WHAT IT TESTS: maintaining identity across gaps and occlusions. OUTLINE: Re-ID matches an object to its prior id using appearance embeddings, not just position; store track features and match re-entering detections by embedding similarity.
3D CNNs vs two-stream action recognition
WHAT IT TESTS: how architectures capture temporal motion. OUTLINE: 3D CNNs learn spatiotemporal filters end to end but are heavy; two-stream splits RGB appearance and precomputed optical flow, strong but costly to compute flow.
Kalman filter for bounding-box tracking
WHAT IT TESTS: predict-update recursion applied to tracking. OUTLINE: state, transition, measurement models, and process plus measurement noise; predict then correct each frame. State holds box position and velocity; measurement is the detected box.
Brightness constancy and small-motion assumptions
WHAT IT TESTS: foundations and limits of optical flow. OUTLINE: brightness constancy says a point's intensity is invariant under motion; small motion lets you linearize via Taylor expansion.