Deep learning

Explain Denoising Diffusion models and forward/reverse processes.

This tests if you see diffusion as iterative latent generation, not GANs. Forward: add Gaussian noise over T steps until data is pure noise. Reverse: a network iteratively denoises random noise into data.

What is overfitting and how does Dropout prevent it?

Tests generalization intuition: overfitting is low train error but high test error. Good answers say dropout randomly zeros hidden units during training to stop co-adaptation. Bad answers say dropout permanently deletes neurons or just reduces capacity.

Explain vanishing and exploding gradients and common mitigation techniques.

Why deep backpropagation causes diverging gradient magnitudes. Repeated multiplication across layers shrinks or explodes gradients; cite tanh [0,1] range; list ReLU, batch norm, and gradient clipping. Blaming activation choice alone without citing depth.

Zero-padding vs reflect vs replicate padding and their visual artifacts

This tests boundary assumptions in convolution. Zero-padding adds black borders causing dark vignettes; reflect padding mirrors edges for continuity; replicate padding repeats edge values outward. A red flag is saying padding choice does not affect outputs.

SSD: Real-Time Detection Without Region Proposals

SSD scores default boxes across multiple scales in one forward pass. It runs real-time robotics and mobile vision where two-stage detectors lag. The footgun is ignoring shallow feature maps, which destroys small object accuracy as early layers carry fine…

Model Compilation: Bridging Models and Hardware

An ML compiler translates a model's abstract math into optimized instructions for specific hardware. This lets you run the same model efficiently on cloud GPUs, mobile CPUs, or edge devices.

Model Pruning: Making ML Models Smaller and Faster

Model pruning is like trimming a bonsai tree; you remove the least important weights to create a smaller, faster model. It's essential for running large models on devices like smartphones, but over-pruning can irreversibly damage accuracy.

Horovod: Scale ML Training Across Many GPUs

Horovod scales a single-GPU training script to hundreds of GPUs with minimal code changes, slashing training time. It's used when models are too big for one machine.

Parameter Servers for Distributed ML Training

A parameter server splits the work in distributed training: central servers hold the model's parameters, while worker nodes pull parameters, compute gradients on data subsets, and push updates back. This enables training models too large for one machine.

Multimodal Models: Beyond Just Text

A multimodal model understands the world by connecting different data types, like images and text, instead of just one. It's how AI generates images from descriptions or answers questions about a photo. The footgun is assuming more data types always helps.

AdamW: Decoupling Weight Decay for Better Generalization

AdamW fixes a flaw in the Adam optimizer by decoupling weight decay from the gradient update, improving model generalization. It's a go-to for training large networks like Transformers. The footgun is thinking it's the same as Adam with L2 regularization.

Mixed-Precision Training: Faster Training with Less Memory

Mixed-precision training is like using rough estimates (FP16) for most math and a calculator (FP32) for critical steps. This speeds up deep learning on GPUs by cutting memory use, but naively switching can cause training to fail as small gradients vanish.

LSTMs: Giving Neural Networks a Longer Memory

LSTMs give neural networks a longer memory, letting them connect events across long sequences. They excel at tasks like language translation or time-series analysis where distant context is key.

The Vanishing Gradient Problem

Training a deep network is like a game of telephone; the error signal (gradient) gets weaker as it's passed back through layers. This happens in deep networks using sigmoid or tanh activations.

Activation Functions: Making Neural Networks Nonlinear

An activation function acts as a gatekeeper for a neuron, deciding what signal to pass on. It introduces non-linearity, allowing networks to learn complex patterns. A network with only linear activations collapses into a simple, less powerful model.

Transfer Learning: Don't Train Models from Scratch

Transfer learning means not training a model from zero. You start with a model pre-trained on a large, general dataset, then fine-tune it for your specific task. This is common in image recognition, using a general model to learn a niche classification.

Convolutional Neural Networks: Finding Patterns with Filters

A CNN learns to spot features by sliding optimized filters over data like images, audio, or text. It's the go-to for computer vision, but a common mistake is thinking it's the only modern tool, as transformers sometimes replace it.

Image Convolution: A Sliding Feature Detector

An image convolution is a sliding filter that scans an image to detect features like edges or textures. It's the core building block of modern computer vision, used in image classification and object detection.

Neural Network Pruning: Making Models Smaller and Faster

Neural network pruning makes models smaller and faster by removing unimportant connections, like trimming a bonsai tree. It's essential for deploying large models on devices with limited memory, like phones.

Semantic Scene Classification: Understanding Context, Not Just Objects

Scene classification tells you the context of an image ("this is a forest"), not just the objects in it ("there's a tree"). It's used by self-driving cars to identify a highway vs. a residential street and by apps to organize photos.

U-Net: Encoder-Decoder for Image Segmentation

U-Net segments images by first compressing them to capture context, then expanding to localize features precisely. It excels in biomedical imaging where annotated data is scarce.

Anchor Boxes: Pre-defined Guesses for Object Detection

Anchor boxes are predefined 'template' boxes of various sizes and shapes. Object detection models use them as a starting point, predicting how to shift and scale these templates to fit actual objects, making detection faster.

TensorRT: From Trained Model to Production Speed

TensorRT is a compiler that turns a trained model into a specialized, high-speed engine for a specific NVIDIA GPU. It's used to deploy models in production where low latency is critical.