Diffusion-based image inpainting design

WHAT THIS TESTS The interviewer wants a concrete plan to constrain a generative diffusion model so it fills only a masked region while preserving the rest, covering both input changes and the inference loop.

A GOOD ANSWER COVERS Two viable approaches. The training-free method, like RePaint, reuses a standard pretrained model and modifies only the reverse loop: at each denoising timestep, you denoise the full latent normally, but then overwrite the unmasked region with the original image diffused to that same noise level, so the known pixels are forced to remain correct while the masked region is generated freely. This keeps the context consistent and needs no new training, though it benefits from resampling steps to harmonize boundaries. The fine-tuned method, used by Stable Diffusion inpainting, adds extra input channels to the U-Net: the masked image latent and the binary mask are concatenated to the noisy latent, and the model is trained to fill masks, giving cleaner, faster results. Both can be conditioned on a text prompt for guided fills.

COMMON WRONG ANSWERS Regenerating the whole image and pasting it back, which loses the original unmasked pixels and creates seams. Another error is feeding only the prompt with no mask, so the model has no idea which region to preserve.

LIKELY FOLLOW-UPS Expect questions on boundary blending and seam artifacts, on the resampling jumps in RePaint, on why latent-space masking needs careful mask downscaling, and on outpainting as a generalization.

ONE CONCRETE EXAMPLE To remove a person from a beach photo, you mask that region. Each denoising step generates plausible sand and water inside the mask while the loop re-injects the noised original ocean and sky outside it, so the final image keeps the real background untouched and the filled area blends seamlessly.