Evaluating image generation: FID and IS

WHAT THIS TESTS The question checks whether you can quantitatively evaluate a generative model and articulate the blind spots of the standard metrics rather than reciting names.

A GOOD ANSWER COVERS Frechet Inception Distance passes real and generated images through a pretrained Inception network, models each set's activations as a multivariate Gaussian, and computes the Frechet distance between them. Lower FID means the generated distribution is closer to real in feature space, capturing both fidelity and diversity, which is why it correlates reasonably with human judgments. Inception Score feeds generated images to an Inception classifier and rewards two properties at once: each image should yield a confident, low-entropy label, and the set should cover many classes. Higher is better. Its key flaw is that it never looks at the real dataset, so a model can score well while ignoring the target distribution, and it is gameable. Both metrics depend on Inception features trained on ImageNet, are biased by sample size, and say nothing about prompt alignment or aesthetic quality.

COMMON WRONG ANSWERS Claiming higher FID is better, or that either metric measures human-perceived quality or text-image alignment. FID is a distance, so lower wins, and neither captures whether an image matches a caption, which needs metrics like CLIPScore.

LIKELY FOLLOW-UPS Expect questions on sample-size bias and clean-FID, on why FID is preferred over IS, on CLIPScore for text-to-image alignment, and on precision-recall metrics that separate fidelity from diversity.

ONE CONCRETE EXAMPLE A model that memorizes a few sharp images can earn a strong Inception Score because each image is classified confidently, yet its FID is poor because the generated distribution collapses and diverges from the real data spread, exposing the mode collapse that IS alone would hide.