Evaluating faithfulness and compositionality in multimodal models

WHAT THIS TESTS Whether you recognize the limits of overlap metrics and can design targeted tests for grounding and the binding of attributes to objects and their relations.

A GOOD ANSWER COVERS WHY BLEU AND CIDEr ARE INSUFFICIENT. They reward n-gram overlap with reference captions, so a fluent caption can score well while hallucinating objects, and a correct caption phrased differently can score poorly. They do not measure whether claims are true of the image or whether relationships are right. FAITHFULNESS. Decompose each generated statement into atomic claims and verify each against the image, for example via a grounding model or human check, then score the fraction of claims that are supported and penalize hallucinated objects, attributes, or counts. Question-answering style probes about specific facts in the scene also expose unfaithfulness. COMPOSITIONALITY. The key is hard negatives that differ minimally. Build contrastive sets where a caption and a distractor swap which object has which attribute or reverse a relation, such as the red cube is on the blue sphere versus the blue cube is on the red sphere, and require the model to match the image to the correct caption. Benchmarks like Winoground and ARO use this design to defeat bag-of-words shortcuts. Include attribute binding, spatial relations, counting, and negation. Combine automated scoring with targeted human evaluation and report per-phenomenon breakdowns rather than one aggregate.

COMMON WRONG ANSWERS Trusting BLEU or CIDEr as understanding. Using only easy negatives the model can solve by keyword matching. Evaluating only overall caption quality, not per-claim grounding. Ignoring attribute-to-object binding and relations. Reporting a single number with no per-skill breakdown.

LIKELY FOLLOW-UPS Why do bag-of-words models still score well on weak benchmarks? How do you build minimal-edit hard negatives at scale? How do you measure hallucination rate precisely? Where does human evaluation remain necessary?

ONE CONCRETE EXAMPLE For a scene with a red cube on a blue sphere, you present two captions differing only by swapped attributes and relations and require the model to pick the right one. A model that merely detects cube, sphere, red, blue as a bag of words cannot choose correctly, exposing weak compositionality, while a faithfulness probe separately asks what color is the cube and checks the answer against the ground truth.