The Cloze Test: Measuring Comprehension, Not Just Readability

Why it exists

Automated readability formulas, like Flesch-Kincaid, only measure the complexity of words and sentences. They can't tell you if a user actually understands the meaning. A text can be simple to read but difficult to comprehend without specific background knowledge, creating a dangerous gap between what the metrics say and what users experience.

The mental model

A Cloze test is an empirical check on user comprehension, not a prediction. Think of it as the difference between a car's spec sheet and a real-world test drive. Readability formulas give you the spec sheet (word length, sentence complexity), but a Cloze test puts your target audience in the driver's seat to see if they can actually operate the text as intended.

How it works

The procedure is straightforward. First, take a piece of text and replace every Nth word (typically the 6th) with a blank space. Second, give this modified text to a group of participants from your target audience and ask them to fill in the missing words. Third, calculate the percentage of correctly guessed words. Synonyms and minor misspellings are usually counted as correct. If the average score is 60% or higher, the text is considered reasonably comprehensible for that audience.

When to use it

Use a Cloze test when you need to be certain that a specific audience understands critical information. This is vital for user instructions, terms of service, privacy policies, or any content where misinterpretation carries a high cost. It validates that your writing is effective for the people who matter most.

When not to use it

Because it requires recruiting and testing real users, a Cloze test is more time-consuming and expensive than running an automated formula. It's overkill for low-stakes content like a routine blog post. Use automated readability scores for a quick first-pass assessment, and reserve Cloze testing for your most important copy.

One canonical example

Consider two sentences: "He waved his hands" and "He waived his rights." A readability formula would score both as very easy to read due to their simple words and short structure. However, while everyone understands the first sentence, the second requires legal knowledge to fully grasp its significant implications. A Cloze test would immediately reveal this comprehension gap where a readability score would not.