Async Generators: `yield` in an `async` World

Why it exists

Before async generators, creating an asynchronous data producer was verbose. You had to write a full class implementing the aiter and anext methods just to make it work with async for, which was cumbersome for simple streaming logic.

The mental model

Think of an async generator as a regular Python generator that can await. It's a function that produces a sequence of values over time, but it can pause for I/O operations (like a network request or database query) between yields without blocking the entire program's event loop.

How it works

You define a function with async def and use one or more yield expressions inside it. Calling this function doesn't run it; it immediately returns an asynchronous generator object. This object implements the asynchronous iteration protocol, allowing it to be used in an async for loop. Each time the loop requests the next item, the generator's code runs until it hits a yield, provides the value, and then pauses its state until the next item is requested.

When to use it

Use async generators when you need to produce a sequence of items, and the production of each item involves an I/O-bound operation that needs to be awaited. This is perfect for streaming rows from a database cursor, reading a large file in chunks from a remote server, or processing data from a WebSocket connection as it arrives.

When not to use it

If your data generation is purely CPU-bound and involves no await calls, a regular generator is simpler and more efficient. Also, if you need to return a final value from the function after iteration is complete, an async generator is not the right tool, as a non-empty return statement is a SyntaxError.

One canonical example

Instead of a complex class, you can write a simple function to stream numbers with a delay. async def ticker(delay, to): for i in range(to): yield i; await asyncio.sleep(delay). This is consumed in another async function with async for num in ticker(1, 5): print(num). This is significantly cleaner and, according to the PEP, about twice as fast as the manual class-based implementation.