Performance Profiling in Tests

Why it exists

Manually running an app and eyeballing whether scrolling feels smooth does not scale. A developer might not notice a regression that drops frame time from 8 milliseconds to 20, especially on a fast development machine, and by the time a user complains about jank in production it is expensive to trace back to the commit that caused it. Performance profiling inside tests exists to turn subjective smoothness into a number that a script, and eventually a CI pipeline, can check automatically on every change.

The mental model

Treat it like a fitness tracker for your app's frames. Instead of trusting how a run felt, you get a log of every step: how long each frame took to build its widget tree and how long the GPU took to rasterize it. A baseline recording becomes your resting heart rate, and any test run that drifts far from it flags a regression before a human ever has to notice the jank by feel.

How it works

Flutter's integration_test package, the successor to flutter_driver, runs a scripted interaction, opening a screen, scrolling a list, playing an animation, on a real device or emulator in profile mode, and records a timeline of events. Wrapping the interaction in a call like binding.traceAction or watchPerformance captures frame build and raster durations for every frame in that window, summarized into a TimelineSummary: average frame build time, 90th percentile rasterizer time, worst frame time, and the count of frames that missed the frame budget. CI stores this JSON output and compares it against a saved baseline, failing the build if a metric regresses past a threshold.

When it matters

It matters most for scroll-heavy screens, image-laden lists, and custom animations, exactly the places jank is most visible to users. The footgun is running these tests in debug mode: debug builds carry assertion checks and JIT overhead that make every timing number meaningless, so profiling only means something in profile or release builds. A second footgun is an emulator without proper GPU acceleration, which skews raster numbers unrelated to real device performance.

A concrete example

After adding a drop shadow behind each card on a feed screen, an integration test scripted to scroll through 50 cards shows average raster time jumping from 8.1 to 21.6 milliseconds per frame, well past the 16.6 millisecond budget for 60Hz. The CI job fails before merge, pointing straight at the new BackdropFilter-based shadow as the cause, instead of that regression shipping and only surfacing later as a vague feed feels laggy bug report.