Pull vs. Push: How Your Metrics Get to the Collector

WHY IT EXISTS: Systems need a way to transport telemetry from where it's generated (the application) to where it's stored and analyzed (the monitoring backend). The choice of who initiates this transfer—the application or the collector—has major architectural implications for reliability, discovery, and network configuration.

THE MENTAL MODEL: Push is like mailing a letter—the sender is responsible for finding a mailbox and sending it. The recipient just waits for mail to arrive. Pull is like a scheduled package pickup—the recipient comes to your address at a set time to get the package. You just have to have it ready at the door.

HOW IT WORKS: In a push system, the instrumented application or its agent opens a connection and sends metrics to a known collector endpoint. This is an outbound connection from the app's perspective. Systems like Statsd or OpenTelemetry's OTLP (OpenTelemetry Protocol) use this model. In a pull system, the application exposes its current metrics on an HTTP endpoint, typically /metrics. The central monitoring system, like a Prometheus server, is configured to periodically connect to this endpoint and 'scrape' the current values. This requires an inbound connection to the application.

WHEN TO USE IT: Use push for short-lived or ephemeral workloads like serverless functions or batch jobs, as they might terminate before a pull-based scraper can reach them. Push is also simpler in networks where applications are behind strict firewalls that prevent incoming connections. Use pull for long-running services, especially in dynamic environments like Kubernetes. Pull simplifies service discovery, as the central system can automatically find and scrape new targets. It also provides a natural health check; if a scrape fails, the target is likely down.

WHEN NOT TO USE IT: The risk with push is overwhelming the collector if many agents send data at once, creating a 'thundering herd' problem. It's also harder to definitively know if an agent is down or just not sending data. The main drawback of pull is that it's unsuitable for ephemeral jobs. It can also introduce network complexity, as you must ensure the central scraper has a network path to every application endpoint.

ONE CANONICAL EXAMPLE: Prometheus is the quintessential pull-based system, scraping HTTP endpoints on a configured interval. In contrast, the OpenTelemetry Protocol (OTLP) is push-based, where instrumented services send data to an OpenTelemetry Collector. A collector can bridge these worlds by accepting pushed OTLP data and then exposing it as a pull-style endpoint for Prometheus to scrape.