Prometheus: Monitoring with a Dimensional Data Model
Prometheus models system health as labeled time series, not just flat metrics. It pulls data from targets, making it ideal for dynamic cloud environments like Kubernetes. The footgun is using high-cardinality labels, which can overwhelm its storage.
Why it exists
Traditional monitoring was often host-centric. In the cloud-native world of ephemeral containers and dynamic services, you need to monitor the service, not a specific machine that could disappear at any moment. Prometheus was built to solve this by continuously discovering and monitoring services in environments like Kubernetes.
The mental model
Think of Prometheus not as a simple metric collector, but as a time series database with a powerful query language. Its core idea is the dimensional data model. A metric is not just a name like http_requests; it is a name plus a set of key-value pairs called labels, like http_requests_total{job="api-server", method="GET"}. These labels let you slice and aggregate your data across any dimension.
How it works
A central Prometheus server periodically scrapes (pulls) metrics over HTTP from configured endpoints on your services, called targets. These targets expose their current state as text-based metrics. The server stores this data locally in an efficient time series database. You then use the PromQL query language to analyze the data, create dashboards, or define alerting rules. A separate component, the Alertmanager, handles deduplicating, grouping, and routing alerts based on these rules.
When to use it
Use Prometheus for white-box monitoring of your own services, especially in cloud-native and containerized environments. It excels at tracking numeric time series like request rates, error counts, CPU usage, and queue depths. Its tight integration with Kubernetes for automatic service discovery is a primary use case.
When not to use it
Prometheus is not for logging or event data; it is a metrics system. Do not use it for tracing individual requests (use a distributed tracing tool for that). Avoid high-cardinality data in labels, such as user IDs, email addresses, or request UUIDs. Every unique combination of labels creates a new time series, and a cardinality explosion will destroy performance and storage.
One canonical example
A web service exposes a metric http_requests_total. Prometheus scrapes this metric and stores it with labels it adds, like job="my-app" and instance="pod-xyz-123". The service can also add its own labels, like method="POST" and status_code="500". A PromQL query like sum(rate(http_requests_total{job="my-app", status_code=~"5.."}[5m])) can then calculate the rate of server errors across all instances of that app over the last 5 minutes, triggering an alert if it crosses a threshold.
Interview question
Which statement accurately describes a core aspect of Prometheus's data model and a potential pitfall?
- a.It relies on a push-based agent model, which can struggle with dynamic service discovery in cloud environments.
- b.It stores metrics as raw event logs, making it suitable for detailed tracing of individual requests.
- c.It uses a dimensional model with labels, where high-cardinality labels can cause performance and storage issues.Correct
- d.It aggregates all data into pre-defined, fixed-schema tables, which limits its flexibility for custom metrics.
Why? this is the answer
Prometheus uses a dimensional data model with key-value labels, allowing flexible data slicing and aggregation. However, using high-cardinality labels creates too many unique time series, leading to performance and storage problems. Option A is incorrect because Prometheus is pull-based and excels in dynamic cloud environments.
Just read this? Test yourself on what you have been reading.
Read the original → prometheus.io
- #monitoring
- #sre
- #cloud-native
- #cncf
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on monitoring — each one lists the topics its interview covers.
See open roles