API Pagination: Serving Big Datasets in Chunks
API pagination breaks large result sets into smaller chunks to prevent server overload. It's essential for any endpoint returning many records, like a list of users or products.
Why it exists
An API might work fine with a few hundred records, but it will crumble as the dataset grows to thousands or millions. Pagination exists to solve this by breaking down a large dataset into smaller, manageable chunks that can be fetched incrementally.
The mental model
Think of pagination as a server-side control for breaking a large list into smaller pieces. Instead of sending the entire phone book at once, you send one page at a time. The client requests a specific chunk, not the whole thing.
How it works
The two most common strategies are page-based and offset-based. Page-based uses page and size parameters (e.g., ?page=2&size=10). Offset-based uses offset and limit (e.g., ?offset=10&limit=10), which maps directly to SQL clauses. A robust implementation includes hypermedia links (next, prev) in the response. This lets the client navigate without having to construct pagination URLs itself, keeping the logic on the server.
When to use it
Implement pagination from the start for any collection endpoint (e.g., /items, /users) that could grow. Adding it after launch is usually a breaking change. It improves server performance and provides a better user experience, as clients get the first chunk of data quickly without waiting for the entire dataset.
When not to use it
You don't need pagination for endpoints guaranteed to return a small, finite number of items. For large, rapidly changing datasets, simple page/offset pagination is problematic. If a user requests page 2 just after a new item was added to the list, they might see a record from page 1 again, creating data consistency issues.
One canonical example
A client requests the second page of 10 items using GET /items?page=2&size=10. The server, instead of just returning data, includes hypermedia controls. The response contains the 10 items, but also a links object with URLs for self (/items?page=2&size=10), next (/items?page=3&size=10), and prev (/items?page=1&size=10). The client application can then use these provided URLs to build "Next" and "Previous" buttons without needing to know the pagination logic.
Interview question
What is the primary benefit of including hypermedia links (next, prev) in a paginated API response?
- a.It reduces the client's need to construct pagination URLs, simplifying navigation logic.Correct
- b.It allows the server to dynamically adjust the page size based on client demand.
- c.It prevents server overload by offloading pagination logic to the client.
- d.It ensures data consistency even when the dataset is rapidly changing.
Why? this is the answer
The card explicitly states that hypermedia links allow the client to navigate without having to construct pagination URLs itself, keeping the logic on the server, which simplifies client-side navigation. Option D is incorrect because hypermedia links do not address data consistency issues in rapidly changing datasets; that's a separate challenge for simple pagination.
Just read this? Test yourself on what you have been reading.
Read the original → apisyouwonthate.com
- #api design
- #rest
- #pagination
- #performance
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on api design — each one lists the topics its interview covers.
See open roles