Database Sharding: Splitting Data for Scale
Sharding splits a database across multiple servers, like dividing a phone book into A-M and N-Z volumes. It's used when a single server can't handle the data size or write load. The footgun is that querying across shards is complex and slow.
Why it exists
A single database server can only get so big or so fast, a process called vertical scaling. When you hit the physical limits of hardware or the cost becomes prohibitive, you need a way to distribute the load across multiple machines. Sharding is the primary method for this horizontal scaling.
The mental model
Imagine a library with a single, enormous dictionary. It's slow for multiple people to use, and eventually, it will be full. Sharding is like replacing it with multiple volumes (A-F, G-L, etc.), each on its own table. Each volume is a shard—smaller, faster to search, and you can add more volumes as your dictionary grows.
How it works
You choose a "shard key" from your data, like a user_id or region. A function, often based on hashing or ranges, uses this key to decide which shard (which server) a row of data lives on. For example, all users with IDs 1-1000 go to Shard 1, and users 1001-2000 go to Shard 2. The application or a routing proxy directs reads and writes to the correct shard based on the key.
When to use it
Use sharding when your database is bottlenecked by write volume, storage capacity, or CPU on a single machine. It's essential for applications with massive datasets or extremely high transaction rates, like large-scale social media platforms or e-commerce sites.
When not to use it
Avoid sharding for small to medium-sized applications where a single powerful server is sufficient and far less complex. It's a poor fit if your queries frequently require joining data that would end up on different shards, as this negates the performance benefits and adds significant complexity.
One canonical example
A ride-sharing app shards its data by geographic region. All trips, drivers, and users in North America are on one set of shards, while data for Europe is on another. This keeps queries fast for most operations (e.g., finding a nearby driver) because the data is co-located. A query for "all trips globally" would be very slow and complex.
Interview question
According to the card, which scenario makes database sharding a poor fit?
- a.When the application requires extremely high transaction rates.
- b.When queries frequently involve joining data that would reside on different shards.Correct
- c.When the database is bottlenecked by write volume or storage capacity.
- d.When the system needs to support massive datasets.
Why? this is the answer
The card explicitly states that sharding is a poor fit "if your queries frequently require joining data that would end up on different shards, as this negates the performance benefits and adds significant complexity." Options A, B, and D describe situations where sharding is beneficial and recommended.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #databases
- #architecture
- #scalability
- #sharding
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on databases — each one lists the topics its interview covers.
See open roles