Distributed Systems
31 bites tagged Distributed Systems — interview questions with model answers, and 60-second explainers.
Design a tiered API rate limiter
Pick token bucket or sliding-window, key limits by partner tier, track counters in a shared store like Redis, decide at the edge. rate-limiting algorithms and distributed counting. per-node counters bypassed across instances.
Capacity planning for distributed stateful systems
Account for replication and cross-region network, IOPS and disk throughput, connection limits, partition/shard balance, and N+1 region failover headroom; validate with load and chaos… Whether you plan capacity beyond raw resources.
How does chaos engineering differ from other testing?
It experiments on real systems by injecting faults to test a steady-state hypothesis, versus verifying known behaviors like integration or load tests. Conceptual grasp of chaos engineering.
Why tail latency (p99) matters
Averages hide the worst experiences, and fan-out amplifies tails so most requests touch a slow path; causes include GC pauses, queueing, contention, and noisy neighbors. Tail-latency reasoning.
How does distributed tracing work?
Trace ID ties one request together, span IDs are individual operations, context propagates via headers. Whether you understand request correlation across services. Conflating traces with logs or omitting propagation entirely.
Predicate pushdown and why it speeds queries
Apply WHERE conditions at the scan or remote source, prune partitions and rows early, shrink data movement. moving filters close to the data.
Mitigating a database shard hot spot
Short-term, add read replicas or cache the hot keys; long-term, fix the partition key with hashing, salting, or finer-grained splitting. load-distribution remedies. only adding hardware without addressing the skewed key.
Split-brain, consensus, and quorum
Split-brain is two nodes both believing they are leader during a partition; Raft/Paxos require a majority quorum to elect a leader and commit, so the minority side cannot make progress. partition handling.
Durable write path in a sharded KV store
Route by key hash to the shard leader, append to WAL and fsync, replicate to two followers, ack on quorum, then confirm. end-to-end durable write design. confirming the client before any durable persistence.
What is eventual consistency?
Replicas converge to the same value if writes stop, allowing temporary staleness for higher availability and lower latency. consistency-model tradeoffs. claiming it means data is wrong or never converges.
Leader-follower vs multi-leader replication
Single-writer leader-follower is simple but a write bottleneck; multi-leader accepts writes in many regions for latency and availability. replication topology tradeoffs. ignoring that multi-leader needs conflict resolution.
Apply the CAP theorem to a real system
Define C, A, P; note partitions are unavoidable, so the real choice during one is consistency versus availability; then classify a system as CP or AP with reasoning. applying CAP to concrete systems.
What consistency do you sacrifice in an AP system?
You give up linearizability and often sequential consistency, accepting stale reads and conflicts, then mitigate with quorums, vector clocks or CRDTs, and… precise reasoning about consistency models and anomaly handling.
How do you keep consistency without multi-document transactions?
A Saga runs a sequence of local transactions, each with a compensating action to undo on failure, coordinated via choreography or orchestration. maintaining integrity across services without distributed ACID.
What are eventual consistency and the BASE model?
Eventual consistency means replicas converge given no new writes; BASE is Basically Available, Soft state, Eventually consistent. trading consistency for availability per feature.
What is the CAP theorem?
Consistency, Availability, Partition tolerance; during a network partition you must choose between staying consistent or staying available. grasp of fundamental distributed-systems limits.
HDFS purpose and fault tolerance
HDFS stores huge files across many commodity machines as large blocks, replicating each block across nodes for fault tolerance; unlike NTFS or ext4 it is distributed, write-once, and optimized for… distributed storage fundamentals.
Diagnosing and fixing data skew in Spark
This is data skew, caused by uneven key distribution concentrating rows on few partitions; mitigate with salting, broadcast joins, repartitioning, or adaptive execution. distributed processing skew. just adding more executors.
CAP theorem and real database tradeoffs
During a partition you pick consistency or availability, CP systems reject requests, AP systems stay available but stale. CAP as a partition-time choice. thinking you pick two of three at all times.
Strong versus eventual consistency in NoSQL
Strong reads see the latest write at higher latency and cost, eventual reads may be stale but are cheaper and faster, match the choice to stakes. consistency tradeoffs. calling eventual consistency simply broken.
Object store vs NFS consistency models
S3 gives strong read-after-write per object with no partial updates; NFS offers close-to-open with shared mutable files. distributed consistency depth.
Publish/Subscribe Pattern
Publish/subscribe decouples senders from receivers: publishers emit messages to a topic without knowing who consumes them, and subscribers receive messages from topics they care about.
How do you broadcast WebSocket messages to all clients across server nodes?
Tests WebSocket horizontal scaling and pub/sub backplanes. A strong answer names a broker like Redis, describes cross-node fan-out, and keeps connection state purely local.
Monitor p99 improvement from 500ms to 200ms and side effects
Tests systems-thinking on tail-latency instrumentation and metric tradeoffs. Strong answers cover histogram metrics, distributed tracing for fan-out bottlenecks, and guarding error rates, cost.
Get Distributed Systems bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.