Impact of losing etcd quorum

WHAT THIS TESTS Whether you understand that Kubernetes separates the control plane, which manages desired state in etcd, from the data plane, which runs workloads, and that they degrade independently.

A GOOD ANSWER COVERS etcd is a Raft-based store that needs a majority quorum to elect a leader and commit writes. Lose quorum, for example two of three members down, and no writes can be committed. The control-plane impact is immediate: the kube-apiserver can no longer persist changes, so creating, updating, scaling or deleting objects fails, deployments and rollouts stall, the scheduler cannot record bindings, and controllers cannot reconcile. Reads may still partly serve from cache or a lingering member but are unreliable, so treat the cluster as frozen for changes. The data-plane impact is far gentler: kubelets on worker nodes keep running the pods already scheduled to them because they operate from their last-known local spec, containers keep serving traffic, and kube-proxy keeps applying existing service routing rules. What you lose is self-healing: a crashed pod will not be rescheduled, a failed node's pods will not move, autoscaling stops, and new endpoints are not reflected. So existing traffic flows but the cluster cannot adapt.

COMMON WRONG ANSWERS Saying all workloads crash immediately; running pods persist. Claiming reads work perfectly; without quorum consistency guarantees are lost. Forgetting that recovery, rescheduling and scaling are blocked even though current pods survive. Conflating an apiserver outage with workload outage.

LIKELY FOLLOW-UPS Why three or five members and never an even number? How do you recover from quorum loss, restore from snapshot or remove failed members? What happens if a node fails while etcd is down? How long can the data plane coast, and what eventually breaks?

ONE CONCRETE EXAMPLE A three-member etcd loses two members. A web app already running keeps serving users and its Service keeps routing. But when one of its pods OOMKills, it is not recreated because the controller cannot write to etcd, and a kubectl scale command hangs and fails. Once quorum is restored from the surviving member or a snapshot, the API server resumes writes and the missing pod is finally rescheduled.