Debug intermittent pod-to-pod connectivity
systematic network debugging.
scope the failure by path, rule out DNS, inspect kube-proxy iptables and conntrack, check the CNI, then verify cloud security groups and MTU.
restarting pods with no hypothesis.
What's really being asked
This evaluates structured troubleshooting across the Kubernetes networking stack and the underlying cloud substrate, and whether you can isolate intermittent faults, which are far harder than total outages because they are timing-dependent and do not reproduce on demand. The interviewer wants a hypothesis-driven method, not random restarts.
The full answer
First scope the problem: is it pod-to-pod on the same node, cross-node, pod-to-service, or DNS? Use a debug container such as netshoot to run curl, dig, ping, and tcpdump. Confirm DNS resolution, since intermittent failures often trace to CoreDNS load or ndots search-domain expansion. Inspect Services: kube-proxy programs iptables or IPVS, and a classic cause of intermittent resets is the conntrack table filling or SNAT source-port collisions on DNAT. Check the CNI for IP address exhaustion in the pod subnet, overlay encapsulation problems, or stale routes. At the cloud layer verify security groups, NACLs, and route tables, and especially MTU: an overlay adds headers, so an MTU mismatch drops only large packets, which looks intermittent. Watch node metrics for packet drops and conntrack counts.
The mistakes people make
Restarting pods or the CNI without forming a hypothesis. Blaming the application when the failure is path-dependent. Ignoring DNS, conntrack, and MTU, the three most common culprits. Not distinguishing same-node from cross-node traffic, which immediately localizes the fault.
What usually comes next
How does conntrack exhaustion cause connection resets? What MTU would you set on a VXLAN overlay? How do you diagnose SNAT port exhaustion? How would CoreDNS caching or NodeLocal DNS help?
A concrete example
Large POST requests between nodes time out while small ones succeed. tcpdump shows fragmentation-needed ICMP messages. The overlay reduced the effective MTU below 1500, so oversized packets are silently dropped; lowering the pod MTU to match the encapsulated path resolves the intermittent failures without touching the application.
Interview question
Large requests between pods on different nodes intermittently fail while small ones always work. What is the most likely root cause?
- a.RBAC denying large payloads
- b.An MTU mismatch from overlay encapsulation dropping oversized packetsCorrect
- c.A missing ResourceQuota on bandwidth
- d.The application thread pool exhausting under load
Why? this is the answer
Size-dependent failures point to MTU: overlay headers shrink the usable MTU, so large packets are dropped while small ones pass. RBAC and quotas do not inspect packet size, and a thread-pool issue would not correlate with payload size.
Just read this? Test yourself on what you have been reading.
Read the original → kubernetes.io
- #kubernetes
- #networking
- #cni
- #debugging
- #troubleshooting
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on kubernetes — each one lists the topics its interview covers.
See open roles