Canary a shared downstream microservice

WHAT THIS TESTS: Whether you can safely roll out a change to a service many others depend on, and measure its blast radius across the full distributed call graph.

A GOOD ANSWER COVERS: Use a service mesh to control traffic at the network layer. Deploy the canary alongside the stable version and use mesh routing rules to send a small, controlled fraction of requests to it, by weight or by matching a header so you can target specific cohorts. Propagate context, a canary marker and trace identifiers, through every hop so the change is observable end to end. Distributed tracing then lets you attribute latency and errors specifically to spans hitting the canary, and crucially lets you watch the SLIs of the upstream services that call it, because a regression in a shared dependency manifests as failures in its consumers, not necessarily in the canary's own metrics.

COMMON WRONG ANSWERS: Judging the canary solely by its local error and latency, ignoring the callers it can break, routing inconsistently so one user's request bounces between versions, or having no tracing to localize where in the path the regression appears.

LIKELY FOLLOW-UPS: How do you keep a request consistent within one version? How do you canary by cohort? How do you separate the canary's contribution from baseline noise in traces? What about retries amplifying canary errors?

ONE CONCRETE EXAMPLE: An authentication service is a dependency for dozens of services. You deploy its canary and configure the mesh to route five percent of auth calls to it, tagging those requests. Distributed tracing shows that requests routed to the canary add fifteen milliseconds and a small error rate that surfaces as elevated checkout failures upstream. Because you instrumented the full path, you catch the downstream impact, not just the canary's own healthy-looking dashboard, and roll back the mesh routing before the change reaches everyone.