Managing model-as-a-feature pipelines

WHAT THIS TESTS Whether you grasp that an embedding feature turns one model into a hard dependency of another, and the cascading operational problems that follow.

A GOOD ANSWER COVERS The upstream embedding model becomes a versioned dependency of the downstream model. The central challenge is version coupling: the downstream model learns the geometry of a specific embedding version, so retraining or updating the upstream model silently shifts the feature space and can degrade the downstream model unless they are retrained together. You must pin and track which embedding version produced training data and ensure the exact same version serves at inference, or you get embedding version skew, a form of training-serving skew. There is also cascading retraining: an upstream change should trigger downstream revalidation. Latency stacking matters online, since each prediction now requires an upstream inference call or an embedding lookup, adding to the budget; precomputing and caching embeddings helps. Monitoring must cover drift in the upstream output distribution, not just inputs. Lineage, rollback, and ownership across teams become harder because a regression may originate one or two models upstream.

COMMON WRONG ANSWERS Treating the embedding as a static, frozen column; ignoring that upstream retraining changes feature semantics; or having no version pinning, drift monitoring, or coordinated rollback.

LIKELY FOLLOW-UPS How to coordinate retraining cadence across teams, whether to precompute versus compute embeddings online, and how to detect upstream-induced regressions.

ONE CONCRETE EXAMPLE A recommender uses user embeddings from an upstream model. The upstream team ships a new embedding model with a rotated space; the recommender's accuracy quietly drops because it was trained on the old space. With version pinning, the recommender keeps consuming the pinned version until it is retrained and validated against the new embeddings, then both are promoted together.