How do you create a 'golden record' from fragmented data?
Tests your ability to design a data reconciliation system. A great answer outlines a process: profiling sources, defining survivorship rules, implementing a matching engine, and creating a stewardship process. A red flag is suggesting a simple one-off script.
WHAT THIS TESTS: This question evaluates your understanding of Master Data Management (MDM) as a system, not just a task. The interviewer is looking for your ability to think through the entire lifecycle of a data entity: discovery, ingestion, standardization, conflict resolution (survivorship), and governance. It tests if you appreciate the socio-technical challenges, like getting business stakeholders to agree on rules, as much as the technical ones, like probabilistic matching.
A GOOD ANSWER COVERS: A senior-level answer should walk through a systematic approach. First, DISCOVERY AND PROFILING: Identify all systems holding customer data (CRM, billing, support) and analyze their quality, completeness, and update frequency. Second, DEFINE SURVIVORSHIP RULES: Work with business owners to create a hierarchy of trust. For example, the billing system's address is the 'winner' over the CRM's, but the CRM's phone number might be more current. These rules can be attribute-specific. Third, IMPLEMENT THE MDM ENGINE: This involves a) Standardizing data (e.g., 'St.' vs 'Street'), b) Matching records using deterministic (e.g., email) and probabilistic (e.g., fuzzy name + address) logic, and c) Merging matched records into a 'golden record' based on the survivorship rules. Fourth, ESTABLISH GOVERNANCE AND STEWARDSHIP: Create a process for handling records that the engine can't resolve automatically, often with a UI for data stewards to manually review, merge, or un-merge records. This feedback loop also improves the matching algorithm over time.
COMMON WRONG ANSWERS: A junior answer focuses only on the technology, for example, "I'd write a Python script with Pandas to join the tables." This completely misses the core challenges: what's the join key when there isn't one? How do you handle (Jon Smith, 123 Main) vs (Jonathan Smith, 123 Main St)? Another red flag is treating it as a one-time cleanup project. A golden record system is a living product that requires continuous monitoring and governance. Failing to mention the need for business stakeholder buy-in on survivorship rules is a major omission.
LIKELY FOLLOW-UPS: "How would you handle a situation where two records are incorrectly merged? What's the rollback or un-merge strategy?" (Tests understanding of data lineage and auditability). "How would you measure the success of this MDM system?" (Tests focus on business value, e.g., "reduction in returned mail by 15%", "increase in match rate from 80% to 95%"). "What if the 'source of truth' for an attribute changes? How does your system adapt?" (Tests system flexibility).
ONE CONCRETE EXAMPLE: For a customer entity, we might find our Salesforce CRM has the most up-to-date email addresses because the sales team constantly verifies them. However, our Zuora billing system has the most accurate physical addresses, as they are validated by payment processors. My survivorship rule would be: for the 'email' attribute, trust Salesforce. For the 'physical_address' attribute, trust Zuora. For 'phone_number', we might take the most recently updated one across all systems. A record from Zendesk might be given the lowest priority for all attributes unless it's the only source available.
Read the original → en.wikipedia.org
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.