Design an LLM ad copy system with human-in-the-loop

WHAT THIS TESTS: Ability to design a full ML lifecycle for a high-stakes generative application. Interviewers want to see you understand data curation, efficient fine-tuning, inference safety, human oversight integration, and closed-loop improvement. They also care about practical constraints like cost, latency, and brand risk.

A GOOD ANSWER COVERS: First, data preparation and fine-tuning strategy. You should mention curating a dataset of past ad copy paired with metadata like campaign goals and audience segments, then using parameter-efficient fine-tuning such as LoRA or QLoRA to avoid the cost and instability of full fine-tuning. Second, inference architecture with guardrails. Describe a serving layer that applies automated checks for brand voice, policy compliance, and toxicity before any human sees the output. Third, human-in-the-loop workflow. Explain a review interface where copywriters or brand editors can approve, reject, or edit generations. The system should route high-uncertainty or high-stakes outputs to humans while auto-approving low-risk variants based on a calibrated confidence score. Fourth, feedback conversion and retraining. Detail how human actions become structured data. Rejections and edits create preference pairs for reward modeling or direct RLHF updates. Approved copy enters the golden dataset for future supervised fine-tuning. Fifth, evaluation and monitoring. Mention offline metrics like perplexity and relevance scores, plus online metrics such as human approval rate and time-to-publish.

COMMON WRONG ANSWERS: Proposing full fine-tuning of a large base model without discussing GPU cost, checkpoint storage, or catastrophic forgetting. Treating human review as a final sign-off with no feedback loop into model training. Ignoring inference guardrails and assuming the fine-tuned model will always be safe. Failing to separate training data from test campaigns, which leaks future information into training. Neglecting latency implications of synchronous human review in a real-time ad serving path.

LIKELY FOLLOW-UPS: How do you handle disagreements between human reviewers? How do you measure if the model is actually improving versus simply overfitting to one editor's style? What is your rollback strategy if a bad generation slips past the human reviewers? How do you scale this from ten campaigns to ten thousand without linearly scaling human headcount?

ONE CONCRETE EXAMPLE: Imagine a retail company generating holiday ad headlines. The pipeline starts by fine-tuning a seven billion parameter model with LoRA on five thousand historically approved headlines. At inference, the system generates ten variants per product, filters them through a toxicity classifier and a brand-style checker, then sends the top three to a copywriter dashboard. The copywriter approves one headline and tweaks another. The approved headline is stored as positive preference data. The gap between the generated and tweaked version is logged as a correction pair. Every weekend, a reward model is retrained on the past week's preference pairs, and the LoRA adapter is updated via RLHF. Over a quarter, the human edit rate drops from sixty percent to fifteen percent while click-through rates on AI-generated copy match manually written baselines.