Intermediate everything in AI & ML, page 4
Prompt injection versus jailbreak, and defenses
Injection hijacks the model via untrusted data overriding developer instructions; jailbreak coaxes a model past its safety policy. Defense: separate trusted instructions from untrusted data and filter.
Designing input and output guardrails for a chatbot
Input guardrails filter or classify user prompts (injection, off-topic, PII) before the model; output guardrails validate responses for toxicity, leakage, and policy before sending.
Reference-free evaluation for open-ended dialogue
ROUGE punishes valid paraphrases; use reference-free LLM-as-judge or learned scorers rating coherence, relevance, and groundedness.
How MMLU works and the contamination problem
MMLU is multiple-choice across 57 subjects scored by accuracy; contamination means test items leaked into pretraining, inflating scores.
How to evaluate a RAG system end to end
Measure retrieval (recall, precision, MRR, NDCG) and generation (faithfulness, answer relevance) separately, plus end-to-end correctness.
How FID is calculated versus Inception Score
FID fits Gaussians to Inception features of real and fake images then measures Frechet distance; it uses real references and detects mode collapse.
Aligning text and image representations
Contrastive learning like CLIP pulls matched image-text pairs together and pushes mismatches apart; alternatively projection layers map one modality into a frozen model's space.
How Stable Diffusion generates images
The text encoder turns the prompt into embeddings, the U-Net predicts noise to remove conditioned on those embeddings, and the scheduler controls how noise is stepped down over iterations in…
Designing a Visual Question Answering system
Encode the image with a vision backbone, encode the question with a text encoder, fuse them via cross-attention into a joint representation, then decode or classify the answer.
Evaluating image generation: FID and IS
FID compares feature distributions of real and generated images, lower is better; Inception Score rewards confident, diverse classes but ignores real data.
Evaluating a RAG system end to end
Measure retrieval with context recall or precision, and generation with faithfulness and answer relevance, attributing failures to the right stage.
Reward models in RLHF and PPO
It learns from human preference comparisons to score responses, then supplies the reward signal that PPO maximizes while a KL penalty keeps the policy near the reference.
Demographic Parity versus Equalized Odds in hiring
Demographic parity equalizes selection rates regardless of qualification; equalized odds equalizes true and false positive rates across groups, conditioning on the true label.
Explain an interaction effect to a non-statistician
Define interaction as it depends on, show separate slope lines per age group, give the business takeaway on targeting.
Present a small but significant A/B test lift
Hypothesis, design and validity checks, result with effect size and interval, business impact of 0.5%, then a clear recommendation.
Interactive versus static plots for EDA
Interactive libraries win for exploring dense, high-cardinality, or multi-dimensional data via zoom, hover, and filtering; static plots win for reproducible, publication output.
Parquet versus CSV for analytical data lakes
Parquet stores by column enabling projection pushdown, compression, and predicate skipping; CSV is row-based, untyped, and slow to scan.
Audit an ML pipeline for GDPR compliance
Inventory data and check minimization, verify processing matches stated purpose, build lineage to trace any prediction's inputs.
A/B test two fraud models in production
Randomize by entity, consider shadow mode first, collect precision/recall and business loss, decide with significance and guardrails.
Communicate a forecast interval to an executive
Give the point estimate but frame the range as scenarios, use a fan chart, tie the interval to planning decisions and risk.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles