More in Data Science & Analytics — page 6
Star schema vs snowflake schema: differences and trade-offs
Tests dimensional modeling: star schemas flatten dimensions for fast joins and simple queries, while snowflakes normalize them to cut redundancy at the cost of extra joins. Red flag: praising snowflake storage savings without admitting query overhead.
Describe the difference between ETL and ELT and when to choose each
Tests transform timing and compute location. ETL cleans data before loading via external engines; ELT loads raw data first, then transforms in the warehouse. Pick ETL when pre-load cleansing is needed, ELT when warehouse compute is cheaper.
How would you standardize a 500GB dataset that does not fit in RAM?
This tests two-pass statistics for out-of-core scaling. A good answer outlines: first compute mean and variance via sums and counts; second apply z = (x - mean) / std; mention Dask-ML or PySpark. A red flag is averaging chunk-wise means without weighting.
How do you prevent future leakage in time-series preprocessing?
This tests temporal causality in feature engineering and validation. Use only past data for lags and rolling windows and enforce a rolling validation split without shuffling. Red flags are random k-fold CV and global standardization leaking future information.
Why is scaling unnecessary for trees but critical for SVM or K-Means?
Tests whether you understand model internals. Trees split on rank order, so scale is irrelevant. SVM and K-Means rely on distance or margin geometry, making magnitude dominate.
What is data leakage in preprocessing and cross-validation?
This tests recognition of data leakage through preprocessing statistics. A strong answer defines leakage, describes scaling using global statistics before CV splits, and states transformers must be fit per training fold.
Why avoid one-hot encoding for high cardinality and what are alternatives?
This tests dimensionality explosion and encoding alternatives. A strong answer notes one-hot creates hundreds of sparse binary columns, causing memory bloat and overfitting, then names two strategies like target encoding and count encoding.
Min-Max scaling vs Z-score standardization: differences and algorithm preferences
Tests if you know how feature scaling works and can pair a scaler with algorithmic assumptions. Contrast [0,1] Min-Max against mean-zero Z-score, then defend standardization for PCA or gradient descent.
Design a scalable, fault-tolerant real-time IoT data ingestion system
This tests separation of edge connectivity, buffering, and processing. A strong answer names an edge gateway, Kafka as the backplane, stream processing, and cold storage, plus backpressure and partitioning.
How would you evade an advanced anti-bot system while scraping?
WHAT IT TESTS: Your grasp of transport and behavioral fingerprinting beyond IP rotation. ANSWER OUTLINE: Discuss JA3/TLS spoofing, CDP-based browser automation, human-like mouse paths and delays, and session consistency. RED FLAG: Only proxies, user-agents.
Design an incremental load pipeline from a transactional DB to a warehouse
WHAT IT TESTS: OLTP-to-OLAP sync without full dumps. ANSWER OUTLINE: Contrast timestamp watermarking, CDC from transaction logs, and open-table incremental reads; cite merge logic and idempotency.
Implement OAuth 2.0 flow to get an access token for API requests
Tests your grasp of OAuth 2.0 grant-type selection and token lifecycle. Strong answers match the script context to client credentials or authorization code flow, detail the token endpoint exchange, and address refresh and expiry.
SQL or NoSQL for high-volume semi-structured event ingestion?
WHAT IT TESTS: schema flexibility and write throughput for raw event ingestion. ANSWER OUTLINE: Choose NoSQL for schema-less landing; use SQL downstream for structured analytics. RED FLAG: Picking SQL for raw clicks because ACID is needed.
How would you scrape a page with dynamically loaded JavaScript content?
It tests if you know dynamic pages need a real renderer. A great answer names Playwright or Selenium, uses explicit waits for elements, and extracts via DOM or network interception. Red flag: suggesting only static parsers like BeautifulSoup or blind sleeps.
Design a rate-limited REST API data collection script
Tests client-side throttling discipline versus reactive 429 handling. Strong answers proactively pace calls using rate-limit headers, cap concurrency, and apply exponential backoff with jitter. Red flag: tight-loop retries or ignoring headers.
What is robots.txt, why respect it, and consequences of ignoring it?
WHAT IT TESTS: Your grasp of ethical and legal guardrails in data collection. ANSWER OUTLINE: It disallows crawler paths via the Robots Exclusion Protocol; honoring it prevents server strain, legal risk, and broken trust. RED FLAG: Calling it optional.
How do you fetch JSON from a REST API and parse it?
This tests practical fluency with HTTP mechanics and JSON deserialization. A strong answer names the method, URL, and headers; checks the status code; then parses with r.json() or json.loads. A red flag is skipping error handling or confusing GET with POST.
Find customers who have not placed any orders
Tests SQL anti-join logic. Great answers show two paths: LEFT JOIN plus IS NULL on orders.customer_id, or NOT EXISTS, and mention NULL safety with NOT IN. Red flag: INNER JOIN with DISTINCT, which silently drops customers without orders.
How do you analyze and reduce large pandas DataFrame memory usage?
This tests in-memory representation and systematic optimization. Start with df.info(memory_usage='deep'), downcast numerics with to_numeric, convert low-cardinality strings to category, and use nullable dtypes.
Process a 50GB CSV with only 16GB RAM
WHAT IT TESTS: Streaming aggregation under memory constraints. ANSWER OUTLINE: Chunk with read_csv chunksize, filter columns via usecols, downcast int64 to int32/int16, skip rows. RED FLAG: Loading everything into one DataFrame or using default dtypes.