Skip to content
tezvyn:

SQL or NoSQL for high-volume semi-structured event ingestion?

Source: aws.amazon.comMediumHow cards are made

SQL or NoSQL for high-volume semi-structured event ingestion?
Summary

schema flexibility and write throughput for raw event ingestion.

Key points

Choose NoSQL for schema-less landing; use SQL downstream for structured analytics.

Watch out for

Picking SQL for raw clicks because ACID is needed.

What's really being asked

This question probes your ability to match storage technology to data shape and access patterns. The interviewer wants to see that you distinguish between a high-velocity, schema-flexible ingestion layer and a structured analytics layer. They are looking for pragmatism over dogma: can you justify why rigid schemas and ACID guarantees are often liabilities for raw event firehoses?

The full answer

First, pick NoSQL for the landing zone and justify it with three specific traits: schema flexibility because mobile app events add new fields without warning; horizontal scalability to absorb millions of writes per second across many nodes; and low-latency ingestion because the pipeline should not block on complex transactions. Second, name a concrete NoSQL variant, such as a document store like MongoDB or DynamoDB, or a wide-column store like Cassandra, explaining that JSON-like or key-value shapes map naturally to semi-structured click payloads. Third, explicitly state that SQL is not wrong forever; it belongs downstream after cleaning and schema enforcement, where relational algebra, joins, and strong consistency become valuable. Fourth, mention cost and operational realities: sharding a relational database for pure write throughput is harder than scaling out a NoSQL cluster.

The mistakes people make

The biggest red flag is choosing SQL for the raw landing zone and defending it with generic ACID arguments. Event logs are typically append-only and immutable; they do not need multi-row transactions or strict relational integrity at ingestion time. Another red flag is treating NoSQL as a magic word without explaining which type fits: saying NoSQL without distinguishing between document, key-value, or columnar stores shows shallow knowledge. A third mistake is ignoring the two-tier architecture entirely and insisting one database must handle both ingestion and analytics.

What usually comes next

The interviewer may ask how you would handle late-arriving events or schema evolution. They might probe how you guarantee at-least-once delivery before the landing zone. They could also ask for a concrete comparison of DynamoDB versus Cassandra for this use case, or how you would migrate from the NoSQL landing zone to a relational data warehouse.

A concrete example

Imagine a mobile game sending JSON click events with fields like timestamp, user_id, event_type, and a nested properties bag. On launch day the app adds a new field called ab_test_variant. A document database ingests the new field immediately with no migration. A relational database would require an ALTER TABLE command that locks millions of rows and breaks the ingestion pipeline. After twenty-four hours a batch job normalizes the most common fields into a PostgreSQL warehouse for cohort analysis, leaving the raw JSON in cold storage for replay.

Interview question

When designing a pipeline to ingest millions of semi-structured mobile events per second, why is NoSQL typically chosen for the initial landing zone?

  • a.It requires multi-row ACID transactions to guarantee exactly-once ingestion of immutable logs.
  • b.It performs relational joins and complex aggregations faster than a data warehouse on raw JSON.
  • c.It enforces strict schemas upfront to prevent bad data from entering the pipeline.
  • d.It allows schema flexibility and horizontal write scaling without blocking ingestion during field additions.Correct
Why?

NoSQL is preferred because document stores handle new fields instantly and scale writes horizontally, avoiding the locking migrations a relational schema would require. The tempting ACID distractor is wrong because immutable append-only event logs do not need multi-row transactions at ingestion time.

Just read this? Test yourself on what you have been reading.

Read the original → aws.amazon.com

You just looked this up. Could you explain it out loud?

That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.

The iPhone app is on the way

We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.

Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.

Get it on Google PlayiPhone app coming soon

We are hiring for this. Open roles that interview on system design — each one lists the topics its interview covers.

See open roles