More in Backend Dev — page 53
Row vs. Columnar Storage: Organizing Data for Speed
Row stores group data by record, like a phone book entry. Column stores group by attribute, like separate lists for all names. Use row stores for transactions (OLTP), but for analytics (OLAP), they force you to read unneeded data from disk.
Storage Engine: The Database's Filing System
A database's storage engine is its specialized filing system, handling how data is physically written to and read from disk. Different engines optimize for different tasks, from fast writes to complex queries.
Predicate Pushdown: Filter Data at the Source
Predicate pushdown tells the database to filter data at the source, not after fetching it. This speeds up queries in data warehouses and lakehouses by reducing network traffic. The main footgun: not all data sources can execute all types of filters.
Cardinality Estimation: How Databases Guess Query Costs
A database's query optimizer guesses how many rows each part of a query will return to pick the fastest execution plan. This guess, cardinality estimation, is key for choosing join strategies.
Database Statistics: The Query Optimizer's Internal Map
Database statistics are the raw data the query optimizer uses to guess the cheapest way to run your query. It uses stats like row counts and value distribution to decide between a full table scan and an index seek.
Composite Indexes: One Index for Multiple Columns
A composite index is like a phone book sorted by last name, then first name. It's for queries filtering on multiple columns, like finding a specific person. The footgun is creating separate indexes, which is far less efficient than a single composite one.
The Query Optimizer: Your Database's Internal GPS
A query optimizer is your database's internal GPS, turning your SQL "what" into the fastest "how." It chooses the best execution plan—like join order or index usage—for every query. The footgun: stale statistics can trick it into picking a slow route.
Row Mode vs. Batch Mode Execution in SQL Server
Row mode processes data one row at a time, like a checklist. Batch mode processes chunks of rows together for vectorized speed. Row mode is classic for OLTP, while batch mode shines in data warehousing for large scans.
B-Tree: The Workhorse of Database Indexes
A B-tree is a self-balancing tree that keeps data sorted for fast lookups, generalizing a binary search tree by allowing nodes to have many children. It enables searches, insertions, and deletions in logarithmic time, making it ideal for large datasets.
Query Execution Plan: The Database's Road Map
A query execution plan is the database's internal strategy for fetching your data. It's the recipe it creates before running your SQL. This plan determines whether to use an index or scan a whole table, directly impacting performance.
Serializable Snapshot Isolation: True Serializability Without Heavy Locking
SSI upgrades Snapshot Isolation to true serializability. It optimistically lets transactions run, but aborts one if a dangerous read-write dependency arises. This prevents subtle data corruption in systems like PostgreSQL without heavy locking.
Write Skew: The Phantom Anomaly of Snapshot Isolation
Write skew is when two transactions read the same data, make decisions, and then update *different* data, violating a business rule. It's common in booking systems or when enforcing multi-row constraints under Snapshot Isolation.
Snapshot Isolation: A 'Photo' of Your Database
Snapshot Isolation gives a transaction a private 'photo' of the database from when it started, ensuring consistent reads. It's used in high-concurrency systems to prevent readers from blocking writers. The footgun is that it doesn't prevent all anomalies.
Timestamp Concurrency Control: No Locks, Just Time
Timestamp-based concurrency control bets that transaction conflicts are rare, using timestamps to order operations instead of locking data. It's used where lock overhead is high, but the footgun is that frequent conflicts can cause transaction starvation.
Strict Two-Phase Locking (S2PL): Safety Over Speed
Strict Two-Phase Locking (S2PL) forces a transaction to hold all its locks until it fully commits or aborts. This prevents cascading aborts in databases but at the cost of concurrency, as other transactions are blocked for longer periods.
MVCC: Read and Write Data Without Blocking Each Other
MVCC avoids slow, traditional locks by giving each transaction its own consistent data snapshot. This allows readers and writers to work at the same time without blocking each other, boosting performance in databases like PostgreSQL.
Database Deadlock: The Two-Way Standoff
A deadlock is a 'Mexican standoff' where two transactions can't finish because each is waiting for a resource the other has locked. This happens in systems with concurrent writes. The database will kill one transaction, forcing your app to handle the retry.
Two-Phase Locking (2PL): Preventing Database Race Conditions
2PL is a database's pessimistic strategy for safe concurrency. A transaction acquires all necessary locks before releasing any, ensuring operations don't clash. It's used to guarantee consistency.
Shared & Exclusive Locks: The Read vs. Write Rule
A Shared (S) lock is like many people reading a library book at once; an Exclusive (X) lock is one person writing in it alone. Databases use S/X locks to manage concurrency, preventing writes from corrupting reads.
Database Transaction Log: Your System's Safety Net
A transaction log is your database's safety journal. Before changing data, it records the intended action in a durable file. This is vital for crash recovery, ensuring data isn't left corrupt. The footgun: its primary role is integrity, not just auditing.