All bites
The whole library, newest first. Filter by what you are here for, or pick a topic if you already know.
4330 bites
Page 80
Attribute cloud costs to teams
Tag resources with team and project metadata, activate them as cost-allocation tags, group the cost report by that tag, and enforce tagging with policy.
On-Demand vs Reserved vs Spot pricing models
On-Demand is flexible but priciest, Reserved trades a 1-3 year commitment for discounts, Spot is cheapest but interruptible.
Rightsizing an underutilized VM fleet safely
Gather multi-week percentile metrics across CPU, memory, network and disk; pick smaller or right-family types; roll out gradually with monitoring.
Lifecycle storage tiering for compliance logs
Hot tier for 7-day query window, lifecycle rules transitioning to infrequent-access then archive, expiration at one year.
Reducing cross-region data transfer costs
Identify cross-region, cross-AZ, and internet egress; co-locate chatty components; add VPC endpoints, CDN caching, and compression.
Savings Plans vs Reserved Instances for mixed compute
Compute Savings Plans cover EC2, Fargate, and Lambda flexibly; EC2 Instance Plans and RIs trade flexibility for slightly deeper discounts.
Designing a multi-account cloud chargeback model
Account-per-team or mandatory cost-allocation tags enforced by SCPs and tag policies, plus a pipeline over the cost and usage report grouped by tag/account.
Resilient stateful batch on Spot Instances
Externalize state and checkpoint to durable storage, react to interruption and rebalance notices to drain gracefully, diversify instance pools.
Data lake versus data warehouse
Lakes store raw, schema-on-read data of any type cheaply; warehouses store curated, schema-on-write structured data for fast SQL; choose a lake for varied raw data and ML.
ETL versus ELT in cloud data platforms
ETL transforms before loading into the target; ELT loads raw first then transforms in the warehouse, leveraging cheap storage and elastic compute.
CSV vs JSON vs Parquet for analytics
CSV and JSON are row-based, human-readable, and bulky; columnar Parquet/ORC compress well and read only needed columns; choose columnar for analytics.
Partitioning order events in a data lake
Partition by the columns queries filter on, typically date hierarchy and category, balancing granularity to avoid too many tiny files.
Diagnosing and fixing data skew in Spark
This is data skew, caused by uneven key distribution concentrating rows on few partitions; mitigate with salting, broadcast joins, repartitioning, or adaptive execution.
Handling late data in streaming windows
Use event-time windows with watermarks to bound lateness, allow a grace period before finalizing, and route data later than that to a side output.
Idempotency in data ingestion pipelines
Idempotency means re-running a step yields the same result with no duplicates; it matters because retries and at-least-once delivery are inevitable; achieve it with deduplication keys or upserts.
The small files problem in data lakes
Too many tiny files inflate metadata and per-file overhead, slowing queries; caused by streaming micro-batches and over-partitioning; fix with compaction and table formats like Iceberg, Delta, or Hudi.
Schema evolution without rewriting history
Use a table format with metadata-level evolution, add a new column rather than mutating the old, and reconcile types at read time; avoid rewriting petabytes.
Pre-built AI service vs custom model
Choose a managed service for speed, no ML expertise, and common tasks; build custom for domain-specific needs, control, or cost at scale.
Feeding large object-store data into training
Stream data instead of copying it all to disk, use streaming/pipe modes, shard and prefetch in parallel, and pack many small images into larger files.
Deploying a real-time inference endpoint
Package the model artifact and inference code in a container, choose instance type and autoscaling, configure the endpoint with health checks, and plan safe rollout like canary plus monitoring.