How would you design an auto-generated sitemap.xml for a large blog?

What's really being asked

This question evaluates whether you can design a production-ready system for automated sitemap generation at scale rather than treating a sitemap as a static artifact. The interviewer cares about your understanding of the sitemaps.org protocol limits, event-driven architecture, and how to guide search engine crawlers without over-promising behavior.

The full answer

First, an async generation pipeline triggered by CMS publish, update, or delete events rather than a nightly cron job, so the sitemap stays fresh without unnecessary rebuilds. Second, a sharding strategy: when the blog exceeds 50,000 URLs or the file nears 50MB, you must split entries across multiple sitemap files and reference them in a sitemap index file. Third, required and optional fields per sitemaps.org: every URL needs a loc tag with a fully qualified, properly URL-escaped and entity-escaped URI under 2,048 characters; lastmod should reflect the actual article modification date in W3C Datetime format; changefreq and priority are optional hints, not commands, with priority ranging from 0.0 to 1.0 relative to your own site. Fourth, hosting and delivery: the file must be UTF-8 encoded, placed at the root or a valid path, and served with a 200 OK so crawlers can fetch it reliably.

The mistakes people make

Treating changefreq or priority as direct instructions that force crawler behavior; they are hints and search engines may ignore them. Generating the sitemap manually or checking it into version control, which creates a maintenance bottleneck and stale data. Ignoring the 50,000 URL or 50MB limit and dumping every article into a single file. Using the sitemap generation timestamp instead of the actual article lastmod date. Failing to entity-escape ampersands or other XML special characters in URLs, which breaks XML parsing.

What usually comes next

How would you handle a multi-region site with hreflang variants in the sitemap? What is your strategy if a sudden bulk import adds 100,000 articles at once? How do you verify the sitemap is valid and monitor whether search engines are actually consuming it? Would you use a static file, an API endpoint that streams XML, or a hybrid approach?

A concrete example

Consider a blog with 80,000 articles. You maintain a database table tracking article_id, canonical_url, updated_at, and is_published. On publish or update, an event triggers a worker that regenerates the affected sitemap shard. Articles are ordered by updated_at and split into two sitemap files of 40,000 URLs each, referenced by a sitemap index at example.com/sitemap_index.xml. Each url entry contains loc with the canonical HTTPS URL, lastmod in YYYY-MM-DD format derived from the articles updated_at column, changefreq set to weekly for active blogs, and priority set to 0.5 for posts and 1.0 for the homepage category landing pages. The worker writes the files to object storage and purges the CDN cache so search engines always receive the latest version.