# withadi.dev # # The line that matters is the last one: without a Sitemap directive there is # nothing on this domain telling a crawler what pages exist. dist/sitemap.xml is # generated from the pages that actually built (src/integrations/for-agents.ts), # and the build fails if this URL and astro.config.mjs's `site` ever disagree. # # The AI-crawler policy is NOT here on purpose. Cloudflare prepends its own # managed block — Content-Signal, and Disallow for GPTBot, ClaudeBot, CCBot, # Bytespider and the rest — to whatever this file says; verified live on # 2026-08-29, its "BEGIN Cloudflare Managed content" marker sitting above this # comment in the served file. A copy of that list lived here for one deploy, to # be safe about which file won, and came straight back out: Cloudflare keeps # theirs current and ours would have gone stale in place. User-agent: * Allow: / Sitemap: https://withadi.dev/sitemap.xml