# ============================================================================== # SAIKATDAS.COM - PRODUCTION ROBOTS.TXT (2026 AEO, GEO & AI SEARCH COMPLIANT) # ============================================================================== # 1. πŸš€ Allow Real-Time AI Answer Engines & Generative Search Bots (Perplexity / ChatGPT Search / Google AI / Apple / DuckDuckGo) User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: DuckAssistBot Allow: / User-agent: Google-Extended Allow: /$ Allow: /sports-photographer-india Allow: /advertising-photographer-india Allow: /identity Allow: /collaborate Allow: /editorial Allow: /commercial-photographer- Allow: /llms.txt Disallow: /uploads/ Disallow: /webp/ User-agent: Applebot-Extended Allow: /$ Allow: /sports-photographer-india Allow: /advertising-photographer-india Allow: /identity Allow: /collaborate Allow: /editorial Allow: /commercial-photographer- Allow: /llms.txt Disallow: /uploads/ Disallow: /webp/ # 2. 🌐 Standard Search Engines (Google, Bing, Apple) User-agent: * Allow: / Allow: /llms.txt Disallow: /dashboard/ Disallow: /client/ Disallow: /photographer/ Disallow: /includes/ Disallow: /ftp_ingest/ Disallow: /uploads/ Disallow: /_sys/ Disallow: /scratch/ Disallow: /*?search= Disallow: /*?ajax= # 3. πŸ›‘οΈ Selective GEO Text Grounding & Raw Image Scraper Quarantine User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Meta-ExternalAgent Allow: /$ Allow: /sports-photographer-india Allow: /advertising-photographer-india Allow: /identity Allow: /collaborate Allow: /editorial Allow: /commercial-photographer- Allow: /llms.txt Disallow: /uploads/ Disallow: /webp/ Disallow: /archive/ Disallow: /dashboard/ Disallow: /client/ User-agent: CCBot User-agent: Bytespider User-agent: Amazonbot User-agent: FacebookBot User-agent: Diffbot User-agent: Scrapy Disallow: / # 4. πŸ—ΊοΈ Dynamic XML Sitemap Index Sitemap: https://www.saikatdas.com/sitemap.xml