# ========================================================== # B2B Optimized Robots.txt for minute-ly.com # Allows Amazon Ads and AI Discovery while protecting data. # ========================================================== # 1. Amazon Ad Verification (Your specific request) User-agent: AmazonAdBot Disallow: Crawl-Delay: 1 # 2. AI Search & Discovery (Allowing GPT and Claude to "know" you) # OAI-SearchBot is for real-time search in ChatGPT User-agent: OAI-SearchBot Allow: / # GPTBot is OpenAI's general crawler User-agent: GPTBot Allow: / # ClaudeBot is for Anthropic's AI User-agent: ClaudeBot Allow: / # Perplexity is a major AI search engine for B2B User-agent: PerplexityBot Allow: / # 3. Content Signals (The 2026 standard for intent) # We tell bots: Search is YES, AI Training is NO. # This means they can summarize you for users, but not "steal" your data. User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / # 4. Standard Search Engines User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / # 5. Sitemap link Sitemap: https://www.minute-ly.com/sitemap_index.xml