CabooBot identifies public business facts for Gateway setup.
CabooBot is the crawler Caboo uses when a business owner asks us to set up or maintain an AI booking Gateway. It reads public pages to understand the business name, services, location, structured data, sitemap, robots policy, and booking-readiness signals.
User agent
CabooBot/0.1 (+https://getcaboo.com/bot)
What it fetches
For Gateway setup, the bot can fetch the homepage, robots.txt, sitemap.xml, favicon.ico, and a small number of public pages needed to draft the approved catalog. It also looks for llms.txt because agent-readable pages help AI assistants understand the business. The bot uses bounded timeouts and byte limits.
Provider crawlers Caboo monitors
Beyond fetching the site itself, Caboo may evaluate whether major AI providers' own crawlers can reach the site. Caboo does not impersonate these bots. It reads the site's robots policy and reports which of them are allowed.
OAI-SearchBot OpenAI · ChatGPT Search index ChatGPT-User OpenAI · live user-triggered fetches GPTBot OpenAI · model training Claude-SearchBot Anthropic · Claude search index Claude-User Anthropic · live user-triggered fetches ClaudeBot Anthropic · model training Google-Extended Google · Gemini API grounding (separate from Googlebot) PerplexityBot Perplexity · search index
Blocking OAI-SearchBot, Claude-SearchBot, Google-Extended, or PerplexityBot can reduce whether provider systems can inspect the public site. Blocking the training-only bots (GPTBot, ClaudeBot) is a different decision.
Robots policy
CabooBot respects robots.txt for unverified domains. If a verified site owner wants Caboo to maintain a Gateway from public site content, they can allow CabooBot explicitly or provide approved catalog information directly.
User-agent: CabooBot Allow: /
How to block it
User-agent: CabooBot Disallow: /
Contact
Questions or crawl issues: [email protected].