BotBase now gives bot operators a way in — and a way to see where things stand

Last month, Cloudflare introduced BotBase, a searchable directory of known bots, alongside Business Insights for understanding crawler behavior. The directory was built for website owners deciding which automated traffic to allow. Bot operators, however, were left with a one-way submission form and no visibility into what happened after they clicked submit — no status checks, no rejection reasons, no way to update an existing entry.

BotBase for Operators addresses that gap, starting with transparency. The new tools are available in the Cloudflare dashboard under Protect & Connect → Application Security → BotBase, a new home for bot-related tooling that sits alongside other trust and security features.

A clearer submission workflow

The previous submission form was tucked under Manage Account → Configurations, tying a bot to an account but isolating it from the broader ecosystem. The new BotBase section splits operator tools into three views:

  • Bots directory — browse, search, and filter the bots Cloudflare already tracks (the same catalogue available on Cloudflare Radar).
  • Submission form — submit a new bot.
  • Submission history — track everything you have submitted.

Submission history is the key change. Operators previously had to email support to learn whether a bot had been reviewed or to check progress. Now every submission shows a status: Waiting for review, Accepted, or Rejected. Rejected submissions include the reason and actionable steps for fixing and resubmitting. If Cloudflare adjusted how an accepted bot is classified, the full details show what changed. A "My bots" filter on the directory screen shows all submissions tied to the account you're logged in with.

Editable entries and cancellations

Bot identification details can change — a website redesign might move an IP list to a new endpoint, or an operator might switch from an IP allowlist to Web Bot Auth signing. Previously, reflecting either change meant submitting a brand-new entry via the full form. Operators can now edit a submission they've already made and cancel one that is still waiting for review.

Keeping entries current matters for maintaining Verified status, which increasingly determines whether sites across Cloudflare's network can allow a bot based on its behavior. Ultimately, individual site owners decide what traffic to permit.

A form that describes real behavior

The updated intake form follows the behavior and content use model introduced in July, asking operators to describe their bot across three dimensions rather than assigning a single label.

  • What the bot does — indexing pages for search, acting as an agent on a user's behalf, collecting data, training models, supporting SEO tools, or any combination. Operators can select every applicable behavior.
  • How it uses content — the level of content use, using the same Content Signals model website owners use to set rules. A robot's declaration is checked against preferences like Content-Signal: search=yes, ai-train=no, use=reference.
  • Who runs it — whether the operator is direct (running the bot from their own infrastructure, e.g., a search engine building its own index) or an intermediary (running a platform that other companies build on, where traffic is sent by a third party's product, e.g., an AI assistant API fetching a page because someone typed a query into a different app built on it).

Faster, automated review

Bot submissions have grown roughly 7 times in volume since 2023, making fully manual review unsustainable. The review process now runs automatically, checking for duplicates, user-agent pattern specificity, and — most importantly — whether the claimed verification method holds up. Cloudflare fetches IP lists, confirms reverse DNS, or validates Web Bot Auth signatures automatically. Submissions that pass can be tracked immediately; those needing a closer look are routed to the team with the specific reason flagged, instead of landing as a blank entry in a queue.

What's next

This launch targets visibility. Upcoming goals include ownership and observability — claiming bot ownership, managing live directory entries, and understanding how websites treat a bot — followed by a longer-term goal of conversation: a sustainable way for bot operators to request access from websites when they can demonstrate value rather than harm.

The BotBase roadmap points toward operators understanding exactly how their bot is treated and getting guidance on more polite crawling, turning a one-way submission into an ongoing relationship.