Cloudflare AI Search: give your agents a search engine for your data

Cloudflare AI Search turns a namespace into a ready-made agent search layer

Cloudflare has expanded AI Search from a set of primitives developers had to stitch together manually into a managed offering that handles crawling, ingestion, embedding, and retrieval in one step. The service now supports indexing entire collections of structured or unstructured data, and promises embedding and reranking at no cost when using the default Workers AI models. The goal: give an agent a search engine scoped to your own data, with a single endpoint to query it.

New capabilities in AI Search include:

• Index any collection of data: Point the service at files or websites you own so your agent can query across them. Website sources must currently be zones on your Cloudflare account, though additional ownership verification methods are planned.
• No sitemap required: A new Discover parsing option lets you add a website as a source by following links, powered by Browser Run's /crawl, instead of requiring a sitemap.
• Single public endpoint per namespace: Enabling public URLs gives you unauthenticated /search and /mcp endpoints that query across all instances in the namespace — useful for sharing with customers directly.
• Custom domains and access control: You can map your own domain to public endpoints (e.g., search.example.com/mcp) and optionally front them with Cloudflare Access to keep the search private.
• EmDash plugin: Sites built on EmDash, Cloudflare's open-source CMS, can add semantic search over their content through an official AI Search plugin.
• Preview pricing: A new billing model makes embedding and reranking free when you use select models from the Workers AI catalog, removing token-count prediction from the cost equation.

Inside the Cloudflare Dev Stack MCP

Cloudflare's own Dev Stack MCP server, available in the AI Playground, is built on these features. It supplies coding agents with current, cited documentation from the Cloudflare developer ecosystem. The implementation shows how the pieces fit together.

Index each surface

Cloudflare created one AI Search instance per owned surface: Docs, Blog, API Docs, Community, Astro, Vite, Vitest, Hono, Replicate, and OpenNext. Because Cloudflare owns that data, AI Search treats them as one set for ingestion. Creating an instance is a single command, and adding --parse-type discover handles sites without a sitemap:

npx wrangler ai-search create cloudflare-community \
  --namespace dev-stack \
  --source https://community.cloudflare.com \
  --type web-crawler \
  --parse-type discover

Combine instances into one query

Two approaches answer a query across all ten instances:

Option A: In a Worker. Used for Dev Stack MCP, this binds the namespace to a Worker that exposes a remote MCP server, letting search ship as a tool alongside the existing Cloudflare MCP tools. The binding lives in wrangler.jsonc:

{
  "ai_search_namespaces": [
    { "binding": "AI_SEARCH", "namespace": "cloudflare-stack" }
  ]
}

A single tool call then fans out across the named instances:

// One tool, one call that searches every surface in the namespace at once.
context.registerTool(
  'search_dev_stack',
  {
    description: 'Search current docs across the Cloudflare stack.',
    inputSchema: z.object({ query: z.string() }),
  },
  async ({ query }) => {
    const res = await context.env.AI_SEARCH.search({
      query,
      ai_search_options: {
        instance_ids: ['developers-cloudflare-com', 'astro', /* ...every surface */],
        retrieval: { max_num_results: 10 },
        reranking: { enabled: true },
      },
    })
    // res.chunks come back cited and tagged with the instance they came from.
    return { content: [{ type: 'text', text: format(res.chunks) }] }
  }
)

Option B: Public endpoints, no code. Enabling public URLs on the namespace immediately provides /search and /mcp endpoints that query every instance with no authentication or deployment required:

Choose the Worker when folding search into an existing app or MCP server; choose the public endpoint for a one-click shareable search.

Brand and secure the endpoint

Public endpoints support custom domains, so you can expose search.example.com/mcp instead of the default URL. Adding Cloudflare Access in front of the domain requires a login, which keeps queries limited to authorized people or agents:

Using the Dev Stack MCP with your agent

The Dev Stack MCP gives coding agents current, cited answers about building on Cloudflare tools. The practical value comes from wiring it directly into your agent: instead of falling back to web search and then fetching pages — a slow, token-heavy process that can land on stale sources — the agent gets documented answers in one tool call. Drop the Dev Stack MCP URL into your agent's MCP configuration:

{
  "mcpServers": {
    "dev-stack": { "url": "https://stack.mcp.cloudflare.com/mcp" }
  }
}

Search on Cloudflare's own properties

Cloudflare uses AI Search for the Blog, Developer Docs, and Cloudflare.com. All instances run hybrid search, combining semantic and keyword matching in a single query to handle both open-ended questions and exact keyword lookups. The Blog was recently rebuilt on EmDash, and its search is powered by the EmDash AI Search integration; the same plugin is available for any EmDash site.

Bot policies are respected

AI Search identifies itself with its own bot user agent, Cloudflare-AI-Search. It follows robots.txt, uses an immutable user agent, and honors other bot controls, consistent with Browser Run behavior.

Preview pricing: predictable at the model layer

AI Search remains free during beta; billing is not yet enabled and will be announced via email before it starts. The preview pricing model covers ingestion, storage, and queries separately, with embedding and reranking free when using defaults or select Workers AI models. Answer generation and query rewriting are optional, billed as Workers AI usage or through AI Gateway credits with any model or provider.

† A single pool of 5M ingestion tokens per month covers any supported file type (e.g., text, images). ‡ A single pool of 2,000 queries per month is shared across query types.

Example monthly bill

For a new instance indexing a 20,000-document source (~20M text tokens) plus 1,000 images (~1,000 tokens each), with 30,000 semantic queries per month using the default embedding and reranking model, the Workers Paid plan bill looks like this:

Images count toward base ingestion and incur an add-on cost. Storage assumes ~10 KB per document and 1 MB per image. Since indexing is largely one-time, subsequent months are mostly query costs, closer to $21.

Getting started

AI Search is available now. Create an instance with one command, enable hybrid search for semantic and keyword matching, and you have a searchable index ready for agent queries:

npx wrangler ai-search create my-search \
  --namespace my-namespace \
  --source https://my-website.com \
  --type web-crawler \
  --hybrid-search

From there you can query directly, wire it into an agent over /mcp, or put a custom domain on a public /search endpoint to share with users. The AI Search docs cover the full setup.