Cloudflare AI Search connects websites, files, and MCP retrieval to agents

Cloudflare’s AI Search update makes it easier to index sites and structured or unstructured files, then expose public or Access-protected search and MCP endpoints per namespace.

On August 6, 2026, Cloudflare introduced a set of developer-experience updates for AI Search. The product builds an index from websites, files, and other content so applications and AI agents can retrieve relevant material with less custom ingestion and retrieval code. The update covers index creation, search endpoints, and MCP integration.

The new Discover flow can find and parse website content without a sitemap, while AI Search also supports structured and unstructured files. That lowers the barrier to creating an index, but it does not make source content automatically safe or useful for an agent. Teams still need to handle versions, duplicate pages, access permissions, sensitive data, and refresh speed.

Each namespace can expose a public search endpoint and an MCP endpoint, with custom domains available as well. For private data, Cloudflare offers Access protection. Public search and private retrieval should be designed as separate boundaries; an endpoint that is easy to connect is not the same thing as a completed authorization model.

Cloudflare says AI Search uses hybrid semantic and keyword search and follows robots.txt through the Cloudflare-AI-Search bot. Hybrid retrieval can handle both semantically related questions and exact matches for product documentation, API names, and error messages. Its real-world quality will still depend on chunking, metadata, index freshness, and query construction.

Cloudflare also showcased Cloudflare Dev Stack MCP, which uses AI Search across ten documentation, blog, and other content instances. An agent can make one tool call and receive cross-site results with cited chunks. The example highlights that MCP is not only a wrapper around search; it can carry source scope, citations, and retrieval policy into the agent context.

Cloudflare labels AI Search beta and provides a pricing preview. The announcement lists preview prices of $0.75 per million ingestion tokens, an additional $0.50 per million tokens for image ingestion, $2 per GB-month for storage with 10 GB free, $0.75 per 1,000 semantic queries, and $0.10 per 1,000 full-text queries, plus 2,000 free queries. These are preview terms and may change, so production planning should use the latest pricing and quota documentation.

For enterprises, the hard problem is not creating a demo in minutes; it is making retrieval auditable. Teams need to track source URLs, crawl time, document versions, Access policy, cited chunks, deletion and withdrawal procedures, and the agent’s behavior when no source is found. Public endpoints simplify integration, while private endpoints require identity, rate limiting, and sensitive-data classification.

The update places search infrastructure and an agent interface in one product. That may shorten the distance from content to an MCP tool, but quality still depends on index management and data governance. A sensible rollout is to test recall, precision, citation completeness, and authorization on a limited corpus before expanding to a whole site or internal document library.

MODULE.002 //

More insights

Ideas on websites, AI automation, digital marketing, AI news, and VMTS updates.