Cloudflare Makes Web Search a Control Plane Primitive for Agents

Cloudflare shipped a Web Search API inside AI Gateway today, giving agents a proper retrieval entry point instead of the URL-guessing that has quietly become the default failure mode of browsing agents. The product opens in beta with three search backends - Ceramic.ai, Exa, and Linkup - behind a single provider parameter, and it runs through the same gateway that already carries model inference traffic.

What Cloudflare actually shipped

The pitch starts from an observed pathology. As the Cloudflare team puts it, when an agent needs a live page it "usually just guesses the URL of the page and then makes a tool call to curl it," which is why 404s keep showing up in agent traces. Web search replaces guesswork with a query, and returns structured results - titles, URLs, and descriptions - that can be dropped straight into a model's context.

The API is callable three ways: a REST endpoint, a Workers binding, and through AI Gateway itself, which Cloudflare describes as "the control plane for your applications." Search calls appear in the same logs and analytics as inference calls, draw down AI Gateway credits at each provider's list price with no additional markup, and support bring-your-own-key. All three launch partners - Ceramic.ai, the default, with an independent index Cloudflare cites at more than 40 billion pages; Exa; and Linkup - support zero data retention on requests made through Cloudflare. It lands in the same week that every major cloud raced to build sandboxes for agents, and it follows the same logic: own the layer agents have to pass through.

Cloudflare also said server tools are coming: tools defined and hosted by the gateway, so developers stop declaring them in the harness. Web search will be among the first.

Why interchangeable backends matter more than the search

Published pricing is the part builders should read twice. Linkup runs $5.00 per 1,000 requests and Exa $7.00 per 1,000 at list, with Ceramic.ai positioned as the default. That turns grounding from a vague line in an AI budget into arithmetic: a thousand retrievals costs less than a single mid-size inference job, and the number now lives in the same billing system as your model spend.

Equally important, all three providers return results in the same format, so swapping search backends is a string change rather than a refactor. That is the commoditisation that already happened at the model layer, where a gateway routes across vendors, applied one layer down to retrieval. It also contrasts with the provider-native approach AI Gateway already proxied - Anthropic's web search tool, OpenAI's web search preview, xAI's web search, and Alibaba's search flag - where the search capability is welded to the model vendor you picked. The new API decouples the two: the model can come from anywhere and the grounding can come from anywhere else.

The crawler deals are the real strategy

Every provider in the launch committed to Cloudflare's verified bot requirements, and results must carry a link to the source of the crawled content. On its own that reads as a compliance footnote. In context, it is the more consequential half of the announcement.

Cloudflare is the company whose bot management layer sits in front of a large share of the public web, and it has been among the loudest voices arguing that AI crawlers need identity, rules, and consequences. Its verified-bots documentation now classifies crawlers by behaviour - Search, Agent, Training, Transact, Data Collection and more - and by operator type, separating direct operators from intermediary agentic services that act for many end users at once, an arrangement Cloudflare flags as creating transitive trust problems. By making gateway participation conditional on that regime, Cloudflare converts a policy argument into a distribution requirement: search vendors that want access to agent traffic through the busiest control plane would rather meet the standard than be shut out of it.

The result is a two-sided trade. Content owners get crawlers that identify themselves, respect robots.txt, and return citations. Search providers get a billing and observability channel into every agent that uses the gateway. Cloudflare gets to be the layer where both sides meet, without shipping a search index of its own for this product.

What this means for builders

• Retrieval is now configuration. One provider field selects the backend and the response contract does not change. Multi-provider fallback for search becomes a weekend task instead of a migration.
• Grounding has a unit price. Five to seven dollars per thousand searches at list, or your own key. Budget retrieval per agent run, cache aggressively, and stop treating search as unlimited.
• Traces converge. Search and inference land in one log stream with shared cost attribution, which is the missing piece for anyone trying to explain what an agent run actually cost.
• Compliance gets easier at launch. Zero data retention across all three providers, plus per-provider access control, means security review answers itself for this slice of the stack.
• Lock-in moves up a layer. Models were already portable and retrieval just became portable, but policy, logs, and budget now accumulate in the gateway. That is where switching costs will live.
• Server tools are the thing to watch. If tools migrate from your harness into the control plane, orchestration work shifts again, this time away from client-side tool registries - the same drift toward a platform-owned operating system for agents that has been building all year.

What to watch next

Whether competing gateways match search routing, whether verified-bot compliance quietly becomes mandatory for content that wants to be visible to agents, and how fast prices move now that three search vendors compete on identical footing inside one API. Cloudflare's own framing - more time building rather than orchestrating the harness - previews the next fight: not whose model wins, but who owns the plumbing between the model and the web.