Updated 2026-10-05
A web search API is a live web index your agent queries for ranked URLs and snippets. A scrape or fetch tool reads pages you already know. RAG over your own docs answers from private material, not the open web. Pick search when the agent must discover sources, scrape when you have the URL, RAG when the corpus is yours. Cloudflare's Web Search API (via AI Gateway, open beta since Oct 2, 2026) and standalone options like Tavily, Exa, and Brave all solve discovery. They do not replace each other one for one.
Last verified: Oct 5, 2026. Prices and launch facts below are tied to the official links on this page (Cloudflare providers and Birthday Week blog, Exa pricing, Tavily credits, Brave Search API). Native Cloudflare "server tools" are still listed as coming soon, so you orchestrate the tool call yourself today. Treat any comparison older than about two months as stale, including this one if you find it later.
Search vs scrape vs RAG
Before you buy an API key, name the job. Most "my agent needs the web" tickets are actually one of these three.
| Approach | What it does | When to pick it |
|---|---|---|
| Web search API | Query a live index; get ranked titles, URLs, snippets | Agent must find sources it does not already know |
| Scrape / fetch | Fetch and clean a known URL (or crawl from a seed) | You already have the page; you need its body |
| RAG over your docs | Retrieve chunks from your corpus, then generate | Answers must come from private or controlled material |
Search discovers. Scrape reads. RAG remembers your stuff. An agent that guesses URLs needs search first. An agent that already holds a permalink needs fetch. An agent answering from SOPs and tickets needs RAG, not a public index. For scrape oriented tooling, see Firecrawl Free Keyless. Broader tool context lives in the AI tools hub.
Steal this rule: if the model invents links, add search. If the model has the link but invents the body, add fetch. If the answer must stay inside your files, add RAG.
Category table: how the search products differ
| Option | Category | How you call it | Billing shape (verified Oct 5, 2026) | Best fit |
|---|---|---|---|---|
| Cloudflare Web Search API | Aggregator via AI Gateway | REST or Workers env.AI.websearch | Partner list price, no Cloudflare markup; BYOK allowed | You already use AI Gateway and want one bill plus logs |
| Ceramic.ai (via Cloudflare) | AI oriented web index | provider: ceramic (default) | $0.25 per 1,000 requests | Many agent loops; cheap grounding |
| Linkup (via Cloudflare) | AI search with sourced snippets | provider: linkup | $5.00 per 1,000 requests | Fast cited results for tool calls |
| Exa (via Cloudflare or direct) | Neural + keyword AI search | Cloudflare provider: exa, or Exa /search | $7.00 per 1,000 requests (up to 10 results on auto/fast) | Query relevant highlights into context |
| Tavily | Agent first search + extract | Tavily /search (and related) | Credits: free 1,000/mo; basic/fast/ultra-fast = 1 credit; advanced = 2; pay as you go $0.008 per credit | LangChain style agents that want search + extract in one vendor |
| Brave Search API | Independent web index | Brave REST (/web/search, LLM context, Answers) | Search $5 per 1,000 requests | You want an independent index outside Cloudflare's partners |
Cloudflare partners at launch are Ceramic.ai, Exa, and Linkup (providers docs). Responses must include a link to the source. Partners commit to Cloudflare's Verified bot crawling rules. Ceramic and Linkup are marked Zero Data Retention Yes on that providers page; Exa is marked No for ZDR there, which is the detail most launch posts gloss over.
Cloudflare vs Tavily / Exa / Brave
Cloudflare Web Search API (via AI Gateway)
Cloudflare's product is not a fourth index. It is a control plane: one REST shape, one Workers binding, AI Gateway logs, and credits that debit at each partner's list price with no extra markup (Web Search overview, Birthday Week announcement). You can also bring your own provider key.
What doesn't get said. Native server tools are still "coming soon," so today your app (or agent runtime) must call web search, then pass results back into the model as context. You also inherit each provider's quirks: Ceramic returns long descriptions (up to 8,000 characters), Exa through the gateway uses Exa's auto mode with highlights as the description, Linkup uses fast depth with raw results (no generated answer). Switching provider keeps the response shape, not the ranking quality.
Verdict: pick Cloudflare when AI Gateway is already your inference front door and you want search in the same observability and credit pool. Avoid treating it as a quality ranking across Ceramic, Exa, and Linkup; run your own evals.
Ceramic.ai through Cloudflare
Default provider. Independent index oriented at agents and LLMs, more than 40 billion pages per Cloudflare's docs. Price is the standout: $0.25 per 1,000 requests.
What doesn't get said. Cheap search still burns budget when an agent loops twenty queries per task. Long descriptions are generous for grounding and expensive for context windows if you dump five full descriptions every turn.
Verdict: best default for high volume grounding and "did the model invent this URL?" checks. Avoid it if you need Exa style neural retrieval features that only show up on Exa's own API surface.
Linkup through Cloudflare
$5.00 per 1,000 requests, Zero Data Retention yes, fast search depth, sourced snippets without a vendor written answer.
What doesn't get said. Fast mode is the gateway path; if you need Linkup's slower or answer generating modes, call Linkup directly. Cloudflare's wrapper is optimized for tool call style results, not deep research reports.
Verdict: solid middle price cited results for agents that must show sources. Avoid it if you already standardized on Exa highlights or Brave's LLM context endpoint.
Exa through Cloudflare (and direct)
Through Cloudflare: $7.00 per 1,000, auto search, highlights as descriptions. Direct Exa /search on fast or auto is the same $7 per 1,000 for up to 10 results with text and highlights included (Exa's instant mode is cheaper at $4 / 1k on Exa's own pricing page; extra results and summaries bill separately).
What doesn't get said. Cloudflare's providers table marks Exa Zero Data Retention as No, unlike Ceramic and Linkup. If ZDR is a hard requirement, do not assume the gateway magically adds it. Deep modes (Deep Lite, Deep, Deep Reasoning) live on Exa's own API, not on the Cloudflare wrapper's launch surface.
Verdict: pick Exa when natural language queries and tight highlights matter more than the lowest unit price. Prefer direct Exa when you need deep modes or content controls the gateway does not expose yet.
Tavily
Built as search for LLM agents, with extract, crawl, and map siblings. Search depth trades cost for relevance: per Tavily's credits docs and search API, basic, fast, and ultra-fast cost 1 credit; advanced costs 2. Free plan includes 1,000 credits per month; pay as you go is $0.008 per credit (plans).
What doesn't get said. Credit math is easy to undercount once you turn on include_raw_content, advanced depth, or Research endpoints (Research has its own min/max credit bounds). Tavily is a full retrieval toolkit, not only a SERP JSON dump.
Verdict: pick Tavily when your stack already expects an agent search SDK and you want search plus extract from one vendor. Avoid it if you only need an independent index and already pay Brave or Cloudflare credits.
Brave Search API
Independent index (Brave quotes 40+ billion pages). Search plan list price $5 per 1,000 requests. Separate Answers product is $4 per 1,000 plus $5 per million input/output tokens on that same page. LLM context endpoint exists for grounding style payloads.
What doesn't get said. Brave is not a Cloudflare AI Gateway partner in this launch. You manage a separate key, bill, and rate limit. Storage rights for training require a plan that explicitly grants them.
Verdict: pick Brave when independence from Google/Bing scrapers and from Cloudflare's partner set matters. Avoid it if one consolidated AI Gateway bill is the whole point.
Grounding with citations
The useful pattern is boring on purpose:
- The model decides it needs live evidence and calls a
web_searchtool with a concrete query. - Your runtime hits Cloudflare Web Search, Tavily, Exa, or Brave and gets titles, URLs, and snippets.
- You pass those results back as tool results (or a short context block), not as silent system prompt folklore.
- The model answers and cites the URLs it actually used.
That is the same discipline as RAG, except the corpus is the public web for this turn. Search without citations just moves the hallucination from facts to "sources." Scrape without search only helps after discovery. Keep the three layers separate in your agent graph: discover (search), read (fetch), remember (RAG).
Which should I pick?
| If you need... | Start with | Skip when... |
|---|---|---|
| One bill and logs next to model calls | Cloudflare Web Search API (Ceramic default) | You refuse any Cloudflare dependency |
| Cheapest high volume grounding | Ceramic via Cloudflare ($0.25 / 1k) | You need Exa deep modes |
| Cited fast tool results at mid price | Linkup via Cloudflare or Brave Search ($5 / 1k) | You need ZDR and were about to pick Exa via Cloudflare without reading the ZDR column |
| Neural search + highlights | Exa ($7 / 1k on auto/fast) | ZDR is mandatory on Cloudflare's path |
| Search + extract + crawl in one agent SDK | Tavily | You only need SERP style JSON |
| Private company answers | RAG over your docs | You are trying to replace a shared drive with Google results |
| Known URL body text | Scrape / fetch | The agent still does not know which URL to open |
Soft next step if you are building agents for real
If you already ship production code and the gap is agents, memory, and retrieval rather than "what is an LLM," compare self paced paths that start at that gap. AI Flex is built for developers who want agents and RAG without a fixed cohort lock in. Use the program comparison to see which program fits before you buy another API key you will not wire into a real loop.
FAQ
What is a web search API for an AI agent?
It is an HTTP API that returns ranked live web results (titles, URLs, snippets) your agent can pass into a model as evidence. It is discovery, not a full page download and not private document search.
How is a web search API different from scraping?
Search finds candidate URLs from a query. Scraping fetches and cleans pages you already chose. Agents that skip search and guess URLs fail differently from agents that skip fetch and invent page contents.
How is web search different from RAG?
RAG retrieves from a corpus you control (handbooks, tickets, specs). A web search API retrieves from a public index. Many production agents use both: RAG for internal truth, search for fresh external context.
What did Cloudflare launch on Oct 2, 2026?
A Web Search API in open beta through AI Gateway, with partners Ceramic.ai, Exa, and Linkup, callable via REST or the Workers AI binding (announcement, docs). Billing uses AI Gateway credits at partner list prices with no Cloudflare markup, or bring your own key.
How much do Cloudflare's search providers cost?
Per Cloudflare's providers docs (checked Oct 5, 2026): Ceramic $0.25, Linkup $5.00, Exa $7.00 per 1,000 requests when paid with AI Gateway credits.
Should I use Cloudflare or call Tavily, Exa, or Brave directly?
Use Cloudflare when you want gateway logging and a shared credit pool across Ceramic, Exa, and Linkup. Call vendors directly when you need features the gateway does not expose (Exa deep modes, Tavily extract/crawl, Brave Answers) or when you want Brave's independent index outside that partner set.