Can agents get better answers from your site using a purpose-built RAG endpoint vs raw web scraping? We ran 60 test cases across 43 real sites to find out.
60
Test cases
43
Real-world sites
July 16, 2026
Run date
GPT-4o (per-fact)
Judge
Verdict
AgentReady wins on citation rate, hallucination rate, and JS-heavy sites. web_fetch wins on static API docs and latency. Overall fact coverage is nearly identical — category breakdown is where they diverge.
+20pp
fact coverage on JS-heavy sites
Sites like Notion, Webflow, and Figma use client-side rendering. web_fetch fetches an HTML shell and misses content. AgentReady's 4-layer pipeline (llms.txt → HTTP → __NEXT_DATA__ extraction → Jina Reader) reaches actual content.
100%
citation rate
Every ask_site answer includes source URLs. web_fetch returns raw text with no attribution. Citations let downstream agents verify facts and detect staleness.
6×
fewer high-hallucination answers
web_fetch returns full page HTML — navbars, footers, cookie banners and all — extra noise that confuses the model. AgentReady pre-chunks and retrieves only relevant passages.
85%
useful answer rate
Judged by GPT-4o: "would this answer help a developer?" ask_site wins by 9pp — driven by multi-page and JS-heavy categories where web_fetch returns incomplete or garbled content.
60 cases · 43 sites · judged by GPT-4o
| Metric | web_fetch | AgentReady |
|---|---|---|
| Fact coverage (avg) Overall similar — category breakdown tells the real story | 78% | 75.9% |
| Useful answers | 46/60 | 51/60 |
| Answers with citations | 0/60 | 60/60 |
| High hallucination risk | 6 | 1 |
| Wrong claims flagged | 10 | 8 |
| p50 latency | 4.2s | 7.8s |
| p95 latency | 7.2s | 12.9s |
Fact coverage — % of expected facts present in the answer
Sites using client-side rendering (Notion, Figma, Webflow). web_fetch fetches an HTML shell with no content. AgentReady's 4-layer pipeline reaches the rendered DOM.
Questions whose answers span multiple pages — pricing tiers, feature comparisons, policy details. web_fetch can only fetch one URL.
Server-rendered docs (MDX, Sphinx, VitePress). web_fetch works well here — both approaches score highly.
Dense API spec pages where the full parameter list is on one page. web_fetch reads the entire spec directly; chunking can lose detail.
43 real-world developer documentation and SaaS sites
Test cases
60 questions across 43 sites, hand-written to require specific factual knowledge. Each question has 2–6 expected facts (e.g. "25,000 MAU", "free tier", "social login included").
Judging
GPT-4o judges each expected fact independently — present (✓) or absent (✗). This avoids holistic "which answer sounds better" bias and produces a reproducible, auditable score.
Comparison
web_fetch is the baseline available to any AI agent with a browser tool — one URL, raw HTML. ask_site uses AgentReady's 4-layer pipeline: llms.txt → HTTP+cheerio → __NEXT_DATA__ extraction → Jina Reader headless.
Categories
Cases are classified as: JS/Playwright-heavy (client-rendered, needs JS execution), Multi-page Synthesis (answer spans multiple URLs), Static Docs (server-rendered), API Reference (dense single-page specs).
Raw results: eval/results/benchmark-2026-07-16-final.json · Evaluation harness available on request.
Make your site agent-queryable in 60 seconds
Index any public website. Get a per-site MCP endpoint, cited RAG answers, and analytics to see which agents are querying it.