July 7, 2026 · 6 min read
You ask Claude to look up pricing from a SaaS pricing page. It uses web_fetch, comes back with something vague, or worse — confidently wrong numbers. You open the URL in your browser and the page loads fine. What happened?
The answer is almost always JavaScript rendering. The page works in your browser because your browser runs JavaScript. The agent's fetch tool doesn't.
web_fetch — the tool Claude and other agents use to read web pages — sends an HTTP GET request and returns the response body. It's essentially curl with some HTML-to-text conversion.
For a server-rendered page (old-school PHP, Rails, Django, or a Next.js app with SSR), this works fine. The server sends HTML with the content already in it. The agent reads the HTML, strips the tags, and gets the text.
For a JavaScript single-page app (React, Vue, Angular in client-side mode, or Next.js with client-side routing), the server sends something like this:
<!DOCTYPE html>
<html>
<head>
<title>Pricing — Acme</title>
</head>
<body>
<div id="root"></div>
<script src="/static/js/main.abc123.js"></script>
</body>
</html>The content — pricing tiers, feature lists, anything useful — doesn't exist in this HTML. It gets rendered by the JavaScript after the page loads in a browser. The agent gets an empty <div id="root"></div> and nothing else.
A surprising amount of the web that matters to developers is JavaScript-rendered:
— SaaS pricing pages (Linear, Vercel, Notion, Figma, Loom)
— Developer dashboards and docs portals built on React
— Product changelog pages
— Any site that went heavily into a React-based framework without enabling SSR
Even sites built on Next.js — which supports SSR — often have client-side-only routes where the content isn't server-rendered. Whether a specific page is JS-only or not isn't visible from the URL.
The result: agents that rely on web_fetch silently get no content, then hallucinate an answer based on their training data. The agent doesn't know it got an empty page — it just got fewer than 200 characters and has to do something with the query.
1. Headless browser rendering
Run a real Chromium instance (via Playwright or Puppeteer), load the page, wait for JavaScript to execute, then extract the text. This is what Google's crawler does. It works, but it's slow (5-15 seconds per page), expensive to operate at scale, and requires significant infrastructure.
For an agent that needs to read one page on demand, this might be acceptable. For indexing hundreds of pages across dozens of sites, it's the wrong tool.
2. Jina Reader and similar services
Services like Jina Reader (r.jina.ai/your-url) handle the JavaScript rendering for you — you hit their API with a URL and get back clean text. They run the headless browser infrastructure so you don't have to.
This works well as a fallback. AgentReady uses Jina as a fallback when a page returns thin content — if our initial HTTP fetch gets less than 150 characters from a page, we automatically retry via Jina to get the rendered version.
3. Pre-index with an embeddings-based RAG system
Rather than fetching pages on demand, crawl and index them up front. This is what AgentReady does: when you submit a site, we crawl it (with JS rendering fallback), chunk the content, embed it into a vector index, and store it. When an agent asks a question, we run a semantic search over the stored chunks — we never fetch the live page again.
Pre-indexing solves several problems at once. JS rendering is handled at index time, not at query time, so queries are fast regardless of how the page is built. The content is available even if the live site is down or rate-limiting. And repeated queries hit the index, not the origin server.
Consider an agent trying to answer "What's the meeting time limit on Zoom's free plan?" Zoom's pricing page is a React app — web_fetch returns about 1,300 characters of nearly empty HTML.
In our benchmark of 37 developer sites, web_fetch returned no useful content for Zoom at all. AgentReady's indexed RAG answered correctly (40-minute limit for free accounts) with a source citation.
The pattern holds across JS-heavy sites: Airtable, monday.com, Miro, Loom. For these categories, raw web fetching fails silently. The agent hallucinates or admits it can't answer — neither is useful.
If you own the site, you have options that users don't:
Add server-side rendering. If your site is Next.js, enabling SSR or static generation for key pages means web_fetch works without any special handling. This is the most robust long-term fix.
Add an llms.txt. Serve a plain-text summary of your content at /llms.txt. Agents that know to check it can get structured information without JS rendering. It's lightweight to add and increasingly expected.
Expose an MCP endpoint. An MCP server that serves your content as queryable tools bypasses the rendering problem entirely. Agents call your tool directly instead of trying to fetch and parse a webpage. AgentReady can serve as this layer if you don't want to build it yourself — submit your site once and it's queryable via MCP for any agent that connects.
JS rendering is not a niche edge case. A significant chunk of the developer-facing web — pricing, docs, product pages — is built on client-side frameworks where web_fetch returns nothing useful.
If you're building an agent, don't rely on web_fetch as your only strategy for reading the web. If you're a site owner, rendering content server-side and exposing an MCP endpoint are the two things that make your site actually agent-friendly.