The short answer: start with an HTTP fetch. Escalate to a headless browser only when the page needs JavaScript, interaction, a frame boundary, or a rendered widget. Use an agent when the task needs planning across steps or recovering from failures — not as a CAPTCHA bypass.
We ran the same tasks through HTTP fetch and headless Chrome. The agent adapter is wired but not yet configured, so its row stays pending. The local suite is the capability test; the 20 real pages are a reachability check; the 3 interactions test whether the method can follow a click.
What we measured
12 local capability tasks
| Method | Passed | Failed | What it shows |
|---|---|---|---|
| HTTP fetch | 3 / 12 | 9 | Static HTML and PDFs only |
| Headless Chrome | 12 / 12 | 0 | JavaScript, clicks, typing, scroll, frames |
| Agent endpoint | pending | pending | Adapter is wired; results await an endpoint |
The HTTP method passed the static article, PDF download, and raw PDF text. It failed the JS-rendered pricing, pagination, login, cross-origin iframe, infinite scroll, cookie consent, multi-step form, JS challenge, and dashboard tasks.
20 real public pages
| Method | Passed | Failed | What it shows |
|---|---|---|---|
| HTTP fetch | 16 / 20 | 4 | Most public pages still serve usable HTML |
| Headless Chrome | 17 / 20 | 3 | Adds JS-rendered pages, not anti-bot walls |
| Agent endpoint | pending | pending | Not evaluated |
HTTP failed npm (403), Stack Overflow (403), Reddit (JS shell), and YouTube (short shell). Headless failed npm, Stack Overflow (challenge), and Reddit (interstitial). Headless added YouTube; neither method bypassed npm, Stack Overflow, or Reddit.
3 real interaction tasks
| Method | Passed | Failed | What it shows |
|---|---|---|---|
| HTTP fetch | 0 / 3 | 3 | Stayed on the landing page |
| Headless Chrome | 3 / 3 | 0 | Followed category and pagination clicks |
| Agent endpoint | pending | pending | Not evaluated |
Books to Scrape Travel category, Quotes to Scrape page 2, and Hacker News “More.” HTTP failed all three; on Hacker News it had the keyword but stayed on the landing URL. Headless passed all three and landed on the expected post-click URL.
Decision matrix
| Job | Start with | Why |
|---|---|---|
| Static article, docs, PDF | HTTP fetch | Cheapest; no execution needed |
| JS-rendered pricing, dashboard, widget | Headless browser | The answer exists only after rendering |
| Login, multi-step form, consent | Headless browser | The flow needs interaction |
| Cross-origin iframe | Headless browser | The content lives in another frame |
| Infinite scroll / “load more” | Headless browser | The page needs repeated interaction |
| Simple JS challenge | Headless browser | The gate runs in the browser |
| Real anti-bot / CAPTCHA | Human-in-the-loop | Do not promise a bypass |
Escalation path: HTTP → headless browser → agent or human. Move up only when the lower layer cannot complete the task; an agent is for planning and recovery, not for bypassing a challenge.
How to read the numbers
Most public pages still return usable server-rendered HTML; a plain fetch handled 16 of 20 real pages. The browser becomes valuable when the answer is rendered, behind an interaction, or inside a frame. In the local suite, that difference was decisive: 3/12 vs 12/12. The interaction suite made the same point with real pages: 0/3 vs 3/3.
Latency also moves in the expected direction: HTTP was typically tens to hundreds of milliseconds; headless was hundreds to several seconds, with the slowest real pages taking up to ~14 seconds. Use HTTP first when it can finish the job.
Limitations
These are capability and reachability results, not a market benchmark. The local tasks are deterministic fixtures; the real checks use a keyword and a minimum text length, and public pages change by region and over time. The headless method is not a stealth browser. The agent adapter is wired but not evaluated; no agent result is claimed here. Timings are a single run. We did not measure token cost or setup time.
Method and raw data
Twelve deterministic local fixtures test capability; 20 public pages test reachability; 3 public interactions test task completion after a click. HTTP and headless received the same task input and check. HTTP strips scripts and styles before checking visible text; headless uses rendered text, frame content, and the final URL.
Download the task definitions, exact checks, runner commands, and raw results (JSON). The file includes all 12 local tasks, all 20 real sites, all 3 interaction tasks, every raw result record, the environment, and the agent-not-configured status.
If you want to inspect a result, use the task ID from the JSON. The numbers above are the summary; the agent row stays pending until an endpoint is evaluated. The harness source is available on request.
What’s next
Wire the agent endpoint, add more real interactions where sites permit, and rerun quarterly. The next version should measure token cost, setup time, and retries.