Headless browser scraping earns its price on 15 of the 53 sites we measured and costs words on 12. Between 26 and 28 September 2026 we fetched every target through a residential exit twice, as a plain HTTP client and fully rendered, and counted the words each view returned. Fifteen sites need the browser, four gain a little from it, fourteen gain nothing for up to 15 times the bytes, twelve return fewer words rendered than plain, and eight return no rows in either mode and want a collector. The table below has all 53 with the setting for each.
The search results teach how to render; none of them say where
The first page for "headless browser scraping" is tutorials and listicles: Selenium, Playwright and Pyppeteer walkthroughs, rankings of browsers, and a Reddit thread from r/webscraping whose title is the real question: headless browsers are killing my wallet, render or not to render? The poster runs a browser on every page of thousands of domains and wants a rule to decide per site. The top tutorial puts self-hosted browser infrastructure at 200 to 500 MB per instance and 200 to 800 dollars a month, and still opens a browser for every example.
Google's own documentation describes the same split from the crawler's side: Googlebot fetches the HTML first, queues the page for rendering, and indexes the rendered result later, which is why a page that only exists after JavaScript is a page the crawler sees late. A scraper has the same two phases and the same choice. What nobody on the first page does is fetch the page both ways and count. So the rest of this post is the count.
Fifteen sites need the browser, twenty-six do worse or no better with it
Each row was audited from the exit country in the second column, as a pure HTTP client with no JavaScript and then fully rendered, in one call. The words are the extractable body text of the home page or the public page named in the per-target post. Verdicts are on the measured word delta; the setting column is what to buy, and the two disagree on a handful of rows where the head of the page or a JSON endpoint carries the fact you want, which the setting says.
| Verdict | Sites | What it means |
|---|---|---|
| Browser required | 15 | The plain fetch returns 0 to 463 words and the render multiplies it: bet365 0 to 987, Trustpilot 19 to 3,588, Deliveroo 194 to 2,247, Target 317 to 2,163. |
| Render optional | 4 | Plain HTTP returns most of the page: Walmart 2,190 of 2,975 words, Ticketmaster 494 of 1,021, with canonical and JSON-LD. |
| Render useless | 14 | The render adds under 2 percent or nothing: Shopee identical to the byte, Best Buy one word fewer for 9.8x the bytes, Shopify +4, Lazada +9. |
| Render loses words | 12 | The browser returns less than the plain client: Steam 2,802 to 1,396, Indeed 2,394 to 683, Glassdoor 624 to 54, Zillow 759 to 0, Etsy 678 to 0, Idealista 1,180 to 0. |
| Collector needed | 8 | Neither mode returns rows: Amazon 0 and 0, eBay 24 and 24, Booking.com 26 and 0, Skyscanner 0 and 0. |
Two counts matter for a budget. Twenty-six of 53 sites, the useless and the losing rows together, are places where the browser is a cost with no return or a negative one. Fifteen are places where skipping it means missing the page. The remaining twelve are optional renders and collector cases. The rule that falls out of that is not "render when in doubt"; it is "fetch plain first, render on evidence", and the evidence is the table.
All 53 targets, one line each
The last column is the setting: network, fetch mode, exit, and the collector with rows and seconds from a run we made. Collector prices are per delivered row; the browser fetches cost bandwidth and time, both measured.
| Target | Exit | Plain HTTP, words | Rendered, words | Verdict | The setting |
|---|---|---|---|---|---|
| Steam | US | 2,802 | 1,396 | No browser, the render loses words | Residential US, plain HTTP: 2,802 words; prices from the appdetails JSON with cc=; never render, it halves the page |
| Walmart | US | 2,190 | 2,975 | Render optional | Residential US, plain HTTP: 2,190 words with canonical and JSON-LD; the render costs 7.5x the bytes |
| US | 1,928 | 2,588 | No browser | Residential US, plain HTTP: follower count and Organization JSON-LD in the head; the render removes words | |
| PayPal | US | 1,693 | n/a | No browser | Residential, plain HTTP: 1,693 words; the real job is a static ISP IP for the Payflow allowlist |
| Shopee | SG | 1,432 | 1,432 | No browser | Residential SG, plain HTTP: 1,432 words, identical when rendered; never an Indonesian exit |
| Lazada | SG | 1,198 | 1,207 | No browser | Residential SG, plain HTTP: 1,198 words; the render adds 9 words for 3.9x the bytes |
| Idealista | IT | 1,180 | 0 | No browser, the render returns nothing usable | Residential IT, plain HTTP: 1,180 words in 3.8 s; never render (0 words); listings via idealista_search, 10 rows in 5.5 s |
| Autotrader | GB | 1,113 | 1,129 | No browser | Residential GB, plain HTTP: 1,113 words; the render adds 16; autotrader_search covers the US site, 10 rows in 15.3 s |
| Rakuten | JP | 885 | 1,029 | No browser | Residential JP, plain HTTP: 885 words, identical from the US; catalogue via the official Ichiba Item Search API |
| Pop Mart | US | 841 | 867 | No browser | Residential US, plain HTTP: 841 words; product pages are no-store, poll them; never render |
| Shopify stores | US | 772 | 776 | No browser | Rotating residential, plain HTTP: /products.json?limit=250 on any storefront, 250 products per call |
| Etsy | US | 678 | 0 | No browser, the render returns nothing usable | Residential US, plain HTTP with a browser TLS profile: 678 words in 205,911 bytes; never render (0 words); never an EU exit |
| DraftKings | US | 647 | 642 | No browser, the render loses words | Residential US, plain HTTP, never render: 647 words plain, 642 rendered; one account per person in their terms |
| OKX | DE | 623 | 667 | No browser | Residential, plain HTTP, exit pinned: DE 623, SG 464, US 371 words; public ticker in 364 bytes |
| US | 574 | 0 | No browser, the render returns nothing usable | Residential US, plain HTTP for listings: 574 words; comments via reddit_comments, 20 rows in 7.9 s; never render | |
| Subito | IT | 602 | 848 | Render optional | Residential IT, plain HTTP: 602 words in 183 KB; listings via subito_search, 10 rows in 2.7 s |
| AliExpress | US | 538 | 2,628 | Browser required | Residential, rendered for home and categories: 2,628 words against 538; listings via aliexpress_search, 20 rows in 4.5 s |
| Kleinanzeigen | DE | 512 | 903 | Render optional | Residential DE, plain HTTP: 512 words with JSON-LD; listings via kleinanzeigen_search, 10 rows in 3.5 s |
| Ticketmaster | US | 494 | 1,021 | Render optional | Residential US, plain HTTP: category pages carry MusicEvent JSON-LD; prices from the Discovery API |
| Netflix | US | 463 | 1,204 | Browser required | Residential, plain HTTP, one exit per market: 463 words with the prices ($8.99 US, £7.99 GB, 6,99 € DE); never render |
| Target | US | 317 | 2,163 | Browser required | Residential US, rendered: 317 words plain, 2,163 rendered; store and ZIP follow the IP, pin the state |
| Best Buy | US | 285 | 284 | No browser | Residential US, plain HTTP: 285 words from the edge cache; the render costs 9.8x the bytes for one word fewer |
| Zalando | IT | 266 | 275 | No browser | Residential IT, plain HTTP: 266 words; catalogue with prices via zalando_search, 10 rows in 3.6 s |
| Vinted | IT | 230 | 217 | No browser, the render loses words | Residential IT, plain HTTP: 230 words; the render loses 13; the locale is the domain, not the exit |
| StubHub | US | 0 | 157 | Browser required | Residential US, rendered through the web scraping API at $0.001 per page: 0 words plain, 157 rendered |
| bet365 (UK exit) | GB | 0 | 987 | Browser required | Residential, rendered, exit pinned to the market you read: 987 words from the United Kingdom |
| bet365 (DE exit) | DE | 0 | 149 | Browser required | Same URL from a German exit: 149 words, a different licensed catalogue, not a translation |
| US | 40 | 40 | No browser | Residential US, plain HTTP: the counts are in the head (104M followers); the render adds nothing | |
| Taobao | SG | 36 | 56 | Collector | Residential SG, plain HTTP on /lang/ and item pages (410 words), not the home (36); no collector |
| Binance | DE | 26 | 1,196 | Browser required | Plain HTTP on data-api.binance.vision from any exit, 558 bytes per ticker; one static ISP IP per API key |
| Booking.com | IT | 26 | 26 | Collector | Neither fetch mode (26 words): booking_stays, 20 priced properties in 5.1 s |
| YouTube | US | 15 | 585 | Browser required | Residential US, plain HTTP: the head resolves @handle to the channel ID; videos via youtube_channel, 30 rows in 3.8 s; EU exit lands on consent |
| TikTok | US | 5 | 12 | Collector | Neither fetch mode (5 and 12 words): the TikTok profile collector |
| US | 0 | 350 | Browser required | Residential US, rendered for Page text (350 words); the follower count sits in og:description without a browser; a DE exit rewrites the number separators | |
| Roblox | US | 0 | 128 | Browser required | Residential US, plain HTTP on game pages (213 words) and the games.roblox.com JSON; never render, 4.4 MB for 128 words |
| Spotify | US | 0 | 14 | Collector | Web API with client credentials for the catalogue; plain HTTP only on the spotify.com price pages; never render open.spotify.com |
| Twitch | US | 0 | 125 | Browser required | Residential, plain HTTP on channel pages: JSON-LD with 2,461,147 followers in 214 KB; the Helix API for data; never render |
| Amazon | US | 0 | 26 | Collector | Neither fetch mode (2,007 bytes, 0 words): amazon_search, 20 products in 43 s at $0.001 each |
| eBay | US | 24 | 24 | Collector | Neither fetch mode (1,976 bytes, 24 words): ebay_search, 25 listings in 19.5 s; the run declares its exit |
| Pokemon Center | US | 1,622 | 765 | No browser | Residential US, plain HTTP: 1,622 words; the render returns 765 in 62 s; resale prices via ebay_search |
| Indeed | US | 2,394 | 683 | No browser, the render returns nothing usable | Residential US, plain HTTP with a browser TLS profile: 2,394 words in 601 KB on the first request; rows via indeed_jobs, 10 in 7.3 s |
| Zillow | US | 759 | 0 | No browser, the render returns nothing usable | Residential US, plain HTTP: 759 words, price, beds and square feet in the meta description; never render; zillow_search, 20 rows in 3.0 s |
| Tripadvisor | US | 713 | 0 | No browser, the render returns nothing usable | Residential US, plain HTTP: 713 words with 19 LocalBusiness JSON-LD blocks; never render; tripadvisor_search, 30 rows in 8.1 s |
| Glassdoor | US | 624 | 54 | No browser, the render loses words | Residential, plain HTTP: 624 words US, 589 GB, with JSON-LD; never render, it lands on the login form; salaries via indeed_jobs |
| Trustpilot | US | 19 | 3,588 | Browser required | Residential, rendered with a 6 s wait: 3,588 words on the home, 4,389 on a company page, 19 plain; reviews via trustpilot_reviews, 10 in 14.8 s |
| Airbnb | US | 59 | 47 | Collector | Plain HTTP gives 59 words: airbnb_stays, 18 stays with the nightly price in 5.3 s; a DE exit gets a redirect stub |
| Deliveroo | GB | 194 | 2,247 | Browser required | Residential GB, rendered: 2,247 words against 194; the full menu exists only in the browser |
| DoorDash | US | 484 | 26 | No browser, the render returns nothing usable | Residential US, plain HTTP: 484 words on the first request; never render; doordash_restaurants, 50 rows in 6.0 s |
| Uber Eats | US | 99 | 206 | Browser required | Residential US, rendered for restaurant pages (206 words against 99); the home is thin in both modes |
| Kayak | US | 2,449 | 2,564 | No browser | Residential US, plain HTTP: 2,449 words with FAQPage JSON-LD; a GB exit redirects to kayak.co.uk; fares via google_flights, 15 in 19.4 s |
| Expedia | US | 72 | 14 | No browser, the render returns nothing usable | Residential US, plain HTTP: 72 words with h1 and canonical; never render (14 words); rates via hotels (20 rows) and google_flights (15 in 19 s) |
| Skyscanner | GB | 0 | 0 | Collector | Neither fetch mode (0 words in both): google_flights, 15 LHR to JFK fares in 33.7 s |
| bet365 (US exit) | US | 0 | 2,031 | Browser required | Same URL from a US exit, rendered: 2,031 words; three licences, three catalogues |
Where the browser earns its price
The fifteen required rows share one shape: the plain fetch returns a head and a shell, and the rendered page is where the copy lives. bet365 is the extreme: 0 words plain from every exit, then 987 rendered from the United Kingdom, 149 from Germany and 2,031 from the United States, three licensed catalogues behind one URL, so the exit is part of the setting. Trustpilot goes from 19 words to 3,588 with a six-second wait before capture. Deliveroo multiplies by 11.6, 194 to 2,247, because the menu exists only in the browser. Target goes 317 to 2,163 and assigns store and ZIP from the IP, so the state is pinned too.
Four of the fifteen carry a required verdict and a plain-HTTP setting, and the table says so. On Netflix the render returns 1,204 words to the plain client's 463, but the 463 include the prices per market, which is the fact people fetch it for. On YouTube the head resolves an @handle to the channel ID and carries the subscriber count without a browser; on Twitch the channel page ships JSON-LD with the follower count in 214 KB; on Roblox the games JSON is 1,802 bytes where the render is 4.4 MB for 128 words. The verdict measures words. The setting measures what you needed.
Where the browser removes words
Twelve sites return fewer words rendered than plain, and the reasons are two. On four the page itself breaks under the render: Steam halves from 2,802 to 1,396 and its h1 becomes an error message, DraftKings loses five words, Vinted loses thirteen, and Glassdoor lands on its login form with a canonical of /member/profile/login and 54 words where the plain client got 624 with JSON-LD.
On eight the rendered view returns nothing usable while the plain client reads the page: Indeed 2,394 words plain against 683, Zillow 759 against 0, Etsy 678 against 0, Idealista 1,180 against 0, Tripadvisor 713 with 19 LocalBusiness objects against 0, Reddit listings 574 against 0, DoorDash 484 against 26, Expedia 72 against 14. On every one of these the setting is residential, engine: tls, sometimes with a browser TLS profile, and never a render. The common tutorial advice runs the other way: if the plain fetch looks thin, open a browser. On these eight it turns a working fetch into an empty one and doubles the bytes. The four shapes those empty replies take, and what to do with each, are in read the block and what to do.
Where the render is a tax
The fourteen useless rows are the ones that quietly cost the most, because nothing looks wrong. Best Buy returns 285 words plain and 284 rendered for 4,968,555 bytes against 506,484: 9.8 times the bandwidth for one word fewer. Walmart, an optional row, costs 7.5x the bytes for the last quarter of the page. Shopee returns 1,432 words in both modes, identical, for 1.7x the bytes. OKX costs 14.9x for 44 extra words. Roblox costs 43.7x. Over a gigabyte of residential Basic at $0.80, counting a gigabyte as 10^9 bytes, a rendered Best Buy home page costs $0.0040 and a plain one $0.0004; across a 10,000-page price watch that is 36 dollars a run spent on nothing.
The plain fetch is also faster on every row where we recorded both. Walmart's plain page arrives in 2.5 seconds from the edge cache and the render took 59.8 seconds; Booking.com's plain reply took 1.3 seconds and the render 46.7; Glassdoor's render took 22.9 seconds today to reach a login form. Time is the second cost of the browser, and on a pool it is the one that caps throughput.
Eight sites where neither mode returns rows
Amazon (2,007 bytes, 0 words), eBay (1,976 bytes, 24 words from the United States and from Germany), Booking.com (26 words plain, 0 rendered for 857,992 bytes), Skyscanner (0 and 0), Airbnb (59 and 47), Taobao's home (36 and 56), Spotify (0 and 14) and TikTok (5 and 12) are not thin sites. They are sites that do not serve the catalogue to an anonymous first request in any mode. The setting there is the collector, and today's runs are the proof: ebay_search returned 25 priced listings in 19.5 seconds, booking_stays returned 20 priced Rome properties in 5.1 seconds, and the per-target posts have amazon_search at 20 products in 43 seconds and google_flights at 15 fares in 33.7 seconds. The eight are compared row by row in when the collector wins.
The rule that replaces "render when in doubt"
The Reddit poster wanted a detector that works across thousands of domains with one page each. The 53 rows give one. Fetch plain first, through the web scraping API at $0.0002 per page, and read four fields of the reply: the word count, the canonical, the h1 and the JSON-LD types. Then:
plain = fetch(url, engine="tls")
if plain.words >= 200 or plain.jsonld:
use(plain) # 30 of 53 rows: the page, or the fact in the head
elif plain.canonical and plain.h1 and plain.words < 200:
r = fetch(url, engine="render") # 15 of 53: bet365, Trustpilot, Deliveroo, Target
use(r if r.words > plain.words else plain)
elif plain.bytes < 5000 and not plain.canonical:
use(collector(url)) # Amazon, eBay, Booking.com: no page exists to render
else:
use(plain) # thin but real: Expedia, Airbnb headThe comparison in the second branch is the part the tutorials skip. Rendering once and keeping the view with more words is what catches Steam, Indeed, Glassdoor and Zillow, where the render returns less; it costs one browser fetch per new domain, not one per page. After the first pair the site's answer is known, and it holds: the four rows we repeated today returned the same shape as two days ago, eBay 24 words, Booking.com 26, Expedia 72 plain and a refusal rendered, Glassdoor's login canonical rendered.
The setting that works on the 53 targets
- Network: residential, Basic line at $0.80/GB, for all 53. Nothing in the two passes scored the address on a page where the plain client read the page; where the plain client got a shell, the render from the same exit got the same shell. Premium at $2.20/GB and mobile at $2.30/GB bought no extra words anywhere in this table. ISP static IPs, from $2.50 per IP per month at volume, appear only where a platform keys an API to an address: PayPal's Payflow allowlist, Binance and OKX API keys.
- Fetch mode:
engine: tlson 30 of 53, with the word counts in the table; rendered on 15, of which bet365, Trustpilot, Deliveroo, Target, AliExpress and Facebook are the clear cases; a collector on 8. Where the head carries the fact (LinkedIn and Instagram follower counts, YouTube channel IDs, Twitch JSON-LD), the plain fetch is the setting whatever the word delta says. - Country: the market you read, pinned. The same URL returns 987, 149 and 2,031 words on bet365 from the UK, Germany and the US; Amazon from Germany is a different page at /-/de/; Etsy from an EU exit is 8 words; YouTube from the EU is a consent page; a German exit on Facebook rewrites 28,729,208 with dots and breaks a comma parser.
- When the proxy is not enough: the collector named in the row, with the run we made: ebay_search 25 listings in 19.5 s, booking_stays 20 properties in 5.1 s, amazon_search 20 products in 43 s, indeed_jobs 10 in 7.3 s, zillow_search 20 in 3.0 s, reddit_comments 20 in 7.9 s. Where a render is the setting, the web scraping API with rendering at $0.001 per page, billed on success.
- Failed requests are never billed, and every account gets $2 of free API usage per month: 10,000 plain fetches, or 2,000 rendered pages, or one plain-and-rendered pair on every site you care about before the first invoice.