Indeed is the target where the plain request wins and the browser loses. On 26 September 2026 the Indeed home page answered a plain HTTP client through a US residential exit with 1,026 words on the first request; on 28 September the same fetch, sent with a browser TLS fingerprint, returned 200 OK, 601,347 bytes and 2,394 words in 9.4 seconds. Rendering the page in a real browser cost 1,066,178 bytes and 52 seconds for 683 words. From a UK exit the same URL became uk.indeed.com. The setting is residential, plain HTTP with a browser fingerprint, country pinned to the market, and the jobs collector when you want salaries as numbers: ten listings in 7.3 seconds.
The keyword has no demand, the scraper does
Google returns no autocomplete at all for "indeed proxies" or "proxies for indeed", and the first page for the head term is Indeed describing itself: five of ten results are Indeed search pages for jobs with the word proxy in the title, 4,653 of them. The rest is a Reddit thread from someone whose personal Indeed scraper stopped working, a scraper-comparison listicle, two vendor guides and a 2023 post about enhancing your job search. Nothing on that page measures what Indeed returns to a request.
The demand is one word over. "Indeed scraper" autocompletes to github, apify, extension, python, api, reddit, chrome extension, free and n8n; "scrape indeed" to job postings, jobs, python and, tellingly, "how often does indeed scrape jobs", because Indeed is itself an aggregator that crawls employer sites. Those searchers want listings with salaries in a table for labour-market analytics, wage tracking and recruiting research. That is the job this post measures.
Plain HTTP with a browser fingerprint gets the whole home page
We fetched indeed.com from a United States residential exit three ways: as a pure HTTP client through our SEO audit, as a plain HTTP client that presents a real Firefox TLS fingerprint, and rendered in a browser.
| Fetch | Status | Bytes | Words | Time | Canonical |
|---|---|---|---|---|---|
| Plain HTTP, 26 September, first request | 200 | n/a | 1,026 | n/a | indeed.com |
| Plain HTTP, browser TLS fingerprint, 28 September | 200 | 601,347 | 2,394 | 9.4 s | indeed.com |
| Rendered in a browser, 28 September | 200 | 1,066,178 | 683 | 52.1 s | indeed.com |
| Jobs collector, "data engineer", New York | done | n/a | 10 listings | 7.3 s | n/a |
Two numbers carry the whole decision. The plain fetch returns 3.5 times the words of the rendered one, in a fifth of the time, for 56 percent of the bytes. Indeed server-renders its search interface, popular categories, the salary and company links and the footer; the browser then replaces most of that with an application shell and a search box. The rendered word count is not what a job seeker sees, it is what the shell had painted when the page settled, and it is a third of the HTML you already had.
The thing that decides whether the plain fetch works is the TLS handshake, not the IP. Indeed decides at the handshake which client it serves the page to. The same residential exit that receives 19 words when it presents a generic client receives 601,347 bytes and 2,394 words when it presents a Firefox fingerprint. Our web scraping API does that by default; if you run your own client, use a TLS library that impersonates a current browser, or you will spend your residential bandwidth on 19-word responses.
The exit country picks the site, not the language
We ran the identical audit through a United Kingdom residential exit. Indeed did not localise the page; it redirected it. indeed.com became uk.indeed.com/?r=us, canonical https://uk.indeed.com/, 216 words with and without JavaScript, a WebSite JSON-LD block, and a meta description promising "CVs" where the US one promises "resumes".
| Exit | Final URL | Words, no JavaScript | Words, rendered | Description says |
|---|---|---|---|---|
| United States | www.indeed.com | 2,394 | 683 | resumes |
| United Kingdom | uk.indeed.com/?r=us | 216 | 216 | CVs |
Indeed runs a separate site per country, with its own listings, salary formats and currency, and the exit country decides which one you land on. A pool that rotates across countries will silently mix US dollar listings from indeed.com with pound listings from uk.indeed.com and Indian rupee listings from in.indeed.com, and the ?r=us parameter in the redirect is the only trace that the request started somewhere else. Pin the country to the market you are measuring, and if you want several markets, run them as several jobs with several exits. The collector takes the same decision as an input: its country field picks the local site and the proxy exit together.
Ten listings with salaries as numbers in 7.3 seconds
We ran the Indeed jobs collector once, query "data engineer", location "New York", country US, capped at ten. It returned ten listings in 7.3 seconds, each with title, company, location, posting age, remote flag, snippet, the raw salary string and, parsed from it, salary_min, salary_max and salary_period, plus the company's Indeed rating and review count and whether the listing was sponsored.
Three things in those ten rows matter more than the fact that it worked. Three of the ten were sponsored, and the collector flags them; if you are measuring wage levels, sponsored rows are paid placement, not a sample of the market. Salaries came back as ranges from $80,000 to $275,000 a year, and one was a single figure, $128,294, which the collector returns with min equal to max rather than as a missing range. And the company rating field was 0 with 0 reviews for four of the ten, which is a company with no Indeed reviews, not a company rated zero; a pipeline that averages that column without checking the count will invent a bad employer.
For the same role on other boards, the LinkedIn jobs collector and the Google Jobs collector take the same query and location shape, so a three-source salary comparison is three calls with one input.
Indeed's robots.txt is a three-tier policy
indeed.com/robots.txt is 13,695 bytes and unusually explicit about who is who. Generic bots are allowed in, with pagination permitted only up to start=90, ten pages of results, and the radius, alert and RSS parameters disallowed. A second block lists the retrieval and grounding agents by name, ChatGPT-User, PerplexityBot, Claude-User, OAI-SearchBot, Google-Extended, and gives them the same access as Googlebot. A third block, headed "Rules for Foundational (Training) bots", names GPTBot, ClaudeBot, CCBot, Bytespider, DeepSeekBot, GrokBot and Diffbot and disallows them from /jobs, /viewjob, /cmp/, /q- and /l-: every listing and every company page. A fourth block fully disallows Scrapy by name, alongside legacy crawlers.
That is the clearest statement of a platform's position we have read this year: search engines and answer engines may index and cite listings, training crawlers may not read them, and a client that announces itself as a scraping framework is refused at the door. Our AI crawler user-agent list shows how the same names are treated elsewhere. And the Terms of Service back the file: the Site Rules say do not access the site through any means other than the public interfaces Indeed provides, do not access any data by automated means without permission, and, in the section on AI connectors, that the prohibitions against scraping, bots and other automated activity remain in full effect.
So be accurate about what a proxied fetch is. Indeed publishes its listings to anyone with a browser, and a residential exit with a browser fingerprint reads what a browser reads; that is a technical fact, and every number here came from one. Indeed's terms say that automated collection needs permission; that is a contractual fact, and the sanctioned route is the partner programme at docs.indeed.com, whose GraphQL APIs manage job postings, candidates and employers for integrators and whose Publisher JavaScript plugin puts Indeed search on your own site. The two facts do not line up, and no proxy makes them.
Cost per thousand pages
Prices from our pricing page: residential Basic $0.80/GB, the web scraping API from $0.0002 per page ($0.001 rendered), the jobs collector $0.001 per delivered listing. A gigabyte is counted as 10^9 bytes.
| Approach | Bytes per page | Pages per GB | Cost per 1,000 pages | What you get |
|---|---|---|---|---|
| Plain HTTP, browser TLS, residential Basic | 601,347 | 1,663 | $0.48 | 2,394 words, full server-rendered page |
| Rendered, residential Basic | 1,066,178 | 938 | $0.85 | 683 words, the application shell |
| Web scraping API, plain fetch | n/a | n/a | $0.20 | Markdown or HTML, browser fingerprint included, failures never billed |
| Jobs collector | n/a | n/a | $1.00 | 1,000 listings with parsed salary min, max and period |
Rendering costs 1.77 times the bytes and 5.5 times the time for 29 percent of the words. That is the sentence to put in front of anyone who says a job board needs a headless browser. Byte figures are decoded bodies and exclude TLS overhead. The collector row costs twice the raw page and returns the thing the raw page does not: a salary as two integers and a period, which is what a wage-tracking pipeline actually stores. A search results page holds a fixed number of listings; the collector bills per listing delivered, so a run that returns fewer rows costs less, and a run that returns none costs nothing.
Where we stop
Everything above is about public listings and public company pages: what Indeed serves any visitor, for labour-market research, wage tracking, aggregation you have permission for, and checking how a posting appears in another market. It does not cover accounts. We will not help with automating applications, creating multiple accounts, mass-messaging candidates or scraping the resume database; Indeed's Site Rules forbid fake accounts and automated account creation, and its Smart Sourcing terms say scraping the resume database ends access. Job seekers' resumes are personal data in every jurisdiction, and no proxy changes that. For the sister site of the same operator, whose measurement points the same way, see what we measured on Glassdoor through residential proxies.
The setting that works on Indeed
- Network: residential proxies, Basic line at $0.80/GB. A residential exit with a browser TLS fingerprint received the full home page on the first request in the United States and the UK site from the United Kingdom; the fingerprint decided, not the address, so Premium at $2.20/GB, mobile at $2.30/GB and ISP at $2.50 per IP per month buy nothing here.
- Fetch mode:
engine: tls, plain HTTP with a current browser fingerprint. 1,026 words on the first request on 26 September, 601,347 bytes and 2,394 words in 9.4 seconds on 28 September. Rendered: 1,066,178 bytes and 52 seconds for 683 words. - Country: pin
country=usfor indeed.com. From a UK exit the same URL redirects to uk.indeed.com, 216 words, CVs instead of resumes, pounds instead of dollars. - When the proxy is not enough: the indeed_jobs collector, 10 listings with parsed salary ranges in 7.3 seconds at $0.001 per listing, up to 300 per run. For a raw page as Markdown, the web scraping API from $0.0002 per page, plain fetch, browser fingerprint included.
- Free tier: every account gets $2 of free API usage per month, which is 2,000 parsed listings from the collector or 10,000 plain-fetch pages before you pay anything.