How to bypass Cloudflare when you scrape

Cloudflare answers in several ways, and each one says something different about why the request was stopped. Look at the status, one header and the text of the page.

Cloudflare sits in front of the site and scores every request before the site sees it. A plain HTTP client fails on things a browser gets right without trying.

• The IP: addresses of hosting providers score worse than home and mobile connections.
• The handshake: the TLS and HTTP/2 fingerprint of the client has to match the browser named in the User-Agent. A Python or Go client that says it is Chrome does not match.
• JavaScript: the challenge page runs a script and expects a browser to answer it. A client that cannot run it never leaves the 403.
• The clearance: cf_clearance belongs to the visitor that earned it. Sent from another IP or another client, it is ignored and the challenge comes back.
• The settings of the site: the countries, paths and rates its owner chose are behind the 1020, 1015 and 1009 errors. No client change lifts a block on your country: the request has to leave from an allowed one.

Send the URL to the REST endpoint. With mode: "auto", the default, the first attempt is a browser-grade request from a residential IP. If Cloudflare answers with a challenge, a real browser opens the page, waits for the check to clear and returns what it sees.

curl -X POST https://app.quanticdata.io/api/v1/scraper/unlock \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/", "country": "us"}'

import requests

r = requests.post(
    "https://app.quanticdata.io/api/v1/scraper/unlock",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"url": "https://example.com/", "country": "us"},
    timeout=130,
)
page = r.json()["payload"]
print(page["status"], page["blocked"], page["latency_ms"], page["cost"])
print(page["body"][:500])

const res = await fetch("https://app.quanticdata.io/api/v1/scraper/unlock", {
  method: "POST",
  headers: { Authorization: "Bearer YOUR_API_KEY", "Content-Type": "application/json" },
  body: JSON.stringify({ url: "https://example.com/", country: "us" }),
});
const { payload: page } = await res.json();
console.log(page.status, page.blocked, page.latency_ms, page.cost);
console.log(page.body.slice(0, 500));

Set country to a country the site serves: error 1009 is a country rule. For many pages of one site, keep calling from the same account: once a challenge is passed, the next calls reuse that session and come back faster.

What the answer tells you

In the canary of 11 November 2026, all 9 calls to Cloudflare-protected pages came back with the real page. Across the whole run, 72 of 81 calls on 27 protected targets did, with a median response of 4.1 s.

The first request to a protected site waits for the site's own check. The next ones from the same account reuse the session: in our test of 11 November 2026, four requests in a row to the same Cloudflare-protected page took 12.4 s, 1.5 s, 1.5 s and 1.4 s.

A call counts only when the answer is 2xx and carries the real page, not a challenge. The list, the method and every run are on the Web Unlocker status page.

Getting past Cloudflare once is an afternoon. Staying past it every week is the work. This is what has to stay right, and who keeps it right.

• It is for pages anyone can open in a browser without signing in: prices, listings, reviews, search results, public profiles.
• Challenges that need no human input are handled by the browser tier. An interactive captcha that asks a person to click, pick images or type is not solved: the answer comes back with blocked: true and blockClass: "captcha", so your code can skip the page or try later.
• It does not log in for you and it does not create accounts. Pages behind a login stay behind it.
• Whether you may collect a page depends on the terms of the site and on the law that covers the data. A protection system does not change that, in either direction.