# How to Use Rotating Proxies

> How to use rotating proxies: gateway endpoint setup, rotation cadence, sticky sessions, retry logic and honest cost math for scraping, with code examples.

[Home](https://quanticdata.io/)/[Blog](https://quanticdata.io/blog/)/How to Use Rotating Proxies

# How to use rotating proxies (without burning your budget)

ProxiesJul 29, 2026·9 min read·QuanticData Team

On this page [What "using rotating proxies" actually means](/blog/how-to-use-rotating-proxies/#what-using-rotating-proxies-actually-means) [Gateway endpoint or your own rotator?](/blog/how-to-use-rotating-proxies/#gateway-endpoint-or-your-own-rotator) [Step by step: rotating proxies in a scraper](/blog/how-to-use-rotating-proxies/#step-by-step-rotating-proxies-in-a-scraper) [Choosing a rotation cadence](/blog/how-to-use-rotating-proxies/#choosing-a-rotation-cadence) [Which network to rotate](/blog/how-to-use-rotating-proxies/#which-network-to-rotate) [Rotation hygiene: subnets, headers, retries](/blog/how-to-use-rotating-proxies/#rotation-hygiene-subnets-headers-retries) [The cost math nobody puts in the docs](/blog/how-to-use-rotating-proxies/#the-cost-math-nobody-puts-in-the-docs) [Rotating proxies inside agent workflows](/blog/how-to-use-rotating-proxies/#rotating-proxies-inside-agent-workflows) [Common mistakes to avoid](/blog/how-to-use-rotating-proxies/#common-mistakes-to-avoid)

To use rotating proxies you point your HTTP client at one gateway endpoint, authenticate with a username and password, and let the provider assign a fresh exit IP per request — or hold one IP for a sticky session. Everything after that is choosing rotation cadence, network type, and retry logic that reacts to blocks.

## What "using rotating proxies" actually means

There are two different things people call rotating proxies, and they are configured in opposite ways.

The first is a **backconnect gateway**: a single host and port you send all traffic to. Behind it sits a pool of exit IPs and a management layer that decides which one your request leaves from. Your code never sees the pool. You change behaviour with credentials or query-style parameters (country, session ID, network type) rather than by editing a list.

The second is a **list of individual proxies** that you rotate yourself in code — the pattern most Python tutorials show, cycling a text file with `itertools.cycle` or `random.choice` ([ZenRows](https://www.zenrows.com/blog/rotate-proxies-python)). That works, but you own the health checks, the retries, the subnet distribution and the dead-IP bookkeeping.

Modern providers push the gateway model because the rotation logic lives closer to the pool, where reputation data actually is. Providers advertise pools in the tens to hundreds of millions of addresses — 40M+ in one vendor's case, 190M across 92+ countries in another ([ScraperAPI](https://www.scraperapi.com/blog/what-are-rotating-proxies/), [Scrapfly](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping)) — which no self-managed list can imitate.

## Gateway endpoint or your own rotator?

| Dimension | Provider gateway | Your own rotator |
| --- | --- | --- |
| Setup | One host:port plus credentials | Fetch, validate and refresh a proxy list |
| Who picks the IP | Provider's management layer | Your selection function |
| Dead IP handling | Transparent; retried inside the pool | You track failures and cooldowns |
| Geo targeting | Parameter per request | Only if your list has that geography |
| Billing unit | Usually GB of traffic | Per IP or per port, sometimes flat |
| Best for | Scraping at scale, mixed targets | Fixed IP needs, niche subnets, audits |

In practice most teams use both: a gateway such as ../../rotating-proxies/ for high-volume crawling, and a small set of dedicated static IPs — ../../isp-proxies/ — for logged-in accounts that must not change address.

## Step by step: rotating proxies in a scraper

1. **Get the endpoint and credentials.** You will receive a hostname, a port, a username and a password. Note whether rotation is per request by default or controlled by a session parameter.

2. **Verify the tunnel.** Send one request to an IP echo endpoint and confirm the address returned is not yours.

3. **Confirm rotation.** Send five requests and print five addresses. If they are identical, you are on a sticky port; check the docs for the rotating port or session syntax.

4. **Set a timeout and a retry budget.** Rotating pools contain slow nodes. A 15-30s timeout with two or three retries is a sane starting point.

5. **Treat 403, 429 and CAPTCHA HTML as failures**, not successes. Status 200 with a challenge page is the most common silent data-quality bug.

6. **Add country targeting only where needed.** Geo-pinning shrinks the effective pool, so use it for locale-specific content, not by habit.

7. **Log the exit IP, status and latency per request.** Without that, you cannot tell a bad target from a bad pool.

```
import requests

# rotating gateway: same host every time, different exit IP per request
PROXY = "http://USER:PASS@GATEWAY_HOST:PORT"
proxies = {"http": PROXY, "https": PROXY}

for i in range(5):
    r = requests.get("https://httpbin.io/ip", proxies=proxies, timeout=20)
    print(i, r.status_code, r.json())

# sticky session: same exit IP for a multi-step flow (session id in the username)
STICKY = "http://USER-session-a19f:PASS@GATEWAY_HOST:PORT"
s = requests.Session()
s.proxies = {"http": STICKY, "https": STICKY}
```

### Sticky sessions when the flow has state

Per-request rotation breaks anything stateful. If a login, a cart or a paginated search token is bound to an IP, switching addresses mid-flow triggers a re-authentication or an outright block. The fix is a session identifier: the gateway pins one exit node for the life of that session — up to 120 minutes on QuanticData's rotating pools — and you rotate between sessions instead of between requests.

Browser automation follows the same rule. Managed browser services expose rotation as connection parameters, and pair sticky sessions with locale alignment so a US exit IP does not advertise a German language preference ([Browserless](https://www.browserless.io/blog/proxy-rotation)). If you drive Playwright or Puppeteer yourself, keep one browser context per proxy session and destroy the context when the session ends.

## Choosing a rotation cadence

Three policies cover nearly everything:

- **Per request** — a new IP on every call. Right for stateless page pulls, sitemaps, product pages, SERP fetches.

- **Time based** — hold an IP for N minutes, then swap. Useful for moderate crawls where connection reuse matters more than raw IP diversity.

- **Session based** — one IP per logical user journey. Mandatory for logins, checkouts and multi-step forms.

Rotating too aggressively is a real failure mode, not just a cost question: sites that fingerprint sessions see a request chain jumping between countries and challenge it. Rotating too little re-creates the rate limit you were escaping.

## Which network to rotate

| Network | Rotation style | From | Use it when |
| --- | --- | --- | --- |
| Rotating datacenter | Per request, huge throughput | $0.50/GB | Tolerant targets, internal APIs, sitemaps, bulk HTML |
| Rotating residential | Per request or sticky | $0.80/GB | Retail, travel, classifieds, anything IP-reputation aware |
| Static ISP | No rotation, dedicated IP | from $2.50/IP/month | Logged-in accounts, long sessions, speed with trust |
| Mobile 4G/5G | Carrier-level NAT, sticky or rotating | $2.50/GB | The hardest defences; mobile-only app endpoints |
| IPv6 | Per request, very cheap bandwidth | $0.20/GB | IPv6-ready targets and high-volume, low-value pages |

Start on the cheapest network the target tolerates and escalate only on measured block rate. Many teams pay residential rates for pages a ../../datacenter-proxies/ pool would serve at full speed, and only need ../../cheap-residential-proxies/ for the 10-20% of routes that actually fight back.

### Free rotating proxies: what you are actually paying

Public lists exist, and they are fine for a throwaway test. They are not fine for production: nodes disappear within hours, throughput is unpredictable, and the operator sits in the middle of your TLS termination if you accept a broken chain. Vendor guidance is unanimous that free pools carry real data-exposure risk and should not handle credentials or anything commercial ([ScraperAPI](https://www.scraperapi.com/blog/what-are-rotating-proxies/)). "Free rotating residential proxy" offers are the same trade with a better name. Budget rotation is a legitimate goal; anonymous free rotation is not a route to it.

## Rotation hygiene: subnets, headers, retries

If you build your own rotator, three details separate a working one from a list that gets banned as a block.

**Distribute by subnet, not just by IP.** Anti-bot systems reason about the third IPv4 octet and about ASN ownership, so consecutive requests from `198.51.100.4` and `198.51.100.72` look like one actor. Track the last subnet used and avoid repeating it ([Scrapfly](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping)).

**Rotate identity, not only IP.** A fresh residential IP that reuses one User-Agent, one TLS fingerprint and one header order is still one client. Keep header sets bound to sessions.

**Weight and cool down.** Penalise recently used and failing nodes, promote healthy ones, give blocked nodes a recovery window instead of deleting them.

```
import time, random
from collections import defaultdict

cooldown = {}          # ip -> unix ts when usable again
fails = defaultdict(int)

def pick(pool, last_subnet):
    now = time.time()
    live = [p for p in pool
            if cooldown.get(p, 0) < now
            and p.split(".")[2] != last_subnet]
    if not live:
        live = [p for p in pool if cooldown.get(p, 0) < now]
    weights = [1.0 / (1 + fails[p]) for p in live]
    return random.choices(live, weights=weights, k=1)[0]

def penalise(ip, seconds=120):
    fails[ip] += 1
    cooldown[ip] = time.time() + seconds
```

A fuller Python implementation, including async validation, is in our walkthrough on ../how-to-rotate-proxies-python/.

## The cost math nobody puts in the docs

Bandwidth billing charges you for bytes, including the bytes of every block page and every retry. Take 100,000 product pages at roughly 250 KB of HTML each: that is about 25 GB, so around $20 on residential at $0.80/GB — if every request succeeds. At a 30% block rate with two retries the same job moves 35-40 GB and you still have to build parsing, rendering and challenge handling around it.

Per-success pricing inverts the risk. The same 100,000 pages through the ../../web-scraping-api/ at $0.0002 per page is $20 for clean Markdown with residential rotation, retries and parsing already inside the call — and failed calls are not billed at all. Add JS rendering only where you need it ($0.001/page) instead of rendering everything.

```
curl https://api.quanticdata.io/v1/scrape \
  -H "Authorization: Bearer $QD_API_KEY" \
  -d '{ "url": "https://example.com/product/123" }'

# { "success": true,
#   "data": { "markdown": "# Product 123\n…" },
#   "usage": { "cost_usd": 0.0002 } }
```

The honest split: raw rotating proxies win when you need full control of the request, unusual protocols (../../socks5-proxies/ for non-HTTP traffic), or you are already running a mature scraping stack. A per-success API wins when the deliverable is parsed data and you would rather not maintain a rotator, a challenge solver and a renderer.

## Rotating proxies inside agent workflows

Agents change the access pattern. An LLM does not issue 10,000 uniform requests; it issues a handful of unpredictable ones, mid-conversation, and then wants clean text rather than HTML. Handing a model raw proxy credentials means it also inherits your retry logic, your bandwidth meter and your block handling.

The cleaner shape is to expose rotation as a tool. Our ../../mcp-server/ gives Claude, Cursor and other MCP clients search, scrape, map, crawl, batch and SEO audit tools with residential rotation underneath, billed per success, with $2 of free usage every month. The agent asks for a page; IP selection, sticky handling and retries stay on the server side where they belong. If you are wiring this into a RAG pipeline instead of a chat client, the same primitives are available through the ../../web-data-api-for-ai/.

## Common mistakes to avoid

- **Counting 200s as successes.** Validate content, not status codes.

- **Rotating during a login.** Bind the flow to a sticky session first.

- **Over-pinning geography.** Country targeting on every request shrinks your usable pool and raises failure rates.

- **Ignoring concurrency limits.** A large pool does not mean unlimited parallel connections; ramp up and watch latency.

- **Chasing "unlimited bandwidth".** Unlimited plans usually cap concurrency or IP freshness; measure cost per successful record instead.

- **Skipping the legal read.** Rotation is a technical control, not permission. Check terms, robots directives and personal-data rules for your target — see ../is-web-scraping-legal-in-us/. This is general information, not legal advice.

### Sources & further reading

- [Proxy Rotation: How It Works and How to Configure It — Browserless](https://www.browserless.io/blog/proxy-rotation)

- [How to Rotate Proxies in Python — ZenRows](https://www.zenrows.com/blog/rotate-proxies-python)

- [What Are Rotating Proxies? (Functions, Types, and Security) — ScraperAPI](https://www.scraperapi.com/blog/what-are-rotating-proxies/)

- [How to Rotate Proxies in Web Scraping — Scrapfly](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping)

## FAQ

Quick answers on how to use rotating proxies.

[Something else? Ask us →](mailto:hello@quanticdata.io)

### How do I use rotating proxies in Python?

Point your `requests` session at the gateway host and port with your credentials in the URL, set a timeout, and send. The provider swaps the exit IP per request. Verify with an IP echo endpoint, treat 403, 429 and challenge pages as failures, and retry with a fresh connection rather than the same one.

### Are free rotating proxies safe to use?

For a throwaway connectivity test, yes. For production, no. Free nodes die within hours, throughput is unpredictable, and an unknown operator sits between you and the target. Vendor guidance consistently warns against passing credentials or commercial traffic through public lists. Budget rotation is better solved with cheap paid residential or datacenter bandwidth.

### Should I use rotating datacenter or residential proxies?

Start with rotating datacenter proxies: they are faster and cheaper, from $0.50/GB, and plenty of targets accept them. Escalate to residential from $0.80/GB only for routes where you measure elevated block rates — typically retail, travel and classifieds, where IP reputation is scored. Escalating by measurement, not by default, is what keeps costs sane.

### How often should the IP rotate?

Per request for stateless pulls, per session for anything with login, cart or token state, and time-based (a few minutes) as a middle ground. Rotating too fast breaks sessions and triggers challenges; rotating too slowly re-creates rate limits. Sticky sessions on QuanticData rotating pools can hold one IP for up to 120 minutes.

### Do rotating proxies with unlimited bandwidth make sense?

Sometimes, but read the constraint that replaces the bandwidth cap: concurrency limits, restricted pools, or slower IP refresh. The metric that matters is cost per successful record, not per GB. Compare an unlimited plan against per-GB rotation and against per-success API pricing using your own measured block and retry rates.

### What is the cheapest way to run rotating proxies at scale?

Match the network to each route, cache aggressively, and stop paying for retries. Cheap IPv6 or datacenter bandwidth handles tolerant pages; residential covers the hard minority. If most of your spend is retries and parsing, per-success pricing — $0.0002 per scraped page, nothing for failures — usually beats raw bandwidth.

## Rotate IPs, or skip the rotator entirely

Run rotating proxies from $0.50/GB across residential, datacenter and mobile networks with sticky sessions up to 120 minutes on one endpoint. Or let our Web Scraping API do the rotation, retries and parsing for you at $0.0002 per successful page — failed calls cost nothing, with $2 free every month.

[Start free — $2/month included](https://app.quanticdata.io/register)[Explore Rotating Proxies from $0.50/GB](https://quanticdata.io/rotating-proxies/)

## Related reading

[Proxies How to Rotate Proxies in Python Rotate proxies in Python three ways — a cycled list, an async pool with cooldowns, or a rotating gateway — plus the cost math nobody publishes. Read →](https://quanticdata.io/blog/how-to-rotate-proxies-python/) [Proxies How to Rotate Proxy in Selenium Python Chrome fixes its proxy at launch. Here are the three real ways to rotate IPs in Selenium Python, with code, auth handling and honest bandwidth math. Read →](https://quanticdata.io/blog/how-to-rotate-proxy-in-selenium-python/) [SEO data How to Perform an SEO Audit A practical six-step SEO audit process with a checklist, the crawler-vs-user diff most audits skip, and how to run the whole thing programmatically. Read →](https://quanticdata.io/blog/how-to-perform-an-seo-audit/)

---

Source: https://quanticdata.io/blog/how-to-use-rotating-proxies/ · Site index for AI: https://quanticdata.io/llms.txt