# Retry-After Header: Read It, Wait, Retry

> The Retry-After header says how long to wait after a 429 or 503: seconds or an HTTP date. Python and Node code tested on a real 429, plus batching.

[Home](https://quanticdata.io/) / [Blog](https://quanticdata.io/blog/) / Retry-After Header: Read It, Wait, Retry

# Retry-After header: how to read it after a 429, with code that waits

Web scraping · Nov 20, 2026 · 6 min read · By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData

![A client receives 429 with Retry-After: 10, waits 10 seconds and gets 200 on the next try; the card shows our two tests of 20 November 2026: the Python and Node helpers each succeeded on the second try, and one batch job fetched 20 of 20 URLs in 17 seconds.](https://quanticdata.io/blog/retry-after-header/hero.webp?v=20261120h)

On this page

- [The two formats: seconds or a date](https://quanticdata.io/blog/retry-after-header/#the-two-formats-seconds-or-a-date)
- [A real 429 with Retry-After](https://quanticdata.io/blog/retry-after-header/#a-real-429-with-retry-after)
- [In Python: wait what the server asks](https://quanticdata.io/blog/retry-after-header/#in-python-wait-what-the-server-asks)
- [The same helper in JavaScript, with fetch](https://quanticdata.io/blog/retry-after-header/#the-same-helper-in-javascript-with-fetch)
- [Rate-limit headers: see the limit before you reach it](https://quanticdata.io/blog/retry-after-header/#rate-limit-headers-see-the-limit-before-you-reach-it)
- [Need fewer calls: one batch job for a list of URLs](https://quanticdata.io/blog/retry-after-header/#need-fewer-calls-one-batch-job-for-a-list-of-urls)

The `Retry-After` header tells a client how long to wait before sending the request again. Servers send it with 429 Too Many Requests and 503 Service Unavailable. Its value is either a whole number of seconds, such as `Retry-After: 10`, or an HTTP date. A well-behaved client waits that long, then tries once more.

**Run it on the free credit.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-lead&utm_content=retry-after-header) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

We tested the code in this post on 20 November 2026 against a real 429 from the QuanticData API, then ran a list of 20 URLs as one batch job. Both runs are below.

## The two formats: seconds or a date

| Form | Example | Meaning |
| --- | --- | --- |
| Delay in seconds | `Retry-After: 10` | Wait 10 seconds from when you received the response |
| HTTP date | `Retry-After: Fri, 20 Nov 2026 19:00:00 GMT` | Do not retry before that moment |

The number is always seconds, never milliseconds, and it has no decimals. That is the most common mistake in client code: passing the value straight to a function that sleeps in milliseconds turns a 10-second wait into 10 milliseconds, and the retry arrives while the limit still applies. The date form is rarer but the standard allows it, so a client should handle both.

The header is defined in RFC 9110. It can accompany 429, 503 and redirects; this post is about the first two.

## A real 429 with Retry-After

To have a real one to test against, we used an endpoint with a documented pause: a QuanticData [rotation link](https://quanticdata.io/blog/change-proxy-ip-by-link/), which restarts a sticky proxy session and can be called once every 10 seconds per session. Two calls in a row:

```bash
$ curl -s -i https://api.quanticdata.io/v1/public/rotate/rl_TOKEN
HTTP/1.1 200 OK
{"type":"response","message":"Session rotated. The next request on this proxy string gets a new IP.", ...}

$ curl -s -i https://api.quanticdata.io/v1/public/rotate/rl_TOKEN
HTTP/1.1 429 Too Many Requests
retry-after: 10
{"type":"error","message":"This session was rotated less than 10 seconds ago. Try again in 10 s.",
 "payload":{"code":"RATE_LIMITED","scope":"session","retryAfter":10}}
```

The body repeats the wait in `payload.retryAfter` and says which limit was reached in `scope`. The header is what generic HTTP clients and retry libraries read.

## In Python: wait what the server asks

```python
import time, email.utils, datetime, requests

def wait_seconds(resp, attempt, cap=60):
    """Retry-After if the server sent it, else 1, 2, 4, 8 seconds."""
    value = resp.headers.get("Retry-After")
    if value:
        if value.isdigit():                                   # delay in seconds
            return min(int(value), cap)
        when = email.utils.parsedate_to_datetime(value)       # HTTP date
        now = datetime.datetime.now(datetime.timezone.utc)
        return min(max(0, (when - now).total_seconds()), cap)
    return min(2 ** attempt, cap)

def call(method, url, tries=4, **kwargs):
    for attempt in range(tries):
        resp = requests.request(method, url, timeout=60, **kwargs)
        if resp.status_code not in (429, 502, 503, 504):
            return resp
        time.sleep(wait_seconds(resp, attempt))
    return resp
```

Run against the link twice in a row (Python 3.12, with a print added inside the loop):

```
call 1: 200 after 0.5 s Session rotated. The next request on this proxy string gets a new IP.
  429, Retry-After=10, waiting 10 s
call 2: 200 after 10.9 s Session rotated. The next request on this proxy string gets a new IP.
```

If you would rather configure than write, `urllib3.util.Retry` honours `Retry-After` by default when 429 is in its `status_forcelist`; see [retries in Python requests](https://quanticdata.io/blog/python-requests-retry/).

**Try it on your own targets.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-inline&utm_content=retry-after-header) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

## The same helper in JavaScript, with fetch

```
function waitSeconds(res, attempt, cap = 60) {
  const value = res.headers.get("Retry-After");
  if (value) {
    const seconds = /^\d+$/.test(value) ? Number(value) : (Date.parse(value) - Date.now()) / 1000;
    return Math.min(Math.max(0, seconds), cap);
  }
  return Math.min(2 ** attempt, cap);
}

async function call(url, options = {}, tries = 4) {
  let res;
  for (let attempt = 0; attempt < tries; attempt++) {
    res = await fetch(url, options);
    if (![429, 502, 503, 504].includes(res.status)) return res;
    await new Promise((r) => setTimeout(r, waitSeconds(res, attempt) * 1000));   // seconds to milliseconds
  }
  return res;
}
```

Same test on Node.js 22: the first call returned 200 in 0.4 s, the second met the 429, waited 10 seconds and returned 200 after 10.6 s.

Three rules are built into both helpers:

- **Retry only what can succeed later:** 429 and the transient 502, 503 and 504. A 400, 401, 402 or 404 will answer the same way again, so retrying it only spends your quota.
- **Cap the tries.** Four attempts, then hand the error to the caller.
- **Cap the wait** at something your job can afford, and stop if the server asks for more.

## Rate-limit headers: see the limit before you reach it

`Retry-After` arrives when the limit is already reached. Many APIs also say how much room is left on every response. The QuanticData API sends three headers on each call and a fourth on a 429:

| Header | What it holds |
| --- | --- |
| `X-RateLimit-Limit` | Requests allowed in the hourly window of your key |
| `X-RateLimit-Remaining` | Requests left in that window |
| `X-RateLimit-Reset` | When the window starts again, as a timestamp |
| `Retry-After` | On a 429: seconds to wait |

Two windows apply to a key: requests per minute, set by the tier, and an hourly window of 1,000 requests or 60 times the per-minute figure, whichever is larger. The hourly window counts every call made with the key. The published tiers:

| Tier | Requests per minute | Batch concurrency |
| --- | --- | --- |
| Pay as you go | 20 | 5 |
| Starter | 300 | 10 |
| Growth | 600 | 20 |
| Scale | 1200 | 20 |

A client that reads `X-RateLimit-Remaining` and slows down as it nears zero rarely sees a 429 at all. What each tier costs is on the [pricing page](https://quanticdata.io/pricing/).

## Need fewer calls: one batch job for a list of URLs

The best way to stay inside a per-minute limit is to need fewer requests. If you already have the URLs, the [web scraping API](https://quanticdata.io/web-scraping-api/) takes up to 5,000 of them in one batch call, fetches them concurrently on its side and lets you poll one job. We sent 20 catalogue pages of a public practice site:

```bash
$ curl -s -X POST https://api.quanticdata.io/v1/scraper/batch \
    -H "Authorization: Bearer $QD_API_KEY" -H "Content-Type: application/json" \
    -d '{"urls":["https://books.toscrape.com/catalogue/page-1.html", ... 20 URLs ...],"format":"markdown","mode":"summary","concurrency":5}'
{"type":"response","message":"Batch started",
 "payload":{"id":"JOB_ID","status":"running","total":20,"statusUrl":"/api/v1/scraper/batch/JOB_ID"}}

$ curl -s https://api.quanticdata.io/v1/scraper/batch/JOB_ID -H "Authorization: Bearer $QD_API_KEY"
{"status":"running","total":20,"completed":9,"failed":0, ...}      # 7 seconds in
{"status":"completed","total":20,"completed":20,"failed":0, ...}   # 17 seconds in
```

Twenty pages, 20 of 20 completed, in 17 seconds, with five HTTP requests from our side: one to submit and four polls. Sent one by one, the same list is twenty requests, which is a full minute of the pay-as-you-go allowance. Each item in the finished job carries its URL, status, title and content length; add `include_content=true` to a poll to get the Markdown, or pass a `webhook` and skip polling.

For search results the equivalent is the bulk search job, and for a whole site the crawl job; a list built by following [pagination](https://quanticdata.io/blog/web-scraping-pagination/) fits a batch well. All three are in the [API reference](https://quanticdata.io/docs/).

When the 429 comes from a website you are scraping and not from an API you pay for, the reasoning changes: there the limit is the site's own, and the answer is to slow down. That case is covered in [429 Too Many Requests when scraping](https://quanticdata.io/blog/429-too-many-requests-web-scraping/).

### Sources & further reading

- [RFC 9110: Retry-After](https://www.rfc-editor.org/rfc/rfc9110#field.retry-after)
- [RFC 6585: 429 Too Many Requests](https://www.rfc-editor.org/rfc/rfc6585#section-4)
- [MDN: Retry-After header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Retry-After)
- [urllib3: Retry](https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry)
- [Books to Scrape (practice site)](https://books.toscrape.com/)

## FAQ

Quick answers on Retry-After header.

[Something else? Ask us](mailto:hello@quanticdata.io)

### What is the Retry-After header?

An HTTP response header that tells the client how long to wait before the next request. Servers send it with 429 Too Many Requests and 503 Service Unavailable, and sometimes with redirects.

### Is Retry-After in seconds or milliseconds?

Seconds, as a whole number, or an HTTP date. It is never milliseconds. In JavaScript multiply by 1000 before passing it to `setTimeout`.

### What does a Retry-After header look like?

`Retry-After: 10` for a delay, or `Retry-After: Fri, 20 Nov 2026 19:00:00 GMT` for a date. The 429 we triggered on the QuanticData API carried `retry-after: 10`.

### What should a client do after a 429?

Stop sending, wait the time in `Retry-After`, and try again a limited number of times. In our test both helpers waited 10 seconds and received 200 on the second try.

### What if there is no Retry-After header?

Back off on your own: wait 1 second, then 2, 4 and 8, with a cap. If the API sends `X-RateLimit-Reset`, wait until that time.

### How do I avoid 429 errors when I have many URLs?

Send them as one batch job instead of one request each. Our 20-URL batch finished 20 of 20 in 17 seconds with five requests in total: one to submit and four to poll.

## Send the list, not twenty requests

The batch endpoint takes up to 5,000 URLs in one call and returns one job to poll, with a webhook when it is done.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-end&utm_content=retry-after-header) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

## Related reading

- [Screen Scraping: Meaning, Example, vs API](https://quanticdata.io/blog/screen-scraping/): Web scraping · Screen scraping reads data from what a program shows on screen instead of from an API or the page source. We read the same ten records from one public page three ways through a residential proxy: off the rendered screen, out of the HTML, and from the JSON API.
- [Web Scraping Project Ideas, Started for Real](https://quanticdata.io/blog/web-scraping-project-ideas/): Web scraping · Five web scraping project ideas, each one started for real on 20 November 2026: a price tracker, a library release dashboard, a question analysis, an open-source landscape and a topic timeline, with the code, the first run and what to build next.
- [How to Use an API Key: curl, Python and JS](https://quanticdata.io/blog/how-to-use-an-api-key/): Web scraping · An API key is a secret string you send with every request so the service knows which account is calling. Nothing is installed: you add one header. Below: the same call in curl, Python, JavaScript and PowerShell, run on 20 November 2026 against the QuanticData API, the exact 401 messages for the usual mistakes, and how to keep the key out of your code.

---

Source: https://quanticdata.io/blog/retry-after-header/ · Site index for AI: https://quanticdata.io/llms.txt
