Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Retry-After header: how to read it after a 429, with code that waits

A client receives 429 with Retry-After: 10, waits 10 seconds and gets 200 on the next try; the card shows our two tests of 20 November 2026: the Python and Node helpers each succeeded on the second try, and one batch job fetched 20 of 20 URLs in 17 seconds.
A client receives 429 with Retry-After: 10, waits 10 seconds and gets 200 on the next try; the card shows our two tests of 20 November 2026: the Python and Node helpers each succeeded on the second try, and one batch job fetched 20 of 20 URLs in 17 seconds.

The Retry-After header tells a client how long to wait before sending the request again. Servers send it with 429 Too Many Requests and 503 Service Unavailable. Its value is either a whole number of seconds, such as Retry-After: 10, or an HTTP date. A well-behaved client waits that long, then tries once more.

We tested the code in this post on 20 November 2026 against a real 429 from the QuanticData API, then ran a list of 20 URLs as one batch job. Both runs are below.

The two formats: seconds or a date

FormExampleMeaning
Delay in secondsRetry-After: 10Wait 10 seconds from when you received the response
HTTP dateRetry-After: Fri, 20 Nov 2026 19:00:00 GMTDo not retry before that moment

The number is always seconds, never milliseconds, and it has no decimals. That is the most common mistake in client code: passing the value straight to a function that sleeps in milliseconds turns a 10-second wait into 10 milliseconds, and the retry arrives while the limit still applies. The date form is rarer but the standard allows it, so a client should handle both.

The header is defined in RFC 9110. It can accompany 429, 503 and redirects; this post is about the first two.

A real 429 with Retry-After

To have a real one to test against, we used an endpoint with a documented pause: a QuanticData rotation link, which restarts a sticky proxy session and can be called once every 10 seconds per session. Two calls in a row:

$ curl -s -i https://api.quanticdata.io/v1/public/rotate/rl_TOKEN
HTTP/1.1 200 OK
{"type":"response","message":"Session rotated. The next request on this proxy string gets a new IP.", ...}

$ curl -s -i https://api.quanticdata.io/v1/public/rotate/rl_TOKEN
HTTP/1.1 429 Too Many Requests
retry-after: 10
{"type":"error","message":"This session was rotated less than 10 seconds ago. Try again in 10 s.",
 "payload":{"code":"RATE_LIMITED","scope":"session","retryAfter":10}}

The body repeats the wait in payload.retryAfter and says which limit was reached in scope. The header is what generic HTTP clients and retry libraries read.

In Python: wait what the server asks

import time, email.utils, datetime, requests

def wait_seconds(resp, attempt, cap=60):
    """Retry-After if the server sent it, else 1, 2, 4, 8 seconds."""
    value = resp.headers.get("Retry-After")
    if value:
        if value.isdigit():                                   # delay in seconds
            return min(int(value), cap)
        when = email.utils.parsedate_to_datetime(value)       # HTTP date
        now = datetime.datetime.now(datetime.timezone.utc)
        return min(max(0, (when - now).total_seconds()), cap)
    return min(2 ** attempt, cap)

def call(method, url, tries=4, **kwargs):
    for attempt in range(tries):
        resp = requests.request(method, url, timeout=60, **kwargs)
        if resp.status_code not in (429, 502, 503, 504):
            return resp
        time.sleep(wait_seconds(resp, attempt))
    return resp

Run against the link twice in a row (Python 3.12, with a print added inside the loop):

call 1: 200 after 0.5 s Session rotated. The next request on this proxy string gets a new IP.
  429, Retry-After=10, waiting 10 s
call 2: 200 after 10.9 s Session rotated. The next request on this proxy string gets a new IP.

If you would rather configure than write, urllib3.util.Retry honours Retry-After by default when 429 is in its status_forcelist; see retries in Python requests.

The same helper in JavaScript, with fetch

function waitSeconds(res, attempt, cap = 60) {
  const value = res.headers.get("Retry-After");
  if (value) {
    const seconds = /^\d+$/.test(value) ? Number(value) : (Date.parse(value) - Date.now()) / 1000;
    return Math.min(Math.max(0, seconds), cap);
  }
  return Math.min(2 ** attempt, cap);
}

async function call(url, options = {}, tries = 4) {
  let res;
  for (let attempt = 0; attempt < tries; attempt++) {
    res = await fetch(url, options);
    if (![429, 502, 503, 504].includes(res.status)) return res;
    await new Promise((r) => setTimeout(r, waitSeconds(res, attempt) * 1000));   // seconds to milliseconds
  }
  return res;
}

Same test on Node.js 22: the first call returned 200 in 0.4 s, the second met the 429, waited 10 seconds and returned 200 after 10.6 s.

Three rules are built into both helpers:

  • Retry only what can succeed later: 429 and the transient 502, 503 and 504. A 400, 401, 402 or 404 will answer the same way again, so retrying it only spends your quota.
  • Cap the tries. Four attempts, then hand the error to the caller.
  • Cap the wait at something your job can afford, and stop if the server asks for more.

Rate-limit headers: see the limit before you reach it

Retry-After arrives when the limit is already reached. Many APIs also say how much room is left on every response. The QuanticData API sends three headers on each call and a fourth on a 429:

HeaderWhat it holds
X-RateLimit-LimitRequests allowed in the hourly window of your key
X-RateLimit-RemainingRequests left in that window
X-RateLimit-ResetWhen the window starts again, as a timestamp
Retry-AfterOn a 429: seconds to wait

Two windows apply to a key: requests per minute, set by the tier, and an hourly window of 1,000 requests or 60 times the per-minute figure, whichever is larger. The hourly window counts every call made with the key. The published tiers:

TierRequests per minuteBatch concurrency
Pay as you go205
Starter30010
Growth60020
Scale120020

A client that reads X-RateLimit-Remaining and slows down as it nears zero rarely sees a 429 at all. What each tier costs is on the pricing page.

Need fewer calls: one batch job for a list of URLs

The best way to stay inside a per-minute limit is to need fewer requests. If you already have the URLs, the web scraping API takes up to 5,000 of them in one batch call, fetches them concurrently on its side and lets you poll one job. We sent 20 catalogue pages of a public practice site:

$ curl -s -X POST https://api.quanticdata.io/v1/scraper/batch \
    -H "Authorization: Bearer $QD_API_KEY" -H "Content-Type: application/json" \
    -d '{"urls":["https://books.toscrape.com/catalogue/page-1.html", ... 20 URLs ...],"format":"markdown","mode":"summary","concurrency":5}'
{"type":"response","message":"Batch started",
 "payload":{"id":"JOB_ID","status":"running","total":20,"statusUrl":"/api/v1/scraper/batch/JOB_ID"}}

$ curl -s https://api.quanticdata.io/v1/scraper/batch/JOB_ID -H "Authorization: Bearer $QD_API_KEY"
{"status":"running","total":20,"completed":9,"failed":0, ...}      # 7 seconds in
{"status":"completed","total":20,"completed":20,"failed":0, ...}   # 17 seconds in

Twenty pages, 20 of 20 completed, in 17 seconds, with five HTTP requests from our side: one to submit and four polls. Sent one by one, the same list is twenty requests, which is a full minute of the pay-as-you-go allowance. Each item in the finished job carries its URL, status, title and content length; add include_content=true to a poll to get the Markdown, or pass a webhook and skip polling.

For search results the equivalent is the bulk search job, and for a whole site the crawl job; a list built by following pagination fits a batch well. All three are in the API reference.

When the 429 comes from a website you are scraping and not from an API you pay for, the reasoning changes: there the limit is the site's own, and the answer is to slow down. That case is covered in 429 Too Many Requests when scraping.

Sources & further reading

FAQ

Quick answers on Retry-After header.

Something else? Ask us

What is the Retry-After header?

An HTTP response header that tells the client how long to wait before the next request. Servers send it with 429 Too Many Requests and 503 Service Unavailable, and sometimes with redirects.

Is Retry-After in seconds or milliseconds?

Seconds, as a whole number, or an HTTP date. It is never milliseconds. In JavaScript multiply by 1000 before passing it to setTimeout.

What does a Retry-After header look like?

Retry-After: 10 for a delay, or Retry-After: Fri, 20 Nov 2026 19:00:00 GMT for a date. The 429 we triggered on the QuanticData API carried retry-after: 10.

What should a client do after a 429?

Stop sending, wait the time in Retry-After, and try again a limited number of times. In our test both helpers waited 10 seconds and received 200 on the second try.

What if there is no Retry-After header?

Back off on your own: wait 1 second, then 2, 4 and 8, with a cap. If the API sends X-RateLimit-Reset, wait until that time.

How do I avoid 429 errors when I have many URLs?

Send them as one batch job instead of one request each. Our 20-URL batch finished 20 of 20 in 17 seconds with five requests in total: one to submit and four to poll.

Send the list, not twenty requests

The batch endpoint takes up to 5,000 URLs in one call and returns one job to poll, with a webhook when it is done.

Related reading