Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Playwright wait for page to load in Python: every wait timed on a slow page

Playwright waits timed in Python on a page that adds its quotes after a delay: goto with commit, domcontentloaded, load and networkidle all returned within 1.67 seconds with 0 quotes on the page, while wait_for_selector returned after 11.24 seconds with all 10; the card recommends waiting for the element, the response or the URL.
Playwright waits timed in Python on a page that adds its quotes after a delay: goto with commit, domcontentloaded, load and networkidle all returned within 1.67 seconds with 0 quotes on the page, while wait_for_selector returned after 11.24 seconds with all 10; the card recommends waiting for the element, the response or the URL.

In Playwright, page.goto() already waits for the page's load event. To wait for the content you need, wait for that content: page.wait_for_selector("css") or expect(locator).to_be_visible() for an element, page.expect_response() for an API call, page.wait_for_url() after a click. networkidle and fixed sleeps are unreliable.

We timed every option on 20 November 2026 with Python 3.12.10, Playwright 1.64.1 and Chromium 156, through a QuanticData Residential Premium session in the US, on quotes.toscrape.com/js-delayed/, a public practice page that adds its quotes after a delay, like many real sites that load data after the first paint.

What goto() waits for, and why it was not enough

for wu in ("commit", "domcontentloaded", "load", "networkidle"):
    page.goto("https://quotes.toscrape.com/js-delayed/", wait_until=wu)
    print(wu, page.locator("div.quote").count())
wait_untilReturns whenOur timeQuotes on the page
commitThe response starts arriving1.11 s0
domcontentloadedThe HTML is parsed1.36 s0
load (default)The page and its resources are loaded1.38 s0
networkidleNo network activity for a short while1.67 s0

All four returned with nothing to scrape. The page waits on a timer before adding its quotes, so the network goes quiet first and the content comes later. Playwright's own documentation marks networkidle as discouraged and recommends waiting on the elements you need instead, which is what the next sections do.

Wait for the element

page.goto(URL, wait_until="domcontentloaded")
page.wait_for_selector("div.quote")                    # 11.24 s, then 10 quotes

from playwright.sync_api import expect
expect(page.locator("div.quote").first).to_be_visible(timeout=30000)   # 10.76 s

page.locator("div.quote").first.wait_for(state="visible")              # 10.01 s

All three returned once the quotes existed, and all 10 were there. Combine them with wait_until="domcontentloaded": there is no reason to also wait for images and fonts when the element is what you need. The element waits measured the real delay of the page, about ten seconds, instead of a guess, so they also tell you how slow a site is.

When one element is not enough, wait for a condition in the page: page.wait_for_function("document.querySelectorAll('div.quote').length >= 10") returned in 10.03 s with the full list.

Wait for the data itself: expect_response

Often the content comes from a JSON call. Waiting for that response gives you the data without parsing the page at all. On the infinite-scroll page of the same site:

page.goto("https://quotes.toscrape.com/scroll", wait_until="domcontentloaded")
with page.expect_response(lambda r: "/api/quotes" in r.url and r.status == 200) as info:
    page.mouse.wheel(0, 20000)
data = info.value.json()
print(info.value.url, data["page"], len(data["quotes"]))
# https://quotes.toscrape.com/api/quotes?page=1  1  10

Start the with block before the action that triggers the request, as above, or you can miss a fast response. More on finding these calls in scraping dynamic web pages in Python.

Wait after a click: wait_for_url

page.click("li.next a")
page.wait_for_url("**/page/2/")      # 0.66 s
print(page.url)                      # https://quotes.toscrape.com/page/2/

After a click that navigates, wait_for_url with a glob pattern waits until the address changes; in our run it returned in 0.66 s. Follow it with an element wait when you need the new page's content, since a changed address does not mean the data has arrived.

Why not just sleep?

page.wait_for_timeout(3000) paused three seconds and the page still had 0 quotes: the data came after about ten. A fixed sleep is either too short, and you scrape an empty page, or too long, and every page costs the full delay. Keep sleeps for debugging. The same goes for time.sleep() in Selenium, where WebDriverWait plays the role of wait_for_selector (see Selenium find_element).

Which wait for which job

  • You scrape a list or a detail block: goto(url, wait_until="domcontentloaded"), then wait_for_selector on the block. It returned the moment our quotes appeared and never earlier.
  • You need every item, not the first: wait_for_function with a count, or wait for a "last item" or footer element that renders after the list.
  • The page loads its data from an API: expect_response, and read the JSON instead of the DOM.
  • You clicked and the page changes: wait_for_url, then an element wait on the new page.
  • You only need the HTML the server sent: the default load is enough, and a plain HTTP client would be cheaper still.

Each of these waits for evidence that the thing you want exists. That is the whole rule: a page is loaded, for a scraper, when the data is on it.

Timeouts

try:
    page.wait_for_selector("div.quote", timeout=2000)
except PlaywrightTimeoutError as e:
    print(e)   # Page.wait_for_selector: Timeout 2000ms exceeded.

page.set_default_timeout(45000)   # for every wait on this page

Waits default to 30 seconds; when one runs out, Playwright raises "Timeout 30000ms exceeded". Import the error with from playwright.sync_api import TimeoutError as PlaywrightTimeoutError. When a wait times out, take a screenshot before closing, page.screenshot(path="debug.png", full_page=True): it usually shows a consent banner, an error page or a block page where you expected content. We saved one of the rendered page and one of a single quote with locator.screenshot().

Waiting through a proxy

Pass the proxy once in chromium.launch(proxy={"server": ..., "username": ..., "password": ...}); see Playwright proxy setup. QuanticData residential proxies give a home IP by country, with a session ID in the username, and mobile proxies put the browser on a carrier network. When you need rendered pages at volume without running browsers, our web scraping API renders them for you. The rest of a Playwright scraper is in Playwright for scraping.

Sources & further reading

FAQ

Quick answers on playwright wait for page to load.

Something else? Ask us

How do I wait for a page to load in Playwright Python?

page.goto(url) waits for the load event by default. To wait for the content you need, call page.wait_for_selector("css") or expect(locator).to_be_visible() after it. In our test, load returned with 0 items and wait_for_selector with all 10.

Should I use networkidle in Playwright?

Rarely. On our delayed page networkidle returned in 1.67 s before any data appeared, and the Playwright documentation marks it as discouraged. Wait for an element or a response instead.

How do I wait for an element to be visible in Playwright?

page.wait_for_selector("div.quote"), page.locator("div.quote").first.wait_for(state="visible") or expect(locator).to_be_visible(). All three returned in 10 to 11.3 s on our test page, once the quotes existed.

How do I wait for an API response in Playwright?

Wrap the action in with page.expect_response(lambda r: "/api/" in r.url) as info:, then read info.value.json(). On our scroll page it returned page 1 with 10 quotes.

How do I change the timeout in Playwright?

Pass timeout= in milliseconds to a single call, or call page.set_default_timeout(45000) for the page. A wait with timeout 2000 raised "Timeout 2000ms exceeded" in our test.

Run Playwright from a home or mobile IP

Residential and mobile proxies with username and password for chromium.launch, and a session ID in the username.

Related reading