pandas.read_html() reads every <table> in an HTML document and returns a list of DataFrames. Give it a URL, a file, or HTML wrapped in io.StringIO; keep only the table you want with match="text" and attrs={"class": "..."}; then clean the columns. It needs lxml, or Beautiful Soup with html5lib, installed.
We ran every call below on 20 November 2026 with Python 3.12.10 and pandas 3.0.3, on the Wikipedia list of countries by GDP (nominal). For the whole job of finding a table on a page and saving it, start with our guide How to scrape a table from a website with Python; this post is about read_html itself.
The basic call, and the 403
import pandas as pd
URL = "https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)"
pd.read_html(URL)
# urllib.error.HTTPError: HTTP Error 403: Forbidden
Given a URL, pandas downloads it with Python's urllib and its generic User-Agent. Wikimedia asks automated clients to identify themselves with a User-Agent that names the tool and a way to contact its owner, and refuses generic ones. Send one with storage_options:
UA = {"User-Agent": "table-reader-example/1.0 (+https://example.com/contact)"}
tables = pd.read_html(URL, storage_options=UA)
print(len(tables)) # 8
Put your own tool name and contact in it. That fixed the 403 and returned the 8 tables of the page.
Pick the table: match and attrs
read_html always returns a list. On this page it held 8 DataFrames; two were article tables with the class wikitable, shaped (222, 4) and (14, 3). Two arguments narrow it down:
import io
df = pd.read_html(io.StringIO(html), match="IMF", attrs={"class": "wikitable"})[0]
print(df.shape) # (222, 4)
print(list(df.columns))
# ['Country/Territory', 'IMF (2026)[1]', 'World Bank (2025)[6]', 'United Nations (2024)[7]']
| Argument | What it does |
|---|---|
match="IMF" | Keeps tables whose text contains the string or regex |
attrs={"class": "wikitable"} | Keeps tables with these HTML attributes, such as an id |
header=0 | Uses that row as column names when the table has no <th> |
skiprows=1 | Drops rows at the top |
thousands=, decimal= | Number format: on a test cell 1.234,5, thousands="." and decimal="," gave 1234.5 |
extract_links="body" | Returns cells as (text, link) |
flavor="bs4" | Parses with Beautiful Soup and html5lib instead of lxml |
When nothing matches, read_html raises instead of returning an empty list: match="zzz-no-such-text" stopped with ValueError: No tables found matching pattern 'zzz-no-such-text'. Catch that error when the page you read changes often, and log the page you got, so a renamed column header does not silently stop a scheduled job.
pandas 3: wrap HTML strings in StringIO
When you download the page yourself, pass the HTML in a StringIO. On pandas 3.0.3 a plain string failed:
pd.read_html(r.text)
# FileNotFoundError: [Errno 2] No such file or directory: <!DOCTYPE html> ...
pd.read_html(io.StringIO(r.text)) # works
pandas treats a string as a path or URL. io.StringIO(r.text) or io.BytesIO(r.content) tells it the content is the document.
read_html through a proxy
read_html has no proxy argument. To fetch through one, download with requests, with the same identifying User-Agent, and hand the HTML over:
import requests
PROXY = "http://USER-country-us-sid-t02-ttl-30m:PASS@residentialpremium.quanticdata.io:8080"
r = requests.get(URL, headers=UA, proxies={"http": PROXY, "https": PROXY}, timeout=30)
print(r.status_code, len(r.content)) # 200 683373
html = r.text
tables = pd.read_html(io.StringIO(html)) # 8 tables
This also gives you timeouts, retries and cookies, which read_html's own download does not. The requests proxy setup is in a proxy in Python requests.
From text to numbers
All four columns of the GDP table came back as text (dtype str in pandas 3). The cells carry footnote marks, a few say —N/a, and some carry a year, such as 98,964 (2024). Clean the names, then the values:
import re
df.columns = [re.sub(r"\[.*?\]", "", c).strip() for c in df.columns]
df["Country/Territory"] = df["Country/Territory"].str.replace(r"\[.*?\]", "", regex=True).str.strip()
col = "IMF (2026)"
df[col] = pd.to_numeric(df[col].str.extract(r"^([\d,]+)")[0].str.replace(",", ""),
errors="coerce")
print(df.dtypes[col]) # float64
print(df[df["Country/Territory"] != "World"].nlargest(3, col)["Country/Territory"].tolist())
# ['United States', 'China', 'Germany']
Taking the leading number first matters. A plain pd.to_numeric(..., errors="coerce") turned 30 cells into NaN, because "98,964 (2024)" is not a number; extracting the digits before converting left 24, the cells that really have no value. Count the NaN after every conversion so you know what you lost.
Keep the links
lk = pd.read_html(io.StringIO(html), match="IMF", attrs={"class": "wikitable"},
extract_links="body")[0]
print(lk.iloc[1, 0])
# ('United States', 'https://en.wikipedia.org/wiki/Economy_of_United_States')
Each body cell becomes a (text, href) pair, None where the cell has no link. Split them into two columns with lk[c].str[0] and lk[c].str[1]: for the first country that gave "United States" and the URL of its economy article, ready to follow for the next level of data.
Fix "No tables found"
pd.read_html(io.StringIO("<html><body><div id='app'></div></body></html>"))
# ValueError: No tables found
read_html only sees <table> elements in the HTML it receives. The usual causes: the table is drawn by JavaScript (render it first, see scraping dynamic web pages), the "table" is built from div elements (parse it with Beautiful Soup web scraping), or you downloaded an error or consent page instead of the content. Print the first 500 characters of what you passed in before changing the arguments.
Where the HTML comes from
We fetched the GDP page through a QuanticData Residential Premium session in the US. residential proxies and mobile proxies take the country and a session ID in the username and work with requests exactly as above; the Residential Basic free trial lets you run the code first.


