Data parsing is the process of reading raw input, such as an HTML page, a log line, a CSV file or a PDF, and converting it into structured records that a program can use, typically JSON, rows or objects. A parser splits the input into parts, maps them to named fields and converts text into proper types such as numbers and dates.
Definitions are easy to find. Here is the process on a real page instead: on 20 November 2026 we fetched a live web page, parsed 34,167 characters of HTML into 30 records with Python, typed the strings into numbers, and then ran the same parse server-side in one request. All output below is from that run.
How a data parser works
Every parser, from a JSON reader to a compiler, does the same three things.
- Tokenise. Split the input into meaningful pieces: tags and text in HTML, fields in a CSV line, words and numbers in a log line.
- Build structure. Arrange the pieces according to the format's rules. An HTML parser builds a tree of elements; a JSON parser builds objects and arrays.
- Map and type. Pick the parts you need, give them names, and convert them: "376 points" becomes the integer 376, a date string becomes a timestamp.
The first two steps are generic and come from a library. The third is specific to your source and is where most parsing code lives, and where it breaks.
Data parsing techniques
| Technique | Good for | Breaks when |
|---|---|---|
| Format parsers (json, csv, an HTML parser) | Input that already follows a standard format | The input is malformed or the format is wrong |
| CSS selectors and XPath on an HTML tree | Web pages with stable markup | The site renames a class or moves a block |
| Regular expressions | Short, predictable strings such as prices, counts, dates | The pattern varies more than expected |
| Grammar-based parsers | Languages and query syntax | The input does not fit the grammar |
| Machine learning and LLMs | Free text and documents with varying layouts | Confidence is low and nobody checks the output |
Real jobs combine them. The example below uses an HTML parser for structure, CSS selectors to find fields and regular expressions to type them.
A data parsing example in Python
The input is the Hacker News front page, a public page whose markup is simple and stable. The script gets the raw HTML through our Web Scraping API, then parses it with Beautiful Soup and the lxml parser.
import os, re, sys, json, requests
from bs4 import BeautifulSoup
sys.stdout.reconfigure(encoding="utf-8")
API = "https://api.quanticdata.io/v1/scrape"
headers = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
URL = "https://news.ycombinator.com/"
# 1. Get the raw input: the HTML of the page, as a string
html = requests.post(API, headers=headers, timeout=120,
json={"url": URL, "format": "html"}).json()["payload"]["content"]
print(f"raw input: {len(html):,} characters of HTML")
# 2. Parse the HTML into a tree, then walk it with CSS selectors
soup = BeautifulSoup(html, "lxml")
stories = []
for row in soup.select("tr.athing"):
link = row.select_one(".titleline > a")
sub = row.find_next_sibling("tr").select_one(".subtext")
score = sub.select_one(".score")
age = sub.select_one(".age")
comments = sub.find_all("a")[-1].get_text(" ", strip=True)
stories.append({
"rank": row.select_one(".rank").get_text(),
"title": link.get_text(),
"url": link["href"],
"points": score.get_text() if score else None,
"posted": age["title"] if age else None,
"comments": comments,
})
print(f"parsed: {len(stories)} stories\n", json.dumps(stories[2], indent=2))
# 3. Type the strings: '1.' -> 1, '245 points' -> 245, '87\xa0comments' -> 87, 'discuss' -> 0
def to_int(text):
m = re.search(r"\d+", text or "")
return int(m.group()) if m else 0
for s in stories:
s["rank"] = to_int(s["rank"])
s["points"] = to_int(s["points"])
s["comments"] = to_int(s["comments"])
s["posted"] = s["posted"].split()[0] if s["posted"] else None # ISO time, epoch dropped
print("typed:", json.dumps(stories[2]))
top = sorted(stories, key=lambda s: s["comments"], reverse=True)[:3]
print("most discussed:", [(s["rank"], s["points"], s["comments"]) for s in top])
# 4. The same parse, done server-side: selectors and type functions in the request
r = requests.post(API, headers=headers, timeout=120, json={
"url": URL,
"extract": {
"titles": {"selector": ".titleline > a", "all": True},
"points": {"selector": ".score", "all": True, "fns": ["amount_from_string"]},
},
}).json()["payload"]
data = r["data"]
print("server-side:", len(data["titles"]), "titles,", len(data["points"]), "scores, third:",
data["titles"][2], "/", data["points"][2], type(data["points"][2]).__name__)
Raw, parsed, typed: the output
The output of the run at 16:39 CET on 20 November 2026, unedited:
raw input: 34,167 characters of HTML
parsed: 30 stories
{
"rank": "3.",
"title": "Frostbite: A new CPU architecture discovered, designed and verified by AI",
"url": "https://architectlabs.com/blog/frostbite",
"points": "376 points",
"posted": "2026-11-20T10:01:41",
"comments": "241\u00a0comments"
}
typed: {"rank": 3, "title": "Frostbite: A new CPU architecture discovered, designed and verified by AI", "url": "https://architectlabs.com/blog/frostbite", "points": 376, "posted": "2026-11-20T10:01:41", "comments": 241}
most discussed: [(1, 579, 318), (25, 318, 305), (10, 327, 276)]
server-side: 30 titles, 30 scores, third: Frostbite: A new CPU architecture discovered, designed and verified by AI / 376 int
Look at the parsed record before typing. Every value is still a string, and two of them hide traps. The rank is "3." with a trailing dot. The comment count is "241\u00a0comments": the space is a non-breaking space (U+00A0), which a parser splitting on ordinary spaces would miss. The posted time is read from an attribute of the age element rather than from the visible relative age ("hours ago"), because the attribute holds a full ISO timestamp.
After typing, the record can be sorted and summed. The three most discussed stories were ranks 1, 25 and 10, with 318, 305 and 276 comments. A story ranked 25th had almost as many comments as the first: a fact you cannot get from the strings.
Parsing on your side or on the server
Step 4 sends the selectors with the request. The API fetches the page, applies them and returns the fields, and the amount_from_string function returns the score as an integer (the output says 376 int). Both routes returned 30 records.
| Parse locally (steps 1 to 3) | Parse server-side (step 4) | |
|---|---|---|
| What comes back | The whole page, 34,167 characters | Only the fields you asked for |
| Control | Any Python logic, any library | Selectors plus a list of transform functions |
| Best for | Complex pages, custom rules, debugging | Many pages with the same layout, agents and no-code tools |
| Cost on our API | $0.0015 per successful page either way | |
For sources we already parse every day, a collector returns typed rows directly: see our Hacker News collector.
What a data parsing error means
"Data parsing error" is one of the most searched variants of the term. It means the input did not match what the parser expected, so it could not build the structure. Common causes:
- Wrong format. A JSON parser receives an HTML error page, often because the request was blocked or redirected. Check the status code and the first characters before parsing.
- Encoding. Bytes decoded with the wrong character set, so "Weiß" turns into garbage and a pattern no longer matches.
- Changed layout. A selector that worked yesterday finds nothing today. Fail loudly on empty fields rather than writing nulls into your data.
- Unexpected values. "discuss" where a number was expected, a non-breaking space, a thousands separator. Type conversion functions must handle them, as
to_intdoes above. - A version the parser does not support. Some apps report a parsing error when a file was written by a newer version than the reader understands.
Data parsing tools
| Input | Python tool |
|---|---|
| HTML | Beautiful Soup with lxml, or lxml directly with XPath (see the Beautiful Soup documentation) |
| JSON | The standard json module, or response.json() in requests |
| CSV and spreadsheets | csv, or pandas read_csv and read_excel |
| Text patterns | re |
| Web pages at scale | A scraping API that returns Markdown, HTML or extracted fields |
Parsing is the first step of turning pages into data. The next one, combining and cleaning parsed rows from several sources, is data wrangling in Python. For a complete worked case with numbers, see scraping Google Finance, and for the basics of fetching pages, web scraping with Python.


