# Parse a JSON Response in Python, Tested

> Parse JSON in Python from an API response with r.json(), from a string with json.loads, from a file with json.load; nested data, errors and json.dump.

[Home](https://quanticdata.io/) / [Blog](https://quanticdata.io/blog/) / Parse a JSON Response in Python, Tested

# Parse JSON in Python: API responses, strings, files and nested data, with the real errors

Guides · Nov 20, 2026 · 5 min read · By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData

![Parsing JSON in Python from an API called through a mobile proxy: r.json() turns the response into a dict with keys such as city, region and org, nested quote data flattens into a table of 10 rows and 5 columns with pandas json_normalize, and the card lists the three errors met in the test, an HTML page, single quotes and a date object.](https://quanticdata.io/blog/parse-json-python/hero.webp?v=20261120h)

On this page

- [Parse a JSON response from an API](https://quanticdata.io/blog/parse-json-python/#parse-a-json-response-from-an-api)
- [Nested JSON: dictionaries inside lists](https://quanticdata.io/blog/parse-json-python/#nested-json-dictionaries-inside-lists)
- [Nested JSON to a table with pandas](https://quanticdata.io/blog/parse-json-python/#nested-json-to-a-table-with-pandas)
- [Parse a JSON string and a JSON file](https://quanticdata.io/blog/parse-json-python/#parse-a-json-string-and-a-json-file)
- [The errors, and what each one means](https://quanticdata.io/blog/parse-json-python/#the-errors-and-what-each-one-means)
- [JSON hidden inside an HTML page](https://quanticdata.io/blog/parse-json-python/#json-hidden-inside-an-html-page)
- [JSON Lines for scraped data](https://quanticdata.io/blog/parse-json-python/#json-lines-for-scraped-data)
- [Where the JSON comes from](https://quanticdata.io/blog/parse-json-python/#where-the-json-comes-from)

To parse JSON in Python, call `r.json()` on a requests response, `json.loads(text)` on a string, or `json.load(f)` on an open file. Each returns Python objects: objects become dict, arrays list, true and false become True and False, null becomes None. Then read values with keys and indexes.

**Run it on the free credit.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-lead&utm_content=parse-json-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

Everything below ran on 20 November 2026 with Python 3.12.10 and requests 2.34.2, against two public JSON endpoints (ipinfo.io and the practice API of quotes.toscrape.com), through a QuanticData mobile proxy in the US.

## Parse a JSON response from an API

```python
import requests

PROXY = "http://USER-country-us-type-mobile-sid-j01-ttl-30m:PASS@mobile.quanticdata.io:8080"
px = {"http": PROXY, "https": PROXY}

r = requests.get("https://ipinfo.io/json", proxies=px, timeout=30)
print(r.headers["content-type"])     # application/json
data = r.json()
print(type(data).__name__, list(data))
# dict ['ip', 'city', 'region', 'country', 'loc', 'org', 'postal', 'timezone', 'readme']
print(data["city"], data["region"], data["org"])
# Detroit Michigan AS21928 T-Mobile USA, Inc.
print(data.get("postal_code", "n/a"))  # n/a
```

- **r.json()** parses the body and returns a dict or list. No need to import json.
- **data["key"]** raises KeyError when the key is missing; **data.get("key", default)** returns the default. Use get for optional fields.
- **Check the content-type** before trusting the body: `application/json` here.

## Nested JSON: dictionaries inside lists

Real APIs nest. The quotes API returns a page object whose `quotes` key is a list of dicts, each with an `author` dict and a `tags` list:

```
d = requests.get("https://quotes.toscrape.com/api/quotes", params={"page": 1},
                 proxies=px, timeout=30).json()
first = d["quotes"][0]
print(first["author"]["name"], first["tags"])
# Albert Einstein ['change', 'deep-thoughts', 'thinking', 'world']

for q in d["quotes"]:
    print(q["author"]["name"], len(q["tags"]))
```

Read the path from the outside in: key, index, key. When you do not know the structure, print `json.dumps(d, indent=2)[:1000]` and follow the brackets. `d.keys()` at each level shows what you can ask for next, and `type()` tells you whether the next step is a key or a list index.

## Nested JSON to a table with pandas

```python
import pandas as pd
df = pd.json_normalize(d["quotes"])
print(list(df.columns), df.shape)
# ['tags', 'text', 'author.goodreads_link', 'author.name', 'author.slug'] (10, 5)
```

`json_normalize` flattens nested dicts into dotted column names; lists such as tags stay lists in one cell. From there, `df.to_csv()` gives a spreadsheet. To collect every page of such an API, see [paginating a JSON API](https://quanticdata.io/blog/web-scraping-pagination/).

## Parse a JSON string and a JSON file

```python
import json
obj = json.loads('{"name": "Café", "price": 51.77, "in_stock": true, "tags": null}')
print(obj["in_stock"], obj["tags"])     # True None

with open("quotes.json", "w", encoding="utf-8") as f:
    json.dump(d["quotes"], f, ensure_ascii=False, indent=2)
with open("quotes.json", encoding="utf-8") as f:
    back = json.load(f)
print(len(back))                        # 10
```

| Function | From / to |
| --- | --- |
| `json.loads(s)` | String to Python |
| `json.load(f)` | File to Python |
| `json.dumps(obj)` | Python to string |
| `json.dump(obj, f)` | Python to file |

The s stands for string. By default dumps escapes non-ASCII: `json.dumps({"name": "Café"})` printed `{"name": "Caf\u00e9"}`; with `ensure_ascii=False` it kept `Café`. We opened the files with `encoding="utf-8"`, which keeps the text the same on every system.

**Try it on your own targets.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-inline&utm_content=parse-json-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

## The errors, and what each one means

| What we did | Error we got | Fix |
| --- | --- | --- |
| `r.json()` on an HTML page | JSONDecodeError: Expecting value: line 1 column 1 (char 0) | Check status and content-type first; you got a page, not data |
| `json.loads("{'name': 'x'}")` | Expecting property name enclosed in double quotes: line 1 column 2 (char 1) | That is a Python repr, not JSON; fix the source or use `ast.literal_eval` |
| `json.dumps({"t": date(2026, 11, 20)})` | TypeError: Object of type date is not JSON serializable | `default=str` gave `{"t": "2026-11-20"}` |

The first one deserves care when scraping. "Expecting value at char 0" means the body does not start with JSON at all, often because the server sent an error page, a login page or a block page instead of data. Guard the call:

```
r = requests.get(url, proxies=px, timeout=30)
if r.ok and "json" in r.headers.get("content-type", ""):
    data = r.json()
else:
    print(r.status_code, r.headers.get("content-type"), r.text[:200])
```

## JSON hidden inside an HTML page

Pages built with JavaScript often carry their data as JSON inside a script tag. quotes.toscrape.com/js/ answered with 5,808 bytes of HTML and no visible quotes, but the script held a JavaScript array. A regular expression cut it out and json.loads read it:

```python
import re
html = requests.get("https://quotes.toscrape.com/js/", proxies=px, timeout=30).text
m = re.search(r"var data = (\[.*?\]);", html, re.S)
quotes = json.loads(m.group(1))
print(len(quotes), sorted(quotes[0]))     # 10 ['author', 'tags', 'text']
```

Framework sites use the same idea with tags such as `<script id="__NEXT_DATA__" type="application/json">`: select the tag with Beautiful Soup, take its text and pass it to json.loads. The block has to be strict JSON; in our checks, unquoted keys (`{a: 1}`) and a trailing comma (`{"a": 1,}`) both raised the same "Expecting property name enclosed in double quotes" error as single quotes.

## JSON Lines for scraped data

For results you append to over time, write one JSON object per line. Each line parses alone, so a crash loses at most one record:

```
with open("quotes.jsonl", "w", encoding="utf-8") as f:
    for q in d["quotes"]:
        f.write(json.dumps(q, ensure_ascii=False) + "\n")
print(pd.read_json("quotes.jsonl", lines=True).shape)   # (10, 3)
```

## Where the JSON comes from

Many sites have JSON behind their pages: an API the page calls, or a JSON block inside the HTML. Finding it beats parsing HTML; see [scraping dynamic pages](https://quanticdata.io/blog/dynamic-web-scraping-python/). Our test calls went through a QuanticData mobile line in Detroit; [mobile proxies](https://quanticdata.io/mobile-proxies/) and [residential proxies](https://quanticdata.io/residential-proxies/) take a country and a session in the username. For JSON from pages that have no API, our [web scraping API](https://quanticdata.io/web-scraping-api/) returns structured data, and the reverse direction, posting JSON, is in [sending JSON with requests POST](https://quanticdata.io/blog/python-requests-post/). XML responses: [parsing XML in Python](https://quanticdata.io/blog/parse-xml-python/).

### Sources & further reading

- [Python docs: json](https://docs.python.org/3/library/json.html)
- [Requests: JSON response content](https://requests.readthedocs.io/en/latest/user/quickstart/#json-response-content)
- [pandas.json_normalize](https://pandas.pydata.org/docs/reference/api/pandas.json_normalize.html)
- [RFC 8259: The JSON Data Interchange Format](https://www.rfc-editor.org/rfc/rfc8259)
- [JSON Lines](https://jsonlines.org/)

## FAQ

Quick answers on python parse json response.

[Something else? Ask us](mailto:hello@quanticdata.io)

### How do I parse a JSON response in Python?

With requests, call `r.json()`: it returns a dict or list. Check `r.ok` and the content-type first, because an HTML error page raises JSONDecodeError: Expecting value: line 1 column 1 (char 0).

### What is the difference between json.load and json.loads?

`json.loads` parses a string; `json.load` reads an open file. Likewise `json.dumps` returns a string and `json.dump` writes to a file.

### How do I get a value from nested JSON in Python?

Chain keys and indexes from the outside in: `d["quotes"][0]["author"]["name"]` returned Albert Einstein in our test. Use `.get()` at each level for keys that may be missing.

### How do I convert nested JSON to a DataFrame?

`pd.json_normalize(list_of_dicts)` flattens nested dicts into columns such as `author.name`. Our 10 quotes became a table of 10 rows and 5 columns.

### Why does json.loads fail with "Expecting property name enclosed in double quotes"?

The text uses single quotes, so it is not JSON; it is often a printed Python dict. Fix the source, or read it with `ast.literal_eval` if it really is a Python literal.

## Call JSON APIs from a mobile or home IP

Mobile and residential proxies for requests and httpx, with the country and a session ID set in the username.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-end&utm_content=parse-json-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

## Collectors with Python examples

- [AliExpress scraper API in Python](https://quanticdata.io/collectors/aliexpress-scraper-api/python/): Collector · Install, first call, async runs, output fields and errors of the AliExpress scraper collector, with Python examples.
- [eBay scraper API in Python](https://quanticdata.io/collectors/ebay-scraper-api/python/): Collector · Install, first call, async runs, output fields and errors of the eBay scraper collector, with Python examples.
- [Google Images scraper API in Python](https://quanticdata.io/collectors/image-scraper-api/python/): Collector · Install, first call, async runs, output fields and errors of the Google Images scraper collector, with Python examples.
- [Video search API in Python](https://quanticdata.io/collectors/search-videos-api/python/): Collector · Install, first call, async runs, output fields and errors of the Video search collector, with Python examples.

## Related reading

- [urllib3 vs requests: Speed and Bandwidth](https://quanticdata.io/blog/urllib3-vs-requests/): Guides · requests is built on urllib3 and adds sessions, JSON, exceptions on errors and compression by default. urllib3 alone was faster on a local server, the same through a proxy, and downloaded 9.5 times more bytes until we asked for compression. Measured, with the code.
- [Python requests POST: JSON, Form Data, Files](https://quanticdata.io/blog/python-requests-post/): Guides · requests.post(url, json=payload) sends a JSON body with the right Content-Type; data= sends form fields; files= uploads. We posted each one through a residential proxy to a test server and print what the server received, including the redirect that silently turns a POST into a GET.
- [Concurrency vs Parallelism, Measured in Python](https://quanticdata.io/blog/concurrency-vs-parallelism/): Guides · Concurrency is making progress on several tasks by switching between them; parallelism is running several at the same instant on different CPU cores. We timed both on a real scraping job in Python on 20 November 2026: threads cut 30 downloads from 10.3 s to 3.5 s, and only processes cut the parsing, from 3.5 s to 0.8 s.

---

Source: https://quanticdata.io/blog/parse-json-python/ · Site index for AI: https://quanticdata.io/llms.txt
