Collectors · Research & science

PDF scraper API in Python

Every PDF and Office document a domain publishes, converted to clean LLM-ready Markdown. One POST from Python with domain, rows back as JSON with 12 fields, $0.005 per delivered document. No proxies or parsers to maintain.

Get an API key See the collector

A Python requests snippet that posts { — A Python requests call to the site_documents collector returns one dict per document, at $0.005 each.

First call

Install, call, read the rows.

pip install requests

The only dependency.

import os, requests

r = requests.post(
    "https://api.quanticdata.io/v1/scraper/collectors/site_documents/run",
    headers={"Authorization": f"Bearer {os.environ['QD_API_KEY']}"},
    json={
        "domain": "bitcoin.org",
        "doc_types": ["pdf"],
        "max_results": 5,
    },
    timeout=120,
)
r.raise_for_status()
run = r.json()["payload"]
print(run["count"], "rows,", run["cost"]["usd"], "USD")
for row in run["results"]:
    print(row.get("url"), row.get("source"), row.get("kind"), row.get("title"))

Python: one Site documents → Markdown run (Whitepapers of a domain), printing url, source, kind, title

The key comes from an environment variable, never from the code. The run answers in the same request when it finishes in time; payload.results holds the rows and payload.cost what they cost.

Request parameters

What the collector takes.

Input of the Site documents → Markdown collector
Parameter	Type	Required	Meaning
`domain`	string	yes	Domain or site URL to harvest documents from.
`doc_types`	array	no	Which document types to collect (default pdf + docx).
`max_chars_per_doc`	integer	no (default 40000)	Cap on the Markdown returned per document (1,000–200,000).
`country`	string	no	ISO 3166-1 alpha-2 code — proxy exit geo and Google locale (gl). Omit for the default pool.
`max_results`	integer	no (default 10)	How many documents to deliver at most (1–50). You pay only for delivered documents.

Response fields

12 fields per document.

Fields of each row (document)
Field	Type	What it holds
`url`	string	Document URL.
`source`	string	Where the link was found: page \| sitemap.
`kind`	string	pdf \| docx \| xlsx \| csv.
`title`	string, nullable	Document title (PDF metadata or first heading).
`markdown`	string	The document converted to Markdown (headings, tables, lists, links).
`chars`	integer	Markdown length in characters.
`words`	integer	Word count of the Markdown.
`pages`	integer, nullable	Page count (PDF only).
`author`	string, nullable	Author from document metadata, when present.
`scanned_pages`	integer	Image-only pages with no text layer (no OCR applied).
`truncated`	boolean	True when the content hit max_chars_per_doc or the page cap.
`found_on`	string, nullable	The page of the site that links the document (null when it came from the sitemap).

Add ?format=csv to the runs endpoint to get the same rows as CSV.

Long runs

Run in the background, then poll.

A run that needs more time can go to the background: send "async": true, get 202 with the run id, and poll the runs endpoint until the status is done. Reading a run is free.

import os, time, requests

H = {"Authorization": f"Bearer {os.environ['QD_API_KEY']}"}
r = requests.post("https://api.quanticdata.io/v1/scraper/collectors/site_documents/run", headers=H,
                  json={**{"domain":"bitcoin.org","doc_types":["pdf"],"max_results":5}, "async": True}, timeout=30)
run_id = r.json()["payload"]["run_id"]
while True:
    run = requests.get(f"https://api.quanticdata.io/v1/scraper/collectors/runs/{run_id}", headers=H, timeout=30).json()["payload"]
    if run["status"] in ("done", "failed"):
        break
    time.sleep(5)
print(run["status"], run["count"])

Python: background run and polling

Errors

What can come back, and what is billed.

Collector run errors
Status	Meaning	Billed
`400`	The input does not match the collector (`payload.errors`)	No
`401`	Missing or wrong API key	No
`402`	Balance and free credit do not cover the run: top up	No
`422`	The input can never resolve; retrying the same input cannot help	No
`424`	The run failed at the source (blocked, timed out, layout changed); safe to retry	No
`429`	Rate limit or a per-account brake; wait for `Retry-After`	No

Only delivered rows are billed: a run that delivers nothing costs nothing.

Price

$0.005 per delivered document.

What Site documents → Markdown rows cost at list price
Rows	Cost
1	$0.005
1,000	$5.00
10,000	$50.00

Up to 50 documents per run; split larger jobs into several runs. The $2 free credit every month covers about 400 documents. No subscription; paid tiers lower the unit price. All collector prices: collector pricing.

Before you run it

Questions developers ask

Is there a free Site documents → Markdown API?

Every account gets $2 of free usage each month, about 400 documents of this collector at $0.005 each, with no card.

Do I have to handle proxies, retries or blocks?

No. The collector runs on our network and handles them; a run blocked at the source returns 424 and is not billed.

How many rows can one run return?

Up to 50 documents per run (the max_results input, where the collector has one).

Can an AI agent run it?

Yes: the MCP server exposes it as the run_collector tool, with the same input and price. See the MCP server.

Run it on the free credit.

Create an API key, export it as QD_API_KEY and run the example. $0.005 per delivered document, no subscription.

Get an API key PDF scraper API