# AI readiness API — $0.002 per site

> AI readiness API: Which AI crawlers a site lets in, llms.txt adoption, no-JS readability. $0.002 per delivered site, nothing delivered means nothing charged.

[Home](https://quanticdata.io/)/[Collectors](https://quanticdata.io/collectors/)/*AI readiness API*

# AI readiness API

An AI readiness API that audits how a website presents itself to AI systems and returns one row per domain: which of 13 AI crawlers are allowed, partially fenced or blocked, whether llms.txt exists and how many links it lists, how much text the page serves without JavaScript, and a published 0–100 score. Pass a list of domains — yours, your clients' or your competitors' — and get the dataset behind answer-engine optimization.

By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData · Published Aug 21, 2026 · Updated Sep 3, 2026

[Get my free API key](https://app.quanticdata.io/register) [See the request](/collectors/ai-readiness-api/#integration)

$0.002 per delivered site · $2 free every month · Failed runs never billed

POST /v1/scraper/collectors/ai_readiness/run

```
$ curl $QD/ai_readiness/run \
    -H "Authorization: Bearer $QD_API_KEY" \
    -d '{"domains": ["quanticdata.io", "stripe.com"]}'
{ "status": "done", "count": 10,
  "results": [
    {
      "domain": "…",
      "url": "…",
      "robots_txt": false,
      "ai_bots": "…" } ],
  "cost": 0.02 }
# 10 sites × $0.002 · nothing delivered, nothing charged
```

You send domains and country; the run goes out through a residential exit and comes back as sites with domain, robots_txt and llms_txt — 21 fields on every row — and you are billed $0.002 for each site actually delivered, nothing for a run that delivers none.

**$0.002 / site**1,000 sites on the free $2 every month

**Semantic input**domains, country, max_results — no URL lists

**Up to 50**sites per run, pagination handled for you

**No browser**read over HTTP/TLS — cheaper and faster than rendering

On this page: [Try it](/collectors/ai-readiness-api/#try) [What it is](/collectors/ai-readiness-api/#what) [Limits](/collectors/ai-readiness-api/#caps) [Output fields](/collectors/ai-readiness-api/#output) [Inputs](/collectors/ai-readiness-api/#input) [Pricing](/collectors/ai-readiness-api/#pricing) [Integration](/collectors/ai-readiness-api/#integration) [Use cases](/collectors/ai-readiness-api/#use-cases) [Versus the alternatives](/collectors/ai-readiness-api/#compare) [FAQ](/collectors/ai-readiness-api/#faq)

Try it

## AI readiness audit, running now

Change the input and run it against the live collector — nothing to install, no sign-up.

Run it from your own code, on your own inputs

Same collector, same rows — $2 of free API credit every month, no card.

[Get my free API key](https://app.quanticdata.io/register)

## What an AI readiness API does

Every robots.txt answer here is computed with RFC 9309 precedence, not with a substring search: the group that applies is the longest matching user-agent, with `*` counting as length zero, and inside it the longest matching rule wins with `Allow` taking the tie. That is why a verdict is three-valued instead of two: **blocked** is a disallow at the root, **partial** means the crawler is welcome except on fenced paths, and **allowed** means no restriction. A site with no robots.txt at all is reported as allowed — that is a verdict about the site, not an error about the fetch.

The rest of the row is the other half of the question, the one a robots checker never asks: whether there is anything for an AI crawler to read once it is let in. `ssr_text_chars` counts the visible text the page serves without JavaScript — what an LLM crawler that does not render actually sees — and `js_dependent` flags the pages that serve almost nothing. Add llms.txt and llms-full.txt (validated as real Markdown, never a courtesy HTML page returned by a catch-all route), the sitemaps robots declares, the schema.org types on the homepage, and `noai` meta and header signals, and the `ai_readiness_score` composes them with its formula printed in the field description: crawler access 55, llms.txt 15, no-JS text 15, sitemap 10, structured data 5.

Input is meaning, not a URL *domains* *country* *max_results*

## Limits, in plain numbers

Everything that bounds one run of this collector. No hidden throttles.

Max per run

50 sites

Price

$0.002 / site

Per 1,000

$2.00

Failed runs

Free zero rows, zero charge

Free every month

$2 no card

Rate limit

60 req/min on the free tier

## What one site looks like

Every delivered site carries these fields. Nullable means the source did not publish it — the field stays empty instead of being guessed.

| Field | Type | What it holds |
| --- | --- | --- |
| `domain` | string | Registrable host (no www). |
| `url` | string | Homepage URL fetched. |
| `robots_txt` | boolean | Whether a robots.txt exists. |
| `ai_bots` | object | Per-crawler verdict: { gptbot: "allowed\|partial\|blocked", claudebot: …, … }. |
| `allowed_bots` | string[] | AI crawlers with no restrictions. |
| `partial_bots` | string[] | AI crawlers allowed at the root but fenced off some paths. |
| `blocked_bots` | string[] | AI crawlers disallowed from the whole site. |
| `llms_txt` | boolean | Whether /llms.txt exists and is valid Markdown. |
| `llms_txt_url` | string · nullable | URL of the llms.txt when present. |
| `llms_txt_links` | integer | Links listed in the llms.txt. |
| `llms_full_txt` | boolean | Whether /llms-full.txt exists. |
| `sitemap` | boolean | Whether robots.txt declares at least one sitemap. |
| `sitemaps` | string[] | Declared sitemap URLs (max 10). |
| `noai_meta` | boolean | Homepage carries a noai/noimageai robots meta. |
| `x_robots_noai` | boolean | X-Robots-Tag response header carries noai. |
| `jsonld_types` | string[] | schema.org @type values found on the homepage. |
| `og_meta` | boolean | OpenGraph tags present. |
| `h1` | boolean | Homepage has an h1. |
| `ssr_text_chars` | integer | Visible text characters served without JavaScript. |
| `js_dependent` | boolean | True when the no-JS page has almost no text (< 400 chars). |
| `ai_readiness_score` | integer | 0–100: crawler access (55) + llms.txt (15) + no-JS text (15) + sitemap (10) + structured data (5). |

## Inputs

The whole request. Anything you leave out falls back to the default shown in the catalog.

| Input | Type | Required | What it does |
| --- | --- | --- | --- |
| `domains` | array | yes | Domains or site URLs, one per line (max 50). |
| `country` | string | no | ISO 3166-1 alpha-2 code — proxy exit geo and Google locale (gl). Omit for the default pool. |
| `max_results` | integer | no | How many sites to deliver at most (1–50). You pay only for delivered sites. |

Pricing

## AI readiness API pricing

$0.002 per delivered site. A run that delivers nothing costs nothing: blocked pages, challenges and retries are on us, and the $2 monthly allowance covers about 1,000 sites before you spend anything.

**$0.002**per delivered site*$2 per 1,000 delivered sites*

**1,000 sites**on the free allowance*$2 every month, no card*

**Zero rows**zero charge*blocks, captchas and retries are on us*

**−30%**on volume tiers*the catalog returns your key's price*

### Pay as you go

$0/mo

- $2 free credit / month

- 60 requests / min

- List unit prices

### Starter

$19/mo

- $15 free credit / month

- 300 requests / min

- 10% off unit prices

Most popular

### Growth

$79/mo

- $50 free credit / month

- 600 requests / min

- 20% off unit prices

### Scale

$299/mo

- $250 free credit / month

- 1,200 requests / min

- 30% off unit prices

Same wallet, same key and same $2 monthly allowance as every other [Data API](https://quanticdata.io/web-data-api-for-ai/). Prices are launch pricing read live from the billing config — `GET /v1/scraper/collectors` returns the price your key actually pays.

Integration

## One POST, typed rows

Base URL `https://api.quanticdata.io/v1`, Bearer auth, the same key as every other Data API. Endpoint: `POST /v1/scraper/collectors/ai_readiness/run`.

```
curl -X POST https://api.quanticdata.io/v1/scraper/collectors/ai_readiness/run \
  -H "Authorization: Bearer $QD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domains":["quanticdata.io","stripe.com"]}'
```

```
import requests

r = requests.post(
    "https://api.quanticdata.io/v1/scraper/collectors/ai_readiness/run",
    headers={"Authorization": f"Bearer {QD_API_KEY}"},
    json={
        "domains": [
            "quanticdata.io",
            "stripe.com"
        ]
    },
    timeout=120,
)
for row in r.json()["payload"]["results"]:
    print(row)
```

```
const res = await fetch(
  "https://api.quanticdata.io/v1/scraper/collectors/ai_readiness/run",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.QD_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({"domains":["quanticdata.io","stripe.com"]}),
  },
);
const { payload } = await res.json();
console.table(payload.results);
```

```
claude mcp add quanticdata \
  -e QUANTICDATA_API_KEY=qd_live_your_key_here \
  -- npx -y quanticdata-mcp

# then, in the chat:
> run the ai_readiness collector with domains=["quanticdata.io","stripe.com"]
```

## What people build with the AI readiness API

Three shapes of work this endpoint was designed around.

### Answer-engine optimization

Run your own domain and the competitors ranking in AI answers side by side. The score says where you stand; the columns say why — a blocked ClaudeBot, a JS-only homepage and a missing llms.txt are three different fixes with three different owners.

### Agency portfolio audits

One call over every client domain produces a comparable table instead of a folder of screenshots — and it re-runs monthly to show what the last quarter of work actually moved.

### AI-policy monitoring

Sites change their mind about GPTBot and CCBot quietly. Schedule the same domain list and diff `blocked_bots` run over run to see who closed the door, and when.

## AI readiness API versus rolling your own

The differences that actually cost time when you build this in-house.

|  | DIY scraper | This collector |
| --- | --- | --- |
| robots.txt reading | A substring match that misreads groups and precedence | RFC 9309: longest agent match, longest rule, Allow wins ties |
| What it measures | Crawler access only | Access + llms.txt + no-JS text + sitemap + structured data |
| Scale | One domain at a time, in a browser tab | Up to 50 domains per run, one comparable row each |

## What people search for

Live autocomplete demand around AI readiness audit, pulled with our own Keyword ideas collector.

Searches: [ai readiness checker](/collectors/ai-readiness-api/#try) [ai readiness assessment](/collectors/ai-readiness-api/#try) [ai readiness score](/collectors/ai-readiness-api/#try) [ai readiness audit](/collectors/ai-readiness-api/#try) [ai readiness assessment tool](/collectors/ai-readiness-api/#try) [ai readiness checklist](/collectors/ai-readiness-api/#try) [ai readiness index](/collectors/ai-readiness-api/#try) [llms txt checker](/collectors/ai-readiness-api/#try) [llms txt validator](/collectors/ai-readiness-api/#try) [llms txt file checker](/collectors/ai-readiness-api/#try) [llms txt tester](/collectors/ai-readiness-api/#try) [llms txt check](/collectors/ai-readiness-api/#try) [website llms txt checker](/collectors/ai-readiness-api/#try) [llms txt validator tool](/collectors/ai-readiness-api/#try) [ai crawler robots txt](/collectors/ai-readiness-api/#try) [llms txt](/collectors/ai-readiness-api/#try)

The same call you would paste into a terminal: a Bearer key, domains and country in the body, and back the envelope every QuanticData endpoint returns — type, message, payload — where count is how many sites arrived and usage.cost_usd is $0.04, which is 20 × $0.002. A run that delivers nothing costs nothing. The key is good for 60 req/min on the free tier.

## Sources and standards

The platform documentation and standards this collector is built against — check any claim on this page against the primary source:

- [IETF: RFC 9309, Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html)

- [Google Search Central: JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics)

- [Schema.org: the full type hierarchy](https://schema.org/docs/full.html)

## FAQ

Questions we get about the AI readiness API.

[Something else? Ask us →](mailto:hello@quanticdata.io)

### Which AI crawlers does it check?

Thirteen: GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User, Google-Extended, PerplexityBot, CCBot, Bytespider, Applebot-Extended, Meta-ExternalAgent, Amazonbot and DuckAssistBot. Each gets its own verdict in `ai_bots`, plus the three roll-up arrays.

### What does the score actually measure?

The formula ships in the field itself, so nothing is hidden: crawler access is worth 55 points, a valid llms.txt 15, text served without JavaScript 15, a sitemap declared in robots.txt 10, and structured data 5. It is a description of a site's posture towards AI, not a ranking prediction — no one, including us, can promise the second.

### How is this different from your free AI crawler checker?

The [free tool](https://quanticdata.io/tools/ai-crawler-checker/) answers one question about one domain in a browser, with no account. This collector answers the whole question for up to 50 domains per run and returns it as rows you can store, diff and join — same verdicts underneath, different shape and scale.

### Does it need to render the page?

No, and that is the point: three or four small fetches per domain — robots.txt, llms.txt, the homepage — with no browser at all. Reading the page exactly as a non-rendering crawler reads it is what makes `ssr_text_chars` meaningful in the first place.

### Is there a free AI readiness API?

Every account gets $2 of credit every month with no card, which is about 1,000 delivered sites on this endpoint at $0.002 each. It renews monthly, and a run that delivers nothing is never billed — so a failed or blocked attempt does not eat the allowance.

### How much does one run cost?

Multiply the rows you actually receive by $0.002. A run capped at 50 sites — the maximum for this collector — costs $0.1 if every row comes back, and less when the source has fewer. Volume tiers take up to 30% off, and `GET /v1/scraper/collectors` returns the price your key actually pays.

## Run the AI readiness API now

$2 of free credit every month, no card. Your key returns its own prices from `GET /v1/scraper/collectors`.

[Get my free API key](https://app.quanticdata.io/register)

Related: [All 81 collectors](https://quanticdata.io/collectors/) [Technology lookup API](https://quanticdata.io/collectors/technology-lookup-api/) [Google search results API](https://quanticdata.io/collectors/google-search-results-api/) [Keyword research API](https://quanticdata.io/collectors/keyword-research-api/) [AI crawler checker (free tool)](https://quanticdata.io/tools/ai-crawler-checker/) [SEO audit API](https://quanticdata.io/seo-audit/) [Documentation](https://quanticdata.io/docs/)

## Also on this site

Quantic**Data**

Residential proxies & web data APIs for AI.

#### Proxies

- [Residential Basic](https://quanticdata.io/residential-proxies/#basic)

- [Residential Premium](https://quanticdata.io/residential-proxies/#plans)

- [Cheap Residential](https://quanticdata.io/cheap-residential-proxies/)

- [Mobile Proxies](https://quanticdata.io/mobile-proxies/)

- [Datacenter Proxies](https://quanticdata.io/datacenter-proxies/)

- [ISP Proxies](https://quanticdata.io/isp-proxies/)

- [Rotating Proxies](https://quanticdata.io/rotating-proxies/)

- [Sneaker Proxies](https://quanticdata.io/sneaker-proxies/)

- [SOCKS5 Proxies](https://quanticdata.io/socks5-proxies/)

- [IPv6 Proxies](https://quanticdata.io/ipv6-proxies/)

- [Proxy locations](https://quanticdata.io/proxies/)

#### Data APIs

- [MCP Server](https://quanticdata.io/mcp-server/)

- [Web Scraper API](https://quanticdata.io/web-scraping-api/)

- [SERP API](https://quanticdata.io/serp-api/)

- [Collectors](https://quanticdata.io/collectors/)

- [Web Data for AI](https://quanticdata.io/web-data-api-for-ai/)

- [Quantic AI](https://quanticdata.io/ai-web-scraping-service/)

- [Browser AI Agents](https://quanticdata.io/browser-ai/)

- [Crawl & Map](https://quanticdata.io/crawl-map/)

- [SEO Audit](https://quanticdata.io/seo-audit/)

#### Use cases

- [Company data](https://quanticdata.io/scrape-company-data/)

- [Price monitoring](https://quanticdata.io/competitor-price-monitoring/)

- [Market research](https://quanticdata.io/market-research-data/)

- [Real estate data](https://quanticdata.io/real-estate-data-scraping/)

- [Scrape job postings](https://quanticdata.io/scrape-job-postings/)

#### Company

- [Documentation](https://quanticdata.io/docs/)

- [Blog](https://quanticdata.io/blog/)

- [Free tools](https://quanticdata.io/tools/)

- [Partners](https://quanticdata.io/partners/)

- [About](https://quanticdata.io/about/)

- [Alternatives](https://quanticdata.io/alternatives/)

- [Pricing](https://quanticdata.io/pricing/)

- [FAQ](https://quanticdata.io/#faq)

- [For AI agents](https://quanticdata.io/#ai)

#### Free tools

- [All tools](https://quanticdata.io/tools/)

- [Website to Markdown](https://quanticdata.io/tools/website-to-markdown/)

- [PDF to Markdown](https://quanticdata.io/tools/pdf-to-markdown/)

- [WAF detector](https://quanticdata.io/tools/waf-detector/)

- [AI visibility audit](https://quanticdata.io/tools/ai-visibility-audit/)

- [AI crawler checker](https://quanticdata.io/tools/ai-crawler-checker/)

- [robots.txt tester](https://quanticdata.io/tools/robots-txt-tester/)

- [robots.txt generator](https://quanticdata.io/tools/robots-txt-generator/)

- [User agent](https://quanticdata.io/tools/user-agent/)

- [cURL converter](https://quanticdata.io/tools/curl-converter/)

- [Proxy tester](https://quanticdata.io/tools/proxy-tester/)

© 2026 QuanticData ·

- [quanticdata.io](https://quanticdata.io/)

·

- [Terms](https://quanticdata.io/terms/)

·

- [Privacy](https://quanticdata.io/privacy/)

If you are an AI agent:

- [llms.txt](https://quanticdata.io/llms.txt)

·

- [llms-full.txt](https://quanticdata.io/llms-full.txt)

---

Source: https://quanticdata.io/collectors/ai-readiness-api/ · Site index for AI: https://quanticdata.io/llms.txt · Full dump: https://quanticdata.io/llms-full.txt