To scrape a web page from a Supabase Edge Function, call a scraping API with fetch and return its JSON. Edge Functions run on Deno, where fetch is built in, so there is nothing to install: the function sends the URL, the API fetches the page through its own proxies and answers with the title and the content as Markdown.
We ran the code in this post on 20 November 2026 with Deno 2.9.8, Bun 1.4.5 and Node.js 22.23.2 on Windows 11. The Edge Function was served locally with Deno; the deployment commands below are the ones in the Supabase documentation.
Why call an API instead of scraping inside the function
An Edge Function is a short-lived worker close to your users. That shapes what it is good at:
| Inside an Edge Function | What it means for scraping |
|---|---|
| No headless browser | Pages that need JavaScript rendering cannot be opened there |
| Requests leave from the platform's addresses | The page you get can differ from the one a visitor in the target country sees |
| A request must answer within 150 seconds (Supabase limits page) | Long crawls do not fit in one invocation |
fetch, JSON and secrets are built in | Calling an HTTP API is the natural job |
So the function does the part it is good at, receiving a request and returning JSON, and hands the fetching to a web scraping API that runs the request through residential proxies, renders when the page needs it and returns clean text.
One file for Deno, Bun and Node.js
Start outside Supabase, with the smallest script that proves your key works. It uses only what the three runtimes share: top-level await, fetch and AbortSignal.timeout.
// scrape.mjs
const key = globalThis.Deno ? Deno.env.get("QD_API_KEY") : process.env.QD_API_KEY;
const res = await fetch("https://api.quanticdata.io/v1/scraper/extract", {
method: "POST",
headers: { Authorization: "Bearer " + key, "Content-Type": "application/json" },
body: JSON.stringify({ url: "https://books.toscrape.com/", format: "markdown" }),
signal: AbortSignal.timeout(90_000),
});
const body = await res.json();
if (body.type === "error") throw new Error(res.status + " " + body.message);
console.log(res.status, body.payload.title, body.payload.content.length + " characters of Markdown");
The same file, three commands, three identical answers:
$ deno run --allow-net=api.quanticdata.io --allow-env=QD_API_KEY scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown
$ bun run scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown
$ node scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown
Two differences are worth knowing. Deno reads environment variables with Deno.env.get and asks for permission to use the network and the environment, which is what the two --allow flags grant; naming the host and the variable keeps the permission narrow. Bun and Node read process.env and need no flags.
Every answer from the API has the same envelope: type is "response" or "error", message says what happened, and payload holds the data. For a scrape, payload.title, payload.content and payload.metadata are the fields most functions return.
The Edge Function
A Supabase Edge Function is a file that calls Deno.serve with a handler. This one accepts {"url": "..."} and answers with the title and the Markdown:
// supabase/functions/scrape/index.ts
const API = "https://api.quanticdata.io/v1/scraper/extract";
Deno.serve(async (req) => {
if (req.method !== "POST") return new Response("POST a JSON body with a url", { status: 405 });
const { url } = await req.json();
const upstream = await fetch(API, {
method: "POST",
headers: {
Authorization: "Bearer " + Deno.env.get("QD_API_KEY"),
"Content-Type": "application/json",
},
body: JSON.stringify({ url, format: "markdown" }),
signal: AbortSignal.timeout(90_000),
});
const body = await upstream.json();
if (body.type === "error") {
// Pass the status and Retry-After through, so the caller can wait and try again on a 429.
const headers = new Headers({ "Content-Type": "application/json" });
const retryAfter = upstream.headers.get("Retry-After");
if (retryAfter) headers.set("Retry-After", retryAfter);
return new Response(JSON.stringify({ error: body.message }), { status: upstream.status, headers });
}
const { title, content, finalUrl } = body.payload;
return Response.json({ title, url: finalUrl, markdown: content });
});
Served with plain Deno on our machine and called with curl:
$ deno run --allow-net --allow-env=QD_API_KEY index.ts
Listening on http://localhost:8000/
$ curl -s -X POST http://localhost:8000/ -H "Content-Type: application/json" -d '{"url":"https://books.toscrape.com/"}'
{"title":"All products | Books to Scrape - Sandbox","url":"https://books.toscrape.com/",
"markdown":"[Books to Scrape](https://books.toscrape.com/index.html) We love being scraped!\n\n- [Home](...) ..."}
$ curl -s http://localhost:8000/
POST a JSON body with a url
The POST came back with status 200, and the GET with 405; in our run the answer came back in about 3 seconds.
Run it on Supabase: secrets, serve, deploy
The Supabase documentation gives the steps; they are short.
- Create the function:
supabase functions new scrapecreatessupabase/functions/scrape/index.ts. Replace its starter code with the function above. - Local secret: put
QD_API_KEY=qd_live_...insupabase/functions/.env, which the local stack loads, and add that file to.gitignore. - Serve locally:
supabase functions serve scrape. - Production secret:
supabase secrets set --env-file .env, or set the single value. The function reads it with the sameDeno.env.get("QD_API_KEY"). - Deploy:
supabase functions deploy scrape.
The key belongs in the function's secrets and nowhere else. A browser or mobile app calls your function; it never sees the key, because the function adds the Authorization header on the server side.
Timeouts, rate limits and bigger jobs
- Timeout. The function above gives the API 90 seconds, inside the 150-second window Supabase documents for a request.
- Rate limits. If the API answers 429, the function returns the same status with the
Retry-Afterheader, so your caller knows how long to wait. How to read that header is in Retry-After header. - Many URLs. Do not loop over a long list inside one invocation. Submit the list as one batch job (
POST /scraper/batch, up to 5,000 URLs), return the job id, and let a second function or a webhook collect the result. - Structured fields. To get fields instead of Markdown, send an
extractobject that maps field names to CSS selectors, or parse the HTML yourself with Cheerio on Bun or Node.
The same pattern on Bun as a server
Bun has its own built-in server. The handler is the same code; only the entry point and the way to read the key change:
// server.ts, run with: bun run server.ts
const API = "https://api.quanticdata.io/v1/scraper/extract";
Bun.serve({
port: 8001,
async fetch(req) {
if (req.method !== "POST") return new Response("POST a JSON body with a url", { status: 405 });
const { url } = await req.json();
const upstream = await fetch(API, {
method: "POST",
headers: { Authorization: "Bearer " + process.env.QD_API_KEY, "Content-Type": "application/json" },
body: JSON.stringify({ url, format: "markdown" }),
signal: AbortSignal.timeout(90_000),
});
const body = await upstream.json();
if (body.type === "error") return Response.json({ error: body.message }, { status: upstream.status });
return Response.json({ title: body.payload.title, url: body.payload.finalUrl, markdown: body.payload.content });
},
});
console.log("Listening on http://localhost:8001/");
On Bun 1.4.5 the same curl POST returned the same JSON with status 200, and a GET returned 405. Request options beyond url and format, such as country or rendering, are in the API reference, and more JavaScript samples are in the Node.js examples.


