# Supabase Edge Function Web Scraping, Deno, Bun

> Scrape a page from a Supabase Edge Function with one fetch call to a scraping API. The same file runs on Deno, Bun and Node.js; code and real output.

[Home](https://quanticdata.io/) / [Blog](https://quanticdata.io/blog/) / Supabase Edge Function Web Scraping, Deno, Bun

# Web scraping from a Supabase Edge Function, Deno and Bun

Guides · Nov 20, 2026 · 6 min read · By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData

![One fetch call to a scraping API from three JavaScript runtimes: the same script ran on Deno 2.9.8, Bun 1.4.5 and Node.js 22 and each printed status 200 with 8402 characters of Markdown; the card lists the three runtimes and the size of the answer each one printed.](https://quanticdata.io/blog/supabase-edge-function-web-scraping/hero.webp?v=20261120h)

On this page

- [Why call an API instead of scraping inside the function](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#why-call-an-api-instead-of-scraping-inside-the-function)
- [One file for Deno, Bun and Node.js](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#one-file-for-deno-bun-and-node-js)
- [The Edge Function](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#the-edge-function)
- [Run it on Supabase: secrets, serve, deploy](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#run-it-on-supabase-secrets-serve-deploy)
- [Timeouts, rate limits and bigger jobs](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#timeouts-rate-limits-and-bigger-jobs)
- [The same pattern on Bun as a server](https://quanticdata.io/blog/supabase-edge-function-web-scraping/#the-same-pattern-on-bun-as-a-server)

To scrape a web page from a Supabase Edge Function, call a scraping API with `fetch` and return its JSON. Edge Functions run on Deno, where `fetch` is built in, so there is nothing to install: the function sends the URL, the API fetches the page through its own proxies and answers with the title and the content as Markdown.

**Run it on the free credit.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-lead&utm_content=supabase-edge-function-web-scraping) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

We ran the code in this post on 20 November 2026 with Deno 2.9.8, Bun 1.4.5 and Node.js 22.23.2 on Windows 11. The Edge Function was served locally with Deno; the deployment commands below are the ones in the Supabase documentation.

## Why call an API instead of scraping inside the function

An Edge Function is a short-lived worker close to your users. That shapes what it is good at:

| Inside an Edge Function | What it means for scraping |
| --- | --- |
| No headless browser | Pages that need JavaScript rendering cannot be opened there |
| Requests leave from the platform's addresses | The page you get can differ from the one a visitor in the target country sees |
| A request must answer within 150 seconds (Supabase limits page) | Long crawls do not fit in one invocation |
| `fetch`, JSON and secrets are built in | Calling an HTTP API is the natural job |

So the function does the part it is good at, receiving a request and returning JSON, and hands the fetching to a [web scraping API](https://quanticdata.io/web-scraping-api/) that runs the request through [residential proxies](https://quanticdata.io/residential-proxies/), renders when the page needs it and returns clean text.

## One file for Deno, Bun and Node.js

Start outside Supabase, with the smallest script that proves your key works. It uses only what the three runtimes share: top-level `await`, `fetch` and `AbortSignal.timeout`.

```javascript
// scrape.mjs
const key = globalThis.Deno ? Deno.env.get("QD_API_KEY") : process.env.QD_API_KEY;

const res = await fetch("https://api.quanticdata.io/v1/scraper/extract", {
  method: "POST",
  headers: { Authorization: "Bearer " + key, "Content-Type": "application/json" },
  body: JSON.stringify({ url: "https://books.toscrape.com/", format: "markdown" }),
  signal: AbortSignal.timeout(90_000),
});

const body = await res.json();
if (body.type === "error") throw new Error(res.status + " " + body.message);

console.log(res.status, body.payload.title, body.payload.content.length + " characters of Markdown");
```

The same file, three commands, three identical answers:

```bash
$ deno run --allow-net=api.quanticdata.io --allow-env=QD_API_KEY scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown

$ bun run scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown

$ node scrape.mjs
200 All products | Books to Scrape - Sandbox 8402 characters of Markdown
```

Two differences are worth knowing. Deno reads environment variables with `Deno.env.get` and asks for permission to use the network and the environment, which is what the two `--allow` flags grant; naming the host and the variable keeps the permission narrow. Bun and Node read `process.env` and need no flags.

Every answer from the API has the same envelope: `type` is `"response"` or `"error"`, `message` says what happened, and `payload` holds the data. For a scrape, `payload.title`, `payload.content` and `payload.metadata` are the fields most functions return.

## The Edge Function

A Supabase Edge Function is a file that calls `Deno.serve` with a handler. This one accepts `{"url": "..."}` and answers with the title and the Markdown:

```javascript
// supabase/functions/scrape/index.ts
const API = "https://api.quanticdata.io/v1/scraper/extract";

Deno.serve(async (req) => {
  if (req.method !== "POST") return new Response("POST a JSON body with a url", { status: 405 });

  const { url } = await req.json();
  const upstream = await fetch(API, {
    method: "POST",
    headers: {
      Authorization: "Bearer " + Deno.env.get("QD_API_KEY"),
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ url, format: "markdown" }),
    signal: AbortSignal.timeout(90_000),
  });

  const body = await upstream.json();
  if (body.type === "error") {
    // Pass the status and Retry-After through, so the caller can wait and try again on a 429.
    const headers = new Headers({ "Content-Type": "application/json" });
    const retryAfter = upstream.headers.get("Retry-After");
    if (retryAfter) headers.set("Retry-After", retryAfter);
    return new Response(JSON.stringify({ error: body.message }), { status: upstream.status, headers });
  }

  const { title, content, finalUrl } = body.payload;
  return Response.json({ title, url: finalUrl, markdown: content });
});
```

Served with plain Deno on our machine and called with curl:

```bash
$ deno run --allow-net --allow-env=QD_API_KEY index.ts
Listening on http://localhost:8000/

$ curl -s -X POST http://localhost:8000/ -H "Content-Type: application/json" -d '{"url":"https://books.toscrape.com/"}'
{"title":"All products | Books to Scrape - Sandbox","url":"https://books.toscrape.com/",
 "markdown":"[Books to Scrape](https://books.toscrape.com/index.html) We love being scraped!\n\n-   [Home](...) ..."}

$ curl -s http://localhost:8000/
POST a JSON body with a url
```

The POST came back with status 200, and the GET with 405; in our run the answer came back in about 3 seconds.

**Try it on your own targets.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-inline&utm_content=supabase-edge-function-web-scraping) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

## Run it on Supabase: secrets, serve, deploy

The Supabase documentation gives the steps; they are short.

- **Create the function:** `supabase functions new scrape` creates `supabase/functions/scrape/index.ts`. Replace its starter code with the function above.
- **Local secret:** put `QD_API_KEY=qd_live_...` in `supabase/functions/.env`, which the local stack loads, and add that file to `.gitignore`.
- **Serve locally:** `supabase functions serve scrape`.
- **Production secret:** `supabase secrets set --env-file .env`, or set the single value. The function reads it with the same `Deno.env.get("QD_API_KEY")`.
- **Deploy:** `supabase functions deploy scrape`.

The key belongs in the function's secrets and nowhere else. A browser or mobile app calls your function; it never sees the key, because the function adds the `Authorization` header on the server side.

## Timeouts, rate limits and bigger jobs

- **Timeout.** The function above gives the API 90 seconds, inside the 150-second window Supabase documents for a request.
- **Rate limits.** If the API answers 429, the function returns the same status with the `Retry-After` header, so your caller knows how long to wait. How to read that header is in [Retry-After header](https://quanticdata.io/blog/retry-after-header/).
- **Many URLs.** Do not loop over a long list inside one invocation. Submit the list as one batch job (`POST /scraper/batch`, up to 5,000 URLs), return the job id, and let a second function or a webhook collect the result.
- **Structured fields.** To get fields instead of Markdown, send an `extract` object that maps field names to CSS selectors, or parse the HTML yourself with [Cheerio](https://quanticdata.io/blog/cheerio-web-scraping/) on Bun or Node.

## The same pattern on Bun as a server

Bun has its own built-in server. The handler is the same code; only the entry point and the way to read the key change:

```javascript
// server.ts, run with: bun run server.ts
const API = "https://api.quanticdata.io/v1/scraper/extract";

Bun.serve({
  port: 8001,
  async fetch(req) {
    if (req.method !== "POST") return new Response("POST a JSON body with a url", { status: 405 });

    const { url } = await req.json();
    const upstream = await fetch(API, {
      method: "POST",
      headers: { Authorization: "Bearer " + process.env.QD_API_KEY, "Content-Type": "application/json" },
      body: JSON.stringify({ url, format: "markdown" }),
      signal: AbortSignal.timeout(90_000),
    });

    const body = await upstream.json();
    if (body.type === "error") return Response.json({ error: body.message }, { status: upstream.status });
    return Response.json({ title: body.payload.title, url: body.payload.finalUrl, markdown: body.payload.content });
  },
});
console.log("Listening on http://localhost:8001/");
```

On Bun 1.4.5 the same curl POST returned the same JSON with status 200, and a GET returned 405. Request options beyond `url` and `format`, such as `country` or rendering, are in the [API reference](https://quanticdata.io/docs/), and more JavaScript samples are in the [Node.js examples](https://quanticdata.io/docs/node/).

### Sources & further reading

- [Supabase Docs: Getting started with Edge Functions](https://supabase.com/docs/guides/functions/quickstart)
- [Supabase Docs: Environment variables](https://supabase.com/docs/guides/functions/secrets)
- [Supabase Docs: Edge Functions limits](https://supabase.com/docs/guides/functions/limits)
- [Deno Docs: Deno.serve](https://docs.deno.com/api/deno/~/Deno.serve)
- [Bun Docs: Bun.serve](https://bun.com/docs/api/http)
- [Books to Scrape (practice site)](https://books.toscrape.com/)

## FAQ

Quick answers on Supabase Edge Function web scraping.

[Something else? Ask us](mailto:hello@quanticdata.io)

### Can a Supabase Edge Function scrape a website?

Yes, by calling a scraping API with `fetch`. The function sends the URL and returns the JSON; the API does the fetching. Our `Deno.serve` function returned the title and Markdown of a page when served locally with Deno.

### Do I need to install a package?

No. `fetch`, `Response` and `AbortSignal.timeout` are built into Deno, Bun and Node.js 18 or later. The script in this post has no imports.

### Where do I store the API key in Supabase?

In the function's secrets: `supabase/functions/.env` for local runs and `supabase secrets set` for production, read with `Deno.env.get("QD_API_KEY")`. Never put it in client-side code.

### Does the same code run on Deno, Bun and Node.js?

Yes. Our `scrape.mjs` ran unchanged on Deno 2.9.8, Bun 1.4.5 and Node.js 22.23.2 and printed the same line each time: status 200 and 8402 characters of Markdown.

### Which permissions does Deno need for this?

Network access to the API host and read access to one environment variable: `--allow-net=api.quanticdata.io --allow-env=QD_API_KEY`. On Supabase the platform grants these to the function.

### How do I scrape many pages from an Edge Function?

Submit them as one batch job and return the job id, then read the result from a second call or a webhook. One invocation should not wait for hundreds of pages.

## One fetch call from your function

The scraping API returns a page as Markdown or structured fields; your Edge Function only needs fetch and a key.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-end&utm_content=supabase-edge-function-web-scraping) · [Explore Web Scraping API](https://quanticdata.io/web-scraping-api/)

## Related reading

- [Read JSON File in JavaScript: Node and Browser](https://quanticdata.io/blog/read-json-file-javascript/): Guides · In Node.js, read the file with fs and pass the text to JSON.parse, or import it with { type: "json" }. In the browser, fetch it from a web server and call response.json(). Below: every method run on Node 22 and Chromium, the exact JSON.parse error messages, the BOM that Windows tools add, and a JSON file of URLs fetched through a proxy.
- [Cheerio Web Scraping in Node.js: 1.x Tutorial](https://quanticdata.io/blog/cheerio-web-scraping/): Guides · Fetch the page with Node's built-in fetch, load it with cheerio.load(), and read it with jQuery-style CSS selectors. Below: a scraper we ran with cheerio 1.2.0 on Node 22 that collected 60 books over 3 pages, the newer $.extract() method, a detail page, CSV output, and requests through residential and mobile proxies.
- [Node Fetch Headers: Set, Read and Defaults](https://quanticdata.io/blog/node-fetch-headers/): Guides · Node's built-in fetch sends six request headers by default, with User-Agent: node. We echoed them on 20 November 2026, directly and through a residential proxy, then set our own, tested every method of the Headers object, and read response headers including several Set-Cookie lines.

---

Source: https://quanticdata.io/blog/supabase-edge-function-web-scraping/ · Site index for AI: https://quanticdata.io/llms.txt
