Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Cheerio web scraping in Node.js: fetch, selectors, pagination and proxies

A Cheerio scraper in Node.js: fetch downloads the HTML through a proxy, cheerio.load parses it, CSS selectors such as article.product_pod and .price_color pick the fields, and the next link is followed page by page; the card shows the real run with cheerio 1.2.0: 60 rows over 3 pages, and 0 quotes on a page built by JavaScript.
A Cheerio scraper in Node.js: fetch downloads the HTML through a proxy, cheerio.load parses it, CSS selectors such as article.product_pod and .price_color pick the fields, and the next link is followed page by page; the card shows the real run with cheerio 1.2.0: 60 rows over 3 pages, and 0 quotes on a page built by JavaScript.

To scrape a website with Cheerio, download the HTML (Node 18 and later have fetch built in), load it with const $ = cheerio.load(html), and read elements with CSS selectors: $('.price_color').text(). Cheerio parses HTML without running JavaScript, so it is fast, but pages built in the browser need a real browser.

We ran every snippet below on 20 November 2026 with Node.js v22.23.2, cheerio 1.2.0 and undici 7.30.0 on Windows 11, against books.toscrape.com and quotes.toscrape.com, two sandboxes made for scraping practice. The output is copied from those runs.

Set up

npm install cheerio undici

Cheerio 1.x is used as an ES module: import * as cheerio from 'cheerio' in an .mjs file, or with "type": "module" in package.json. undici is only needed for the proxy part; plain requests use the built-in fetch, so axios is optional.

Step 1: download and load the page

import * as cheerio from 'cheerio';

async function getHtml(url, dispatcher) {
  const res = await fetch(url, { dispatcher, headers: { 'user-agent': 'my-scraper/1.0 (+https://example.com/bot)' } });
  if (!res.ok) throw new Error(`HTTP ${res.status} for ${url}`);
  return res.text();
}

const $ = cheerio.load(await getHtml('https://books.toscrape.com/', basic));
console.log('title:', $('title').text().trim());
// title: All products | Books to Scrape - Sandbox

The user agent names the scraper and gives a URL where a site owner can read about it. Check res.ok: fetch does not throw on a 403 or 404, and Cheerio will happily parse an error page. The dispatcher argument is the proxy: in our run, the Residential Basic agent built in step 6. Leave it out and the request leaves from your own connection.

Step 2: select the fields

Cheerio uses the selectors you know from CSS and jQuery. Each book on the page is an article.product_pod:

const books = [];
$('article.product_pod').each((i, el) => {
  books.push({
    title: $(el).find('h3 a').attr('title'),
    price: $(el).find('.price_color').text(),
    rating: $(el).find('.star-rating').attr('class').replace('star-rating ', ''),
    stock: $(el).find('.availability').text().trim(),
  });
});
console.log(books.length, books.slice(0, 2));

20 [
  { title: 'A Light in the Attic', price: '£51.77', rating: 'Three', stock: 'In stock' },
  { title: 'Tipping the Velvet', price: '£53.74', rating: 'One', stock: 'In stock' }
]
NeedCheerio callOur result
Text of an element$(el).find('.price_color').text()£51.77
An attribute$(el).find('h3 a').attr('title')A Light in the Attic
A list of values$('.price_color').map((i, el) => $(el).text()).get()[ '£51.77', '£53.74', '£50.10', ... ]
A real array of elements$('h3 a').toArray()20 elements

Step 3: the shorter way, $.extract()

Cheerio 1.x adds $.extract(), which takes a map of fields to selectors and returns plain objects. The same page in one call:

const data = $.extract({
  books: [{
    selector: 'article.product_pod',
    value: { title: { selector: 'h3 a', value: 'title' }, price: '.price_color', url: { selector: 'h3 a', value: 'href' } },
  }],
  next: { selector: 'li.next a', value: 'href' },
});
console.log(data.books.length, data.books[0], 'next =', data.next);

20 { title: 'A Light in the Attic', price: '£51.77', url: 'catalogue/a-light-in-the-attic_1000/index.html' } next = catalogue/page-2.html

A field in square brackets returns every match; without brackets, the first. value picks an attribute instead of the text.

Step 4: follow pagination and save a CSV

import { writeFileSync } from 'node:fs';

let url = 'https://books.toscrape.com/catalogue/page-1.html';
const rows = [];
for (let n = 0; n < 3 && url; n++) {
  const $p = cheerio.load(await getHtml(url, premium));
  $p('article.product_pod').each((i, el) => rows.push({
    title: $p(el).find('h3 a').attr('title'),
    price: Number($p(el).find('.price_color').text().replace(/[^\d.]/g, '')),
  }));
  const next = $p('li.next a').attr('href');
  url = next ? new URL(next, url).href : null;
  console.log('page', n + 1, 'rows so far', rows.length, 'next', url);
}
const csv = ['title,price', ...rows.map((r) => `"${r.title.replace(/"/g, '""')}",${r.price}`)].join('\n');
writeFileSync('books.csv', csv);

page 1 rows so far 20 next https://books.toscrape.com/catalogue/page-2.html
page 2 rows so far 40 next https://books.toscrape.com/catalogue/page-3.html
page 3 rows so far 60 next https://books.toscrape.com/catalogue/page-4.html

new URL(next, url) turns the relative link into a full one, which matters because the site's links change form between the home page and the catalogue pages. The CSV had 61 lines: a header and 60 books. Titles are wrapped in quotes because some contain commas.

Step 5: a detail page and its table

const $d = cheerio.load(await getHtml('https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html', mobile));
const table = Object.fromEntries($d('table.table tr').toArray().map((tr) => [$d(tr).find('th').text(), $d(tr).find('td').text()]));
console.log({ h1: $d('h1').text(), upc: table.UPC, availability: table.Availability, reviews: table['Number of reviews'] });

{ h1: 'A Light in the Attic', upc: 'a897fe39b1053632', availability: 'In stock (22 available)', reviews: '0' }

Step 6: send the requests through a proxy

The built-in fetch takes a dispatcher; undici's ProxyAgent is one. We made one agent per QuanticData network and passed it to getHtml:

import { ProxyAgent } from 'undici';

const agentFor = (host, user, pass) => new ProxyAgent({
  uri: `http://${host}`,
  token: 'Basic ' + Buffer.from(`${user}:${pass}`).toString('base64'),
});
const basic   = agentFor('residentialboson.quanticdata.io:10000', 'USER-type-residential-country-us-session-ch11-lifetime-30', 'PASS');
const premium = agentFor('residentialpremium.quanticdata.io:8080', 'USER-country-us-sid-ch12-ttl-30m', 'PASS');
const mobile  = agentFor('mobile.quanticdata.io:8080', 'USER-country-us-type-mobile-sid-ch13-ttl-30m', 'PASS');
NetworkUsed forExit location (ipinfo.io)Result
Residential BasicHome page, steps 1 to 3Cambridge, Maryland, US20 books
Residential PremiumPagination, step 4Dumont, New Jersey, US60 books over 3 pages
MobileDetail page, step 5Louisville, Kentucky, US, T-Mobile USAAll table fields

Each username carries a session ID and a session time; change the ID for a new session. More ways to set a proxy in Node: a proxy in Node.js, Node fetch with a proxy agent and axios with a proxy, if you prefer axios to fetch.

When Cheerio sees nothing: JavaScript pages

quotes.toscrape.com/js/ shows the same quotes as the home page, but writes them with JavaScript after the page loads. Cheerio gets the HTML the server sends:

div.quote found: 0 | static version: 10

Zero matches on a page that clearly shows data in your browser means the data arrives later. Look in the browser's network tab for a JSON request you can call directly (see reading and parsing JSON in JavaScript), or render the page with a browser, as in Playwright for scraping, and pass await page.content() to cheerio.load().

Cheerio or a browser?

Cheerio with fetchHeadless browser
Runs the page's JavaScriptNoYes
What it downloadsThe HTML document onlyThe page and everything it loads
Good forServer-rendered pages, sitemaps, listingsSingle-page apps, content behind clicks

Keep to each site's terms and robots rules, and pause between requests. For larger jobs, our residential proxies and mobile proxies give your scraper a home or carrier IP in the country you choose, with a Residential Basic free trial; our web scraping API returns rendered pages when you would rather not run browsers yourself.

Sources & further reading

FAQ

Quick answers on cheerio web scraping.

Something else? Ask us

How do I scrape a website with Cheerio?

Download the HTML with fetch, load it with const $ = cheerio.load(html), then select elements with CSS selectors and read them with .text() or .attr(). With cheerio 1.2.0 on Node 22 this collected 20 books per page from a test site.

Do I need axios with Cheerio?

No. Node 18 and later have fetch built in, which is what we used. axios works too if your project already uses it; Cheerio only needs the HTML string.

What is $.extract() in Cheerio?

A Cheerio 1.x method that takes a map of field names to selectors and returns plain objects. Put a field in square brackets to get every match, and use value to read an attribute such as href or title.

Why does Cheerio return nothing for a page I can see in my browser?

The page builds its content with JavaScript, and Cheerio only reads the HTML the server sends. On a JavaScript version of a quotes page we found 0 quotes against 10 on the static version. Call the JSON endpoint the page uses, or render it in a browser first.

How do I use a proxy with Cheerio?

Cheerio does not make requests; the proxy goes on the request. With Node's fetch, pass dispatcher: new ProxyAgent({ uri, token }) from undici, then load the response text into Cheerio. We fetched pages through residential and mobile proxies this way.

A home or carrier IP for your Cheerio scraper

Residential and mobile proxies with country targeting and session control, ready for fetch and undici ProxyAgent.

Related reading