Lemmy is the federated link aggregator of the fediverse: thousands of independent servers, each running the same open-source software, each holding its own copy of the conversation. On 24 September 2026 we read it eleven times through residential exits. The result is not the one the open-source label suggests: the HTML of the largest instance refuses a plain HTTP client, its public JSON API hands over ten posts in 37,702 bytes with no login, and two instances asked about the same post return different vote counts.
Two questions hide behind one search
Search for Lemmy and proxies and Google answers a question you probably did not ask. Every result on the first page is about running an instance: nginx reverse proxy templates, Caddy configuration, the content-negotiation proxy in front of the backend. That is a hosting problem, and the people who wrote those guides solved it well.
The other question, the one that brought you here, is about exit IPs: which addresses you route through when you read Lemmy from outside a browser, for social listening, brand monitoring, fediverse research or an archive. That query lives under phrasings like how to scrape Lemmy posts, and the pages that rank for it are hosted scraper listings and the official API reference. None of them say what an instance returns to an anonymous client, so we measured it.
What a Lemmy instance actually serves a proxied client
Every row below is one fetch on 24 September 2026 through a residential exit, no account, no cookies, no browser unless the row says so.
| Request | Status | Bytes | Time | What came back |
|---|---|---|---|---|
| lemmy.world community page, plain HTTP | 403 | small | - | Nothing. A "Just a moment..." interstitial, 6 words, no canonical, no h1 |
| lemmy.world community page, headless browser | 200 | 690,458 | 13.0 s | The full server-rendered community: title, h1, sidebar rules, 3,970 words |
| The same render, second attempt | 403 | 194,238 | 35.7 s | Nothing. Stuck on the same interstitial, zero post listings parsed |
lemmy.world public API, /api/v3/post/list | 200 | 37,702 | 0.77 s | 10 posts with title, body, author, community, score, upvotes, downvotes, comment count, thumbnail |
| The same API call from a German exit | 200 | 37,702 | 0.32 s | The same 10 post ids, two of them swapped by the Hot ranking |
| lemmy.ml community page, plain HTTP | 200 | 330,894 | 3.5 s | The full community, server-rendered, 3,673 words, no challenge at all |
| lemmy.ml public API, same community | 200 | 42,056 | 1.47 s | The same 10 posts, different local ids, different vote counts |
| Nonexistent community, API | 404 | 48 | 0.64 s | A JSON error object, and an honest status code |
/robots.txt | 200 | 315 | 1.2 s | The crawl rules, quoted later in this post |
Two things in that table are worth reading twice. First, the browser path moved 690,458 bytes in 13.0 seconds to show a community whose posts the API delivers in 37,702 bytes in 0.77 seconds - eighteen times the bandwidth, seventeen times the wall clock, for data that is thinner, because the HTML carries no vote breakdown and no ActivityPub ids. Second, the browser path is not even reliable: our second render of the identical URL spent 35.7 seconds and 194,238 bytes to end up back at the challenge, while both API calls succeeded first try.
The failures have shapes, and they are honest ones
This is where Lemmy is pleasant to work with compared to the commercial networks. A missing community returns HTTP 404 and a two-field JSON body. A challenged HTML request returns HTTP 403. Nothing pretends to be a success. You can branch on the status code, which is not true of every platform in this series - on x.com, as we measured in the Twitter post of this series, even the blocked responses arrive as HTTP 200.
import httpx, time
BASE = "https://lemmy.world/api/v3/post/list"
def page(community, cursor=None, limit=50):
params = {"community_name": community, "sort": "New", "limit": limit}
if cursor:
params["page_cursor"] = cursor
r = httpx.get(BASE, params=params, timeout=30)
if r.status_code == 404:
raise LookupError("no such community on this instance")
if r.status_code == 429:
time.sleep(60) # documented back-off, then retry once
r = httpx.get(BASE, params=params, timeout=30)
if r.status_code == 403:
raise RuntimeError("edge challenge, not a Lemmy response")
r.raise_for_status()
body = r.json()
return body["posts"], body.get("next_page")
The cursor in that snippet is the one the API itself returns. Our ten-post response ended with a next_page token, and that token is how you walk a community without offset drift while people keep posting.
The same post, two instances, two answers
This is the part that has no equivalent on a centralised network, and the reason a Lemmy dataset needs a design decision before it needs a proxy. We read the same community twice, ninety-nine seconds apart: once from lemmy.world, which hosts it, and once from lemmy.ml, which subscribes to it. Both returned the same ten posts in the same order. The numbers attached to those posts did not match.
| Post, by the instance that originated it | lemmy.world score / comments | lemmy.ml score / comments | Gap |
|---|---|---|---|
| Samsung fridges bricked by firmware (lemmy.ca) | 576 / 126 | 528 / 106 | -48 / -20 |
| Protestors ransack an AI lab (lemmy.ml) | 388 / 34 | 363 / 30 | -25 / -4 |
| Microsoft patents in-game ads (lemmy.world) | 347 / 55 | 318 / 52 | -29 / -3 |
| Discord ID photo leak (piefed.social) | 259 / 22 | 240 / 18 | -19 / -4 |
| Readers on searching in the age of AI (lemmy.world) | 195 / 33 | 177 / 34 | -18 / +1 |
The direction is systematic: on every one of the five, the subscribing instance reported a lower score, and on four of five a lower comment count. Some of that spread is the ninety-nine seconds between the two reads, and we are not going to pretend otherwise. But drift over a minute and a half does not explain a gap of forty-eight votes on a post already six hours old, and it does not explain why the remote instance consistently saw fewer downvotes as well as fewer upvotes. What explains it is federation itself: each server holds the activities that reached it, and a server that joined the conversation late, or that has defederated from the instance a voter belongs to, simply does not have those votes.
Two practical consequences follow, and they cost more to discover after you have built the pipeline than before.
- The local id is worthless across instances. The fridges post is id 52279485 on lemmy.world and 53122919 on lemmy.ml. The only stable identifier is
ap_id, the ActivityPub URL of the original post. Key your dataset on that, or you will store the same post several times and never know it. - Engagement numbers are per-instance readings, not facts. If your report says a post scored 576, it should also say which server you asked and when. For brand monitoring the practical fix is to read the originating instance - the host named in
ap_id- because it is the one that sees the most of the conversation.
The byte counts differ too, and for a documented reason: lemmy.ml returned 42,056 bytes against lemmy.world's 37,702 for the same ten posts, because it rewrites every remote thumbnail through its own image proxy. Lemmy's configuration reference describes that mode plainly - it improves privacy for users and increases bandwidth use for the server. It also inflates your payload by about eleven per cent.
Which proxy type Lemmy actually needs
The honest answer is smaller than the listicles want it to be, and it depends on which of the two surfaces you are reading.
- For the JSON API, start with the cheapest exit you have. Every API call in our test succeeded first try, including from two countries, and the endpoint is unauthenticated by design. Datacenter exits are the cheapest bandwidth we sell and the right thing to measure first on a surface this open.
- For the HTML of a challenged instance, use residential. lemmy.world sits behind an edge that greeted our plain HTTP client with an interstitial. Rotating residential IPs with a real browser fingerprint are what got the page rendered at all - and even then one of our two attempts failed. The lesson is not to buy a better IP, it is to stop asking for HTML.
- Mobile is an escalation, not a starting point. 4G and 5G device IPs carry the highest trust on any social platform and the highest price per gigabyte. Nothing we measured on Lemmy calls for them.
- ISP static suits a long-running monitor. A fixed address with unlimited bandwidth per IP is attractive if you are polling a handful of communities on a steady schedule, because the bill stops tracking page weight. It is also the shape an instance admin can most easily recognise and allow.
There is a fourth option the centralised networks never offer: run your own instance, subscribe it to the communities you care about, and read your own database. That costs a server instead of bandwidth, and it makes you a participant in the federation rather than a visitor. For a long-lived research archive it is often the cheaper answer, and it is the one the protocol was designed for.
Geo targeting: country changes latency, not content
We ran the identical API call from a US exit and from a German exit. The response was 37,702 bytes both times, with the same ten post ids; only two adjacent posts had swapped places because the Hot ranking reshuffled between the calls. The difference that did show up was speed: 0.77 seconds from the US, 0.32 seconds from Germany, because the German request landed on a nearer edge.
That is a genuinely different answer from the one we measured on the big commercial platforms, where the exit country changes prices, ads, interface language and sometimes the content itself. On Lemmy there is no per-country feed to discover, because there is no advertising system and no per-market catalogue. What varies is not where you stand, it is which server you ask. Country targeting on this network is a latency and a compliance tool - reading an EU-hosted instance from an EU exit, for instance - not a way to see different data. The same conclusion held when we measured the Mastodon side of the fediverse.
What it costs
The API returns ten posts in 37,702 bytes, which is 3,770 bytes per post including the full community description repeated on every row. Turn that into money with our published per-gigabyte prices, counting a gigabyte as a thousand million bytes.
| Job | Bytes moved | Datacenter at $0.50/GB | Residential Basic at $0.80/GB |
|---|---|---|---|
| 1,000 API calls, 10,000 posts | 37.7 MB | $0.02 | $0.03 |
| 1,000,000 posts through the API | 3.77 GB | $1.89 | $3.02 |
| 1,000 community pages, rendered HTML | 690 MB | $0.35 | $0.55 |
Read the first and third rows together. A thousand rendered page loads cost more bandwidth than a hundred thousand posts read through the API, and return less structured data. This is the whole argument of this post in two numbers.
If you would rather not run the fetching yourself, our Web Scraping API bills $0.0002 per page over plain HTTP and $0.001 with JavaScript rendering - the same five-times ratio the bandwidth shows, priced per request - and the SEO Audit API, which is what produced the no-JavaScript and rendered comparison at the top of this post, costs $0.0012 per URL. Failed requests are never billed, and every account gets $2 of free API usage each month. For the centralised link aggregator next door there is a ready-made collector, our Reddit posts API at $0.0005 per delivered post; for Lemmy there is no collector because the public API needs no unblocking, and we are not going to sell you one for a job that a plain GET already does. All of these tools are also exposed through our MCP server if you would rather drive them from an assistant.
What the rules say, and what they do not
The robots.txt of lemmy.world is 315 bytes and unusually specific. It disallows the paths where an account acts - login, settings, the create endpoints, the inbox, the admin area, password change - plus /search/ and /modlog, and it sets Crawl-delay: 60 for every user agent. Sixty seconds between requests is a real instruction, not a formality: these are volunteer-funded servers, and the file is telling you the pace they can absorb.
Three things that file does not do. It does not mention the API, which is not permission - it simply means the exclusion standard has nothing to say about it, and the obligation shifts to the instance's own terms and to your own restraint. It does not speak for other instances: every server publishes its own robots.txt and its own rules, and they differ as much as the servers do. And it has nothing to say about what you may then do with the data, which is governed by the instance's terms, by the licence the content carries, and, for anything that identifies a person, by data protection law wherever your users live. Lemmy posts are public, and public is not the same as unregulated.
The part that has no legal text at all is the etiquette, and on a federated network it matters more than usual. There is no company on the other end of your requests: there is an admin paying for a server. Announce yourself with a real user agent and a contact address, honour the crawl delay, cache the responses - the API already tells you to, with a sixty-second cache header on every call - and read the originating instance rather than hammering five mirrors for the same post. If you need real volume, ask. On this network, unlike the others in this series, asking often works.
And the line we will not help you cross: none of this is a route to creating accounts in bulk, automating votes or comments, evading an instance ban, or reconstructing the identity of people who chose a pseudonymous platform. Lemmy's whole design assumes participants act as themselves. Read the public record, cite where you read it, and leave the accounts alone.