Concurrency means a program makes progress on several tasks in overlapping periods, usually by switching between them while one is waiting. Parallelism means several tasks run at the same instant on different CPU cores. Concurrency is how a program is organised; parallelism is how it is executed. Waiting work gains from concurrency, computing work only from parallelism.
The usual explanation stops at an analogy about cooks and kitchens. We timed the difference on a job every scraper has, download pages and parse them, on 20 November 2026 in Python 3.12 through one QuanticData residential proxy line. The script and its output are below.
Concurrency vs parallelism in one table
| Concurrency | Parallelism | |
|---|---|---|
| Meaning | Several tasks in progress over the same period | Several tasks executing at the same instant |
| Needs more than one CPU core | No | Yes |
| Helps when the task | Waits: network, disk, database | Computes: parsing, hashing, compression, maths |
| Python tools | threading, ThreadPoolExecutor, asyncio | multiprocessing, ProcessPoolExecutor |
| In our run | 30 downloads: 10.3 s in a row, 3.5 s with 10 threads | 240 parses: 3.5 s on one core, 0.8 s with 8 processes |
The two are not opposites and a program can be both: eight processes, each downloading with ten threads, is concurrent and parallel at once.
I/O-bound and CPU-bound: the question to ask first
Before choosing a tool, ask what the program is doing while time passes.
- I/O-bound: it is waiting for something outside the CPU. A page download is almost all waiting: the request travels, the server works, the answer travels back. During that wait the CPU is free.
- CPU-bound: it is computing. Turning 50,000 characters of HTML into a tree and running selectors over it keeps one core busy from start to finish.
A scraper has both phases, which makes it a good test.
The test script
The target is books.toscrape.com, a shop built for scraping practice: 30 catalogue pages of 20 books each. The script downloads them twice (one after another, then with 10 threads), then parses 240 pages three ways (one after another, 8 threads, 8 processes). Python 3.12.10, Requests 2.34.2, BeautifulSoup 4.14.3 with the pure-Python html.parser, on a Windows 11 PC with 32 logical CPUs.
import os, sys, threading, time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import requests
from bs4 import BeautifulSoup
PROXY = os.environ["QD_PROXY"] # http://USER:PASS@HOST:PORT of your proxy line
PROXIES = {"http": PROXY, "https": PROXY}
URLS = [f"https://books.toscrape.com/catalogue/page-{n}.html" for n in range(1, 31)]
local = threading.local()
def fetch(url):
if not hasattr(local, "session"): # one keep-alive session per thread
local.session = requests.Session()
local.session.proxies = PROXIES
r = local.session.get(url, timeout=30)
r.raise_for_status()
return r.text
def parse(html):
soup = BeautifulSoup(html, "html.parser")
return [(a["title"], p.get_text()) for a, p in
zip(soup.select("article.product_pod h3 a"), soup.select("p.price_color"))]
def timed(label, fn):
t = time.perf_counter(); out = fn(); dt = time.perf_counter() - t
print(f"{label:<34}{dt:6.1f} s {out}")
return dt
if __name__ == "__main__":
print("Python", sys.version.split()[0], "| logical CPUs:", os.cpu_count())
pages = []
def io_seq():
pages[:] = [fetch(u) for u in URLS]; return f"{len(pages)} pages"
def io_threads():
with ThreadPoolExecutor(max_workers=10) as ex:
got = list(ex.map(fetch, URLS))
return f"{len(got)} pages"
print("-- I/O-bound: download 30 pages through one proxy")
timed("one after another", io_seq)
timed("10 threads", io_threads)
work = pages * 8 # 240 parse jobs, pure Python (html.parser)
def cpu_seq():
return f"{sum(len(parse(h)) for h in work)} books"
def cpu_threads():
with ThreadPoolExecutor(max_workers=8) as ex:
return f"{sum(len(x) for x in ex.map(parse, work))} books"
def cpu_procs():
with ProcessPoolExecutor(max_workers=8) as ex:
return f"{sum(len(x) for x in ex.map(parse, work, chunksize=10))} books"
print("-- CPU-bound: parse 240 pages with html.parser")
timed("one after another", cpu_seq)
timed("8 threads", cpu_threads)
timed("8 processes", cpu_procs)
Python 3.12.10 | logical CPUs: 32
-- I/O-bound: download 30 pages through one proxy
one after another 10.3 s 30 pages
10 threads 3.5 s 30 pages
-- CPU-bound: parse 240 pages with html.parser
one after another 3.5 s 4800 books
8 threads 3.3 s 4800 books
8 processes 0.8 s 4800 books
Concurrency at work: the downloads
One after another, the 30 pages took 10.3 seconds. With 10 threads the same 30 pages took 3.5 seconds, about a third of the time, and the script returned 30 of 30 pages both ways.
Nothing ran faster. Each request took as long as before; the threads simply overlapped the waiting. While one thread waits for its answer, Python hands control to another, which sends its own request. This is concurrency, and it would work the same on a single core, because a waiting thread needs no CPU.
asyncio gets the same effect with one thread and an event loop instead of ten threads; the mechanics and a measured example are in async web scraping in Python. For a few dozen simultaneous requests, a thread pool around Requests is the smallest change to an existing script; the proxy settings it needs are in a proxy in Python Requests.
Parallelism at work: the parsing
The parsing phase tells the opposite story. One after another, 240 parses took 3.5 seconds. With 8 threads they took 3.3 seconds: no real gain. With 8 processes they took 0.8 seconds, more than four times faster, and all three returned the same 4,800 books.
The reason threads did not help is the global interpreter lock (GIL). In the standard build of Python 3.12 only one thread executes Python bytecode at a time. Threads can overlap waiting, because a thread that waits on the network releases the lock, but they cannot overlap computing: eight parsing threads take turns on one core.
Processes do not share that lock. ProcessPoolExecutor starts eight Python interpreters, each with its own, and the operating system places them on eight cores. That is parallelism. It has costs that threads do not: each process takes time to start, and every page and every result is copied between processes, which is why the script sends the work in chunks of 10 and why the gain is four times and not eight.
| Phase | One after another | Threads | Processes | What helped |
|---|---|---|---|---|
| Download 30 pages (I/O-bound) | 10.3 s | 3.5 s with 10 | Not needed | Concurrency |
| Parse 240 pages (CPU-bound) | 3.5 s | 3.3 s with 8 | 0.8 s with 8 | Parallelism |
These are the timings of one run on one PC and one network on 20 November 2026; the shape of the result, not the exact seconds, is what carries over to other machines.
Concurrency vs parallelism vs multithreading vs async
The four words are often mixed up because two of them are ideas and two are tools.
| Term | What it is | In Python | Concurrent | Parallel |
|---|---|---|---|---|
| Concurrency | An idea: overlapping tasks | Threads or asyncio | Yes | Not necessarily |
| Parallelism | An idea: simultaneous execution | Processes | Yes | Yes |
| Multithreading | A tool: several threads in one process | threading, ThreadPoolExecutor | Yes | Not for Python code under the GIL: 3.3 s against 3.5 s in our run |
| Async | A tool: one thread, tasks that yield while waiting | asyncio, await | Yes | No |
| Multiprocessing | A tool: several processes | multiprocessing, ProcessPoolExecutor | Yes | Yes: 0.8 s against 3.5 s in our run |
In languages without a global lock, such as Go or Java, threads can run in parallel, so multithreading there covers both columns. The distinction between the two ideas stays the same in every language.
Which to use in a scraper
- Downloading: use concurrency. A thread pool or asyncio, with a fixed number of workers.
- Parsing a lot of HTML in Python: use processes, or a parser written in C such as lxml, which does the heavy part outside the interpreter. The selectors themselves are in our BeautifulSoup guide.
- Both: download with threads into a queue, parse in a process pool. Keep the two phases separate so each gets the tool that fits.
- A handful of pages: a plain loop. In our run the sequential version already parsed 4,800 books in 3.5 seconds; concurrency adds moving parts that a small job does not need.
Concurrency is also a matter of manners
Ten threads mean ten requests in flight at the same site. The site we used exists for practice; a real one has to serve its visitors too. Choose the number of workers on purpose and keep it small, add a pause between requests when you crawl many pages of one site (pagination shows the loop), set timeouts and a bounded retry for transient errors (retries and timeouts in Python Requests), and if the site answers 429, lower the concurrency: that status is the site telling you its limit, as our post on rate limiting explains. The wider list of habits is in crawling a website politely.
Where the proxy comes in
A proxy does not change which tool fits: downloads stay I/O-bound. What matters is that every worker uses the proxy the same way. In the script each thread keeps one requests.Session with the proxy set, so its connection to the gateway is reused from page to page. The run above used one residential proxy line with the country set to the United States in the username. Plans and countries for residential proxies and mobile proxies are on their pages.


