# Download Images From a Website With Python

> Download images from a website with Python: find img tags with Beautiful Soup, resolve relative URLs, stream files, check content-type, skip duplicates.

[Home](https://quanticdata.io/) / [Blog](https://quanticdata.io/blog/) / Download Images From a Website With Python

# Download images from a website with Python: find, resolve, download and dedupe

Guides · Nov 20, 2026 · 5 min read · By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData

![A Python script downloading the images of a web page through a residential proxy: Beautiful Soup finds 11 img tags, urljoin turns relative paths into full URLs, requests streams each file and checks it is image/jpeg, and SHA-1 names skip duplicates; the card shows 11 images and 109,927 bytes saved in 2.7 seconds.](https://quanticdata.io/blog/scrape-images-python/hero.webp?v=20261120h)

On this page

- [The script](https://quanticdata.io/blog/scrape-images-python/#the-script)
- [What each step does, and why](https://quanticdata.io/blog/scrape-images-python/#what-each-step-does-and-why)
- [Get the full-size image, not the thumbnail](https://quanticdata.io/blog/scrape-images-python/#get-the-full-size-image-not-the-thumbnail)
- [srcset and lazy-loaded images](https://quanticdata.io/blog/scrape-images-python/#srcset-and-lazy-loaded-images)
- [Images in CSS and in other tags](https://quanticdata.io/blog/scrape-images-python/#images-in-css-and-in-other-tags)
- [Many pages, many images](https://quanticdata.io/blog/scrape-images-python/#many-pages-many-images)
- [Rights and good manners](https://quanticdata.io/blog/scrape-images-python/#rights-and-good-manners)

To download images from a website with Python, fetch the page with requests, find every `<img>` with Beautiful Soup, turn each `src` into a full URL with `urljoin`, then download each one with `requests.get(url, stream=True)` and write the bytes to a file. Check the content-type and skip duplicates.

**Run it on the free credit.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-lead&utm_content=scrape-images-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

We ran the script below on 20 November 2026 with Python 3.12.10 on a category page of books.toscrape.com, a public shop made for scraping practice, through a QuanticData Residential Basic session in the UK. It saved 11 images, 109,927 bytes, in 2.7 s.

## The script

```python
import hashlib, os, requests
from urllib.parse import urljoin, urlparse
from bs4 import BeautifulSoup

PROXY = "http://USER-type-residential-country-gb-session-img02-lifetime-30:PASS@residentialboson.quanticdata.io:10000"
s = requests.Session()
s.proxies = {"http": PROXY, "https": PROXY}
s.headers["User-Agent"] = "image-fetch-example/1.0 (+https://example.com/contact)"

page = "https://books.toscrape.com/catalogue/category/books/travel_2/index.html"
soup = BeautifulSoup(s.get(page, timeout=30).text, "lxml")
os.makedirs("imgs", exist_ok=True)

seen = set()
for img in soup.select("img"):
    src = img.get("src") or img.get("data-src")
    if not src:
        continue
    url = urljoin(page, src)
    with s.get(url, timeout=30, stream=True) as r:
        if r.status_code != 200 or not r.headers.get("content-type", "").startswith("image/"):
            continue
        data = r.content
    h = hashlib.sha1(data).hexdigest()
    if h in seen:
        continue
    seen.add(h)
    ext = os.path.splitext(urlparse(url).path)[1] or ".jpg"
    with open(os.path.join("imgs", h[:12] + ext), "wb") as f:
        f.write(data)
print(len(seen))   # 11
```

## What each step does, and why

1. **Find the tags.** `soup.select("img")` found 11 on this page, one per book cover.
2. **Resolve the URL.** The first src was `../../../../media/cache/27/a5/...jpg`. Requesting that string fails; `urljoin(page, src)` turned it into `https://books.toscrape.com/media/cache/27/a5/...jpg`. Always join against the URL of the page the tag came from.
3. **Stream the download.** `stream=True` lets you read the headers before the body. Every response here was `image/jpeg`; an HTML error page served under an image URL would be skipped instead of saved as a broken .jpg.
4. **Name by content.** A SHA-1 of the bytes as the file name means the same image saved twice under two URLs is kept once, and reruns do not create copies.

The files opened fine in Pillow: the last thumbnail measured 106 x 155 pixels.

## Get the full-size image, not the thumbnail

List pages usually show thumbnails. The large file is on the product page, or in a `srcset`. Following the first book's link and reading `#product_gallery img` gave a 274 x 400 pixel cover of 51,619 bytes, against 106 x 155 in the list.

```
product = urljoin(page, soup.select_one("article.product_pod h3 a")["href"])
ps = BeautifulSoup(s.get(product, timeout=30).text, "lxml")
big = urljoin(product, ps.select_one("#product_gallery img")["src"])
```

**Try it on your own targets.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-inline&utm_content=scrape-images-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

## srcset and lazy-loaded images

Modern pages list several sizes in `srcset` and keep a placeholder in `src` until the image scrolls into view, with the real URL in `data-src`. This helper picks the widest candidate, then falls back to data-src, then src:

```
def best_image_url(img, base):
    if img.get("srcset"):
        cands = [c.strip().split() for c in img["srcset"].split(",")]
        url = max(cands, key=lambda c: int(c[1].rstrip("w"))
                  if len(c) > 1 and c[1].endswith("w") else 0)[0]
    else:
        url = img.get("data-src") or img.get("src")
    return urljoin(base, url)
```

On a test tag with `400w, 1200w, 800w` candidates it returned the 1200w file, and on a tag whose src was a `data:` placeholder it returned the data-src URL. Some lazy loaders use other attribute names (`data-lazy-src`, `data-original`); print one tag before you write the selector. Images inserted entirely by JavaScript need a rendered page: see [scraping dynamic web pages](https://quanticdata.io/blog/dynamic-web-scraping-python/).

## Images in CSS and in other tags

- **\<picture>** holds `<source srcset>` elements next to the img; `soup.select("picture source")` read them on our test markup. Their srcset may use density descriptors (`1x, 2x`) instead of widths; the helper above only ranks `w` values, so for those take the last candidate, which is normally the largest.
- **Background images** live in `style="background-image: url(...)"` or in CSS files; a regular expression on `url\((.*?)\)` finds them.
- **Open Graph images** sit in `<meta property="og:image">`, often the best single picture of a page.

## Many pages, many images

Images are bigger than HTML, so downloads dominate the time. Run them concurrently with a thread pool or asyncio (patterns in [async web scraping in Python](https://quanticdata.io/blog/async-web-scraping-python/)), keep a timeout on every request, and write files as they arrive. Send a User-Agent that names your script and a way to reach you, as in the code above, and keep the pace modest. We ran it on one session, set in the username; QuanticData [residential proxies](https://quanticdata.io/residential-proxies/) take the country, the rotating gateway or a session ID there, and [mobile proxies](https://quanticdata.io/mobile-proxies/) put the run on carrier IPs. To try it, there is a [Residential Basic free trial](https://quanticdata.io/residential-proxy-free-trial/).

## Rights and good manners

Being able to download an image does not give you the right to reuse it: most images are protected by copyright, and site terms often limit automated downloading. Download what you are allowed to use, keep the request rate modest, and read [is web scraping legal in the US](https://quanticdata.io/blog/is-web-scraping-legal-in-us/) for the legal background. The parsing side of the script is explained in [Beautiful Soup web scraping](https://quanticdata.io/blog/beautifulsoup-web-scraping/).

### Sources & further reading

- [Requests: raw response content and streaming](https://requests.readthedocs.io/en/latest/user/quickstart/#raw-response-content)
- [Python docs: urllib.parse.urljoin](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urljoin)
- [MDN: img srcset attribute](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img#srcset)
- [Pillow documentation](https://pillow.readthedocs.io/en/stable/)

## FAQ

Quick answers on download images from website python.

[Something else? Ask us](mailto:hello@quanticdata.io)

### How do I download all images from a website with Python?

Fetch the page with requests, select every `img` with Beautiful Soup, resolve each src with `urljoin`, download with `stream=True`, check the content-type is `image/`, and save the bytes. Our run saved 11 images, 109,927 bytes, in 2.7 s.

### How do I download an image from a URL in Python?

`r = requests.get(url, timeout=30)`, check `r.headers["content-type"]` starts with `image/`, then `open("file.jpg", "wb").write(r.content)`. Use `stream=True` for large files.

### Why are my downloaded images broken?

Usually the URL was relative and was not joined with the page URL, or the server sent an HTML page (an error or a placeholder) instead of an image. Resolve with urljoin and check the content-type before saving.

### How do I get lazy-loaded images with Python?

Read `data-src` or the largest `srcset` candidate instead of `src`, which often holds a placeholder. If the image URL appears only after JavaScript runs, render the page with Playwright first.

### How do I get full-size images instead of thumbnails?

Follow the link to the item page and read the main image there, or take the widest candidate from `srcset`. On our test shop the list thumbnail was 106 x 155 pixels and the product page image 274 x 400 pixels, 51,619 bytes.

### Is it legal to scrape images from a website?

Downloading is one question and reuse another: images are usually protected by copyright and site terms may restrict automated access. Download what you have the right to use, at a modest rate.

## Download images from a home or mobile IP

Residential and mobile proxies with country, session and rotation settings in the username.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-end&utm_content=scrape-images-python) · [Explore Residential Proxies from $0.80/GB](https://quanticdata.io/residential-proxies/)

## Collectors with Python examples

- [Zalando Scraper API in Python](https://quanticdata.io/collectors/zalando-scraper-api/python/): Collector · Install, first call, async runs, output fields and errors of the Zalando Scraper collector, with Python examples.
- [eBay product data API in Python](https://quanticdata.io/collectors/ebay-product-api/python/): Collector · Install, first call, async runs, output fields and errors of the eBay product data collector, with Python examples.
- [Google Shopping scraper API in Python](https://quanticdata.io/collectors/google-shopping-api/python/): Collector · Install, first call, async runs, output fields and errors of the Google Shopping scraper collector, with Python examples.
- [Reverse image search API in Python](https://quanticdata.io/collectors/reverse-image-search-api/python/): Collector · Install, first call, async runs, output fields and errors of the Reverse image search collector, with Python examples.

## Related reading

- [urllib3 vs requests: Speed and Bandwidth](https://quanticdata.io/blog/urllib3-vs-requests/): Guides · requests is built on urllib3 and adds sessions, JSON, exceptions on errors and compression by default. urllib3 alone was faster on a local server, the same through a proxy, and downloaded 9.5 times more bytes until we asked for compression. Measured, with the code.
- [Python requests POST: JSON, Form Data, Files](https://quanticdata.io/blog/python-requests-post/): Guides · requests.post(url, json=payload) sends a JSON body with the right Content-Type; data= sends form fields; files= uploads. We posted each one through a residential proxy to a test server and print what the server received, including the redirect that silently turns a POST into a GET.
- [Concurrency vs Parallelism, Measured in Python](https://quanticdata.io/blog/concurrency-vs-parallelism/): Guides · Concurrency is making progress on several tasks by switching between them; parallelism is running several at the same instant on different CPU cores. We timed both on a real scraping job in Python on 20 November 2026: threads cut 30 downloads from 10.3 s to 3.5 s, and only processes cut the parsing, from 3.5 s to 0.8 s.

---

Source: https://quanticdata.io/blog/scrape-images-python/ · Site index for AI: https://quanticdata.io/llms.txt
