Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Data wrangling in Python: a real example on 62 scraped listings

Data wrangling in Python on 62 listings scraped on 20 November 2026, 40 from eBay.de in euros and German and 22 from Amazon.com in dollars: the steps gather, structure and clean, normalise currency at the ECB rate of 1.099, filter out 16 off-target rows, then deduplicate, leave 45 comparable rows; the card shows the median price per marketplace and condition.
Data wrangling in Python on 62 listings scraped on 20 November 2026, 40 from eBay.de in euros and German and 22 from Amazon.com in dollars: the steps gather, structure and clean, normalise currency at the ECB rate of 1.099, filter out 16 off-target rows, then deduplicate, leave 45 comparable rows; the card shows the median price per marketplace and condition.

Data wrangling is the work between raw data and analysis: gathering, structuring, cleaning, normalising, filtering, enriching and validating. In Python it is done with pandas. Below, every step runs on 62 real listings scraped on 20 November 2026 from two marketplaces, in two languages and two currencies, ending with 45 comparable rows.

Teaching examples start from a tidy CSV. Real data does not: titles carry emoji and German, prices come in euros and dollars, a quarter of the rows are off target, and a few are duplicates. This post shows the script we ran and its output, so each step has a visible reason.

Data wrangling steps, and how they differ from cleaning

Cleaning is one step of wrangling. Wrangling is the whole route from "rows from several places" to "one table I trust".

StepQuestion it answersIn this example
GatherWhere does each row come from?Three collector runs: eBay.de, Amazon.com, the ECB rate
StructureDo all rows share one schema?Same columns for both marketplaces, plus a source column
CleanIs the text usable?Emoji and symbols out of the titles, spaces collapsed
NormaliseAre values comparable?Every price in US dollars
FilterIs every row on target?Accessories, games and other models removed
EnrichWhat can I derive?Condition and bundle flags from title words in English and German
ValidateAre there duplicates or impossible values?Exact duplicates dropped, outliers cut per source

"Normalisation" means two things in the search results. In databases it is splitting tables into normal forms. In analysis, and here, it is putting values on one scale or unit, such as one currency, so they can be compared.

Gather: three collector runs

We searched both marketplaces for the same product, a Nintendo Switch OLED console, with the eBay collector on the German site and the Amazon collector on the US site, and fetched the day's euro to dollar reference rate with the exchange rates collector. All three ran from Claude through our MCP server:

run_collector { "slug": "ebay_search",    "input": { "query": "nintendo switch oled", "country": "de" } }
run_collector { "slug": "amazon_search",  "input": { "query": "nintendo switch oled", "country": "us" } }
run_collector { "slug": "exchange_rates", "input": { "base": "EUR", "symbols": ["USD"], "max_results": 1 } }

They returned 40 eBay.de listings, 22 Amazon.com products and one rate row (EUR to USD 1.099, dated 20 November 2026). At the published prices that is $0.12, $0.022 and $0.0002. A raw eBay.de title looks like Nintendo Switch OLED | V1/V2 | verschiedene Farben | Auswahl | TOP Zustand!, with a game-controller emoji after "V1/V2" and a check-mark emoji at the end.

The script

The script reads the three runs by id over the REST API and applies the steps in order. Note the cleaning line. With pandas 3.0.3, the version we ran, strings are backed by PyArrow, and in our first attempt a pandas str.replace with \w stripped the German umlauts ("Weiß" became "Wei"). Python's re keeps them, so the title cleaner uses it.

import os, re, sys, requests, pandas as pd
sys.stdout.reconfigure(encoding="utf-8")

API = "https://api.quanticdata.io/v1/scraper/collectors/runs/"
headers = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
def rows(run_id):
    return requests.get(API + run_id, headers=headers, timeout=60).json()["payload"]["results"]

# 1. Gather: two marketplaces, two shapes, two currencies (runs of 20 Nov 2026)
ebay_de = pd.DataFrame(rows("EBAY_RUN_ID"))    # ebay_search, country de
amazon_us = pd.DataFrame(rows("AMAZON_RUN_ID"))  # amazon_search, country us
eur_usd = rows("FX_RUN_ID")[0]["rate"]       # exchange_rates, ECB
print("raw rows:", len(ebay_de), "eBay.de +", len(amazon_us), "Amazon.com")

# 2. Structure: one schema for both sources
cols = ["title", "price_value", "currency"]
df = pd.concat([ebay_de[cols].assign(source="ebay.de"),
                amazon_us[cols].assign(source="amazon.com")], ignore_index=True)

# 3. Clean: strip emoji and symbols, collapse spaces, lowercase copy for matching
clean = lambda s: re.sub(r"\s+", " ", re.sub(r"[^\w\s/+&.,()\-\"']", " ", s)).strip()
df["title"] = df["title"].map(clean)  # Python re: \w keeps umlauts and accents
t = df["title"].str.lower()

# 4. Normalize: one currency
df["price_usd"] = df["price_value"].where(df["currency"] == "USD",
                                          df["price_value"] * eur_usd).round(2)

# 5. Filter: OLED consoles only, no Switch 2, no accessories or games
accessory = r"protector|case for|silicone|accessories kit|charger|exclusive edition"
keep = t.str.contains("oled") & ~t.str.contains("switch 2") & ~t.str.contains(accessory)
print("dropped as off-target:", int((~keep).sum()))
df, t = df[keep].copy(), t[keep]

# 6. Enrich: condition and bundle flags from the title text (EN + DE words)
used = r"renewed|refurbished|used|zustand|b-ware|gebraucht|genutzt|getestet"
df["condition"] = t.str.contains(used).map({True: "used/renewed", False: "not stated"})
df["bundle"] = t.str.contains(r"\+|\bspiele?\b|mario kart|inkl|set mit|zubeh")

# 7. Deduplicate and drop price outliers (IQR rule per source)
before = len(df)
df = df.drop_duplicates(subset=["source", "title", "price_usd"])
q = df.groupby("source")["price_usd"].quantile([.25, .75]).unstack()
lo = df["source"].map(q[0.25] - 1.5 * (q[0.75] - q[0.25]))
hi = df["source"].map(q[0.75] + 1.5 * (q[0.75] - q[0.25]))
df = df[df["price_usd"].between(lo, hi)]
print("duplicates and outliers removed:", before - len(df))
print("clean rows:", len(df), "\n")

summary = (df.groupby(["source", "condition"])["price_usd"]
             .agg(listings="count", median="median", min="min", max="max").round(2))
print(summary.to_string())
print("\nbundles (console + game or accessories):", int(df["bundle"].sum()), "of", len(df))
df.to_csv("switch_oled_clean.csv", index=False)

The output

raw rows: 40 eBay.de + 22 Amazon.com
dropped as off-target: 16
duplicates and outliers removed: 1
clean rows: 45

                         listings  median     min     max
source     condition
amazon.com not stated           3  387.99  341.50  414.99
           used/renewed         4  277.99  268.95  286.00
ebay.de    not stated          29  203.32  158.74  295.63
           used/renewed         9  204.40  124.13  251.67

bundles (console + game or accessories): 16 of 45

And three of the 45 clean rows, as the CSV holds them:

sourcetitle (cleaned)priceprice_usdconditionbundle
ebay.deNintendo Switch OLED Konsole Weiss komplett + 1x Spiel Auswahl (Zelda, Mario..)149.00 EUR163.75not statedTrue
ebay.deNintendo Switch OLED V1/V2 verschiedene Farben Auswahl Zustand "gut"112.95 EUR124.13used/renewedFalse
amazon.comNintendo Switch OLED Model w/ White Joy-Con (Renewed)279.99 USD279.99used/renewedFalse

What each step changed

  • Filtering mattered most. A marketplace search returns what sells next to the product. 16 of 62 rows were screen protectors, cases, chargers, games, the newer Switch 2 or older models.
  • Normalising the currency made the two sources comparable at all. Note what it does not fix: the two marketplaces differ in sellers, taxes and shipping, so a lower median on one site is not by itself a cheaper market. Normalising a unit is not the same as normalising a market.
  • Enrichment from text in two languages flagged 13 used or renewed listings from title words, using "Zustand" (condition) and "B-Ware" on the German side and "Renewed" on the US side, and 16 bundles with a game or accessories.
  • Validation removed one row: a duplicate or an outlier beyond 1.5 times the interquartile range of its marketplace. Small, but on a daily price feed duplicates add up.

Data wrangling tools, and when to change them

ToolUse it when
pandasUp to a few million rows in memory, the default for scraped data
SQL in your warehouseThe rows already live there and the logic is joins and filters
SpreadsheetA one-off check by someone who does not code
Collector output fieldsThe source already gives typed values, such as a numeric price_value and a currency, so parsing is done before wrangling starts

The last row is why we started from collector output instead of raw HTML: prices arrived as numbers with an ISO currency code, which removed a whole parsing step. If you start from raw pages, see our guide to data parsing first. To run the same steps every day, wrap them in a scheduled job as described in how data pipelines work, and for the business side, see competitor price monitoring and how to build a price monitor. The same cleaning applies to text data, as in sentiment analysis of customer reviews.

Sources & further reading

FAQ

Quick answers on data wrangling python.

Something else? Ask us

What is data wrangling in Python?

It is the process of turning raw data into an analysis-ready table with Python, usually with pandas: gathering rows from several sources, giving them one schema, cleaning text, normalising units and currencies, filtering off-target rows, deriving new columns and removing duplicates and outliers.

What is the difference between data wrangling and data cleaning?

Data cleaning fixes the values you have: typos, symbols, missing or impossible entries. Data wrangling is the wider process that includes cleaning plus gathering, restructuring, normalising, filtering and enriching, until data from several sources fits one table.

What are the steps of data wrangling?

Gather, structure, clean, normalise, filter, enrich and validate. In our example: three collector runs, one schema for two marketplaces, emoji removed from titles, euros converted to dollars at the ECB rate, accessories filtered out, condition and bundle flags added, duplicates and outliers dropped.

What is an example of data normalization?

Converting prices in different currencies to one currency is a common example. In our run, 40 eBay.de prices in euros were converted to dollars at the ECB reference rate of 1.099 for 20 November 2026, so they could sit next to 22 Amazon.com prices in dollars.

Which Python libraries are used for data wrangling?

pandas for tables, NumPy for numeric arrays, the standard library re module for text patterns, and requests to fetch data from an API. For data larger than memory, a SQL warehouse or a columnar engine takes over the same steps.

Start from typed rows, not raw pages

Marketplace collectors return prices as numbers with a currency code, so wrangling starts at the useful part. Pay per row delivered, with $2 of free API usage every month.

Collectors with Python examples

Related reading