# Panel Data Example, Built in Python

> Panel data follows the same entities over time. A real example: 5 countries x 9 years of World Bank data, collected 20 November 2026 and analysed in pandas.

[Home](https://quanticdata.io/) / [Blog](https://quanticdata.io/blog/) / Panel Data Example, Built in Python

# Panel data: what it is, and a real example built in Python

Use cases · Nov 20, 2026 · 7 min read · By [Aldo Morese](https://quanticdata.io/about/), founder of QuanticData

![A panel dataset built on 20 November 2026 from two World Bank collector runs: five countries, Germany, Spain, France, Italy and Portugal, observed for nine years from 2015 to 2023, 45 observations in a balanced panel with GDP per capita and inflation; the card shows the slope of growth on inflation, 0.17 pooled and 0.64 once country and year effects are removed.](https://quanticdata.io/blog/panel-data/hero.webp?v=20261120h)

On this page

- [Panel data vs cross-sectional data vs time series](https://quanticdata.io/blog/panel-data/#panel-data-vs-cross-sectional-data-vs-time-series)
- [Building a panel dataset: two collector runs](https://quanticdata.io/blog/panel-data/#building-a-panel-dataset-two-collector-runs)
- [The Python script](https://quanticdata.io/blog/panel-data/#the-python-script)
- [The output: long, wide and the balance check](https://quanticdata.io/blog/panel-data/#the-output-long-wide-and-the-balance-check)
- [What the panel shows that a cross-section cannot](https://quanticdata.io/blog/panel-data/#what-the-panel-shows-that-a-cross-section-cannot)
- [Panel data pitfalls in pandas](https://quanticdata.io/blog/panel-data/#panel-data-pitfalls-in-pandas)

Panel data is a dataset that observes the same entities, such as people, firms, products or countries, at several points in time. Each row is one entity in one period, so it combines a cross-section with a time series. A real example: GDP per capita and inflation for five countries over nine years.

**Run it on the free credit.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-lead&utm_content=panel-data) · [Explore Market Research Data, Built on Demand by AI](https://quanticdata.io/market-research-data/)

That example is the one built below. On 20 November 2026 we pulled two World Bank indicators for Germany, Spain, France, Italy and Portugal from 2015 to 2023 with two collector runs, then built and analysed the panel in pandas. All the tables are the script's real output.

## Panel data vs cross-sectional data vs time series

| Type | Entities | Periods | Example from this post |
| --- | --- | --- | --- |
| Cross-sectional | Many | One | Inflation in all five countries in 2022 |
| Time series | One | Many | Inflation in Italy from 2015 to 2023 |
| Pooled cross-section | Many, different each period | Many | A survey of new households every year |
| Panel | Many, the same each period | Many | Inflation in all five countries, every year from 2015 to 2023 |

The point of following the same entities is that you can separate what differs between them (Germany is richer than Portugal every year) from what changes within each of them over time (every country's inflation jumped in 2022). A cross-section cannot tell those apart, and a single time series has nothing to compare against.

Panels are **balanced** when every entity is observed in every period, and **unbalanced** when some entity-period pairs are missing. They are **long** when each row is one entity-period, and **wide** when each entity or each period becomes a column.

## Building a panel dataset: two collector runs

The [World Bank Indicators API](https://datahelpdesk.worldbank.org/knowledgebase/articles/889392-about-the-indicators-api-documentation) serves each indicator by country and year. Our [World Bank collector](https://quanticdata.io/collectors/world-bank-api/) wraps it: one indicator code, a list of countries and a year range, one row per country-year. We ran it twice from Claude through the QuanticData [MCP server](https://quanticdata.io/mcp-server/):

```
run_collector { "slug": "world_bank", "input": { "indicator": "NY.GDP.PCAP.CD",
  "countries": ["IT", "DE", "FR", "ES", "PT"], "year_from": 2015, "year_to": 2023, "max_results": 45 } }
run_collector { "slug": "world_bank", "input": { "indicator": "FP.CPI.TOTL.ZG",
  "countries": ["IT", "DE", "FR", "ES", "PT"], "year_from": 2015, "year_to": 2023, "max_results": 45 } }
```

Each returned 45 datapoints. NY.GDP.PCAP.CD is GDP per capita in current US dollars; FP.CPI.TOTL.ZG is consumer price inflation in percent. One row as delivered:

```json
{"country": "Germany", "country_code": "DEU", "country_iso2": "DE", "year": "2023",
 "value": 54776.7668235491, "indicator": "NY.GDP.PCAP.CD",
 "indicator_name": "GDP per capita (current US$)", "unit": null, "obs_status": null}
```

At $0.0002 per datapoint the 90 datapoints cost $0.018.

## The Python script

```python
import os, sys, requests, numpy as np, pandas as pd
sys.stdout.reconfigure(encoding="utf-8")

API = "https://api.quanticdata.io/v1/scraper/collectors/runs/"
headers = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
def rows(run_id):
    return requests.get(API + run_id, headers=headers, timeout=60).json()["payload"]["results"]

# Two world_bank runs of 20 Nov 2026: IT, DE, FR, ES, PT, 2015-2023
gdp = pd.DataFrame(rows("GDP_RUN_ID"))  # NY.GDP.PCAP.CD, GDP per capita, current US$
cpi = pd.DataFrame(rows("CPI_RUN_ID"))  # FP.CPI.TOTL.ZG, inflation, consumer prices %

# 1. Long format: one row per entity (country) and period (year)
long = (gdp[["country_code", "year", "value"]].rename(columns={"value": "gdp_pc"})
        .merge(cpi[["country_code", "year", "value"]].rename(columns={"value": "inflation"}),
               on=["country_code", "year"]))
long["year"] = long["year"].astype(int)
panel = long.set_index(["country_code", "year"]).sort_index()
print(panel.shape[0], "observations:", panel.index.get_level_values(0).nunique(), "countries x",
      panel.index.get_level_values(1).nunique(), "years")
counts = panel.groupby(level=0).size()
print("balanced panel:", counts.nunique() == 1, "\n")
print(panel.loc[["DEU", "ITA"]].loc[(slice(None), [2019, 2020, 2021, 2022, 2023]), :].round(1).to_string(), "\n")

# 2. Wide format: countries as columns, years as rows
print(panel["inflation"].unstack(level=0).round(1).to_string(), "\n")

# 3. Growth needs the previous year of the SAME country: shift within each entity
panel["gdp_growth"] = panel.groupby(level=0)["gdp_pc"].pct_change() * 100

# 4. Pooled vs within (fixed effects): demean each variable by country, then regress
d = panel.dropna()
pooled = np.polyfit(d["inflation"], d["gdp_growth"], 1)[0]
within = d - d.groupby(level=0).transform("mean")
fe = np.polyfit(within["inflation"], within["gdp_growth"], 1)[0]
print(f"slope of GDP-per-capita growth on inflation: pooled {pooled:.2f}, within countries {fe:.2f}")
print(f"variance of inflation that is within countries: {within['inflation'].var() / d['inflation'].var():.0%}")

# 5. Two-way: also remove what all five countries share in a given year
two_way = within - within.groupby(level=1).transform("mean")
tw = np.polyfit(two_way["inflation"], two_way["gdp_growth"], 1)[0]
print(f"slope with country and year effects removed: {tw:.2f}")
```

**Try it on your own targets.** Every QuanticData account gets $2 of free API usage per month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-inline&utm_content=panel-data) · [Explore Market Research Data, Built on Demand by AI](https://quanticdata.io/market-research-data/)

## The output: long, wide and the balance check

```
45 observations: 5 countries x 9 years
balanced panel: True

                    gdp_pc  inflation
country_code year
DEU          2019  47656.2        1.4
             2020  47394.9        0.1
             2021  52349.2        3.1
             2022  50506.5        6.9
             2023  54776.8        5.9
ITA          2019  33812.8        0.6
             2020  32091.5       -0.1
             2021  36852.5        1.9
             2022  35653.9        8.2
             2023  39279.7        5.6

country_code  DEU  ESP  FRA  ITA  PRT
year
2015          0.5 -0.5  0.0  0.0  0.5
2016          0.5 -0.2  0.2 -0.1  0.6
2017          1.5  2.0  1.0  1.2  1.4
2018          1.7  1.7  1.9  1.1  1.0
2019          1.4  0.7  1.1  0.6  0.3
2020          0.1 -0.3  0.5 -0.1 -0.0
2021          3.1  3.1  1.6  1.9  1.3
2022          6.9  8.4  5.2  8.2  7.8
2023          5.9  3.5  4.9  5.6  4.3

slope of GDP-per-capita growth on inflation: pooled 0.17, within countries 0.17
variance of inflation that is within countries: 99%
slope with country and year effects removed: 0.64
```

The first block is the long form, indexed by entity and period, which is what panel models expect. The second is the wide form of one variable: read across a row for a cross-section, down a column for a time series. The panel is balanced: all five countries have all nine years.

## What the panel shows that a cross-section cannot

- **Almost all the variation in inflation is over time, not between countries.** 99% of its variance is left after removing each country's average. In the wide table every column moves together: near zero in 2015 and 2020, then 5.2% to 8.4% in 2022.
- **Shared shocks hide the relationship.** The slope of GDP-per-capita growth on inflation is 0.17 both pooled and within countries, because both are dominated by the same year-to-year swings. Removing the year effects as well, so that each country is compared with the other four in the same year, gives 0.64.
- **The unit matters.** GDP per capita here is in current US dollars, so part of each country's yearly change is the euro to dollar exchange rate, which all five share. That is one reason the year effects are so large.

Five countries and nine years make a teaching example, not an estimate: the slopes come from 40 growth observations, and none of these numbers is a causal effect. The method is the same at any size.

## Panel data pitfalls in pandas

1. **Lags and growth must stay inside each entity.** `pct_change()` on the whole column would compute Spain's 2015 growth from Germany's 2023 value, the row above it. Group by the entity level first, as in step 3.
2. **Check balance before you model.** A missing year shifts every lag after it. Count periods per entity, and decide whether to drop or fill.
3. **Keep the year an integer.** The World Bank returns years as strings. String years sort correctly only until you mix formats; convert once, early.
4. **Write down the units.** Current or constant prices, percent or index: the indicator name says, so keep it with the data.

The same structure works for business data: products observed every week with their price and rank, stores observed every month with their reviews, companies observed every quarter with their headcount. Collect the same entities on a schedule and you have a panel. For the cleaning steps that come before, see [data wrangling in Python](https://quanticdata.io/blog/data-wrangling-python/); for prices and quotes over time, see [scraping Google Finance and financial data](https://quanticdata.io/blog/scrape-google-finance/); and for the business side, [market research data](https://quanticdata.io/market-research-data/).

### Sources & further reading

- [World Bank: About the Indicators API](https://datahelpdesk.worldbank.org/knowledgebase/articles/889392-about-the-indicators-api-documentation)
- [World Bank Indicators API: GDP per capita (current US$), NY.GDP.PCAP.CD](https://api.worldbank.org/v2/indicator/NY.GDP.PCAP.CD?format=json)
- [World Bank Indicators API: Inflation, consumer prices (annual %), FP.CPI.TOTL.ZG](https://api.worldbank.org/v2/indicator/FP.CPI.TOTL.ZG?format=json)
- [pandas documentation: MultiIndex and advanced indexing](https://pandas.pydata.org/docs/user_guide/advanced.html)

## FAQ

Quick answers on panel data example.

[Something else? Ask us](mailto:hello@quanticdata.io)

### What is panel data?

Panel data observes the same entities, such as people, firms, products or countries, at several points in time. Each observation is one entity in one period, so a panel combines a cross-section with a time series and lets you separate differences between entities from changes within them.

### What is an example of panel data?

GDP per capita and inflation for Germany, Spain, France, Italy and Portugal for every year from 2015 to 2023: five entities, nine periods, 45 observations. We built exactly that panel from two World Bank collector runs on 20 November 2026.

### What is the difference between panel data and time series?

A time series follows one entity over time, for example Italy's inflation from 2015 to 2023. Panel data follows several entities over the same periods, for example all five countries over those years, so you can compare them with each other as well as over time.

### What is the difference between panel data and cross-sectional data?

Cross-sectional data observes many entities at one point in time, such as inflation in five countries in 2022. Panel data observes the same entities at several points in time. A pooled cross-section also has several periods, but the entities change each time.

### What is a balanced panel?

A panel is balanced when every entity is observed in every period. Ours is balanced: five countries, nine years each. If some entity-period pairs are missing the panel is unbalanced, which most panel models handle, but lags and growth rates need care.

### How do you analyse panel data in Python?

Put the data in long form with a two-level index, entity and period, in pandas. Compute lags and growth within each entity with groupby. For a fixed-effects estimate, subtract each entity's mean, and for two-way effects also each period's mean, then regress. Dedicated libraries add standard errors and tests.

## Build your own panel from public sources

World Bank indicators at $0.0002 per datapoint, and the same entities collected on a schedule from any of our collectors. Start with $2 of free API usage every month.

[Start free — $2/month included](https://quanticdata.io/signup/?utm_source=blog&utm_medium=website&utm_campaign=blog-end&utm_content=panel-data) · [Explore Market Research Data, Built on Demand by AI](https://quanticdata.io/market-research-data/)

## Related reading

- [Sentiment Analysis of Customer Reviews](https://quanticdata.io/blog/sentiment-analysis-of-customer-reviews/): Use cases · Sentiment analysis of customer reviews sorts each review into positive or negative, then tells you which words drive each side. We collected 150 fresh App Store reviews on 20 November 2026, trained a TF-IDF and logistic regression model on the star ratings, and reached 87% accuracy in 5-fold cross-validation. Code and output below.
- [Data Wrangling in Python: A Real Example](https://quanticdata.io/blog/data-wrangling-python/): Use cases · Data wrangling turns raw rows into a table you can analyse: gather, structure, clean, normalise, filter, enrich, validate. We ran every step in Python with pandas on 62 listings collected on 20 November 2026 from eBay.de and Amazon.com, in two languages and two currencies, and ended with 45 comparable rows and a price table.
- [Bulk WHOIS Lookup over RDAP: 23 TLDs Tested](https://quanticdata.io/blog/bulk-whois-lookup-rdap/): Use cases · We sent 23 domains on 23 different TLDs to the WHOIS collector in one call on 20 November 2026: 23 rows came back in about 2 seconds, 15 read over RDAP and 8 from the registry's WHOIS server. Below: the request, the poll, a table of what each TLD returned, and how to run a list of thousands.

---

Source: https://quanticdata.io/blog/panel-data/ · Site index for AI: https://quanticdata.io/llms.txt
