Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Sentiment analysis of customer reviews in Python, on 150 fresh App Store reviews

Sentiment analysis of 150 App Store reviews collected on 20 November 2026: the star ratings label 92 reviews positive and 39 negative, a TF-IDF and logistic regression model classifies them with 87% accuracy in 5-fold cross-validation, negative words include gems, energy and worst, and positive words include love, good and fun.
Sentiment analysis of 150 App Store reviews collected on 20 November 2026: the star ratings label 92 reviews positive and 39 negative, a TF-IDF and logistic regression model classifies them with 87% accuracy in 5-fold cross-validation, negative words include gems, energy and worst, and positive words include love, good and fun.

Sentiment analysis of customer reviews labels each review as positive or negative and shows which words push it either way. With star ratings as labels, a TF-IDF and logistic regression model in Python takes a few lines. On 150 App Store reviews collected on 20 November 2026 it scored 87% accuracy in cross-validation.

Most tutorials train on a review file that is years old. Here the reviews were collected the same afternoon, the model was cross-validated on them, and every number below is from that run. The full script is in the post, so you can point it at your own product.

What sentiment analysis of reviews gives you

A star rating already says how happy a customer was. Sentiment analysis adds three things the stars do not:

  • The why. The words that weigh most on the negative side tell you what customers complain about, without reading every review.
  • A score for text without stars. Support tickets, social posts and survey answers have no rating. A model trained on rated reviews can score them.
  • A split of the middle. Three-star reviews are neither. A model tells you which way each one leans.

There are three common approaches. Lexicon tools score words from a fixed dictionary and need no training. Classic machine learning, used here, learns the words from your own labelled reviews. Large language models read each review in full and can answer open questions about it; our test of a review model on 730 Google reviews compares that approach with the stars. Start with the classic model: it is fast, free to run and easy to inspect.

Step 1: collect the reviews

We picked a large consumer app with many recent reviews, Duolingo on the US App Store, and ran our App Store reviews collector from Claude through the QuanticData MCP server:

run_collector {
  "slug": "app_store_reviews",
  "input": { "app_id": "570060128", "country": "us", "max_results": 150 },
  "async": true
}

It returned 150 reviews, each with the rating, title, text, version and vote counts. At $0.0005 per review the run cost $0.075. Reviews carry the author's display name; the script below drops it at once, because a sentiment model needs the text, not the person.

StarsReviewsLabel used
582positive
410positive
319held out, scored at the end
211negative
128negative

Step 2: the Python script

The script reads the run by its id over the REST API, labels the reviews from their stars, turns the text into TF-IDF features (words and word pairs, as described in the scikit-learn text feature extraction guide) and evaluates a logistic regression with 5-fold stratified cross-validation, so every review is scored by a model that never saw it.

import os, requests, numpy as np, pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_predict, StratifiedKFold
from sklearn.metrics import classification_report

RUN_ID = "YOUR_RUN_ID"  # app_store_reviews, Duolingo, US store, 20 Nov 2026
API = "https://api.quanticdata.io/v1/scraper/collectors/runs/"
headers = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
rows = requests.get(API + RUN_ID, headers=headers, timeout=60).json()["payload"]["results"]

df = pd.DataFrame(rows)[["rating", "title", "text"]]   # no author names kept
df["review"] = df["title"].fillna("") + ". " + df["text"].fillna("")
print(len(df), "reviews; stars:", df["rating"].value_counts().sort_index().to_dict())

# 1. Labels from the stars: 4-5 positive, 1-2 negative, 3 left out for now
lab = df[df["rating"] != 3].copy()
lab["label"] = np.where(lab["rating"] >= 4, "positive", "negative")

# 2. TF-IDF on words and word pairs, logistic regression, 5-fold cross-validation
vec = TfidfVectorizer(ngram_range=(1, 2), min_df=2, sublinear_tf=True, stop_words="english")
X = vec.fit_transform(lab["review"])
clf = LogisticRegression(max_iter=1000, class_weight="balanced")
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
lab["pred"] = cross_val_predict(clf, X, lab["label"], cv=cv)
print(classification_report(lab["label"], lab["pred"], digits=2))

# 3. What drives each side: the heaviest words of the model trained on all labelled rows
clf.fit(X, lab["label"])
terms = np.array(vec.get_feature_names_out())
order = np.argsort(clf.coef_[0])          # classes_ = ['negative', 'positive']
print("negative words:", ", ".join(terms[order[:12]]))
print("positive words:", ", ".join(terms[order[-12:]][::-1]))

# 4. Score the 3-star reviews the model never saw
mid = df[df["rating"] == 3]
p_pos = clf.predict_proba(vec.transform(mid["review"]))[:, 1]
print(f"\n3-star reviews: {len(mid)}, leaning negative: {(p_pos < 0.5).sum()}, leaning positive: {(p_pos >= 0.5).sum()}")

Step 3: the result

The output of that run, unedited:

150 reviews; stars: {1: 28, 2: 11, 3: 19, 4: 10, 5: 82}
              precision    recall  f1-score   support

    negative       0.79      0.77      0.78        39
    positive       0.90      0.91      0.91        92

    accuracy                           0.87       131
   macro avg       0.85      0.84      0.84       131
weighted avg       0.87      0.87      0.87       131

negative words: used, lessons, use, right, guys, time, app, getting, free, gems, energy, worst
positive words: love, good, duolingo, fun, amazing, great, best, spanish, learn, learning, helps, excelente

3-star reviews: 19, leaning negative: 9, leaning positive: 10
MeasureValueWhat it means
Accuracy87% of 131Share of reviews whose predicted side matched the stars
Negative recall0.77Share of the 39 negative reviews the model caught
Positive F10.91Balance of precision and recall on the 92 positive reviews
Macro F10.84Average of both classes, so the larger positive class does not hide the smaller one

Reading the words

The positive side is what you would expect from a language app: love, fun, learn, helps, and "spanish". The negative side is more useful. Next to general words such as "worst" and "time", the model picked out "gems", "energy" and "free": the in-app currency, the limit on lessons, and what users can do without paying. Those three words point to the part of the product the unhappy reviews are about, which the star average alone would never show.

Two details of the data are worth noting. Some reviews are in Spanish ("excelente" made the positive list), so a store's reviews are rarely in one language; filter by language or use a multilingual model if the share is large. And the 19 three-star reviews split almost evenly, 9 leaning negative and 10 positive, which is a reason to keep them out of training and score them separately.

Making it hold up on your own reviews

  1. Collect more than you think. 131 labelled reviews give a working model; a few thousand give stable word lists per topic. The collector returns up to 500 reviews per app per run.
  2. Balance the classes. Happy reviews outnumber unhappy ones more than two to one here. class_weight="balanced" stops the model from calling everything positive.
  3. Report cross-validated numbers. Accuracy on the training rows always looks better than it is. Cross-validation scores each review with a model that did not see it.
  4. Clean before you train. Duplicates, empty texts and mixed languages lower the score. Our guide to data wrangling in Python shows the cleaning steps on scraped rows.
  5. Re-collect on a schedule. A new release changes what people write about. Collect the newest reviews every week and compare the negative word list with the last one.

The same script works for any source with a star rating and text. Google reviews of local businesses are one call away (see how to scrape Google reviews), and product reviews from marketplaces feed the same model. For broader market research data, combine review sentiment with prices and rankings collected on the same day.

Sources & further reading

FAQ

Quick answers on sentiment analysis of customer reviews.

Something else? Ask us

What is sentiment analysis of customer reviews?

It is the automatic classification of each review as positive, negative or neutral, usually followed by the words or topics that drive each side. With star ratings as labels, a classic model learns the vocabulary of happy and unhappy customers and can then score text that has no rating.

How do you do sentiment analysis of reviews in Python?

Collect the reviews with their star ratings, label 4 and 5 stars positive and 1 and 2 stars negative, turn the text into TF-IDF features with scikit-learn, train a logistic regression and evaluate it with cross-validation. Our script does this on 150 App Store reviews and reaches 87% accuracy.

Which model is best for review sentiment analysis?

Start with TF-IDF and logistic regression: it runs on a laptop, needs no GPU and shows which words drive the result. Lexicon tools need no training but only know the words in their dictionary. Large language models read each review in full and can answer open questions about it, at a cost per review.

How many reviews do you need for sentiment analysis?

A hundred or so labelled reviews give a usable classifier, as in our run with 131. A few thousand give stable results per topic and per product version. Cross-validation tells you whether you have enough: if the score swings a lot between folds, collect more.

How do I get customer reviews for sentiment analysis?

Collect them from public review pages with a collector: App Store and Google Play reviews, Google reviews of local businesses, marketplace product reviews and Trustpilot. The App Store reviews collector returns rating, title, text, app version and vote counts, at $0.0005 per review.

What do you do with 3-star reviews?

Leave them out of training, because they mix praise and complaint, then score them with the trained model to see which way each leans. In our run the 19 three-star reviews split 9 leaning negative and 10 leaning positive.

Collect the reviews, then ask the questions

App Store, Google Play, Google, marketplace and Trustpilot reviews as rows with rating and text; App Store reviews cost $0.0005 each. Start with $2 of free API usage every month.

Related reading