Sentiment analysis of customer reviews labels each review as positive or negative and shows which words push it either way. With star ratings as labels, a TF-IDF and logistic regression model in Python takes a few lines. On 150 App Store reviews collected on 20 November 2026 it scored 87% accuracy in cross-validation.
Most tutorials train on a review file that is years old. Here the reviews were collected the same afternoon, the model was cross-validated on them, and every number below is from that run. The full script is in the post, so you can point it at your own product.
What sentiment analysis of reviews gives you
A star rating already says how happy a customer was. Sentiment analysis adds three things the stars do not:
- The why. The words that weigh most on the negative side tell you what customers complain about, without reading every review.
- A score for text without stars. Support tickets, social posts and survey answers have no rating. A model trained on rated reviews can score them.
- A split of the middle. Three-star reviews are neither. A model tells you which way each one leans.
There are three common approaches. Lexicon tools score words from a fixed dictionary and need no training. Classic machine learning, used here, learns the words from your own labelled reviews. Large language models read each review in full and can answer open questions about it; our test of a review model on 730 Google reviews compares that approach with the stars. Start with the classic model: it is fast, free to run and easy to inspect.
Step 1: collect the reviews
We picked a large consumer app with many recent reviews, Duolingo on the US App Store, and ran our App Store reviews collector from Claude through the QuanticData MCP server:
run_collector {
"slug": "app_store_reviews",
"input": { "app_id": "570060128", "country": "us", "max_results": 150 },
"async": true
}
It returned 150 reviews, each with the rating, title, text, version and vote counts. At $0.0005 per review the run cost $0.075. Reviews carry the author's display name; the script below drops it at once, because a sentiment model needs the text, not the person.
| Stars | Reviews | Label used |
|---|---|---|
| 5 | 82 | positive |
| 4 | 10 | positive |
| 3 | 19 | held out, scored at the end |
| 2 | 11 | negative |
| 1 | 28 | negative |
Step 2: the Python script
The script reads the run by its id over the REST API, labels the reviews from their stars, turns the text into TF-IDF features (words and word pairs, as described in the scikit-learn text feature extraction guide) and evaluates a logistic regression with 5-fold stratified cross-validation, so every review is scored by a model that never saw it.
import os, requests, numpy as np, pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_predict, StratifiedKFold
from sklearn.metrics import classification_report
RUN_ID = "YOUR_RUN_ID" # app_store_reviews, Duolingo, US store, 20 Nov 2026
API = "https://api.quanticdata.io/v1/scraper/collectors/runs/"
headers = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
rows = requests.get(API + RUN_ID, headers=headers, timeout=60).json()["payload"]["results"]
df = pd.DataFrame(rows)[["rating", "title", "text"]] # no author names kept
df["review"] = df["title"].fillna("") + ". " + df["text"].fillna("")
print(len(df), "reviews; stars:", df["rating"].value_counts().sort_index().to_dict())
# 1. Labels from the stars: 4-5 positive, 1-2 negative, 3 left out for now
lab = df[df["rating"] != 3].copy()
lab["label"] = np.where(lab["rating"] >= 4, "positive", "negative")
# 2. TF-IDF on words and word pairs, logistic regression, 5-fold cross-validation
vec = TfidfVectorizer(ngram_range=(1, 2), min_df=2, sublinear_tf=True, stop_words="english")
X = vec.fit_transform(lab["review"])
clf = LogisticRegression(max_iter=1000, class_weight="balanced")
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
lab["pred"] = cross_val_predict(clf, X, lab["label"], cv=cv)
print(classification_report(lab["label"], lab["pred"], digits=2))
# 3. What drives each side: the heaviest words of the model trained on all labelled rows
clf.fit(X, lab["label"])
terms = np.array(vec.get_feature_names_out())
order = np.argsort(clf.coef_[0]) # classes_ = ['negative', 'positive']
print("negative words:", ", ".join(terms[order[:12]]))
print("positive words:", ", ".join(terms[order[-12:]][::-1]))
# 4. Score the 3-star reviews the model never saw
mid = df[df["rating"] == 3]
p_pos = clf.predict_proba(vec.transform(mid["review"]))[:, 1]
print(f"\n3-star reviews: {len(mid)}, leaning negative: {(p_pos < 0.5).sum()}, leaning positive: {(p_pos >= 0.5).sum()}")
Step 3: the result
The output of that run, unedited:
150 reviews; stars: {1: 28, 2: 11, 3: 19, 4: 10, 5: 82}
precision recall f1-score support
negative 0.79 0.77 0.78 39
positive 0.90 0.91 0.91 92
accuracy 0.87 131
macro avg 0.85 0.84 0.84 131
weighted avg 0.87 0.87 0.87 131
negative words: used, lessons, use, right, guys, time, app, getting, free, gems, energy, worst
positive words: love, good, duolingo, fun, amazing, great, best, spanish, learn, learning, helps, excelente
3-star reviews: 19, leaning negative: 9, leaning positive: 10
| Measure | Value | What it means |
|---|---|---|
| Accuracy | 87% of 131 | Share of reviews whose predicted side matched the stars |
| Negative recall | 0.77 | Share of the 39 negative reviews the model caught |
| Positive F1 | 0.91 | Balance of precision and recall on the 92 positive reviews |
| Macro F1 | 0.84 | Average of both classes, so the larger positive class does not hide the smaller one |
Reading the words
The positive side is what you would expect from a language app: love, fun, learn, helps, and "spanish". The negative side is more useful. Next to general words such as "worst" and "time", the model picked out "gems", "energy" and "free": the in-app currency, the limit on lessons, and what users can do without paying. Those three words point to the part of the product the unhappy reviews are about, which the star average alone would never show.
Two details of the data are worth noting. Some reviews are in Spanish ("excelente" made the positive list), so a store's reviews are rarely in one language; filter by language or use a multilingual model if the share is large. And the 19 three-star reviews split almost evenly, 9 leaning negative and 10 positive, which is a reason to keep them out of training and score them separately.
Making it hold up on your own reviews
- Collect more than you think. 131 labelled reviews give a working model; a few thousand give stable word lists per topic. The collector returns up to 500 reviews per app per run.
- Balance the classes. Happy reviews outnumber unhappy ones more than two to one here.
class_weight="balanced"stops the model from calling everything positive. - Report cross-validated numbers. Accuracy on the training rows always looks better than it is. Cross-validation scores each review with a model that did not see it.
- Clean before you train. Duplicates, empty texts and mixed languages lower the score. Our guide to data wrangling in Python shows the cleaning steps on scraped rows.
- Re-collect on a schedule. A new release changes what people write about. Collect the newest reviews every week and compare the negative word list with the last one.
The same script works for any source with a star rating and text. Google reviews of local businesses are one call away (see how to scrape Google reviews), and product reviews from marketplaces feed the same model. For broader market research data, combine review sentiment with prices and rankings collected on the same day.


