review-intel
Provides review intelligence for App Store games by collecting public reviews via RSS and using App Store version history to link complaints and insights to specific releases.
Provides review intelligence for Google Play games by collecting public Google Play reviews and exposing classifications, sentiment, severity, and version-linked insights.
Delivers weekly review-intelligence reports to Slack via webhook.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@review-intel show me this week's biggest complaint spikes for critical-strike"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Player Review Intelligence
Player Review Intelligence collects public app-store reviews for mobile games, classifies them with Claude, links complaints to game updates with RAG, and serves the results as a weekly report, a dashboard and an MCP server. Claude Code skills and subagents run the workflows on top.

The problem
A live-service mobile game gets hundreds of store reviews a week across Google Play and the App Store, in many languages, with a star rating that says little about why. The useful signals are buried: an ad change that annoys players, a crash after an update, a weapon that feels pay-to-win. Game teams usually find these late, by skimming. This project turns the review stream into numbers a team can trust, plus a short weekly read that says what changed, what players say and who should act. Every number comes from the database, never from the model.
Games are configured in config/apps.yaml; adding one needs no code changes.
Related MCP server: google-play-analytics-mcp
What it does
Step | How | Measured on real data |
Collect | Google Play scraper plus the App Store RSS feed: polite pacing, retries, idempotent upserts, no usernames stored | 950 reviews across both stores |
Classify |
| 0.89 test accuracy against a stronger model's labels; ~$0.08 per 1,000 reviews |
Evaluate | Blind reference labels, dev/test split, per-class P/R/F1, confusion matrix, Sonnet judge | README table updated by |
Link to releases | App Store version history, local embeddings in | Answers cite real review ids and the right version |
Report | Code computes the facts, Sonnet writes the prose, code checks it, a judge scores it | Example report, ~$0.03 each |
Serve | FastAPI dashboard, MCP server (tools, resources, prompts), Slack webhook | Works in Claude Code via |
Automate | Claude Code skills and subagents in |
|
Architecture
flowchart LR
subgraph stores["Public stores"]
GP["Google Play"]
AS["App Store feed<br/>and version history"]
end
GP --> COL
AS --> COL
COL["collectors<br/>pacing, retries"] --> DB[("SQLite<br/>reviews, labels, patch notes, reports<br/>plus sqlite-vec vectors")]
DB --> CLS["classify<br/>Haiku 5.5, strict tool<br/>prompt cache, batches"]
CLS --> DB
DB --> IDX["index<br/>local MiniLM embeddings"]
IDX --> DB
EV["evals<br/>blind labels, metrics, judge"] -.->|keeps the prompt honest| CLS
DB --> INS["insights<br/>SQL answers"]
DB --> RAG["ask and spikes<br/>RAG, z-test"]
INS --> REP["weekly report<br/>Sonnet prose, checks, judge"]
INS --> DASH["dashboard<br/>FastAPI, SVG"]
INS --> MCP["MCP server<br/>tools, resources, prompts"]
RAG --> MCP
REP --> SL["Slack"]
MCP --> CC["Claude Code / Desktop<br/>skills, subagents"]Each module is described in two sentences in docs/ARCHITECTURE.md.
Quickstart
Requires uv, which installs Python 3.12 for you. Only an
ANTHROPIC_API_KEY is needed.
git clone https://github.com/LightWonkas/player-review-intelligence && cd player-review-intelligence
uv sync
cp .env.example .env # then set ANTHROPIC_API_KEY in .env
uv run review-intel fetch --app critical-strike --store all --limit 100
uv run review-intel classify --app critical-strike --limit 100 # ~$0.02
uv run review-intel report --app critical-strike --days 7 # ~$0.03
uv run review-intel dashboard # http://127.0.0.1:8765This exact sequence was run from a fresh clone. The database is created on first use, and every
command is safe to re-run. uv run review-intel --help lists the rest: eval, label,
patches, index, ask, spikes.
Example report
docs/example-report.md is a real report from the review data,
unedited. It has headline numbers against the previous week, the top issues with player quotes
and review ids, a version watch, a four-week sentiment table, prioritized actions for dev,
product and marketing, and the judge's score.
Classification
Each review gets one strict tool call (record_classification) to claude-haiku-5-5. The result
is a primary category plus up to three more from the 12 in
config/taxonomy.yaml, along with sentiment, severity (1-5), a
bug-report flag, mentioned features and a one-line summary.
Validation: Pydantic checks every response. Invalid output is retried once and then kept in
classification_failures, never dropped.Untrusted input: Review text goes in as escaped data inside
<review>tags, and a test checks that a malicious review can't break out.Prompt versions:
prompt_version(e.g.v1-10aea4a5) is a hash of the prompt and tool schema, so editing either creates a new version and results never mix.Cost: The ~1,900-token shared prefix is prompt-cached, so each review pays for about 300 input and 210 output tokens.
classify --dry-runprints the count and the estimated cost before you spend anything.
Evals
The reference set has 100 reviews, labeled blind: the labeler never sees the model's answer. The
current labels were set by claude-opus-5-5, a stronger model than the classifier, and recorded
per item in the labeler field. So the scores measure agreement with a stronger reference, not
human accuracy; review-intel label lets a person relabel or extend the set.
The sample is 40% random (mostly praise, like the real mix) and 60% low-rated reviews with real text, where the hard categories are, with at least 25% from the App Store. A hash of each review fixes it in a 70/30 dev/test split: prompts are tuned on dev, and the headline number is test. With 35 test reviews, one review moves accuracy by about 3 points, so every score comes with a 95% bootstrap interval.
review-intel eval reports:
primary-category accuracy, per-category precision/recall/F1 and macro-F1
sentiment accuracy and a confusion matrix
a comparison with the previous prompt or model
an LLM judge's faithfulness score for the per-review summaries (
claude-sonnet-5-5)
Predictions are cached per prompt version, so re-running an unchanged prompt is free.
Date | Prompt | Model | Test acc [95% CI] | Test macro-F1 [95% CI] | Sentiment acc (all) | Judge |
2026-11-19 |
| claude-haiku-5-5 | 0.89 [0.77-0.97] (n=35) | 0.70 [0.52-0.95] | 0.97 | 4.65/5 |
Read the intervals before the point estimates. Accuracy is solid (0.77-0.97), but macro-F1
spans 0.52-0.95: categories with 1-3 examples pull the average down, and the set is too small
to pin down rare categories. The weakest spot is the game's own auto-aim, which gets read as cheating
(controls_ux -> cheating_hackers). The improve-classifier
skill is built for exactly this kind of fix.
Linking complaints to releases (RAG)
uv run review-intel patches fetch --app critical-strike # release notes, one request
uv run review-intel index --app critical-strike # local embeddings, free
uv run review-intel ask "When did matchmaking complaints start?" --app critical-strike
uv run review-intel spikes --app critical-strikePatch notes: The App Store page lists the last ~25 versions with dates and notes, the only public release history with dates. Notes for other stores can be imported from markdown (
patches import). Version numbers are shared across stores, so one note serves both.Embeddings: Reviews and single patch-note lines are embedded locally with
all-MiniLM-L6-v2and stored insqlite-vectables in the same SQLite file, so search results join straight back to SQL. Re-indexing only embeds changed text.ask: Retrieves similar reviews and patch notes, adds the notes for the versions those reviews were written on, and adds a per-version timeline from SQL. Sonnet answers and cites[r<id>]and[v<version>]. Every citation is checked against what was retrieved; invented ones are removed and reported. Asked when complaints about a new weapon started, it named the release that added the weapon and the earliest review that mentions it.spikes: No LLM is involved. Per store, a version's share of reviews mentioning a category is compared with the previous versions using a two-proportion z-test. Stores are kept apart, so a shift in store mix can't fake a spike. Each spike links to the patch note in force, with its lines ranked by similarity to the complaints. When nothing is significant, the closest candidates are shown and labeled as not significant.
Weekly report and dashboard
Code lays out the report and fills in every number. claude-sonnet-5-5 writes only the prose, as
structured output, and code then checks it:
Numbers: every number in the prose must appear in the facts the model was given. If one doesn't, the model gets one retry with the offending numbers listed.
References: review ids and categories that weren't in the facts are dropped.
Judge: a separate call scores the prose for faithfulness, and the score goes in the footer.
On real data the first version scored 2/5: it said "players say" on the strength of one review, and it treated an old version as current. Fixing the facts (per-store versions, marked partial weeks, the severity scale) and asking for claims sized to their evidence brought it to 4/5.
The dashboard is FastAPI and Jinja2 with server-side SVG charts and no JavaScript. It shows
7-day stat tiles, top issues, reviews by category (7/30/90 days), weekly sentiment and the latest
report. Every chart has a table view, and both color modes were run through a color-vision
deficiency check. JSON routes live under /api/apps/{app}/.... The dashboard is read-only and
listens on localhost.

MCP server
review-intel-mcp serves the data over the Model Context Protocol
with the official Python SDK (MCPServer, the SDK 2.x name for FastMCP). It runs over stdio, or
over streamable HTTP with --transport streamable-http.
Kind | Name | What it does |
Tool |
| Top complaint categories vs the previous period, with example reviews |
Tool |
| Sentiment and average rating per day or week |
Tool |
| Category shares with z-scores, rating, sentiment and patch notes |
Tool |
| Semantic search with filters |
Tool |
| Cited answer (calls Claude, ~$0.014) |
Resource |
| One review, the latest report, the categories |
Prompt |
| Step-by-step plans that use the tools |
All tools except ask are read-only and free, and they share the insights
query layer with the report and the dashboard, so all three show the same numbers.
Claude Code: a session opened in this folder picks up .mcp.json; approve it
once. To add the server elsewhere:
claude mcp add --transport stdio review-intel -- uv run --directory /path/to/repo review-intel-mcp.
Claude Desktop: add this to claude_desktop_config.json:
{
"mcpServers": {
"review-intel": {
"command": "uv",
"args": ["run", "--directory", "C:\\path\\to\\player-review-intelligence", "review-intel-mcp"]
}
}
}Inspector: npx @modelcontextprotocol/inspector uv run review-intel-mcp
Claude Code skills and subagents
The repo uses Claude Code to operate the system, not just to build it.
Name | What it does | |
Skill | Fetch, classify and report for a game, then summarize. Prices each paid step before running it. | |
Skill | Adds a game to the config, checks both store ids against the stores, fetches a sample. | |
Skill | Eval, confusion matrix, one proposed change shown as a diff, re-eval, before/after table. | |
Subagent | Deduplicated bug list with versions, severity and example review ids. | |
Subagent | Sentiment by week and version, backed by numbers. | |
Subagent | Checks a report's quotes and claims against the data before it is shared. |
Each subagent only gets the free, read-only MCP tools. CLAUDE.md gives every
session the commands, the conventions and the rule: run tests and evals before claiming a change
works.
Cost
Prices as of November 2026 (pricing.py); check Anthropic's
pricing page before budgeting.
What | Model | Cost |
Classify 1,000 reviews, | Haiku 5.5 | ~$0.08 |
Classify 1,000 reviews, synchronous | Haiku 5.5 | ~$0.17 |
Weekly report, with judge | Sonnet 5.5 | ~$0.03 |
| Sonnet 5.5 | ~$0.014 |
Eval run (100 labels + 20 judged summaries) | Haiku + Sonnet | ~$0.08; ~$0.06 when cached |
Embeddings and index (950 reviews in 19 s on CPU) | local MiniLM | $0 |
A game with 1,000 new reviews a week costs about $0.11 per week: a batch classification plus one report. All the live runs behind this README cost under $1 in total.
Design decisions
Haiku to classify, Sonnet to write and judge. Classification is high-volume and narrow, and Haiku 5.5 reaches 0.89 test accuracy at about $0.08 per 1,000 reviews. Prose a product manager forwards, and grading other models' output, need the stronger model. The judge is never the model it grades.
Tool use for structured output. One forced, strict tool call with enum categories gives parseable JSON every time, and Pydantic covers what JSON Schema can't express. Results are keyed by prompt version and model, so comparisons stay clean.
Code computes, the model writes. Every number in a report, answer or dashboard comes from SQL. The model's prose is checked for numbers and ids that aren't in the facts, and a judge scores what's left. This is what makes the report safe to forward.
Local embeddings in the same database. MiniLM on CPU is free, keeps review text on the machine, and needs no second vendor.
sqlite-veckeeps vectors next to the rows they describe, so retrieval joins straight back to SQL. The trade-off is an English-focused model, which is one setting to change.Evals keep the prompt honest. Labels are blind, the dev/test split is fixed by hash, and predictions are cached per prompt version. The README table updates from real runs, so any prompt change shows up as a number before it ships.
Untrusted input everywhere. Review text is escaped and delimited in every prompt, flattened and escaped in reports, and autoescaped in the dashboard. Tests cover injection attempts at each layer.
Statistics before stories. Spikes need a z-score and a minimum sample, per store. With today's data that honestly means "no significant spikes" plus the closest candidates, rather than a confident narrative.
Limitations and next steps
Reference labels come from a stronger model, not people, and there are only 100. Scores measure agreement with Opus, and the macro-F1 interval is wide (0.52-0.95). A larger, human-labeled set with more rare-category examples would measure real accuracy (
review-intel labelis ready for it).Shallow history. The App Store feed serves only ~500 recent reviews per country, and fetching newest-first covers a few weeks of Google Play. With ~20 reviews per version, spikes rarely reach significance. A scheduled weekly fetch would build the history.
One storefront. Only the US store is tracked. Other countries are a config entry each, and multilingual reviews would want a multilingual embedding model (
EMBEDDING_MODEL).Judge scores vary. Scores for the same week ranged from 3/5 to 4/5 between runs. The
report-reviewersubagent is the second check before sharing.SQLite. Time windows and counts run as SQL filters and aggregates, but per-version timelines and spikes still read a game's full classified history, and vector search uses
sqlite-vec. For hundreds of thousands of reviews: Postgres throughDATABASE_URL, pgvector, and per-version counts in SQL.Next steps: human labels, a scheduled pipeline with Slack alerts when a spike becomes significant, and more storefronts.
Development
uv run pytest # 217 tests, no network, no real LLM
uv run ruff check . && uv run ruff format --check .
uv run mypy # strict
uv run python scripts/record_fixtures.py # re-record store fixtures (by hand)
uv run scripts/make_demo_gif.py # rebuild docs/demo.gif from a running dashboardCI runs ruff, mypy and pytest on every push. The full build plan is in docs/SPEC.md.
License
Available Tools
5 toolsaskARead-only
Answer an open question (e.g. "when did matchmaking complaints start?") from retrieved reviews and patch notes, citing review ids and versions. Calls Claude Sonnet: costs about $0.014 per question, so prefer the other tools first.
Inputs: app (required), question (required). Returns: notes, answer, cost_usd, cited_versions, +2 more
compare_versionsARead-only
Compare two app versions: reviews, rating, sentiment, the share of reviews mentioning each category (with a z-score; |z| >= 2 is roughly significant) and the patch notes in force for each.
Inputs: app (required), version_a (required), version_b (required). Returns: a, b, app, categories, +1 more
search_reviewsARead-only
Semantic search over reviews, optionally filtered by category and app version. Returns review ids, ratings, versions, dates, classification and (untrusted) text.
Inputs: app (required), query (required), limit, version, +1 more. Returns: app, query, results, version, +1 more
sentiment_trendBRead-only
Positive / neutral / negative counts and average rating per day or week.
Inputs: app (required), days, bucket. Returns: app, bucket, points, period_end, +1 more
top_issuesARead-only
The most common complaint categories in the last days days (by primary
category; praise and other excluded), with counts, share, severity, the count in
the previous period of the same length, and example reviews.
Inputs: app (required), days, limit. Returns: app, issues, period_end, period_start, +3 more
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Each tool targets a distinct analysis mode: raw semantic retrieval (search_reviews), complaint aggregation (top_issues), version diffing (compare_versions), time-series sentiment (sentiment_trend), and open-ended Q&A (ask). The main soft spot is `ask`, which is a general-purpose catch-all that could plausibly answer anything the other four handle, but its description explicitly tells agents to prefer the others, which mitigates the overlap.
All names are snake_case and self-descriptive, but the grammatical pattern varies: verb_noun (search_reviews, compare_versions), noun phrases (top_issues, sentiment_trend), and a bare verb (ask). It reads cleanly and an agent can infer intent, with `ask` being the only notably vague outlier.
Five tools is tightly scoped for a review-analytics server, with each tool covering a genuinely different question type (retrieval, ranking, comparison, trend, synthesis). Nothing feels padded or redundant.
The surface covers search, aggregation, version comparison, trends, and open Q&A with cited review ids, which is most of the review-intel lifecycle. Minor gaps: no direct fetch-by-review-id, no standalone patch-notes or category-taxonomy listing (those are only reachable indirectly through compare_versions and ask).
Maintenance
Related MCP Connectors
Ask Claude, ChatGPT, or any MCP client about your game's players, retention, revenue, and win rates.
MCP server unifying ERPs, CRMs, APIs and knowledge base for Claude, ChatGPT and Gemini.
Your agent needs app-store data — what an app looks like on the App Store and Google Play, what reviewers say, and what ranks for a search in a given country. **What you can ask for** • "What does this app's store listing look like, and how is it rated?" • "Pull recent reviews for this app and group the complaints." • "What apps rank for this search term in Japan?" • "List the top apps in this category on both stores." • "Compare this app's listing on iOS and Android." **How to use it** Point any MCP client at https://mcp.aisa.one/seo-apps/mcp and sign in with OAuth — there is no key to create or paste. 37 tools: Apple App Store and Google Play app info, app lists, category listings, reviews and store search results, in live and queued forms, with categories, languages and locations for each. **It is also a door to the rest** The same login reaches 26 sources and 580+ operations. Read the store listing here, then ask the same agent what the app's website ranks for — without adding a second server. **What it costs** Finding and inspecting an operation is free. Running one is billed per call at API prices, with no seat and no monthly minimum, and every call takes max_price_usd so an agent cannot overspend by accident. **Where else it reaches** https://mcp.aisa.one/seo/mcp for all of it at once — rankings, keywords, backlinks, site health and AI-answer visibility across DataForSEO, Semrush and Ahrefs.
ASOMobile MCP server: App Store Optimization data for the iOS App Store and Google Play inside AI agents. 29 tools: keyword research (traffic, competition, suggestions), app keyword indexing, competitors and keyword gaps, top charts and chart history, reviews with rating summaries, Apple Search Ads insights, organic download estimates and Keyword Monitor rank tracking, plus six ASO skills (app audit, keyword research, competitor audit, localization check, review analysis, ASA planning). Remote streamable HTTP with OAuth 2.1 (DCR + PKCE); sign in with your ASOMobile account.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceRemote MCP server that exposes Airspeed/Glyphic call data to Claude web, enabling tools to list, retrieve, and query calls, transcripts, snippets, and playbooks.ISC
- AlicenseNot gradedqualityCmaintenanceAn MCP server that exposes Google Play Console analytics to AI agents, including store-listing acquisitions by traffic source, installs, uninstalls, ratings, crashes, subscriptions, and reviews by reading bulk CSV reports from Cloud Storage.MIT
- AlicenseAqualityDmaintenanceMCP server for Claude Desktop that exposes App Store Connect and RevenueCat APIs, enabling indie iOS developers to query portfolio metrics, sales reports, subscriptions, and customer reviews directly in chat.17MIT

GetAppNiche MCPofficial
AlicenseNot gradedqualityAmaintenanceEnables AI agents to query live App Store and Google Play data, including app search, revenue/download metrics, keyword difficulty, and reviews, via a hosted MCP server with API-key authentication.76 npmMIT