# product-feedback-mcp

[![CI](https://github.com/25andresbernal/product-feedback-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/25andresbernal/product-feedback-mcp/actions/workflows/ci.yml)

An MCP server that lets Claude triage product feedback: search it, group
it into themes, tag its severity, and draft a PRD-style problem
statement, all over a local dataset.

## Why this exists

A PM who wants to know what customers are complaining about usually ends
up pasting a spreadsheet of tickets and reviews into a chat window and
asking for a summary. That works once. It does not scale to "which
themes came up this week," it does not let you ask a follow-up question
against the same data, and every teammate who wants an answer has to
paste the spreadsheet again.

An MCP server fixes that by giving Claude actual tools: `search_feedback`,
`list_themes`, `get_theme`, `tag_severity`, `severity_summary`, and
`draft_problem_statement`, backed by one dataset that lives in the repo.
Ask "what should we look at this week" and Claude calls the tools
instead of guessing from whatever text you happened to paste in.

The dataset here is synthetic feedback for a fictional B2B shift
scheduling product, Shiftly, but the server does not know or care that
it is fictional. Point `FEEDBACK_DATASET_PATH` at a real export in the
same shape and every tool works the same way.

## Demo

`scripts/demo.py` calls the server the same way a real MCP client would
(the SDK's in-memory `Client`, no subprocess) against the committed
200-item dataset. Real output, captured by running it:

```
$ uv run python scripts/demo.py
```

`list_themes(min_items=5)` returned six themes. Each label is a real
member's own opening clause, chosen from the cluster's most central
item, so a PM can read the list without opening anything. `keywords`
stays available for transparency and `representative_item_id` names the
item the label came from:

```json
[
  { "theme_id": "theme-01", "label": "Really solid support experience this week", "size": 7, "representative_item_id": "fb-0032" },
  { "theme_id": "theme-02", "label": "The Shiftly app crashed constantly", "size": 6, "representative_item_id": "fb-0042" },
  { "theme_id": "theme-03", "label": "Sales call: wanted benchmark numbers before rolling this out to the kitchen team", "size": 6, "representative_item_id": "fb-0089" },
  { "theme_id": "theme-04", "label": "Notifications are hit or miss", "size": 5, "representative_item_id": "fb-0049" },
  { "theme_id": "theme-05", "label": "Two-factor rollout for the delivery drivers went smoothly", "size": 5, "representative_item_id": "fb-0035" },
  { "theme_id": "theme-06", "label": "Sales call: asked what reporting looks like out of the box", "size": 5, "representative_item_id": "fb-0036" }
]
```

(`keywords` omitted above for width; the full objects are in the real
output.)

`draft_problem_statement("theme-02")`, the first complaint-majority
theme. The evidence quotes are picked for diversity across phrasing,
source, and segment rather than taken in order, and `kind` tells a
client whether it is looking at a problem or a strength:

```json
{
  "theme_id": "theme-02",
  "theme_label": "The Shiftly app crashed constantly",
  "kind": "problem",
  "representative_item_id": "fb-0042",
  "who": [
    { "customer_segment": "small_business", "count": 4 },
    { "customer_segment": "enterprise", "count": 1 },
    { "customer_segment": "mid_market", "count": 1 }
  ],
  "what": "Customers across 3 segment(s) repeatedly report: The Shiftly app crashed constantly (6 of 200 items, 3.0%).",
  "evidence": [
    { "feedback_id": "fb-0014", "source": "nps_comment", "quote": "The app has crashed on the scheduling admin three times this week, this is getting old." },
    { "feedback_id": "fb-0084", "source": "support_ticket", "quote": "App crashed and the shift I published for our call center agents disappeared. Had to rebuild it from memory." },
    { "feedback_id": "fb-0033", "source": "support_ticket", "quote": "Ugh, the app crashed again this morning right as the nursing unit tried to punch in." },
    { "feedback_id": "fb-0042", "source": "nps_comment", "quote": "The Shiftly app crashed constantly when the delivery drivers tried to clock in for a holiday week." },
    { "feedback_id": "fb-0017", "source": "support_ticket", "quote": "App crashed and the shift I published for our support reps disappeared. Had to rebuild it from memory." }
  ],
  "frequency": {
    "count": 6,
    "percent_of_dataset": 3.0,
    "by_source": { "nps_comment": 2, "support_ticket": 4 },
    "date_range": { "earliest": "2025-11-03", "latest": "2026-06-27" },
    "average_rating": 2.0
  },
  "suggested_success_metric": "Reduce 'The Shiftly app crashed constantly' feedback volume from 6 items (mostly via support_ticket) to 1 or fewer over the same reporting period, with no critical- or high-severity item left unresolved for more than one release cycle."
}
```

For a praise-majority theme such as theme-01, `kind` is `"strength"` and
the wording flips: the statement says what to protect and the metric is a
floor to hold, not a volume to reduce.

Full output, including `severity_summary()` and `get_theme()`, is in
[`scripts/demo.py`](scripts/demo.py); run it yourself to see all of it.

## Architecture

```mermaid
flowchart LR
    subgraph Client
        C[Claude Desktop, Claude Code,\nor any MCP client]
    end
    subgraph Transport
        T1[stdio\nlocal subprocess]
        T2[streamable HTTP\n+ bearer token]
    end
    subgraph Server[product-feedback-mcp]
        TOOLS[Tools\nsearch_feedback, list_themes,\nget_theme, tag_severity,\nseverity_summary,\ndraft_problem_statement]
        RES[Resources\nfeedback://summary\nfeedback://item/id]
        PROMPT[Prompt\ntriage_this_weeks_feedback]
    end
    DATA[(data/feedback.jsonl\n200 synthetic items)]

    C --> T1 --> Server
    C --> T2 --> Server
    TOOLS --> DATA
    RES --> DATA
```

## Quick start

Takes under five minutes, no API key required.

```bash
git clone https://github.com/25andresbernal/product-feedback-mcp.git
cd product-feedback-mcp

export PATH="$HOME/.local/bin:$PATH"   # if uv is not already on PATH
uv venv --python 3.12
uv pip install -e ".[dev]"

# See it work end to end
uv run python scripts/demo.py

# Run the test suite
uv run pytest

# Run the server itself, over stdio, the way an MCP client launches it
uv run product-feedback-mcp
```

The dataset is already committed at `data/feedback.jsonl`. To regenerate
it (or make a different-sized sample):

```bash
uv run python scripts/generate_dataset.py --count 200 --seed 42
```

### Connect it to a client

**Claude Desktop** (`claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "product-feedback": {
      "command": "uv",
      "args": [
        "run",
        "--directory",
        "/absolute/path/to/product-feedback-mcp",
        "product-feedback-mcp"
      ]
    }
  }
}
```

**Claude Code:**

```bash
claude mcp add product-feedback -- uv run --directory /absolute/path/to/product-feedback-mcp product-feedback-mcp
```

**Cursor, or any other MCP client:** the pattern is the same: point the
client's stdio server config at `uv run --directory <path to this repo>
product-feedback-mcp`. No environment variables or auth are needed for
stdio (see "Auth" below).

## Configuration

All configuration is environment variables; see [`.env.example`](.env.example).

| Variable | Required | Default | Purpose |
|---|---|---|---|
| `FEEDBACK_DATASET_PATH` | no | `data/feedback.jsonl` | Which `.jsonl` file the server reads. Point it at a real export in the same shape to use real data. |
| `MCP_AUTH_TOKEN` | only for `--transport http` | none | The bearer token clients must send. The server refuses to start over HTTP without it. |

CLI flags (`product-feedback-mcp --help`): `--transport stdio\|http`
(default `stdio`), `--host`, `--port` (HTTP transport only).

## How it works

- **`dataset.py`**: loads and validates `feedback.jsonl` into
  `FeedbackItem` records, and caches the parsed result per path so every
  tool call in a session reuses the same in-memory data instead of
  re-reading the file.
- **`search.py`**: a from-scratch Okapi BM25 implementation (stdlib
  only) over the feedback text, used by `search_feedback`.
- **`themes.py`**: deterministic clustering with no LLM and no
  randomness. Each item gets its top TF-IDF keywords (unigrams and
  bigrams); two items merge into the same theme only if they share at
  least two of those keywords, using a union-find over the whole
  dataset. A theme's label is a real member's own words, not a
  synthesized phrase: pick the cluster's medoid (the member whose
  keyword set overlaps most, on average, with every other member's,
  using those same keyword sets), then keep that item's first clause
  (`_first_clause`, cut at the first comma, period, or standalone "and"
  after at least four words, capped at 14 words either way, cutting
  before a subordinating word or trailing function words when the cap
  hits mid-sentence). If the
  cluster mixes praise and complaints, the medoid search is restricted
  to whichever side is the majority (by median rating if any member has
  one, otherwise by a small positive/negative phrase list), so the
  label is never a positive sentence pulled from a cluster that reads
  as a complaint or vice versa. `representative_item_id` on every theme
  names exactly which item the label came from. See "Design decisions"
  below for why clustering and labeling both work this way.
- **`text.py`**: also has `select_diverse_items`, a greedy picker used
  by `get_theme` and `draft_problem_statement` to choose representative
  quotes: start from the lowest-id member, then repeatedly add whichever
  remaining item shares the fewest tokens with everything already
  picked, breaking ties toward a source or customer_segment not yet
  represented. Without it, "first five members" tends to surface five
  near-duplicates of the same template with one word swapped.
- **`severity.py`**: rule-based severity tagging from keyword lists, the
  item's rating (if any), and its source, all in one short, readable
  function.
- **`problem_statement.py`**: assembles `draft_problem_statement`'s
  output from a theme's own member items with plain arithmetic; nothing
  in it is generated by a model.
- **`server.py`**: registers all six tools, the two resources
  (`feedback://summary`, `feedback://item/{id}`), and the
  `triage_this_weeks_feedback` prompt on an `MCPServer` instance.
- **`auth.py`**: the bearer-token middleware for the HTTP transport.
- **`cli.py`**: the `product-feedback-mcp` entry point.

## Auth

**stdio** (the default, and what both client configs above use): no auth
at all. The client launches the server as a local subprocess it already
controls and talks to it over that subprocess's own stdin/stdout. There
is no network socket for anyone else to reach, so there is nothing to
authenticate.

**streamable HTTP**: reachable over a socket, so it needs a check before
it will run a tool. This server implements the simplest one that is
still real: a single static token read from `MCP_AUTH_TOKEN`, required
as `Authorization: Bearer <token>` on every request
(`src/product_feedback_mcp/auth.py`). The server refuses to start over
HTTP at all if the variable is unset.

That is a real tradeoff, not a shortcut taken by accident. A static
bearer token has no expiry, no per-client scoping, and no revocation
short of rotating the value and redeploying. The `mcp` SDK also ships a
full OAuth authorization flow (`TokenVerifier`, `AuthSettings`,
protected-resource metadata) for exactly the cases that need those
things: multiple clients with different permissions, tokens that expire,
a real identity provider. For a single-tenant demo server backed by a
static local file, that machinery is a lot of moving parts for no
practical gain. If this server ever needed multiple callers with
different access levels, that is the point to switch.

Run it:

```bash
export MCP_AUTH_TOKEN="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"
uv run product-feedback-mcp --transport http --port 8000
```

`tests/test_auth.py` covers both paths: a request with no token or the
wrong token gets `401`, and a request with the right token gets past the
middleware.

## Design decisions

- **Deterministic keyword clustering instead of an LLM call inside the
  server.** `list_themes` and `get_theme` need to return the same
  answer every time for the same dataset, cheaply and offline, since the
  test suite and the demo script both depend on that. An LLM call would
  make every theme-clustering test either mocked (testing nothing real)
  or slow, flaky, and dependent on a paid API key just to run `pytest`.
  The tradeoff: keyword overlap is a much blunter instrument than an
  embedding model or an LLM's judgment, and it will split or merge
  themes a human would draw differently. `search_feedback` and
  `tag_severity` also stay on the same offline, deterministic footing
  for the same reason.
- **Requiring two shared keywords to merge two items, not one.** The
  first version of `themes.py` merged any two items that shared even one
  top keyword, which chained unrelated complaints together through a
  single common word (a generic verb, or a word two different templates
  both happened to use) into one oversized catch-all cluster. Requiring
  real overlap fixed that at the cost of some recall: two items about
  the same underlying problem, phrased differently enough to share only
  one keyword, end up in separate themes instead of one. Widening the
  dataset's phrasing (more sentence frames per theme, concrete detail
  fillers like a role or a time) then undercut that same fix from the
  other direction: a short sentence's top keywords started skewing
  toward whatever rare filler word it happened to use instead of the
  theme's real anchor term, so `TOP_KEYWORDS_PER_ITEM` went from 3 to 8
  to give that anchor term room to make the top set alongside the
  filler, and the "who/when" noise list (`NOISE_TERMS`) grew to cover
  the day, time, count, and role filler vocabulary the same way it
  already covered team names, so a coincidence like two unrelated items
  both mentioning "Monday morning" cannot count as a shared keyword.
- **A theme's label is a trimmed real quote, not assembled keywords.**
  The first version built labels from the cluster's top TF-IDF unigrams
  joined with `" / "` (`"answer / real / minutes"`), which is exactly
  as informative as it sounds: readable only to someone who already
  knows what the cluster is about. Keyword fragments cannot describe a
  theme in language because they are not language. `_first_clause` on
  the cluster's medoid item fixes that, at a real cost: a sentence with
  no early comma gets cut at a hard word cap, and the cut is made before
  the last subordinating word inside the cap ("...crashed constantly"
  rather than "...tried to clock"), which can drop useful detail from the
  label. The full quote is always one call away through
  `representative_item_id`, so the label trades completeness for
  readability on purpose.
  A sentiment-aware medoid restriction is layered on top of that for the
  same reason: a straight "most central item" medoid on a
  mostly-positive cluster with one complaint mixed in could just as
  easily land on the complaint, producing a negative-sounding label for
  a theme that is mostly praise. Restricting the medoid search to the
  cluster's majority side (median rating, or a small phrase list when
  nobody has one) fixes that at the cost of occasionally picking a
  slightly less central item than the unrestricted medoid would have.
- **Evidence quotes are picked greedily for diversity, not just taken in
  order.** `select_diverse_items` starts from the lowest-id member and
  repeatedly adds whichever remaining item overlaps least with what is
  already picked. The tradeoff: greedy is not globally optimal (a
  different starting point could occasionally produce a more diverse
  set of five), and it is still just a token-overlap heuristic, not a
  read for semantic diversity. It is enough to stop five near-identical
  praise quotes from crowding out the one real complaint in a cluster,
  which is the failure mode that mattered here.
- **BM25 implemented in stdlib instead of a `rank_bm25` dependency.**
  The formula is small and well known, and writing it out means there is
  nothing to configure or version-pin for a dataset this size (a couple
  hundred short documents). The tradeoff is obvious: a real search
  product would use a maintained library, or a real search engine,
  rather than hand-rolled ranking code.
- **A tool interface, not a raw file the client reads itself.** An MCP
  client could just read `feedback.jsonl` directly if it had filesystem
  access. Tools instead give it `search_feedback`, `list_themes`, and
  the rest, which means the ranking, clustering, and severity logic are
  defined once, tested once, and identical no matter which client or
  model is calling them.
- **A single static file as the datastore.** No database, no ingestion
  pipeline. That is right for a demo server whose whole point is
  showing tool design against a fixed, inspectable dataset, and wrong
  for anything that needs to ingest new feedback continuously; swapping
  `dataset.py`'s file read for a real data source would not require
  changing any tool's interface.

## Roadmap

- A `refresh_dataset` tool or resource subscription so a long-running
  server picks up an updated `feedback.jsonl` without restarting.
- An optional embedding-based clustering backend behind the same
  `list_themes` / `get_theme` interface, for datasets where keyword
  overlap clusters too coarsely.
- Pagination for `search_feedback` and `list_themes` on much larger
  datasets than the couple hundred rows this one ships with.
- A `--transport http` example using a real reverse proxy in front of it
  (TLS termination, rate limiting) to show the auth tradeoff section in
  practice rather than only in prose.

## Contributing

Issues and pull requests are welcome. Before opening a pull request:

```bash
uv run ruff check .
uv run ruff format .
uv run pytest
```

If you change `scripts/generate_dataset.py`, regenerate
`data/feedback.jsonl` and confirm `tests/test_dataset.py`'s determinism
tests still pass; the committed dataset should always match what the
script currently produces.

## License

MIT. See [LICENSE](LICENSE).