![CI](https://github.com/subhamjalan-spec/whatsapp-mcp/actions/workflows/ci.yml/badge.svg)
<!-- Replace YOUR_USERNAME/YOUR_REPO once pushed, or the badge will be broken. -->

# WhatsApp MCP Assistant: RAG search + ML analysis + message sending

An MCP server over WhatsApp chat history combining all three things
discussed: hybrid-retrieval RAG search over the conversation, ML
analysis (sentiment, topic clustering, real stats), and WhatsApp Cloud
API message sending -- exposed as five real MCP tools.

Builds directly on the design proven in the docs-RAG project (hybrid
BM25 + TF-IDF/LSA retrieval, real MCP protocol testing), retargeted at
conversational data, which has its own genuinely different challenges
from documentation (short messages, informal language, multi-speaker
attribution, timestamps).

## A real bug found live: iPhone export format wasn't supported

While actually testing this against a real, live-exported chat (not the
synthetic sample data), the pipeline crashed with a cryptic
`ZeroDivisionError` three layers deep inside the BM25 library. Root
cause, confirmed directly: **iPhone WhatsApp exports use a completely
different format than Android**:

```
Android:  12/01/24, 09:15 - Priya: message text
iPhone:   [12/01/24, 9:15:32 AM] Priya: message text
```

Bracketed instead of dash-separated, includes seconds, uses 12-hour
AM/PM instead of 24-hour time -- the parser only handled the Android
format, so every line in a real iPhone export silently failed to match,
producing zero parsed messages, which then crashed deep inside a
third-party library instead of failing with a clear message at the
actual source of the problem.

**Fixed properly, not patched around:** the parser now tries both
formats, correctly converts 12-hour AM/PM to 24-hour time, and handles
a real, easy-to-miss quirk -- iPhone exports commonly use `U+202F`
(a narrow no-break space, invisible when eyeballing the file) instead of
a plain space before "AM"/"PM", which a plain-ASCII regex silently fails
to match. **Also added:** `ChatPipeline` now raises a clear, specific
error ("Parsed zero messages... check that this is really a WhatsApp
export file") if a chat file produces zero messages, instead of letting
that surface as an unrelated crash inside a dependency three call-stack
frames away from the actual cause.

## Attempted improvement that honestly didn't help: a trained reranker

The docs-RAG project's biggest quality lever was a trained reranker on
top of hybrid retrieval. It was missing here, so it was built
(`wa_mcp/reranker.py`, same design: logistic regression trained from
scratch on BM25/dense scores, term overlap, chunk length, plus a
chat-specific "distinct senders in this chunk" feature) and rigorously
measured, not just added and assumed to help.

**Real result, on the held-out test queries:**
```
Hybrid alone:       precision@5=0.200  recall@5=1.000  mrr=0.815
Hybrid + reranker:  precision@5=0.200  recall@5=1.000  mrr=0.726
```

**The reranker measurably does NOT help here, and slightly regresses
MRR.** Most likely cause: only 8 training queries (120 examples, 8
positive) is too small a training set for the reranker to learn
reliable weights -- smaller than the docs project's training set, on a
task where hybrid retrieval alone is already close to its ceiling at
this corpus size.

**Decision: not wired in as the default**, specifically because shipping
something that measures worse while calling it an "improvement" would
be dishonest. The code is real, tested for correctness
(`tests/test_reranker.py` -- training produces valid, imbalanced
labels; predictions are valid probabilities), and available
(`wa_mcp/reranker.py`) for a much larger real chat where a bigger,
real-usage-derived training set might behave differently -- but it's
not claimed as a proven improvement, because it isn't one, measured
honestly.

**The improvement that DID measurably help was the filler-message
filtering described above** -- that one is wired in as the default,
because it was actually measured to help.

## Attempted improvement #2: a trained filler classifier -- helped on its own test, then failed at real scale

The fixed FILLER_PHRASES list has an honest, stated limitation: it can
only catch filler wording someone thought to write down. So a trained
alternative was built (`wa_mcp/filler_classifier.py`, logistic
regression on TF-IDF, same pattern as the sentiment classifier) and
compared directly against the fixed list -- including on filler
phrasings deliberately absent from the fixed list ("hbu", "no cap",
"word", "true story", "for real").

**On its own held-out test set, the trained classifier looked like a
clear win:**
```
accuracy: 0.9   precision: 1.0   recall: 0.8
```
It correctly caught 4 of 5 novel slang variants the fixed list would
have completely missed, while never wrongly flagging a real,
substantive message.

**Then it was tested against the actual 60,000-message stress-test
corpus used earlier -- and it did measurably WORSE than the fixed
list:**
```
Fixed list:          2 chunks remaining after filtering
Trained classifier:  1,728 chunks remaining after filtering
```
The trained classifier let obvious filler through repeatedly ("call me
later", "wait what" appearing in its own top search results,
unfiltered).

**Root cause, confirmed directly, not guessed:** checked exactly which
filler phrases the 60k-message test corpus actually used against what
the classifier's 28-example training set had ever seen --
**14 of the 22 real filler phrases in that corpus were never in the
classifier's training data at all** ("yeah", "wait what", "sounds
good", "call me later", "checking now", and 9 others). The classifier
isn't learning a general concept of "this message carries no
retrieval-worthy content" -- with only 28 examples, it's memorizing its
own narrow training vocabulary, the same fundamental limitation as the
fixed list, just less predictable and harder to inspect/edit than a
plain list you can read and extend directly.

**This is a genuinely important, humbling lesson, not a wasted
attempt:** a small held-out test set drawn from a similar, narrow
vocabulary can make a tiny trained model look like it generalizes well,
right up until it meets real data with a genuinely different
vocabulary distribution -- exactly the gap between "measures well on
its own test set" and "actually works in the real world" that's worth
knowing about as a general ML lesson, not just for this one feature.

**Decision: the fixed list remains the default**
(`use_trained_classifier=False` in `chunk_messages()`), specifically
because it measurably performed better on the real, at-scale test that
actually matters. The trained classifier is kept available
(`use_trained_classifier=True`) and tested for correctness
(`tests/test_filler_classifier.py`) -- but not claimed as an
improvement, because real testing found the opposite. A genuinely
larger training set (hundreds of real labeled examples, not 28) would
likely change this finding; not built here, since that's real,
substantial labeling effort, not a quick fix.

## A real scaling investigation: tested at 60,000 messages, not assumed

This project was originally built and tested against a 66-message
synthetic sample. A real question came up during use: does this actually
work at real chat scale (tens of thousands of messages)? Rather than
guess, a realistic 60,000-message synthetic chat was generated (mostly
short filler messages -- "lol", "ok", "one sec" -- with one real,
findable conversation planted in the middle) and tested directly.

**First result:** it does NOT crash and is NOT slow -- built in 5.9s,
searched in 0.035s even at 60k messages. But retrieval QUALITY
genuinely degraded: the planted conversation about booking "Cotton
Villa" got diluted across noisy chunks full of filler messages, and
none of the top-5 results was a clean match.

**Root cause:** with ~20,000 sliding-window chunks at this scale, a
5-message chunk that's 80% filler ("yep", "one sec", "wait what") has
far less distinguishing content for BM25/TF-IDF to match against than
one where every message carries real content -- filler doesn't just
fail to help, it actively dilutes genuine signal sitting in the same
window.

**Fix:** `wa_mcp/chunking.py` now filters out common filler messages
before chunking (they're still counted in message-count stats, just
excluded from retrieval chunks specifically). Re-tested against the
exact same 60k-message file: chunk count dropped from 19,999 to 2, and
both top results were now clean, correct matches for the planted
conversation.

**Honest limitation, not glossed over:** the filler list is a fixed set
of phrases, not a trained classifier -- it will always miss some
real-world phrasings. One was caught and fixed during this very
investigation ("haha true" wasn't in the initial list, only found by
testing against the large synthetic file and inspecting actual results).
A more robust version would train a small classifier the same way the
sentiment model was trained, rather than hardcode a list -- not
implemented here, stated as a real next step rather than presented as
solved.

**Also worth noting honestly:** the synthetic test chat was closer to a
worst-case (nearly all filler except one planted signal) than a typical
real chat, which usually has a higher proportion of substantive
messages. The *mechanism* (filtering filler improves signal-to-noise at
scale) is validated; the exact magnitude of improvement on any specific
real chat will vary.

## Data: synthetic, not anyone's real private chat

data/sample_chat.txt is a hand-written, fictional conversation (three
people planning and taking a trip), written in the REAL WhatsApp export
text format (DD/MM/YY, HH:MM - Name: message). The parser handles this
real format, so a genuine exported chat (Settings > Export Chat in
WhatsApp) would work too -- deliberately not used here, same reasoning
as writing a fictional docs corpus instead of scraping real documents.

## Part 1: RAG search over chat history

Individual WhatsApp messages are often 1-5 words ("yes", "lol", "on my
way") -- too short to retrieve meaningfully alone. wa_mcp/chunking.py
groups messages into overlapping sliding-window chunks (window=5,
stride=3) so retrieval has something substantive to match against, with
the overlap ensuring a conversational moment near a window boundary is
still captured whole in at least one chunk.

Retrieval reuses the proven hybrid design (BM25 + TF-IDF/LSA + Reciprocal
Rank Fusion, with Porter stemming) from the docs-RAG project.

**Honest note on retrieval quality on chat data specifically:** casual
chat text is measurably harder to retrieve well than clean documentation
prose. A real example found during testing: the query "who was stressed
about work" found the correct chunk at rank 4 out of 22, not rank 1 --
reasonable (top 20%), but not perfect, and reported as such rather than
only testing easy queries that happen to rank first.

## Part 2: ML analysis

- **Sentiment classifier** (wa_mcp/sentiment.py): logistic regression
  TRAINED FROM SCRATCH on hand-labeled, informal chat-style examples
  (not clean prose) -- same "no pretrained model" reasoning as the docs
  project's reranker (this sandbox has no access to huggingface.co).
  **Real, honestly reported result** on a held-out test set: **70%
  accuracy, 62.5% precision, 100% recall.** Clearly better than random
  guessing on a small, hand-labeled dataset -- and clearly limited by
  that small dataset size, not presented as more than it is.
- **Topic clustering** (wa_mcp/topics.py): TF-IDF + KMeans, classical,
  no download needed. Tested against a real, checkable claim: every
  chunk mentioning "Cotton Villa" (the accommodation) lands in the same
  cluster, distinct from the beach-day and stress-venting conversations.
- **Chat statistics** (wa_mcp/stats.py): message counts per sender,
  average response time between different senders, busiest day -- all
  directly computed, including honestly handling a real tie (two
  participants are tied at 23 messages each in the sample data; tested
  as "one of the tied leaders," not asserting a single winner as if the
  tie didn't exist).

## Part 3: WhatsApp message sending -- the honest limitation

wa_mcp/whatsapp_client.py is written to Meta's actual, currently-
documented Cloud API request format (POST /{phone_number_id}/messages,
Bearer token auth, messaging_product/to/type/text JSON body --
verified via Meta's developer docs, not assumed from memory).

**This has never been verified against the live API.** This sandbox has
no network access to graph.facebook.com (confirmed directly: a plain
curl request to it returns 403, same category of block as
huggingface.co earlier), and there's no real WhatsApp Business Account
or access token available regardless. Tests
(tests/test_whatsapp_client.py) mock the HTTP layer to verify the
client builds requests correctly against Meta's documented contract --
that is NOT the same claim as "this works against the real API," and
this README says so explicitly rather than blurring the two.

**One genuinely interesting thing found while building this**: calling
send_whatsapp_message with a placeholder token, without any mocking,
actually attempted a REAL network request and came back with HTTP 403
-- confirming this sandbox's network block directly, and confirming the
client fails cleanly (returns a proper error result) rather than
crashing, even against a real, if blocked, network attempt -- not just
against mocked failures.

## Build & run

```bash
python3 -m venv venv
./venv/bin/pip install -r requirements.txt
```

Run tests:
```bash
./venv/bin/python -m pytest tests/ -v
```

Try it live (uses the fictional sample chat by default):
```bash
./venv/bin/python -m wa_mcp.interactive_demo
```

Or point it at a real chat you've exported yourself (Settings > Export
Chat in WhatsApp, no API needed -- both Android and iPhone export
formats are supported):
```bash
./venv/bin/python -m wa_mcp.interactive_demo ~/Downloads/_chat.txt
```

Run the MCP server:
```bash
./venv/bin/python -m wa_mcp.server
```

To actually send real WhatsApp messages (untested against the live API,
see above), set real credentials first:
```bash
export WHATSAPP_PHONE_NUMBER_ID="your-real-id"
export WHATSAPP_ACCESS_TOKEN="your-real-token"
./venv/bin/python -m wa_mcp.server
```

## Test suite

```
tests/test_parser.py            10/10 -- both Android + iPhone export formats,
                                          incl. multi-line messages and the
                                          narrow-no-break-space quirk
tests/test_chunking.py           6/6  -- sliding window + overlap + filler filtering
tests/test_pipeline.py           2/2  -- clear error on unrecognized format
tests/test_retrieval.py          3/3  -- hybrid search on real chat queries
tests/test_sentiment.py          5/5  -- trained-from-scratch classifier, real train/test split
tests/test_stats_and_topics.py   5/5  -- real computed stats + topic separation
tests/test_whatsapp_client.py    3/3  -- mocked HTTP layer, documented contract only
tests/test_reranker.py           3/3  -- reranker mechanics (see honest finding above:
                                          not wired in as default, doesn't measurably help)
tests/test_filler_classifier.py  4/4  -- trained classifier mechanics (see honest
                                          finding above: fixed list wins at real scale)
tests/test_mcp_server.py         6/6  -- real MCP protocol, all 5 tools
```
**47/47 passing.**

## Known limitations (stated plainly)

- **WhatsApp sending is untested against the live API** -- the single
  biggest, most important honest caveat in this project. See Part 3.
- **Small, synthetic chat data** (66 messages). Real stats/retrieval
  numbers, but from a small sample -- a genuine multi-thousand-message
  export would behave somewhat differently at scale.
- **Small hand-labeled sentiment dataset** (28 train / 10 test examples)
  -- 70% accuracy reflects that data size honestly, not a ceiling on the
  approach itself.
- **No template message support** -- Meta requires pre-approved templates
  for messages sent outside a 24-hour customer service window; only
  free-form messages are implemented here.

## Layout

```
data/sample_chat.txt      hand-written, real-format synthetic chat export
eval/sentiment_data.py    hand-labeled TRAIN/TEST sentiment examples
wa_mcp/
  parser.py               real WhatsApp export format parser
  chunking.py             sliding-window conversational chunking
  retrieval.py            hybrid BM25 + TF-IDF/LSA + RRF
  sentiment.py            logistic regression, trained from scratch
  topics.py               TF-IDF + KMeans clustering
  stats.py                message counts, response times, busiest day
  whatsapp_client.py      Cloud API client (mocked-tested, see Part 3)
  pipeline.py             ties it all together
  server.py               the actual MCP server (FastMCP)
  interactive_demo.py     "watch it work" CLI
tests/                    one file per layer, incl. real MCP protocol tests
```