# MAX MCP

Pull credit card expenses from MAX (Israel), categorize them, and hand the result
to a Claude agent that analyzes spending behaviour against current Israeli market
conditions.

MAX has no public API, so transactions are collected by driving a headless
browser session against max.co.il using
[israeli-bank-scrapers](https://github.com/eshaham/israeli-bank-scrapers).

## Pieces

**Start here:** [run.py](run.py) is the interface. Edit the configuration block at
the top of that file, then `python3 run.py`. Everything else is machinery.

| File | Role |
|---|---|
| [run.py](run.py) | **The file you edit.** Cards, date range, and what to run |
| [display.py](display.py) | Terminal rendering: Hebrew/RTL column alignment |
| [export.py](export.py) | CSV and HTML output |
| [MAX_mcp.py](MAX_mcp.py) | MCP server (stdio) for **MAX** — scrapes, categorizes, caches, and exposes the data as tools |
| [CAL_mcp.py](CAL_mcp.py) | The same server for **CAL / Visa Cal** (cal-online.co.il), with its own cache |
| [providers.py](providers.py) | One façade over the issuer servers, so the flow can run on MAX, CAL, or both |
| [framework.py](framework.py) | The categorization taxonomy and the keyword classifier |
| [scraper/max_scraper.js](scraper/max_scraper.js) | Node wrapper around israeli-bank-scrapers (MAX) |
| [scraper/cal_scraper.js](scraper/cal_scraper.js) | The same wrapper for CAL (`visaCal` provider) |
| [analyst_agent.py](analyst_agent.py) | Claude agent that reads the profile, researches the market, and writes the report |

Data flow: `max.co.il → scraper → framework.categorize_all → data/expenses.json → analyst_agent.py → data/analysis.md`

## Setup

```bash
pip install -r requirements.txt
(cd scraper && npm install)      # downloads Chromium, ~300MB
cp .env.example .env             # then fill in MAX_USERNAME, MAX_PASSWORD, ANTHROPIC_API_KEY
                                 # add CAL_USERNAME / CAL_PASSWORD to use CAL too
```

## Running it

```bash
python3 run.py
```

Open [run.py](run.py), change the settings in the CONFIGURATION block, save, run
again. Every setting has EXAMPLE lines above it showing the forms it accepts;
the examples are commented out, so only the uncommented line takes effect.

The settings, in short:

| Setting | What it controls |
|---|---|
| `PROVIDER` | `"max"`, `"cal"`, or `"both"` — which card company to pull from |
| `CARDS` | `[]` for all cards together, or e.g. `["8810"]` / `["personal"]` |
| `SEPARATE_REPORT_PER_CARD` | Also print one report per card |
| `START_DATE` / `END_DATE` | Exact `"YYYY-MM-DD"` window; `None` leaves that end open |
| `MONTHS_BACK` | How far back to scrape when `START_DATE` is `None` |
| `FETCH` | `True` downloads fresh data; `False` reuses the last download |
| `SHOW_BROWSER` | Watch the login happen |
| `ANALYZE` | Run the Claude advisory report |
| `MARKET_RESEARCH` | Let the analysis check current Israeli prices online |
| `RTL_MODE` | How Hebrew is rendered in the terminal — see below |
| `SAVE_CSV` / `SAVE_HTML` | Where to write the file copies of the report |

Typical loop: run once with `FETCH = True` to download, then set it to `False`
and re-run freely while you change dates and cards — no repeated logins.

Run it with `CARDS = []` first: the card table it prints shows every name you can
put in `CARDS`.

## Dashboard

A browser front end over the same data and helpers `run.py` uses — nothing about
`run.py` or the MCP servers changes because of it.

```bash
streamlit run dashboard.py
```

Flow:

1. Open the page.
2. Configure the scrape in the panel at the top (provider, dates, cards,
   browser visibility, whether to run agent recommendations) — no file editing.
3. Click **Scrape now** (or **Use existing data** to skip straight to what's
   already cached) and get interactive tables and charts across five tabs
   (Overview, Categories, Cross-analysis, Transactions, Categorize), all
   filterable from the sidebar.
4. Chat with the analyst agent about what's on screen, in the collapsible
   panel on the right — it can search the web to back a recommendation.

The chat panel needs `ANTHROPIC_API_KEY` in `.env`, same as `analyst_agent.py`;
without it the panel disables itself but the rest of the dashboard works
normally. The **Categorize** tab is the browser equivalent of
`python3 categorize.py` — same merchant grouping, same `data/category_overrides.json`,
so terminal and dashboard agree.

## Running the MCP server

Register it with your MCP client (Claude Desktop, Claude Code, etc.):

```json
{
  "mcpServers": {
    "max-expenses": {
      "command": "python",
      "args": ["/Users/erezcohen/Documents/MAX MCP/MAX_mcp.py"]
    },
    "cal-expenses": {
      "command": "python",
      "args": ["/Users/erezcohen/Documents/MAX MCP/CAL_mcp.py"]
    }
  }
}
```

Both servers expose the **same tool names** over the same record shape; they
differ only in which site they log into and which cache file they own
(`data/expenses.json` for MAX, `data/cal_expenses.json` for CAL). Register one or
both. The merchant rules file `data/category_overrides.json` is deliberately
shared between them — a merchant is the same merchant whichever card paid.

Tools it exposes:

- `fetch_expenses(months_back, show_browser, timeout_seconds)` — log in, scrape
  every card on the account, categorize, cache. Takes 30–120 seconds. Pass
  `show_browser=true` the first time to watch the login.
- `get_expenses(start_date, end_date, category, min_amount, card, refunds_only, limit)` — read the cache.
- `get_spending_summary(start_date, end_date, card)` — totals per category, per
  budget nature, per card, per month, plus top merchants. The agent's input.
- `list_cards()` — the cards found, with spend and date range per card.
- `list_categories()` — the framework itself.
- `recategorize(merchant_pattern, category)` — permanently map a merchant to a
  category, overriding the keyword classifier. For clearing out `uncategorized` in
  bulk, `python3 categorize.py` is the faster route.

## Two card companies

CAL (cal-online.co.il) is wired in exactly like MAX: its own scraper
([scraper/cal_scraper.js](scraper/cal_scraper.js), `visaCal` provider), its own
MCP server ([CAL_mcp.py](CAL_mcp.py)), its own cache, its own credentials.

```
CAL_USERNAME=...
CAL_PASSWORD=...
CAL_CARD_LABELS={"1234": "personal"}     # optional, same rules as MAX_CARD_LABELS
```

Set `PROVIDER` in [run.py](run.py) to pick which one the report covers:

```python
PROVIDER = "max"     # MAX only
PROVIDER = "cal"     # CAL only
PROVIDER = "both"    # download from both, one merged household report
```

With `"both"`, each company is scraped in turn and one failed login does not
discard the other's data — the run reports the failure and continues with what it
got. Every transaction carries a `provider` field, so the merged view can still
attribute a row.

The same switch exists on the categorizer:

```bash
python3 categorize.py --provider both
```

Direct CLI access to either server, without an MCP client:

```bash
python3 CAL_mcp.py fetch --months 3 --show-browser
python3 CAL_mcp.py cards
python3 CAL_mcp.py summary
```

### Why the CAL scraper does not sweep overlays

[scraper/max_scraper.js](scraper/max_scraper.js) installs a MutationObserver that
clicks anything resembling a close button and deletes full-viewport overlays,
because MAX puts a promo interstitial over its login. **CAL must not get that
treatment**: its login *is* a modal — the library clicks `#ccLoginDesktopBtn` and
the form arrives in a full-screen overlay with its own close control — so the
sweeper closes or deletes the login popup the instant it appears. The symptom is
a scrape that hangs immediately after sign-in, which looks like a captcha but is
not. CAL's own post-login tutorial is already dismissed by the library.

So [scraper/cal_scraper.js](scraper/cal_scraper.js) only accepts the cookie
banner by its specific IDs and never removes an element.

When a CAL run does stall, `CAL_DEBUG=1` prints the library's step-by-step trace
to stderr, which names the step it is waiting on:

```bash
echo '{"startDate":"2026-07-01","showBrowser":false}' \
  | CAL_DEBUG=1 node scraper/cal_scraper.js
```

## Multiple cards

One MAX login covers every card on the account. The scraper returns each card
separately, so every transaction carries `card` (the number MAX reports) and
`card_label`. Nothing needs configuring for this to work — it is the default.

To get readable names instead of card numbers, set `MAX_CARD_LABELS` in `.env` to
one line of JSON. Keys match against the **end** of the card number, so the last
four digits are enough and the full number never has to be stored:

```
MAX_CARD_LABELS={"1234": "personal", "5678": "business", "9012": "spouse"}
```

Anywhere a `card` argument appears, either the label or the trailing digits work.
`get_spending_summary` without a `card` returns the household total plus a
`by_card` breakdown (each card's own category and nature mix) and
`by_month_by_card`, so a rising total can be traced to the card causing it.

The agent gets one extra thing from having several cards: a
`merchants_on_multiple_cards` list. The same subscription billed to two cards is
invisible in a household total and is the most common duplicate charge.

## Running the analyst

```bash
python analyst_agent.py                 # last 3 months, all cards, with market research
python analyst_agent.py --months 6
python analyst_agent.py --card personal # scope to one card
python analyst_agent.py --no-research   # numbers only, no web search
```

Writes `data/analysis.md`.

## Hebrew in the terminal

Merchant names are mostly Hebrew, which reads right-to-left. A report line mixes
a right-to-left run, left-to-right numbers, and the padding spaces between them —
and the Unicode Bidirectional Algorithm resolves those spaces from whatever
surrounds them, which is what drags the amount column out of line on Hebrew rows.

[display.py](display.py) fixes this by wrapping each field in zero-width isolate
characters, so a cell cannot influence its neighbours, and by computing the
padding from the *display* width rather than `len()` — a Python format spec like
`{:<44}` counts the invisible isolates as characters and would leave every
Hebrew cell two columns short.

**Terminals disagree about right-to-left text and there is no way to ask yours
which behaviour it has**, so `RTL_MODE` in [run.py](run.py) picks:

| Mode | Use when |
|---|---|
| `"isolate"` | Columns stay aligned. Correct on terminals that implement bidi, harmless on those that don't. |
| `"visual"` | **The setting this machine needs.** Hebrew reads backwards otherwise, because the terminal has no bidi support and draws it left-to-right. Reverses the text before printing. Needs `python-bidi`. |
| `"off"` | No handling at all. |

In `"visual"` mode the text really is reversed on screen, so anything copied out
of the terminal comes with it — copy from the CSV or HTML instead.

The two are mutually exclusive: applying both reverses the text twice.

**The file exports are the reliable way to read Hebrew**, since Excel and every
browser implement bidi properly:

- `data/report.html` — open in any browser. Each cell carries `dir="auto"`, so
  Hebrew cells lay out right-to-left while the number columns stay pinned
  left-to-right. This one is always correct.
- `data/report.csv` — written with a UTF-8 BOM (`utf-8-sig`), which is what makes
  Excel recognise the encoding; without it Hebrew arrives as mojibake. Amounts
  are bare numbers so they stay numeric, with both English and Hebrew category
  columns.

Files are written raw, in logical order — the isolate characters are strictly a
terminal concern and would be corruption in a file that does its own bidi.

## Signs: charges vs refunds

MAX reports a purchase as a **negative** amount and a refund or credit as a
**positive** one, and that sign is preserved end to end — `amount` in the cache
is exactly what MAX said, plus an `is_refund` convenience flag.

Every total is **net**: spending minus refunds, so a refund reduces the category
and card it landed in instead of inflating them. `gross_spend` and `refunds` are
reported alongside the net figure at the top level, per card, and per category,
because a category that looks cheap only because one large refund landed in it is
a different story from one that was genuinely low-spend.

`framework.cost_of(txn)` is the single place that flips the sign into
"money that left the account" terms. Use it rather than touching `amount`
directly — and never `abs()` an amount, which is what turns a ₪4,000 refund into
₪4,000 of spending.

`get_expenses(refunds_only=True)` lists just the money that came back.
`min_amount` filters on magnitude, so it surfaces large refunds as well as large
charges.

## The categorization framework

Each category in [framework.py](framework.py) carries a **nature** —
`fixed`, `variable`, `discretionary`, or `investment` — because the analysis
reasons very differently about rent than about restaurants. Classification is
deterministic keyword matching against the merchant name in Hebrew or English,
longest keyword first. Anything unmatched lands in `uncategorized`, and the
summary reports what share of spend that is, since a high uncategorized share
undermines every conclusion drawn from the totals.

### Fixing categories by hand

```bash
python3 categorize.py                    # only the uncategorized merchants
python3 categorize.py --all              # review every merchant
python3 categorize.py --category dining  # audit one bucket that looks wrong
```

The tool groups transactions by merchant and walks them **largest net spend
first**, so the merchants that actually distort the report come up first. For
each one it prints the numbered category list; you pick a number, `n` to invent
a new category, Enter to skip, or `q` to stop. One decision covers every charge
from that merchant, past and future.

Each decision is written immediately to `data/category_overrides.json` as a
`{"merchant pattern": "category_key"}` rule — the substring is matched against
the normalized merchant name — and overrides are re-applied every time the cache
is read, so a fix takes effect on the next report **without re-scraping**. If you
pick the category the keyword classifier would have chosen anyway, no rule is
written (and a now-redundant one is removed).

Categories created with `n` land in `data/custom_categories.json` and are merged
into `CATEGORIES` at import, so they show up in the report, both CSVs, the HTML,
and `list_categories` like any built-in. You can also add one in code by
appending a `Category(...)` to `_BUILTIN_CATEGORIES` in
[framework.py](framework.py); to improve an existing one, add keywords to its
tuple so future merchants classify automatically instead of needing a rule.

## Notes

- Credentials live only in `.env` and are passed to the Node subprocess through
  its environment, never on the command line.
- `data/` and `.env` are gitignored. The cache contains your full transaction
  history — treat it as you would a bank statement.
- MAX occasionally changes its login flow, which breaks the scraper. When that
  happens, update `israeli-bank-scrapers` (`cd scraper && npm update`) before
  debugging anything else.