mcp-judge
# mcp-judge
> Unpublished public-release candidate. See [PUBLIC_RELEASE.md](PUBLIC_RELEASE.md)
> for license information, privacy boundaries, verification and known limitations.
**English** | [한국어](README.ko.md) | [日本語](README.ja.md) | [中文](README.zh-CN.md)
A source-neutral **message-triage MCP**. It classifies incoming messages into a small, bounded set of **action categories** with a fast model, then routes each one: dismiss noise, notify what needs you, and (optionally) auto-register clear calendar events. You bring your own source, notifier, and calendar through small adapter interfaces.
It answers *"does this need me to act?"* — not the open-ended *"is this useful?"* — so it stays a filter, not a second inbox.
## Why
A plain keyword/regex filter can miss action items that have no date token ("can you take this?", "please reply"). Model-based classification is an alternative; accuracy and cost depend on the selected provider and have not been benchmarked here.
## Install
```bash
pip install -e . # core (MCP + Claude CLI backend, no API key)
pip install -e ".[anthropic]" # optional: Anthropic SDK backend
pip install -e ".[dev]" # tests + lint
```
The default backend drives a logged-in **Claude CLI** (`claude -p`) — no API key to manage. Set `backend: "anthropic"` in config to use the Anthropic SDK with `ANTHROPIC_API_KEY` instead.
## Quickstart (model service required; may incur charges)
```bash
mcp-judge --source examples/demo.jsonl
```
This reads a JSONL file source, classifies each line, prints notifications for action items, and dismisses the rest. State is stored in a local SQLite DB (`judge.sqlite3`).
This is not an offline demo: it invokes the configured model backend. A CLI login
does not make inference free. Messages and context are sent to that provider.
For offline checks with synthetic models, use the tests instead; see PUBLIC_RELEASE.md.
## Configuration
A JSON file (see `config.example.json`), passed with `--config`:
| key | meaning |
|---|---|
| `preset` | category vocabulary + routing (`schedule-en`, `schedule-ko`) |
| `model` | model id (default `claude-haiku-5-5`) |
| `backend` | `claude-cli` (default) or `anthropic` |
| `claude_exe` | explicit path to the `claude` binary (optional) |
| `db_path` | SQLite path |
| `sources` | allowlist of accepted source names (empty = any) |
| `channels` | free-form config consumed by your adapters |
### Presets
A preset is pure data: the category list, which categories are *action* (notify-worthy), which notify even when unsure (fail-safe), which may auto-register to a calendar, and the model instructions. `schedule-en` and `schedule-ko` ship built in; add your own in `mcp_judge/config.py`.
## MCP server
```bash
MCP_JUDGE_CONFIG=./config.json mcp-judge-server
```
Tools: `judge_ingest(source, item_id, text, context?)`, `judge_pending(limit?)`, `judge_get(item_id, source?)`, `judge_health()`. The server records verdicts and sends notifications inline when a notifier is configured.
### Identity and upgrades
Items are unique by `(source, item_id)`. `judge_get` without `source` remains
compatible for unique IDs; ambiguous IDs return `ambiguous_item_id` rather than
selecting an arbitrary record. Store `get`/`resolve` accept keyword `source` and
raise `AmbiguousItemError` for ambiguous unqualified IDs.
Opening an old database migrates its single-column primary key transactionally,
preserving message text, status, verdict, notification and creation time. Back up
your database before upgrading; old versions cannot safely use the migrated schema.
New calendar idempotency keys are JSON arrays `[source, item_id]`, not bare IDs.
Existing registered records remain terminal and are not replayed. Custom calendar
adapters must accept the new key format; historical external keys are not rewritten.
## Bring your own adapters
The engine depends only on three `Protocol`s in `mcp_judge/adapters.py`:
- **Source** — `items()` yields `{item_id, text, context?}`.
- **Notifier** — `send(text) -> delivery_id`.
- **Calendar** — `create_event(title, date, time, idempotency_key) -> id` (idempotent).
`FileSource` and `StdoutNotifier` are included as runnable demos, plus a built-in **`TelegramNotifier`** (set `channels.notifier: "telegram"` with your own bot `token`/`token_env` and `chat_id` — no extra infra). Implement the Protocols to wire any other source, notifier, or calendar.
## License
[MIT](LICENSE), Copyright (c) 2026 Bum-Boo. The owner confirmed rights to the
code and samples and approved this notice.
> Note: the ko/ja/zh translations are machine-made and not yet native-reviewed.
TDQS
Scored across 4 tools
Each tool targets a distinct operation: reading a single item, checking health counts, ingesting/classifying a message, and listing pending items. The boundaries are clear, with only minor conceptual overlap between judge_get and judge_pending that the descriptions resolve.
All tools use a consistent judge_ prefix and snake_case, making the set predictable. The only minor deviation is judge_pending, which is adjectival rather than a verb like get/health/ingest, but it remains readable and consistent enough.
Four tools are well-scoped for a focused triage/judging server: ingest, inspect, list pending, and health. Each tool earns its place, and the set avoids unnecessary surface area.
The surface covers ingestion, item lookup, pending listing, and health status, but there is no explicit tool to resolve, retry, or manually decide pending items after a model error. This is a notable gap for a judging/triage workflow, even though core read and ingest operations exist.