grounded-mcp

An MCP server for markdown knowledge vaults that ships with its own eval suite.

Plenty of servers expose your notes over the Model Context Protocol. Before you point an agent at a knowledge store, though, you need answers to three questions most of them skip:

1. Can I trust what it returns? Every content-bearing response carries a stable citation id (path#heading + line spans) — nothing is returned that can't be quoted with provenance.

2. Does it know when it doesn't know? Search abstains — an explicit, structured "no adequate answer" — instead of dressing weak matches up as results.

3. Who is allowed to see what? Declarative entitlements, enforced at index level: content a profile can't see is never indexed for it, so it can't leak through scores, snippets, or rankings. A redaction guard refuses to serve notes that look like they contain credentials, regardless of entitlements.

And none of that is a claim — it's a CI-gated scorecard run against the committed demo vault:

retrieval   (staff, n=22):  hit@1 1.00   recall@5 1.00   MRR 1.00
abstention  (n=10):         rate  1.00   (target 1.00)
leakage     : 0 (must be 0)
redaction   : 0 (must be 0)
EVAL GATE: PASS

Weaken the abstention gates and the build fails — try it:

GROUNDED_MIN_SCORE=0 GROUNDED_MIN_COVERAGE=0 python evals/run_evals.py   # EVAL GATE: FAIL, exit 1

Quickstart

pip install -e .
grounded-mcp          # serves the bundled ACME demo vault over stdio

Point it at your own vault (Obsidian works as-is — frontmatter, [[wikilinks]], tags):

GROUNDED_VAULT=~/notes GROUNDED_PROFILE=default grounded-mcp

Claude Code / Claude Desktop

{
  "mcpServers": {
    "grounded": {
      "command": "grounded-mcp",
      "env": { "GROUNDED_VAULT": "/path/to/your/vault" }
    }
  }
}

Related MCP server: Vault Knowledge MCP Server

Tools

Read-only by design — a server that can quote your vault but never rewrite it is a trust feature, not a missing feature.

Configuration

Entitlements (deny wins, then allow, then default-deny):

profiles:
  staff:
    allow: ["public/**", "internal/**"]
    deny:  ["restricted/**"]
  contractor:
    allow: ["public/**"]

Why abstention needs two gates (a measured finding)

BM25 score alone cannot separate true answers from confident-looking misses on a small vault: in our golden set, genuine answers score as low as 2.4 while wrong-but-plausible matches reach 4.3 — a single common word ("policy") landing in a weighted title field looks like a result. The discriminator is term coverage: what fraction of the query's content words the note actually contains. Both gates together take abstention from 0.30 → 1.00 with zero retrieval loss. The eval suite is what made that tuning honest — full details in evals/run_evals.py and the scorecard baseline.

HTTP mode (v0.2): real per-client entitlements

python -m grounded_mcp.authz my-raw-token        # -> sha256 for the tokens file
GROUNDED_TRANSPORT=http GROUNDED_TOKENS=tokens.yaml GROUNDED_VAULT=~/notes grounded-mcp

Each client's bearer token maps to an entitlements profile (see example-tokens.yaml); the server keeps one index per profile, so enforcement stays index-level per VERIFIED identity. Unknown token = 401, never a default. Empty token file = server refuses to start. Tested over the wire: two tokens against one live server get different vaults, and the leakage probe runs as a real client.

Honest boundaries

• stdio = one user. Over stdio, client and server run as the same user, so profiles demonstrate the deployment pattern rather than enforce against a hostile peer — for real multi-client enforcement, use HTTP mode (above). The enforcement machinery is identical either way — index-level, not response-filtering.

• A denied note and a nonexistent note return the same response. The server refuses to be an existence oracle for content outside your entitlements.

• The redaction patterns are high-precision, not exhaustive. They catch key-shaped strings (AWS/GitHub/Slack tokens, private-key blocks, api_key = "..." assignments), not every secret.

• BM25 is the deliberate v0.1 baseline — deterministic, dependency-free, measurable. Hybrid semantic retrieval lands in v0.3 with its eval delta published against this baseline.

The demo vault

A fictional company handbook ("ACME Ltd") with three zones — public/, internal/, restricted/ — plus one deliberately seeded fake-credentials note (the classic AWS documentation example key) that the redaction guard must refuse to serve. All content is synthetic; the vault exists so the eval suite has something real-shaped to prove things against.

Development

pip install -e ".[dev]"
pytest -q                      # unit tests
python evals/run_evals.py      # the eval gate (CI runs both)
python evals/run_evals.py --write-baseline   # accept current scores after a deliberate change

Roadmap

• v0.2 ✅ shipped — streamable HTTP transport with per-client entitlements.

• v0.3 — hybrid semantic retrieval, landing only with its eval delta vs the BM25 baseline published.

• Pluggable store backends (the vault interface is small); community adapters welcome.

MIT © Richard Atkins