footnote-mcp

An MCP server for source-grounded web research. It searches the web, fetches and extracts pages, pulls structured data out of tables/files/APIs, and — the part that sets it apart — verifies that a claim is actually supported by its source instead of trusting a snippet. 42 tools over stdio MCP, driven by any MCP client (Claude Desktop, Cursor) or by the companion Scholiast research agent.

The design priority is trustworthiness over convenience: search snippets are treated as discovery only, every fetched page is cached with provenance, and claims are checked against the source text before they count. It also degrades gracefully — with no API keys and no config it still works (scraped search + an automatic headless-browser fallback + an offline verification heuristic); keys and env vars only make it better.

Quick start

From source (Python ≥ 3.10):

python3 -m venv .venv && source .venv/bin/activate
pip install -e .                        # installs the `footnote-mcp` console script + deps
python -m playwright install chromium   # the headless browser used by the fetch fallback
footnote-mcp                            # start the server (speaks MCP over stdio)

footnote-mcp now waits for an MCP client on stdio. Point a client at it by dropping this into its MCP settings (Claude Desktop: claude_desktop_config.json; Cursor: ~/.cursor/mcp.json):

{
  "mcpServers": {
    "footnote": { "command": "footnote-mcp" }
  }
}

No API keys are required to start — search falls back to scraping Bing + DuckDuckGo. Add keys later under "env" (see Search backends). Pass --headed to watch the browser tier work.

To run without installing, straight from the source tree:

PYTHONPATH=src python -m footnote_mcp

Related MCP server: rust-research-mcp

Verifying claims — the differentiator

The reason to use this over a plain search tool is evidence_entailment and friends: they tell a claim a source supports from one it does not. benchmarks/run_benchmark.py measures that on a labeled set of claim/source pairs (and demos corroborate_claim and locate_claim_span):

python benchmarks/run_benchmark.py                    # offline heuristic (deterministic)
python benchmarks/run_benchmark.py --backend ollama   # LLM judge (needs ollama)

Offline-heuristic result on the labeled set (benchmarks/REPORT.md):

On its design domain — numeric and factual data claims — the offline heuristic never blesses an unsupported claim and never misses one. Its blind spot is purely-semantic negation/paraphrase; for those, evidence_entailment with backend="ollama" (a local LLM judge) closes the gap. Run the --backend ollama line above to score that path on your own machine.

Tool surface (42 tools)

When generic parsers fail, synthesize a sandboxed parser:

A controlled Chromium session for JS-heavy or interactive pages:

Search backends

web_search (and everything built on it) routes through a provider layer. With an API key it uses that provider; otherwise it scrapes Bing + DuckDuckGo. Results are normalized to one shape regardless of backend.

auto (default) tries each keyed provider in order Tavily → Brave → Google, then scrapes. Force one with the provider argument (tavily/brave/google/scrape).

Semantic reranking. Pass semantic: true to web_search to reorder by meaning rather than keyword overlap: it over-fetches, embeds query and results with a local ollama model, and sorts by cosine similarity (each result gains semantic_score). Best-effort — if ollama is unavailable the original order is returned. Model: FOOTNOTE_EMBED_MODEL (default bge-m3).

Fetching & anti-bot ladder

web_read fetches through an escalation ladder (scraper.py): the cheapest method runs first and escalates only when a result looks blocked or empty. A block/quality detector decides when to escalate; a per-domain rate limiter, circuit breaker, and negative cache keep it polite. The tier used and the full attempt trace come back in fetch_tier / scrape_tiers.

With nothing configured it is the plain HTTP path plus an automatic browser fallback for JavaScript-rendered pages.

Runtime data

~/.footnote-mcp/source_cache/        # persistent page cache (with provenance)
~/.footnote-mcp/research_memory.json # persistent research memory

Override the cache location with FOOTNOTE_SOURCE_CACHE=/path/to/cache footnote-mcp.

check_date_completeness supports the calendars calendar, business_day, crypto_24_7, forex_weekday, us_business_day, and ru_business_day (pass explicit holidays for source-specific ones; the us_/ru_ variants use the optional holidays package).

Other install paths

Docker bundles Chromium and tesseract — nothing else to install:

docker build -t footnote-mcp .
docker run -i --rm footnote-mcp        # the client launches this; see MCP config below

{ "mcpServers": { "footnote": { "command": "docker", "args": ["run", "-i", "--rm", "footnote-mcp"] } } }

pipx / uvx (isolated install of the entry point):

pipx install /path/to/footnote-mcp          # or: pipx install git+<repo-url>
uvx --from /path/to/footnote-mcp footnote-mcp   # ad-hoc, no install

OCR. PDF/image OCR uses pytesseract + the system tesseract binary (brew install tesseract on macOS). Local NLI backend for evidence_entailment backend="local_nli": pip install -r requirements-nli.txt (model via FOOTNOTE_NLI_MODEL). Either way, startup_health_check reports what is actually available. Runtime dependency ranges are declared in pyproject.toml and mirrored in requirements.txt.

Tests

pip install -r requirements-dev.txt
python -m pytest -q          # offline unit + smoke tests; no network or keys needed

tests/test_mcp_smoke.py launches the server over real MCP stdio and exercises the tools end to end against a local HTTP fixture; the rest are offline unit tests of the parsers, fetch ladder, search providers, and dispatch. The live search test is opt-in:

RUN_LIVE_WEB_TESTS=1 python -m pytest -m live

CI runs the same suite (.github/workflows/tests.yml).

License

MIT — see LICENSE.