# mcp-citation-checker

An [MCP](https://modelcontextprotocol.io) server that verifies whether a claim is actually supported by the source text at a given citation — independent of what the calling LLM asserts.

## Why this exists

LLM agents that cite sources can still get the citation wrong: paraphrasing loosely, citing the wrong page, or asserting a source says something it doesn't. `check_citation` closes that gap by **independently fetching the real text** at the claimed citation and asking a separate LLM call to judge, strictly from that text, whether the claim holds up. The agent's own paraphrase is never trusted as evidence — only the independently-fetched source is.

## Design: pluggable backends

Looking up "the real text at citation X" depends entirely on where your documents actually live — Weaviate, Postgres, a flat file, something else. Rather than hardcoding one storage system, this package defines a small interface any backend can implement:

```python
class SourceLookup(Protocol):
    def fetch(self, pdf_name: str, page_number: int) -> list[dict]:
        ...
```

`weaviate` is included out of the box as a working, tested implementation (bundled as an optional extra, so installing the base package doesn't force a Weaviate dependency on anyone who doesn't need it). Writing your own backend for another storage system just means implementing that one `fetch` method — see [Writing your own backend](#writing-your-own-backend) below.

## Install

```bash
pip install "mcp-citation-checker[weaviate]"
```

(Omit `[weaviate]` if you're supplying your own backend instead.)

## Configuration

Set via environment variables when the server starts:

| Variable | Default | Purpose |
|---|---|---|
| `CITATION_BACKEND` | `weaviate` | Which backend to use (currently only `weaviate` ships built-in). |
| `ANTHROPIC_API_KEY` | — | Required. Used for the verification LLM call. |
| `WEAVIATE_MODE` | `local` | `local` (self-hosted) or `cloud` (Weaviate Cloud). |
| `WEAVIATE_HOST` / `WEAVIATE_PORT` | `localhost` / `8081` | Used when `WEAVIATE_MODE=local`. |
| `WEAVIATE_CLOUD_URL` / `WEAVIATE_API_KEY` | — | Required when `WEAVIATE_MODE=cloud`. |

## Usage

Run directly:

```bash
mcp-citation-checker
```

Or wire it into an MCP client config (e.g. Claude Desktop, or your own agent) like any other stdio MCP server:

```json
{
  "mcpServers": {
    "citation-checker": {
      "command": "mcp-citation-checker",
      "env": {
        "ANTHROPIC_API_KEY": "...",
        "WEAVIATE_MODE": "cloud",
        "WEAVIATE_CLOUD_URL": "...",
        "WEAVIATE_API_KEY": "..."
      }
    }
  }
}
```

The server exposes one tool, `check_citation(claim, pdf_name, pdf_page)`, returning `"<label>: <one-sentence reasoning>"` where `<label>` is one of `supported`, `contradicted`, or `not_mentioned`.

## Writing your own backend

The verification logic (`mcp_citation_checker.checker.check_citation`) has no storage-specific coupling at all — it only needs `claim` and a list of chunk dicts (`pdf_name`/`page_number`/`text`). Any backend just needs to implement the `SourceLookup` shape:

```python
class MyBackend:
    def fetch(self, pdf_name: str, page_number: int) -> list[dict]:
        # return [{"pdf_name": ..., "page_number": ..., "text": ...}, ...]
        ...
```

No inheritance required — `SourceLookup` is a `typing.Protocol`, so any object with a matching `fetch` method satisfies it.

Two ways to actually use a custom backend:
- **Bypass the bundled server entirely**: call `checker.check_citation(claim, my_backend.fetch(pdf_name, page))` directly from your own code/MCP server — this works today, no changes to this package needed.
- **Wire it into this package's own server**: the current version only registers a `weaviate` backend in `_get_backend()` (`src/mcp_citation_checker/__init__.py`) — using a different backend through the bundled `mcp-citation-checker` command currently means adding a branch there yourself (a small, welcome PR).

## Development

```bash
pip install -e ".[weaviate,dev]"
pytest
```

## License

MIT — see [LICENSE](LICENSE).