Citable

Cited answers on Indian criminal law for AI assistants. Citable is an MCP server that lets Claude (or Cursor, or any MCP client) search the official text of India's three new criminal laws and answer with the exact Act, section and Gazette page for every claim, instead of answering from memory.

It is built as a measured system: a hand-labelled eval set gates search quality in CI, and every tool call is logged for latency and cost.

Why

On 1 July 2024 India replaced its three core criminal laws, and every section number changed:

Cheating was IPC s. 420; it is now BNS s. 318. Language models learned mostly from the old codes, so they quote repealed sections or invent provisions — and in law a wrong section number is worse than no answer. Citable gives the assistant the current text to quote, with a citation the reader can check.

Related MCP server: Law7 MCP

What it looks like

Ask Claude "Someone on a motorbike snatched my gold chain. What offence is that?" and it calls Citable's search_law tool. The same search from the command line:

$ uv run citable search "someone on a motorbike snatched my gold chain" -k 3 --rerank
1. BNS s. 304, p. 80 — Snatching.  [keyword #1, vector #3]
   304. (1) Theft is snatching if, in order to commit theft, the offender suddenly or
   quickly or forcibly seizes or secures or grabs or takes away from any person or from his
   possession any movable property. (2) Whoever ...
2. BNS s. 112, p. 36 — Petty organised crime.  [keyword #2, vector #11]
3. BNS s. 309, pp. 81-82 — Robbery.  [vector #2]

(keyword 2 ms, vector 9 ms, rerank 247 ms, total 258 ms)

Claude then reads the full section with get_section and answers citing "BNS s. 304", with a link straight to page 80 of the Gazette PDF.

At a glance

How it works

flowchart TD
    subgraph build ["Build once"]
        G["Gazette of India PDFs"] -->|"parse the page layout"| S["1,059 sections"]
        S -->|"chunk + embed"| DB[("SQLite<br/>FTS5 + sqlite-vec")]
    end
    subgraph ask ["Every question"]
        A["Claude Desktop / Cursor"] <-->|"MCP"| M["search_law · get_section · list_contents"]
        M --> H["citation lookup + BM25 + vector"]
        H --> F["fusion (RRF) + rerank"]
    end
    DB --> H
    M --> T[("telemetry log")] --> D["cost & latency dashboard"]
    E["85 labelled questions"] --> CI["CI quality gate"]

1. Ingestion (parse.py). The Gazette prints each section's heading as a note in the page margin, level with the section's first line. The parser reads the PDF line by line with coordinates, pairs margin notes with sections, and tracks parts, chapters and sub-headings. Section numbers must follow on, which catches stray lines that look like sections.

2. Search (search.py), in one SQLite file:

   • Citation lookup: "BNS 318" or "section 482 of BNSS" goes straight to the section.

   • Keyword: FTS5 BM25, with section headings weighted 4x over body text.

   • Vector: BAAI/bge-small-en-v1.5 embeddings in sqlite-vec, run locally with fastembed.

   • Fusion: Reciprocal Rank Fusion of the two rankings, per section.

   • Rerank: an ms-marco-MiniLM-L-6-v2 cross-encoder over the top 20.

3. MCP server (server.py): three read-only tools, plus instructions telling the assistant to answer from the returned text and cite every provision.

4. Evaluation (evaluate.py): recall@k and MRR for every search configuration, checked in CI.

5. Telemetry and dashboard (telemetry.py, dashboard.py): every tool call logged; one HTML page of latency and cost.

Quick start

Requires uv. The parsed sections are in the repo, so you don't need to download the PDFs.

git clone https://github.com/Pranava77/citable.git
cd citable
uv sync
uv run citable index                       # embed and build data/citable.db (~1 min; downloads ~150 MB of models once)
uv run citable search "can the police arrest me without a warrant" --rerank
uv run citable show bns 318                # print one section

Connect it to Claude Desktop

Add this to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) and restart Claude Desktop. Use the full path to uv (which uv), because desktop apps don't see your shell's PATH:

{
  "mcpServers": {
    "citable": {
      "command": "/Users/you/.local/bin/uv",
      "args": ["--directory", "/path/to/citable", "run", "citable", "serve"]
    }
  }
}

The tools:

The server reranks by default; append "--no-rerank" to args to trade about 8 points of recall for about 240 ms per search.

Evaluation

evals/questions.jsonl holds 85 questions, each labelled with the sections that answer it. Every label was checked against the parsed text.

uv run citable eval (recall@k = share of questions with a correct section in the top k):

Recall@5 by category:

What the numbers say:

• Citations and the Act's own wording are solved. Every miss is a vocabulary gap.

• The reranker earns its 240 ms: +9 points recall@5, mostly on plain-English and jargon questions. Inside an assistant's tool call that latency is small, so it is on by default.

• Hybrid fusion is no better than vector search alone here: equal at R@5, worse at R@1. The difference is one or two questions, too few to call.

• Practitioners' terms fail most. "FIR", "anticipatory bail", "chargesheet" and "sedition" appear nowhere in the text. The tool description tells the assistant to rephrase in statutory language; how much that recovers isn't measured yet.

The CI gate (ci.yml) builds the index from the committed sections, runs the tests, then citable eval --check, which fails the build if any configuration's recall@5 or MRR drops more than 0.03 below evals/baseline.json. After an intended improvement, refresh it with uv run citable eval --save-baseline.

The questions were written alongside the search code, before its results were known, and nothing has been tuned on them. A held-out set written by someone else would be a stronger test.

Cost & latency dashboard

Every MCP tool call is logged to data/telemetry.db: the arguments, the time spent in each search stage, errors, and the size of the text sent back into the assistant's context. uv run citable dashboard --open turns the log into one self-contained HTML page (no network, no JavaScript libraries), with source and time-range filters, a table view for every chart, and light and dark modes.

uv run citable replay sends the 85 eval questions through the tools the way an assistant would (search, then read the top section). From that run, 181 calls:

Cost is what Citable's results add to the assistant's bill: tool results are input tokens for whichever model the assistant runs, while Citable's own search runs locally and is free. Tokens are estimated at about 4 characters each (exact counts need Anthropic's token-counting API); the logged size is exactly the text the MCP SDK hands the model, which a test checks.

Two conclusions. Reranking is nearly all of the latency, so it is the one stage worth optimising (a smaller cross-encoder, or fewer candidates), and the eval says what each option costs in recall. And context, not compute, is the real cost: a search returns about 1,300 tokens, almost all of it statute text (JSON indentation is about 6%), so limit and snippet length are the levers.

The log contains the questions people asked, so it stays on your machine (data/telemetry.db is gitignored). Set CITABLE_TELEMETRY=off to stop logging.

Limitations

• Not legal advice. Citable finds and quotes the statute; interpreting it is up to the assistant and, ultimately, a lawyer.

• Three Acts, as enacted. No case law, rules, state laws, or amendments after the 2023 Gazette text.

• Old section numbers mislead. "Section 438 CrPC" returns BNSS ss. 438–439, unrelated provisions that share the number. Fixing this needs a correspondence table from the old codes to the new ones.

• Jargon the statute never uses ("FIR", "anticipatory bail") is the weakest category, at 62% recall@5.

Project layout

src/citable/
  sources.py     the three Acts and their Gazette URLs
  download.py    fetch the PDFs (egazette.gov.in serves an incomplete TLS chain; uses the OS trust store)
  parse.py       PDF -> sections, using the Gazette's page layout
  chunk.py       sections -> ~1,200-character passages
  embed.py       local embedding and reranking models (fastembed, ONNX)
  index.py       build the SQLite index (FTS5 + sqlite-vec)
  search.py      citation lookup, BM25, vector, RRF, rerank
  server.py      the MCP server and its three tools
  evaluate.py    recall@k, MRR, the CI gate
  telemetry.py   per-call logging
  dashboard.py   the HTML dashboard
  cli.py         `citable ...` commands
evals/           labelled questions and the CI baseline
data/processed/  parsed sections (committed); everything else in data/ is generated
tests/           63 tests: parser, search, server, eval, telemetry

Development

uv run pytest                    # all tests (a few need the built index or the PDFs, and skip without them)
uv run ruff check src tests
uv run citable eval --misses     # scores, plus every question answered badly
uv run citable download          # re-fetch the Gazette PDFs into data/raw/
uv run citable parse             # re-parse them into data/processed/

Built with Python 3.12, PyMuPDF, SQLite FTS5, sqlite-vec, fastembed, Pydantic, Typer and the MCP Python SDK.

License

The code is released under the MIT License. The statute text in data/processed/ comes from the Gazette of India (egazette.gov.in), the official publication of the Acts, and is not covered by this license.