# farr-cite

farr-cite keeps a searchable library of research papers on your own computer. It finds the
passages in those papers that support a claim, and it writes a citation only for a paper that
is actually in the library. Ask it to cite anything else and it refuses, so an invented or
misremembered reference cannot slip into a manuscript.

It is a command-line tool and an MCP server (so an AI assistant can use the same library, under
the same rules). It runs entirely on your machine; the only network calls are to the open
scholarly services named below.

## Quickstart (5 minutes)

You need Python 3.10 or newer. No PDF of your own is needed: the example paper is fetched under
its open licence.

```bash
pip install farr-cite
farr-cite init --email you@example.org   # use your real address; the open services ask for one
farr-cite fetch 10.1371/journal.pcbi.1003285
farr-cite search "random seed"
farr-cite evidence "record the random seed of every analysis"
farr-cite cite 10.1371/journal.pcbi.1003285
farr-cite cite 10.1000/not-held
```

Until the first release is on PyPI, install from a clone instead: `pip install .` in this folder.

What you should see:

- `fetch` reports `downloaded under cc-by from europepmc`. The example is Sandve et al. (2013),
  *Ten Simple Rules for Reproducible Computational Research*, PLOS Computational Biology,
  published under CC BY 4.0.
- `search` and `evidence` list passages, each with its citation key, section and DOI.
- The first `cite` prints the reference and an inline form, `(Sandve et al., 2013)`.
- The second `cite` prints `REFUSED: 10.1000/not-held is not in your library ...` and exits 1.

To add papers you already have:

```bash
farr-cite add paper.pdf --doi 10.xxxx/yyyy    # metadata comes from Crossref
farr-cite add papers/                          # every PDF; each DOI is read from its first pages
farr-cite add paper.pdf --doi 10.xxxx/yyyy --no-metadata --title "..." --authors "Family, Given; Family, Given" --year 2020
```

The library lives in `~/.farr-cite` (or wherever `FARR_CITE_HOME` or `--home` points).

## Commands

| Command | What it does |
|---|---|
| `init --email ADDR` | create the library; store the contact email the open services ask for |
| `add FILE_OR_FOLDER [--doi DOI]` | add your own PDFs (or JATS XML) |
| `fetch DOI...` | download an openly licensed paper; refuse anything else |
| `search "terms"` | full-text and author search; every passage names its paper |
| `evidence "claim"` | the passages that best match a claim, at most two per paper |
| `sections DOI [NAME]` | a paper's sections, or the text of one |
| `cite DOI... [--style bibtex]` | a citation for a held DOI; refusal for any other |
| `check draft.md [--year Y] [--bib refs.bib]` | check every citation in a draft |
| `bench questions.json [--k 1,5,10]` | retrieval benchmark: recall@k and MRR |
| `list`, `stats`, `remove DOI` | what is held; remove a paper |
| `mcp` | run the MCP server on stdio |

Every command takes `--json`. Exit codes: 0 success, 1 a verdict (a refusal, a problem in a
draft, nothing fetched), 2 a usage or configuration error.

## How it works

Text is extracted with pypdf (or read from JATS XML, the format open-access archives publish
in), split into sections where a heading can be recognised ("Introduction", "2. Methods",
"Results and Discussion" ...) and into passages of about 120 words. Reference lists are not
indexed: a bibliography is not evidence. Passages are indexed with SQLite FTS5 and ranked by
BM25, with the title and author names searchable alongside the text. Metadata (title, authors,
year, journal) comes from Crossref by DOI. Section detection on PDFs is a heuristic; a PDF with
unusual headings ends up with fewer, larger sections, and its text is still searchable.

## The refuse-to-cite rule

`cite` returns a citation only for a DOI the library holds. For any other DOI it refuses, says
so plainly, and exits 1:

```
REFUSED: 10.1000/not-held is not in your library, so farr-cite will not write a citation for it.
Do not cite it from memory. Add it first (...), or mark the claim [CITATION NEEDED].
```

This refusal is the point of the tool. A citation written from memory, by a person or by a
language model, looks exactly like a real one; the only reliable test is whether the paper is in
hand. Because a held paper's text is in the library, the claim it is cited for can also be
checked with `evidence` or `sections` before it goes into a manuscript.

`check` applies the same rule to a whole draft. It finds every DOI and every pandoc-style
`[@key]` citation (keys are the ones `cite` prints, such as `sandve2013`, or come from a BibTeX
file passed with `--bib`) and reports, with line numbers:

- **held**: in the library;
- **not held**: a DOI the library does not have;
- **unknown key**: a `[@key]` that matches nothing;
- **published after the citing year**: a held paper whose year is later than the draft's own
  year, given with `--year` or as `year:` in the draft's YAML front matter. A paper cannot cite
  work that did not exist yet; this usually means a wrong DOI or a mixed-up version.

It exits 0 only when every citation is held and none is from the future.

## The open-access-only rule

`fetch` downloads a paper only when an **open licence** is recorded for the exact copy it
downloads. It asks four official open services, in this order:

1. **arXiv**, for `10.48550/arXiv.*` DOIs: the licence the authors chose on submission.
2. **Europe PMC / PubMed Central**: the licence of the open-access copy; the full text is
   downloaded as JATS XML and the licence stated inside the file is checked again.
3. **Unpaywall**: every open location carries its own licence and version; only a location with
   an open licence and a direct PDF link is used.
4. **Crossref**: a licence record and a full-text link are used together only when they name
   the same content version and any embargo has ended.

Open means a Creative Commons licence (CC BY, CC BY-SA, CC BY-NC, CC BY-ND and their
combinations) or CC0 / public domain. "Free to read", publisher-specific and "implied" licences
are not open, and neither is arXiv's default distribution licence.

If none of the four records an open licence, `fetch` refuses: *the paper is not openly
licensed*. There is no second route. farr-cite has no proxy or VPN support, reads no publisher
web pages, and has no login, cookie, institutional-access or EZproxy path; its HTTP client
ignores proxy settings in the environment. It runs from your own connection, identifies itself
with the contact email you configured, and keeps each service's pace (one request per second to
Crossref and Unpaywall, one every three seconds to arXiv).

Why so strict: a tool that quietly falls back to paywalled copies puts its users in breach of
publisher terms and of their institution's licence agreements, usually without their knowing.
For a paper that is not openly licensed, use your own copy and `farr-cite add` it.

## MCP server

```bash
pip install "farr-cite[mcp]"
```

Then register it with your MCP client, for example in a project's `.mcp.json`:

```json
{"mcpServers": {"farr-cite": {"command": "farr-cite", "args": ["mcp"]}}}
```

Tools: `search`, `evidence`, `sections`, `cite`, `check`, `fetch`, `stats`. They follow the same
rules as the command line: `cite` returns an error object for an unheld DOI, and `fetch` refuses
anything not openly licensed. Use `--home` before `mcp` to serve a library other than the
default.

## Benchmark

`bench` measures how well `evidence` finds the right passage. A question set is a JSON file:

```json
{"questions": [{"id": "seeds", "q": "why record the random seed ...",
                "doi": "10.1371/journal.pcbi.1003285", "section": "Rule 6"}]}
```

A returned passage counts as a hit when it comes from that DOI and its section name contains
`section` (omit `section` to accept any passage from the paper). For each question, the rank of
the first hit is recorded. **recall@k** is the share of questions with a hit in the top k;
**MRR** (mean reciprocal rank) is the average of 1/rank, counting 0 when there is no hit.

The repository ships an example set: 15 questions over three CC BY 4.0 papers. No paper is
committed; `examples/bench/papers.txt` lists the DOIs and `fetch.sh` downloads them through
`farr-cite fetch`, which re-checks each licence.

```bash
farr-cite init --email you@example.org
examples/bench/fetch.sh
farr-cite bench examples/bench/questions.json
```

Measured with version 0.1.0: recall@1 0.800, recall@5 0.933, recall@10 1.000, MRR 0.858. This is
a small smoke test of the ranking, not a claim about retrieval quality in general; build a set
from your own papers to measure what matters to you.

## Development

```bash
pip install -e ".[dev]"
python -m pytest -q
```

The tests run offline: they build a library from small PDFs generated inside the tests and mock
every HTTP call.

## Licence

Apache-2.0. See `LICENSE` and `NOTICE`. Papers you fetch or add keep their own licences.