# PriorArt Triage Agent

Give it a plain-English invention description. It searches granted US patent claims
and returns a prior-art memo with per-finding citations, then verifies every assertion
against the raw source material and quarantines anything it cannot ground.

```bash
python -m agent "a solar-powered soil moisture sensor that predicts irrigation need six hours ahead"
```

Runs entirely locally. No API key, no per-token cost, no identity verification.

## Why the verification pass exists

A model that searches, drafts, and then grades its own work in one context agrees with
itself. The first three iterations of this project proved it: every `supporting_text`
came back as the model's own paraphrase, and the model-based verifier passed all of
them because it was assessing relevance rather than provenance. The safety stage was
theater.

The fix was to take the model out of the quoting path entirely.

**The model returns a patent number and an explanation of relevance. Nothing else.**
The citation text is then read out of the corpus by that ID. Two properties fall out
for free:

- A hallucinated patent number is not in the database, so the finding is dropped
  before verification runs.
- Every surviving quote is byte-exact claim text, because no model ever typed it.

That leaves exactly one question worth a model's judgement: does this claim text
actually support the assertion made about it? Findings that fail land in a **Flagged**
section, never in the memo body.

The general lesson, and the reason this is the interesting part of the project: don't
ask a model to do what a database already does correctly. Retrieval and provenance are
code. Judgement is the model.

## Architecture

```
invention description
        │
   [1] plan_search       schema-constrained call → SearchPlan
        │                 examiner vocabulary, not founder pitch language
   [2] draft_memo        hand-rolled agent loop over MCP tools → DraftMemo
        │                 model returns patent IDs + relevance, never quotes
        │                 code attaches claim text from the corpus by ID
   [3] verify_memo       per-finding: does this claim text support the assertion?
        │
   memo — grounded findings, flagged findings, gaps
```

| Path | What it is |
|---|---|
| `uspto/ingest.py` | Builds the local corpus from USPTO bulk downloads |
| `uspto/corpus.py` | SQLite + FTS5 query layer over granted claims |
| `uspto/server.py` | MCP server exposing three tools over stdio |
| `agent/llm.py` | Ollama backend: chat, tool calling, schema-constrained output |
| `agent/pipeline.py` | The three stages and the agent loop |
| `agent/models.py` | Pydantic schemas |
| `agent/grounding.py` | Deterministic quote-provenance checking |
| `agent/__main__.py` | CLI and Markdown renderer |
| `scripts/smoke_test.py` | Gate: prove the corpus is queryable |
| `evals/run_eval.py` | Grounding rate, hit rate, term coverage |

## Setup

**1. Python deps**

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

**2. Local model**

```bash
brew install ollama
ollama serve &
ollama pull qwen2.5:7b
```

Override with `OLLAMA_MODEL` if you want a different one. It needs tool-calling
support; `qwen2.5` and `llama3.1` both have it.

**3. Corpus**

Sign in to [data.uspto.gov](https://data.uspto.gov) and download two files into `data/`:

| File | Dataset | Size |
|---|---|---|
| `g_claims_2025.tsv.zip` | `pvgpattxt` | 346 MB |
| `g_patent.tsv.zip` | `pvgpatdis` | 232 MB |

Then:

```bash
python -m uspto.ingest      # ~2 minutes, produces data/corpus.db
python scripts/smoke_test.py
```

## Run it

```bash
python -m agent "your invention description"
python -m agent -f invention.txt -o memo.md

python evals/run_eval.py --plan-only    # planning stage only, fast
python evals/run_eval.py                # full run, writes evals/results.json
```

## Why bulk data instead of the ODP API

USPTO splits access into two tiers:

| Access | Requires |
|---|---|
| Bulk data downloads | USPTO.gov account + MFA |
| ODP API key | USPTO.gov account **+ verified and linked ID.me** |

The API key path requires identity verification through ID.me, which collects an SSN
and biometric data. That is a real cost for a side project, and it buys the wrong
data: the ODP API serves patent **application** file-wrapper records (titles,
abstracts, CPC codes), not granted claim text.

Bulk downloads need only the account, and they include full granted claims. Claims are
what a prior-art search actually reads.

**Do not reach for PatentsView as an alternative.** It migrated into ODP on
2026-03-20, its `search.patentsview.org` API is being retired, previously issued keys
do not work against ODP, and there is no announced replacement date.

### API routes, if you do go that way

Base URL `https://api.uspto.gov/api/v1`. Routes confirmed by probing without
credentials: **401 means the route exists and needs a key.** There are two distinct
403s, which matters when debugging. Body `"Missing Authentication Token"` means the
route does not exist. Body `"Forbidden"` means the route is fine and your key was
rejected.

Real: `/patent/applications/search`, `/patent/applications/{id}`, `.../meta-data`,
`.../continuity`, `.../transactions`, `.../associated-documents`,
`/datasets/products/search`.

Not real, despite appearing in older docs: `/patent/applications/{id}/documents`,
`/patent/grants/search`, `/bulk-data/products`. The legacy Developer Hub at
`developer.uspto.gov` was decommissioned 2026-06-05, so most tutorials point at a
dead host.

## Results

Ten invention descriptions, `qwen2.5:7b` throughout, full pipeline per case
(`python evals/run_eval.py`).

```
cases run          10/10
findings drafted   42
findings grounded  19
grounding rate     45%
hit rate           90%
term coverage      88%
```

| Case | Drafted | Grounded | Rate |
|---|---:|---:|---:|
| soil-moisture-pinn | 4 | 4 | 100% |
| rag-retrieval | 5 | 4 | 80% |
| solar-tracker | 5 | 3 | 60% |
| pill-dispenser | 2 | 1 | 50% |
| cold-chain | 5 | 2 | 40% |
| leak-detector | 5 | 2 | 40% |
| handwriting-tablet | 4 | 1 | 25% |
| shift-scheduling | 5 | 1 | 20% |
| hearing-aid | 5 | 1 | 20% |
| bike-lock | 2 | 0 | 0% |

**The 45% is a rejection rate, not an error rate.** The model drafted 42 findings; 23
did not survive verification and none of those reached a memo. Without that stage all
42 would have shipped as prior art. A number closer to 100% would mean the verifier
was rubber-stamping, which is what the first three iterations of this project did.

**Hit rate 90%** — nine of ten cases returned at least one defensible finding.

**The spread tracks the corpus, not the code.** The strong cases (soil moisture at
100%, RAG at 80%) are ones where 2025 grants genuinely contain close prior art in
standard vocabulary. The weak ones are not: shift-scheduling is a web application
rather than a patented device, and bike-lock is a consumer product thinly represented
in a single grant year. The verifier declined to manufacture findings that were not
there, which is the intended behavior.

### What this measurement does not tell you

The rejection rate shows the verifier is filtering, not that it is filtering
*correctly*. A verifier that rejected everything would score 100% here and be
worthless. Establishing that the rejections were right needs one of two things, and
neither is done yet:

- **Known-item retrieval.** Take N patents from the corpus, write each invention
  description from that patent's own claim, and measure whether the agent recovers it.
  Ground truth is known by construction, so the result is a hard recall number with no
  labeling and no judgement call.
- **Verifier confusion matrix.** Hand-label the 42 drafted findings as relevant or
  not, then report the verifier's precision and false-negative rate. Converts "it
  rejected 55%" into "it rejected 55%, and was right about X% of them."

The first measures retrieval; the second measures the verification stage. Both are
stronger claims than anything in the table above.

### Future work: a larger model

Everything here ran on `qwen2.5:7b`. `qwen2.5:14b` is also free and open-weight, needs
about 9 GB, and fits in 16 GB of RAM.

The expected benefit is a **higher grounding rate through a smaller denominator**. The
7b drafts up to five findings per case whether or not five exist, and the surplus is
what verification throws away. A larger model should propose fewer speculative
findings, so more of what it drafts survives.

Worth stating plainly: that is a hypothesis, not a measured result. It has not been
run, and the number is not reported here until it is.

### Known false positive

The example memo cites US 12327263, an AI smart-grid patent, as "moderately relevant"
to an irrigation invention. It is not relevant, and the verifier passed it. Relevance
judgement catches obvious mismatches and misses subtle ones. Provenance is airtight;
relevance is not.

## Stated limitations

**The corpus is 2025 grants only** — about 379k patents, 5.5M claims. Real prior-art
search spans decades. This is a scope choice, not an oversight: adding years is one
`g_claims_YYYY` download each (~350 MB) plus a re-ingest. The agent is required to
call `get_corpus_stats` before drawing conclusions, and the memo footer states the
range, so an empty result reads as "not in this corpus" rather than "does not exist."

**Only exemplary claims are indexed.** USPTO flags one representative claim per
patent; indexing those rather than all ~14 claims cuts the index by an order of
magnitude and keeps the claim that describes the invention. All claims are stored and
retrievable once a patent is found.

**Retrieval is lexical, not semantic.** SQLite FTS5 with Porter stemming. It will miss
a patent that describes the same idea in entirely different vocabulary. Mitigated by
having the planning stage generate several phrasings, not solved.

**A 7B is not a patent attorney.** It drafts roughly twice as many findings as survive
verification (see Results). A frontier model would draft fewer bad ones to begin with.
The architecture is what transfers; the verification stage exists precisely because the
drafting model is not trustworthy on its own.

**Relevance judgement is the weak link.** Provenance is guaranteed by construction.
Whether a genuine patent is genuinely *relevant* is still a model call, and it gets
that wrong sometimes in both directions.

## Implementation notes

**`mcp >= 2.0` renamed `FastMCP` to `MCPServer`**, and `Tool.inputSchema` to
`input_schema`. The decorator and `run()` APIs are otherwise unchanged.

**The agent loop is hand-rolled.** Ollama has no managed tool runner, so
`draft_memo` implements the loop directly: call, execute requested tools through MCP,
append results, repeat to a turn cap. MCP is model-agnostic, so switching backends
never touched the tool layer.

**Tool results are truncated for the model but kept whole for verification.** A 7B
degrades badly past a few thousand tokens of context, so the model sees 6k characters
per result while the verifier gets the full text. Verification would otherwise fail
findings for being unable to see their own evidence.

**Unruled findings default to ungrounded.** A local model sometimes returns fewer
verdicts than findings. Anything the verifier did not rule on is treated as failed,
which is the safe direction.

**Schema-constrained decoding, then validation anyway.** Ollama's `format` parameter
enforces JSON Schema during decoding, which guarantees parseable output but not
sensible field contents. Pydantic validation and a retry still run.

## Scope

Public USPTO bulk data only, downloaded through the documented portal with an account.
No scraping.

This is a triage aid that tells an attorney what is worth reading. It is not a
patentability opinion and does not attempt to be one.