# ClaimCheck

**Checks a health insurance claim settlement against the policy wording and IRDAI's rules, and explains every deduction with the clause it rests on.**

Indian insurers often pay less than the hospital bill, and the settlement letter lists the cuts in a line each:
"proportionate deduction", "non-payable items", "co-payment". Few policyholders can check them against a
60-page policy and the regulator's circulars. ClaimCheck reads the policy, the bill (PDF or phone photo) and
the settlement letter, works out what the policy should pay, judges each deduction, quotes the exact
clause, and drafts the appeal letter.

**[Open the demo](https://pranavkk7.github.io/claimcheck/)**: the report of a real run on a fictional claim,
with every deduction, quote and calculation step. To check your own claim, run it locally (below).

![The demo claim: the insurer approved ₹1,15,080, the policy and IRDAI's rules support ₹1,41,906, so ₹26,826 may be due. Each deduction is stamped Allowed, Too high or Not allowed, with a plain explanation and the quoted rule.](docs/report.png)

In the demo (a fictional gallbladder surgery under India's standard Arogya Sanjeevani policy) the insurer
applied the "proportionate deduction" to pharmacy, consumables and diagnostics. IRDAI's circular
IRDAI/HLT/REG/CIR/151/06/2020, paragraph 4, forbids exactly that. ClaimCheck finds it, quotes the paragraph,
recalculates the deduction (₹34,125, not ₹57,563) and puts it in the appeal letter.

## How it works

```
policy wording (PDF) ─┐
IRDAI circulars (PDF) ┴─► clause-aware passages ─► hybrid index (Gemini embeddings + BM25, fused by rank)
                                                          │
hospital bill (PDF/photo) ─► Gemini reads the lines ─┐    │
                                                     ├─► non-payable check: search IRDAI's lists, the model
                                                     │   picks only among the entries retrieved
policy limits ◄── Gemini reads them, each number must be quoted from the policy and found in the quote
                                                     │
                                         calculator (plain code): what the policy should pay, step by step
                                                     │
settlement letter (PDF/photo) ─► Gemini reads the deductions
                                                     │
                     agent: passages retrieved for each deduction + tools (search the policy and rules,
                     read a passage, look up a list item) ─► a verdict and quotes for every deduction
                                                     │
                     verifier: every quote must be word for word in the passage it names
                                                     │
                     report + appeal letter (written from the verified findings, not by the model)
```

Design decisions worth knowing:

- **Arithmetic in code, judgement in the model.** Room limits, the proportionate deduction, ICU limits,
  co-payment and the sum-insured cap are computed by `calc.py`. The agent never produces an amount; it
  points each deduction at a calculation step, and the recoverable amount is computed from that step.
- **Every quote is checked.** A quote counts only if its words are in the passage it names (ignoring case,
  spacing and punctuation, because the scanned IRDAI circular splits words: "proporti on a te"). A verdict
  left with no verified quote becomes "unclear". A policy limit whose number is not in its quote is not
  applied.
- **Retrieval before reasoning.** The documents are searched for each deduction before the agent starts,
  and the passages go into its brief; the tools are for what is missing. This took the agent from 14+
  turns (searching one passage at a time) to 2.
- **Passages follow the documents' own clauses** ("3.45 Room Rent means…", "9.3 Co-payment"), so a
  citation is a clause number and a page. IRDAI's four non-payable lists are parsed into 146 entries, each
  one citable ("List I, item 56: GLOVES"); the lists are told apart by their numbering, because PDF text
  extraction puts a list's heading in the wrong place.
- **Service charges never go to the list check.** A bill line for the operation theatre, the surgeon or
  the room is not a list item; without this rule the model once matched "operation theatre charges" to
  "ward and theatre booking charges" and removed ₹22,000 from the claim.
- **The most capable model first, and it keeps working on the free tier.** Every step tries Gemini 4, then
  the Pro models (both need a paid plan), then the newest Flash models. Each has its own small free daily
  quota (20 requests), so when one is used up or overloaded the next takes over, and the agent restarts
  cleanly on it (a model's thought signatures do not carry over). If the embedding quota runs out, search
  falls back to keywords.
- **Provider-agnostic.** The model sits behind a two-method interface (`structured`, `run_tools`); Gemini is
  the default and a local model through Ollama works with `CLAIMCHECK_LLM=ollama`.
- **An MCP server** (`python -m claimcheck.mcp_server`) gives any AI assistant three tools:
  `search_rules`, `nonpayable_item` and `check_claim`.

## Results

Measured against labels written before any run (`eval/`).

**Non-payable items** (44 bill lines, 32 of them on IRDAI's lists, 12 payable; the same result on Gemini 3.7 and 3.6 Flash):

| | |
|---|---|
| Retrieval: the right list entry among the 4 passages found | 100% |
| Lines judged correctly (list entry or payable) | 97.7% (43/44) |
| Precision: lines flagged as non-payable that really are | 100% |
| Recall: non-payable lines that were found | 96.9% |

Missed: a combined line, "Toothpaste and toothbrush".

**Insurer deductions** (20 deductions in 4 made-up claims, including a hospital with flat pricing, an
ICU-driven deduction and a room within its limit). ClaimCheck against the same model with no retrieval,
no tools and no calculator, asked two ways (Gemini 3.8 and 3.7 Flash):

| | ClaimCheck | Gemini alone, asked as a policyholder would | Gemini alone, told the policy's limits |
|---|---|---|---|
| Verdicts right | **20/20** | 20/20 | 20/20 |
| Recoverable amounts right to the rupee | **19/19** | 18/19 | 18/19 |
| Verdicts backed by a quote verified in the documents | **20/20** | 0/20 | 0/20 |

What this shows, honestly: these rules (proportionate deductions, the non-payable lists, co-payment) are
well known, and a strong model judges the verdicts correctly on its own. The difference is in what can be
trusted and acted on. Gemini alone gave no source for any verdict, so none of its answers can be checked;
it got a co-payment refund wrong (₹3,471 instead of ₹2,997), and in an earlier run it put the demo's main
refund at ₹22,266 instead of ₹23,438. ClaimCheck's amounts come from the calculator, every verdict carries
a checked quote with its page, and it reads the limits from the policy itself instead of assuming them.

Two changes came from earlier evaluation runs, both general rules rather than fixes for one case: a
deduction no larger than the rules allow can no longer be called "too high" (the amounts decide that, not
the model), and the agent also sees the comparisons that produce no calculation step ("Room: ₹4,500 a day;
the limit is ₹5,000 a day, within the limit").

## Sources

The documents are public and are downloaded from their official links (`python -m claimcheck fetch`); they
are not stored in this repository.

| Document | Publisher |
|---|---|
| [Norms on Proportionate Deductions](https://irdai.gov.in/document-detail?documentId=394680) (IRDAI/HLT/REG/CIR/151/06/2020) | IRDAI |
| [Master Circular on Standardization of Health Insurance Products](https://irdai.gov.in/document-detail?documentId=395035) (2020), with the standard non-payable lists | IRDAI |
| [Master Circular on Health Insurance Business](https://irdai.gov.in/document-detail?documentId=4942918) (2024) and its annexure | IRDAI |
| Arogya Sanjeevani Policy wording | IFFCO-Tokio General Insurance |
| Arogya Sanjeevani Policy wording | SBI General Insurance |

The demo bill and settlement letter in `samples/` are fictional and say so on the page.

## Run it

```bash
python -m venv .venv
.venv/Scripts/python -m pip install -e ".[dev]"
cp .env.example .env            # then add a Gemini API key (free at aistudio.google.com/apikey)
.venv/Scripts/python -m claimcheck fetch
.venv/Scripts/python -m uvicorn claimcheck.api:app --port 8020          # the web page
.venv/Scripts/python -m claimcheck check --bill samples/demo-bill.pdf --settlement samples/demo-settlement.pdf --sum-insured 300000
.venv/Scripts/python -m pytest                                          # stand-ins: no key or network needed
```

The hosted demo is the same page as static files: `python -m claimcheck site _site` builds it from
`samples/demo-report.json`, and a GitHub Actions workflow publishes it to GitHub Pages on every push. With
`CLAIMCHECK_DEMO=1` the server also shows only the saved demo report, without spending the API key.

## Limits

- The calculator implements the standard Arogya Sanjeevani limits and IRDAI's general rules. Other
  policies can be uploaded and their limits are read from the wording, but sub-limits (cataract, modern
  treatments) and waiting periods are not calculated yet.
- Policy wordings must be text PDFs; bills and letters can be photos.
- The verdicts are explanations with sources, not legal or financial advice. The insurer's grievance
  officer and the Insurance Ombudsman decide claims.

## Licence

MIT. The documents keep their own terms; the vendored Motion library has its own
[MIT licence](claimcheck/web/vendor/LICENSE-motion.md).