# NotesToReport

An open-source MCP developer tool for [SuperDocs](https://superdocs.app) —
turns a folder of rough notes into a citation-grounded report, then hands
it off to SuperDocs to create a live document. Every claim in the final
report is either backed by an exact source citation (file + line range) or
flagged as unverified — never silently invented.

Built as part of a job application task for SuperDocs.

## Why this exists

AI-written summaries can quietly add details that were never in the source
material. That's fine for casual use, but risky if the output is going to
a client or a boss. NotesToReport adds a verification layer on top of an
AI-generated report: every sentence is checked against the original notes
before the document is ever created.

## How it works

1. **Read notes** — point the tool at a folder of rough notes (e.g.
   standup notes).
2. **Draft a report** — an LLM reads all the notes and writes a clean,
   structured report.
3. **Ground every claim** — the report is split into individual claims.
   Each one is checked against the source notes; if supported, it gets a
   citation pointing to the exact file and line range. If not, it's
   flagged as unverified instead of silently passing through.
4. **Score faithfulness** — the percentage of claims that are actually
   grounded becomes the report's faithfulness score.
5. **Circuit breaker** — if the score is below 0.85, the tool refuses to
   hand off to SuperDocs at all.
6. **Hand off to SuperDocs** — if it passes, the report is sent to
   SuperDocs via the `chat_async` tool, which creates a live document with
   citations as footnotes and unverified claims as inline comments.

## Project structure

```
connector/    MCP connector — connects to SuperDocs' real MCP server
citation/     Citation grounding engine (chunking, retrieval, entailment)
handoff/      Faithfulness circuit breaker + SuperDocs hand-off logic
engine/       Report synthesis + end-to-end test scripts
sample-notes/ Example notes for testing
```

## Setup

```bash
npm install
cp .env.example .env
# then fill in GROQ_API_KEY and SUPERDOCS_API_KEY in .env
```

## Running the dry-run pipeline

Runs the full pipeline against sample notes without contacting SuperDocs —
prints what _would_ be sent.

```bash
npx ts-node engine/src/testPipeline.ts ./sample-notes
```

## Running the adversarial test

Injects a false claim directly into the AI-generated report (not the
source notes) and confirms the grounding engine correctly rejects it. See
`FINDINGS.md` for full test results and methodology.

```bash
npx ts-node engine/src/testFakeClaimInReport.ts ./sample-notes
```

## Current status

- Citation grounding engine — built and tested, including an adversarial
  test confirming it correctly rejects hallucinated claims not present in
  the source notes.
- Faithfulness circuit breaker — built and tested.
- SuperDocs MCP connector — built, uses only SuperDocs' confirmed tools
  (`chat_async`, `get_job`, `upload_attachment_base64`,
  `get_attachment_status`).
- Report synthesis — built and tested end-to-end.

### Known limitations

- The embedding function used in tests (`simpleEmbed`) is a lightweight
  local word-hash stand-in, not a real semantic embedding model. A real
  embeddings API is the natural next step for grounding quality on larger,
  real-world note sets.
- The connector has not yet been run against a live SuperDocs connection
  end-to-end (dry-run only so far).
- No CLI wrapper yet (`bin/cli.ts` referenced in package.json is not yet
  built) — the pipeline is currently run directly via ts-node.

See `FINDINGS.md` for detailed test results.

## License

MIT