# DocMind — MCP-Native Document Research Agent

An iterative, multi-step research agent built on **LangGraph** state
graphs and **LangChain** retrieval chains, with search, summarization, and
citation exposed as a real **MCP (Model Context Protocol) server** — so
those tools are consumable by any MCP-compatible client, not locked to
this agent. Retrieval combines **hybrid search** (vector embeddings +
keyword) and every research session ends in a **validated structured JSON
report**, not free text.

Runs on a **BYOK (Bring Your Own Key)** model — you supply your own
Anthropic or OpenAI API key via a local `.env` file. DocMind never ships,
stores, or transmits your key anywhere except directly to your chosen LLM
provider.

## Why this is a *research agent*, not just RAG

A single retrieve-then-answer pass often isn't enough for open-ended
questions: the first search may surface part of the answer while missing
another part entirely. DocMind's graph is a loop, not a pipeline:

```
   ┌──────┐      ┌────────┐      ┌──────────┐
   │ plan │ ───▶ │ search │ ───▶ │ evaluate │
   └──────┘      └───┬────┘      └────┬─────┘
                      ▲                 │
                      │  "search" (gap  │  "synthesize" (evidence
                      │   found, budget │   sufficient, or budget
                      │   remains)      │   exhausted)
                      └─────────────────┤
                                         ▼
                                 ┌─────────────┐
                                 │ synthesize  │ ──▶ END
                                 └─────────────┘
```

- **`plan`** turns the raw question into a focused first search query.
- **`search`** calls the MCP `search_documents` tool and accumulates
  evidence across iterations (deduplicated by document + chunk).
- **`evaluate`** — an LLM call — judges whether the evidence gathered so
  far actually answers the question. If not, it proposes a refined query
  and the graph loops back to `search`, up to `MAX_RESEARCH_ITERATIONS`.
- **`synthesize`** produces the final answer as a validated
  `ResearchReport` object via LangChain's `with_structured_output` —
  never a raw string a downstream service has to parse.

## MCP server: search, summarize, cite

`app/mcp_server/server.py` exposes three tools over the corpus:

| Tool | Purpose |
|---|---|
| `search_documents(query, top_k)` | Hybrid vector+keyword search across the whole corpus |
| `summarize_document(source, focus)` | LLM summary of one named document, optionally focused on an aspect |
| `cite_passage(claim, source)` | Finds the best-matching passage *within one document* that supports a claim — the citation/grounding primitive |

Because these are MCP tools rather than plain Python functions, any
MCP-compatible client can connect to this same server and use them —
including, for example, Claude Desktop, once pointed at
`python -m app.mcp_server.server`. The LangGraph agent shipped in this
repo deliberately consumes its own server the same way an external client
would (`app/mcp_client/tools.py`), rather than importing the retrieval
module directly — that's what makes it "MCP-native" end to end.

## Hybrid retrieval

`app/retrieval/hybrid_search.py` fuses Postgres full-text keyword search
with pgvector cosine-similarity search (HNSW index), normalizing and
weighting both signals (`HYBRID_ALPHA` in `.env`). The fusion logic is a
pure function, unit-tested without needing a live database.

## BYOK — Bring Your Own Key

- Set `LLM_PROVIDER=anthropic` or `openai` in `.env`.
- Set the matching `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`.
- `app/config.py` raises an explicit, readable error if the selected
  provider's key is missing.
- Local embeddings (`sentence-transformers`) run entirely on your machine
  and need **no API key** — only the plan/evaluate/synthesize LLM calls
  and the `summarize_document` tool use your BYOK key.

## Project structure

```
DocMind/
├── app/
│   ├── config.py             # BYOK settings, loaded from .env
│   ├── llm.py                 # LLM factory (Anthropic / OpenAI)
│   ├── schemas.py               # ResearchReport structured-output schema
│   ├── main.py                    # CLI entry point
│   ├── api.py                      # optional FastAPI /research endpoint
│   ├── graph/
│   │   ├── state.py                 # ResearchState (question, evidence, iteration...)
│   │   ├── nodes.py                  # plan / search / evaluate / synthesize
│   │   └── build_graph.py             # wires the iterative loop
│   ├── retrieval/
│   │   ├── embeddings.py               # local sentence-transformers embeddings
│   │   ├── vector_store.py              # documents + corpus_chunks (pgvector)
│   │   └── hybrid_search.py              # score fusion (unit-testable, pure fn)
│   ├── mcp_server/
│   │   └── server.py                      # FastMCP: search / summarize / cite tools
│   └── mcp_client/
│       └── tools.py                        # discovers + calls MCP tools at runtime
├── data/corpus/                # sample research docs (RAG, vector DBs, agents)
├── scripts/
│   ├── init_db.sql               # pgvector schema (documents + corpus_chunks)
│   └── seed_corpus.py             # paragraph-aware chunking + embedding + insert
├── tests/                          # pure-logic unit tests (no live DB/LLM needed)
├── examples/example_research_session.md
├── docker-compose.yml                # local Postgres + pgvector (port 5433)
├── requirements.txt
└── .env.example
```

## Setup

**1. Clone and install dependencies**

```bash
git clone <your-fork-url>
cd DocMind
python -m venv venv && source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt
```

**2. Configure your BYOK key**

```bash
cp .env.example .env
# edit .env — set LLM_PROVIDER and the matching API key
```

**3. Start Postgres + pgvector**

```bash
docker-compose up -d
```

**4. Seed the corpus**

```bash
python -m scripts.seed_corpus
```

**5. Run DocMind**

```bash
python -m app.main
```

Or as an API:

```bash
uvicorn app.api:app --reload
# POST http://localhost:8000/research  {"question": "How does HNSW work?"}
```

**Inspecting the MCP server directly** (useful for demoing that these
tools are consumable by any MCP client, not just this agent):

```bash
mcp dev app/mcp_server/server.py
```

See [`examples/example_research_session.md`](examples/example_research_session.md)
for a full sample session showing the loop refine itself across two
iterations.

## Running tests

```bash
pytest
```

Covers the pure-logic pieces — score fusion and loop routing — that don't
require a live database or LLM call, so they run in any environment,
including CI.

## Tech stack

| Layer | Technology |
|---|---|
| Agent orchestration | LangGraph (iterative state graph, conditional looping, checkpointer) |
| Retrieval chains | LangChain |
| Tool exposure | MCP (Model Context Protocol) — `mcp` Python SDK |
| Structured output | LangChain `with_structured_output` + Pydantic |
| Vector store | PostgreSQL + pgvector (HNSW index) |
| Embeddings | sentence-transformers, local, CPU-only |
| LLM | Anthropic Claude or OpenAI (BYOK, user-selected) |
| API | FastAPI (optional) |

## Known limitations

- The MCP client opens a fresh stdio session per tool call rather than a
  persistent connection — simple and reliable for a research workload
  with a handful of tool calls per session, not tuned for high frequency.
- `evaluate_node`'s sufficiency judgment is itself an LLM call and can be
  wrong in either direction (stopping early or looping unnecessarily);
  `MAX_RESEARCH_ITERATIONS` bounds the cost of the latter.
- The sample corpus is small and topic-specific (RAG, vector databases,
  agent architectures) — meant to demonstrate the pipeline end to end, not
  as a production knowledge base.

## License

MIT — see [LICENSE](LICENSE).