# mcp-data-cluster

An extensible MCP server that puts **SQL, vector search, and a knowledge graph**
behind one Model Context Protocol surface. It ships a Company-X reference pack and
connects to other data through portable data packs and `DataTool` adapters.

[한국어 README](README.ko.md)

> Point it at any PostgreSQL database. It reads the catalog, learns its own routing
> vocabulary, and starts answering questions. No hand-written schema files, no
> per-dataset keyword lists.

## Why

Retrieval-Augmented Generation works, but production RAG is famously configuration
sensitive: chunk size, overlap, top-k, reranking thresholds, prompt budget. Each knob
is a failure point.

This project takes the opposite bet — **standardize the interface (MCP), derive the
configuration from the data** — and then ships the benchmark to check whether the bet
pays off. The claim is not repeated from a paper; it is re-measured here, against a
deliberately conventional RAG baseline running on the same corpus.

## Architecture

```
question
   │
   ▼
router ──── scores tools by signals they declare about themselves
   │        (weak signal → MCP Parallel instead of guessing)
   ▼
DataTool registry
   ├── nl2sql          → PostgreSQL (schema context generated from the catalog)
   ├── vector_search   → pgvector
   └── knowledge_graph → ontology traversal
   │
   ▼
TACC curation ──── each tool compresses its own result within a token budget
   │
   ▼
answer (local LLM via Ollama)
```

The core knows nothing about any of these tools. `src/core/` never names a tool.

## Quick start

Requires Docker and [Ollama](https://ollama.com).

```bash
ollama pull gemma4:e2b
ollama pull nomic-embed-text

cp .env.example .env
npm install

npm run fetch:data     # downloads the dataset pack, verifies SHA-256
npm run db:up
npm run embed          # embedding dimension is probed from the model

npm run start                        # MCP server (SSE :3510)
npm run agent -- "your question"     # one-shot CLI
```

## Extending

Implement one interface. The router, the curator and the MCP surface pick it up
automatically — no core file changes.

```ts
import type { DataTool } from './src/core/types.js';

export const logSearchTool: DataTool<{ pattern: string }> = {
  name: 'log_search',
  description: 'Full-text search over operational logs',
  params: { pattern: { type: 'string', description: 'search pattern' } },

  // routing signals this tool owns
  signals: () => ({ patterns: [/log|stacktrace/i] }),
  // vocabulary learned from its own data
  bootstrap: async () => ({ patterns: [], lexicon: await distinctServiceNames() }),

  toParams: (question) => ({ pattern: question }),
  run: async ({ pattern }) => searchLogs(pattern),
  // this tool knows how to compress its own output
  curate: (result, budget) => renderLogTable(result, budget),
};
```

```ts
createRegistry().register(logSearchTool);
```

See [docs/extending.md](docs/extending.md).

## Routing without hand-written vocabulary

Each tool derives its routing vocabulary from its own data source:

| Tool | Vocabulary source |
|------|-------------------|
| `nl2sql` | `information_schema` — table/column names, plus values of low-cardinality columns |
| `vector_search` | `document_chunks` — headings and titles, filtered by document-frequency band |
| `knowledge_graph` | node names and the relation types that actually occur |

The pack also declares how relations and node types are *named in natural language*
(`relationCues`, `typeLabels`). Knowing that `USES` is spoken as "uses / runs / adopted"
is knowledge the dataset owner has; it does not belong in code.

Only *phrasing* patterns are hand-written (`"tell me about ~"`, `"the most ~"`), because
those belong to language, not to a dataset.

```bash
npm run signals                # what each tool learned
npm run signals -- "question"  # why that question routed where it did
npm run demo:foreign           # attach to an unrelated database, zero code changes
```

### Two tuning parameters

| Parameter | Default | Meaning |
|-----------|---------|---------|
| `confidenceFloor` | 2 | Below this score, don't commit — escalate to MCP Parallel |
| `lexiconMaxPoints` | 2 | Ceiling on points earned from lexicon hits |

Tools additionally declare how much their own vocabulary is worth (`lexiconWeight`) —
an exact match against a stored column value is stronger evidence than a word that
merely appears somewhere in a corpus. That is part of the adapter contract, not an
operator knob.

Parallel dispatch is used as an *uncertainty policy*, not just a tie-breaker.

## Benchmark

The headline metric is **Evidence Recall**: did the pipeline put the facts needed to
answer in front of the model? It is deterministic, needs no LLM, and applies equally to
both pipelines (curated tool output for MCP, retrieved chunks for RAG).

```bash
npm run bench                    # evidence recall, both pipelines
npm run bench -- --set holdout   # held-out questions authored for this repo
npm run bench -- --answer        # also score the generated answer (slow)
npm run bench:sweep              # chunk size × top-k sensitivity
```

### Measured results

| Set | Pipeline | Evidence Recall | Routing |
|-----|----------|----------------:|--------:|
| gold (30) | **MCP** | **0.817** | 1.000 |
| gold (30) | naive RAG (pure vector) | 0.342 | — |
| gold (30) | RAG + hybrid retrieval | 0.420 | — |
| holdout (22) | **MCP** | **0.955** | 1.000 |
| holdout (22) | naive RAG (pure vector) | 0.328 | — |
| holdout (22) | RAG + hybrid retrieval | 0.570 | — |

Configuration sensitivity across a chunk-size × top-k grid (spread between best and
worst run): **MCP 0.010, RAG 0.263** — the baseline is 26× more sensitive to tuning.

The third row of each block is a control: giving the baseline the same retriever
separates how much of the gap comes from retrieval technique versus from routing,
tool selection and curation.

The held-out set earned its keep — it caught routing patterns that had memorised the
phrasing of the provided questions (0.636 at first measurement). Because those failures
were then fixed, the final routing figure is fitted, not a blind estimate. Full numbers,
the improvement history, and what is still broken: [docs/results.md](docs/results.md).

Gold answers live in `packs/<pack>/gold.json`; every SQL gold case carries a reference
query that is executed at scoring time, so the expected values track the data.

## Dataset

The bundled pack points at a contest dataset that **may not be redistributed**. It is not
committed here. `npm run fetch:data` downloads it and verifies the SHA-256 declared in
`packs/companyx/pack.json`.

To use your own data, write a new `pack.json` — no code changes.

## References

- Model Context Protocol — https://modelcontextprotocol.io
- air, MCP server framework (Apache-2.0) — https://airmcp.dev
- pgvector — https://github.com/pgvector/pgvector
- Jeon et al. (2026), *MCP context composition for small language models*, Zenodo 18842478
- Jeon (2026), *Pylon-7: A 7-Layer Reference Model for AI Agent Workflows*, Zenodo 18808598
- Liu et al. (2024), *Lost in the Middle*, TACL

## Licensing

This project is licensed under the **Apache License 2.0** — see [LICENSE](LICENSE)
and [NOTICE](NOTICE).

### Dependencies

All runtime and development dependencies are permissively licensed; there is no
copyleft in the tree. They are installed from npm by the user and are not bundled
or redistributed here.

| License | Count |
|---------|------:|
| MIT | 99 |
| ISC | 9 |
| Apache-2.0 | 3 |
| BSD-2-Clause / BSD-3-Clause | 4 |

`npm run check:licenses` enforces this in CI: it fails the build if the declared
licenses disagree with each other, if a copyleft dependency appears, or if the
non-redistributable dataset is tracked by git.

### Dataset — not redistributable

The Company-X dataset is licensed by LIWONACE **for contest participation only**.
It is deliberately absent from this repository and from its git history:

- `data/` is gitignored; `npm run fetch:data` downloads it and verifies the SHA-256
  declared in `packs/companyx/pack.json`.
- The benchmark stores **how to derive** each answer, not the answer itself. Gold
  cases carry a reference SQL query or a graph traversal spec that is evaluated
  against the source data at scoring time, and the provided questions are referenced
  by index rather than copied. Nothing in `packs/` reproduces dataset content.

If you fork this repository, obtain the dataset yourself under its own terms.