# lyra-mcp

Typed pipeline-stage abstraction + MCP tool surface for RAG/agent pipelines — lets Claude read/run/modify individual pipeline stages (retrieve, rerank, generate, eval) as structured, typed calls instead of tracing and re-reading raw code each time.

**Status: Phase 3 done, plus a `wiqas.eval` stage.** The abstraction is validated against a second, differently-built project (Orion, a FastAPI service vs. WiQAS's script/CLI shape) with **zero changes to any shared code** — only new adapter files. A third stage type (`wiqas.eval`, wrapping RAGAS's `answer_similarity` metric) proves the same abstraction also extends to a stage with a real live-LLM dependency, not just local-inference stages like retrieval/reranking.

## Running the MCP server

```powershell
uv run lyra-mcp
```

Registered with this session's Claude Code project (`.mcp.json`, generated via `claude mcp add`):
```powershell
claude mcp add lyra -s project -e WIQAS_REPO_ROOT="D:\GitHub\WiQAS" -e WIQAS_VENV_PYTHON="D:\GitHub\WiQAS\.venv\Scripts\python.exe" -- uv run --project D:\GitHub\lyra-mcp lyra-mcp
```
New MCP servers load at Claude Code session start — restart/reconnect the session for `lyra`'s tools to appear.

### Tools

- **`list_stages()`** — every stage in the loaded graph (`graphs/wiqas_query_pipeline.json` by default, override with `LYRA_GRAPH_PATH`): id, type, current config, and full config/input/output JSON schema. Enough to call the other tools without opening any adapter source file.
- **`get_stage_config(stage_id)`** / **`set_stage_config(stage_id, config_overrides)`** — read/persistently mutate a stage's baseline config (validated before committing).
- **`run_stage(stage_id, input, config_override=None)`** — run one stage. `config_override` merges over the baseline for that call only, without persisting — the way to compare configs on the same input (call repeatedly with a different override each time, nothing to reset).
- **`run_pipeline(external_inputs)`** — run the whole loaded graph end to end via `pipelines/executor.py`'s topological-order executor.

### Real bugs found building this (Phase 2)

Two contract mismatches only surfaced through live protocol testing, not unit tests — both are why the project's test-first-workflow rule treats "touching a real external system" as its own verification step:

- **Client-visible errors were silently generic.** The installed MCP SDK (`mcp==2.1.1`) only forwards a raised exception's own message to the client when it's the SDK's own `ToolError` — anything else (including Lyra's own `LyraError` hierarchy) becomes a bare `"Error executing tool <name>"`, discarding every hand-written error message. Fixed with a `_translate_lyra_errors` decorator in `server.py` that re-raises any `LyraError` as a `ToolError` with the same text, at the MCP boundary only (`core/errors.py` stays independent of the SDK).
- **Schema and actual data disagreed on field names.** `model_json_schema()` defaults to `by_alias=True`; `model_dump()` defaults to `by_alias=False`. `list_stages` advertised `SearchResultModel`'s aliased field as `"id"`, but `run_stage`/`run_pipeline` were returning it as `"document_id"` — a real schema/data mismatch an agent trusting the advertised schema would hit immediately. Fixed by adding `by_alias=True` everywhere a `StageOutput` gets dumped for an external caller (`pipelines/executor.py`, `server.py`).

Also confirmed empirically (not assumed): a bare `dict` return-type annotation does **not** produce structured content from this SDK version — only a parameterized `dict[str, Any]` (or a real pydantic model) does; sync `def` tool functions are automatically offloaded to a worker thread (`anyio.to_thread.run_sync`), so no manual `async`/`await` was needed anywhere in `server.py`.

## Architecture: the subprocess boundary

WiQAS's internal code (`from src.retrieval.retriever import WiQASRetriever`, etc.) only imports cleanly inside **WiQAS's own venv**, because of its heavy ML dependencies (torch, chromadb, sentence-transformers). Rather than running Lyra itself inside WiQAS's venv — which would break down the moment a second, differently-built project (Orion, Phase 3) needs wrapping too — each Lyra stage invokes the target project's **own interpreter as a subprocess**, running a small bundled shim script that talks JSON over stdin/stdout.

```
Claude ──calls──> Lyra typed Stage ──subprocess──> target project's own venv
                   (stable interface,     runs a small runner shim script,
                    same every time)      talks JSON over stdin/stdout
```

Lyra itself stays a small, dependency-light project (pydantic only) — the framework (`src/lyra/core/`) has zero knowledge of WiQAS or Orion specifically, so the same `run_subprocess_stage` helper is what every adapter reuses unchanged.

A real gap this surfaced during implementation, worth knowing if you're wrapping a new project: **don't trust a code investigation's reported function signatures for the exact wire shape** — always confirm against the actual runtime output. Two mismatches turned up in WiQAS that weren't visible from reading signatures alone: `SearchResult.to_dict()` serializes its `document_id` attribute under the JSON key `"id"`, and `RerankerManager.rerank_search_results()`'s output dicts omit `search_type` entirely. Both are handled in `SearchResultModel` (see its docstring in `src/lyra/adapters/wiqas/common.py`). **Orion (Phase 3) turned out to have the identical mismatch**, independently confirmed by reading its own source — not a coincidence worth assuming holds for a third project, but a real, recurring pattern worth checking for every time.

Also found live: WiQAS's own logging (a module-level `rich.Console()`) prints straight to stdout, which would have polluted the runner scripts' JSON-only stdout contract. Fixed on the Lyra side only (no WiQAS changes) with an OS-file-descriptor-level redirect — see `src/lyra/adapters/_shared/shim_io.py`. That helper was deliberately kept project-agnostic and outside any single adapter's folder specifically because this pattern was expected to recur — and it did: Orion has the exact same `rich.Console()`-to-stdout issue, fixed by reusing `shim_io.py` completely unchanged, zero new code needed.

### Phase 3: validated against Orion, zero shared-code changes

Orion (`D:\GitHub\Orion`) is a FastAPI service with a Tauri desktop shell — a genuinely different shape from WiQAS's script/CLI style — wrapped the same way: `adapters/orion/` mirrors `adapters/wiqas/` file-for-file (`common.py`, `retrieval.py`, `reranking.py`, `runners/`), registered as two more `STAGE_REGISTRY` entries. **No changes were needed to `core/`, `adapters/_shared/shim_io.py`, `registry.py`'s shape, `pipelines/executor.py`, or `pipelines/state.py`** — the entire cost of wrapping a second project was the new adapter files, exactly as the architecture was designed to allow. Live integration test (`tests/integration/test_orion_live_pipeline.py`) passed in ~21s against Orion's real ~232-chunk collection — dramatically faster than WiQAS's several minutes, since Orion's corpus is smaller and it does no query-decomposition/cross-lingual translation.

### Eval stage: a stage type with a real live-LLM dependency

`adapters/wiqas/eval.py` wraps WiQAS's RAGAS `answer_similarity` metric (`src/evaluation/[3] ragas/ragas_fixed.py`) as `wiqas.eval`. Two things found by reading that code directly, not guessed:

- **The file WiQAS's own docs point to as the RAGAS entrypoint (`ragas_eval.py`) is broken** — it references `json`, `argparse`, `evaluate`, and the `ragas` metric objects without ever importing them, so it imports cleanly but raises `NameError` at call time. `ragas_fixed.py` is the working one.
- **A real, hard live dependency, unlike retrieval/reranking.** WiQAS's own `setup_ollama_for_ragas()` makes an unconditional chat-completion call to Ollama to test connectivity — even in `answer_similarity`-only mode, where that chat model is never actually used for scoring (`answer_similarity` is embeddings-only). Can't be skipped without modifying WiQAS's own code, which this adapter never does. A running Ollama server with the configured chat model (default `mistral:latest`) and `nomic-embed-text` both pulled is a hard precondition.

Scoped narrowly and deliberately: `answer_similarity` only (`use_all_metrics=False`, WiQAS's own default) — its own code comment calls this "the only one that works reliably," and warns the full 6-metric mode "may timeout." `use_all_metrics=True` is exposed as a config option but untested here. Live integration test (`tests/integration/test_wiqas_eval_live.py`) confirmed a real, discriminating score (0.697 similarity between a Filipino ground-truth answer and a differently-phrased English paraphrase) in ~12s wall time — far faster than WiQAS's own "this may take several minutes" comment, which describes the full-metric path this stage doesn't use. Not wired into `wiqas_query_pipeline.json`: eval needs a *generated* answer as input, which no existing stage produces (no `generate` stage exists yet) — registered in `STAGE_REGISTRY` alone, same precedent as retrieval/reranking being proven directly before Phase 2's MCP layer existed.

## Setup

Requires [uv](https://docs.astral.sh/uv/) and Python 3.12.

```powershell
uv venv --python 3.12
uv sync
```

`.env.example` documents the environment variables Lyra reads if your WiQAS/Orion checkouts live somewhere other than their defaults. It's a template, not auto-loaded — nothing in this codebase reads `.env` — so set these as real environment variables in your shell/session:

```
WIQAS_REPO_ROOT=D:\GitHub\WiQAS
WIQAS_VENV_PYTHON=D:\GitHub\WiQAS\.venv\Scripts\python.exe
ORION_REPO_ROOT=D:\GitHub\Orion
ORION_VENV_PYTHON=D:\GitHub\Orion\.venv\Scripts\python.exe
```

## Running on a different machine

Everything machine-specific is env vars plus one generated config file — nothing is hand-edited:

1. `uv venv --python 3.12 && uv sync` (see Setup above).
2. Set `WIQAS_REPO_ROOT`/`WIQAS_VENV_PYTHON` and/or `ORION_REPO_ROOT`/`ORION_VENV_PYTHON` as real environment variables for whichever projects you're wrapping on this machine (see `.env.example`).
3. Regenerate `.mcp.json` — it's gitignored on purpose (bakes in this machine's absolute `uv.exe` path and repo checkout paths, not portable as-is; see `.mcp.json.example` for the shape). Don't hand-copy the example — run:
   ```powershell
   claude mcp add lyra -s project -e WIQAS_REPO_ROOT="<path>" -e WIQAS_VENV_PYTHON="<path>" -e ORION_REPO_ROOT="<path>" -e ORION_VENV_PYTHON="<path>" -- uv run --project <path to this checkout> lyra-mcp
   ```
   Restart/reconnect the Claude Code session afterward — new MCP servers load at session start.

If a path is wrong, the failure is a clear, actionable `SubprocessLaunchError: Interpreter not found: <path>` (see `core/subprocess_adapter.py`) — not a silent hang or a cryptic import error.

## Running tests

```powershell
# Unit tests (default) — fast, subprocess boundary mocked, no WiQAS/Orion needed
uv run pytest

# Integration tests (opt-in) — need a real WiQAS/Orion venv + an already-ingested
# Chroma collection at WIQAS_REPO_ROOT / ORION_REPO_ROOT respectively
uv run pytest -m integration -s
```

## Layout

```
src/lyra/
  core/            # StageConfig/StageInput/StageOutput, Stage ABC, the shared subprocess-boundary helper
  registry.py      # stage_type -> concrete classes lookup, all wrapped projects register here
  server.py        # the MCP server (list_stages/get_stage_config/set_stage_config/run_stage/run_pipeline)
  mcp_models.py    # tool-output-only models (StageInfo, RunPipelineResult)
  adapters/
    _shared/       # project-agnostic runner-script helpers (e.g. the stdout-redirect fix), reused by every adapter
    wiqas/         # WiQAS-specific typed stages (retrieval, reranking, eval) + their runner shim scripts
    orion/         # Orion-specific typed stages, same shape as wiqas/ — proves the pattern generalizes
  pipelines/       # PipelineGraph (load/validate), executor.py (topological-order execution), state.py (mutable loaded-graph wrapper)
  graphs/          # Example pipeline-as-data graphs, one per wrapped project
tests/
  unit/           # subprocess boundary mocked, run by default
  integration/    # opt-in (`pytest -m integration`), needs real WiQAS/Orion
```