spark-mcp

Spark — the creativity and institutional-learning MCP for quant research organizations. Spark remembers what a research organization has tried (including failures), counts every trial so statistical verdicts are deflated rather than lucky, points at unexplored regions of research space, and turns validated trajectories into replay-gated skills. It never backtests and never owns market data — a peer engine computes, Spark judges.

Built per the Spark v3 master plan through phases 0–7 (hypothesis ledger, method graph + novelty, gates, QuantFlow hook seam, MAP-Elites QD archive, skill evolution, analogy/consolidation/curriculum). Current state and per-phase evidence: STATUS.md; independent milestone reviews: reviews/.

Install

uv venv .venv --python 3.12
uv pip install -e '.[dev]'

Requires Python ≥3.11. The [qd] extra needs pyribs (optional — the QD archive is pure-Python by design).

Related MCP server: APEX Research MCP Server

Quick start

spark init-db          # create ~/.spark/spark.db (schema migrations)
spark doctor           # capability report (degraded mode, FTS5, sqlite-vec)
spark serve            # MCP server over stdio (equivalent: python -m spark_mcp)
python -m pytest -q    # test suite incl. the independence gate

Configuration

All settings live in config.example.yaml (copy to your SPARK_CONFIG path or rely on defaults: DB at ~/.spark/spark.db). Secrets are environment-variable references only — never values in files:

Without an LLM key Spark runs in degraded mode (lexical novelty proxy, deterministic exploration) — fully usable, no network needed.

Use as an MCP server

Any MCP-capable agent or harness can spawn it over stdio:

{ "command": "<venv>/bin/python",
  "args": ["-m", "spark_mcp"],
  "env": { "SPARK_CONFIG": "/path/to/spark.yaml" } }

28 tools, highlights:

• Ledger & memory: register_hypothesis, hypothesis_propose, log_outcome, memory_recall, memory_promote, research_history

• Exploration: explore_methods, suggest_next, archive_status, find_analogies, combine_methods

• Gates & evidence: significance_gate, check_novelty, backtest_run, causal_check, risk_decompose

• Skill evolution: distill_skill, propose_skill_challenger, run_replay, decide_skill_promotion, approve, retire_skill

• Peer seam: run_summary_ingest (QuantFlow Tier-2 hook), report_write, consolidate, generate_exercise, spark_health, spark_doctor

Layout

• src/spark_mcp/ — package (config, storage, llm protocols+fakes, server, cli, gates, QD archive, skill evolution)

• tests/ — unit tests + tests/independence/ (the blocking independence gate)

• reviews/ — independent milestone reviews

• STATUS.md — per-phase verdicts and program-gate evidence

The independence contract

This package works installed alone. The reference repos studied during design are never imported, read, or required at runtime. tests/independence/ enforces this on every test run.

Conventions

• Time fields are ISO-8601 UTC text; IDs are ULIDs with type prefixes (hyp_, ep_, trial_, belief_).

• Storage is SQLite WAL with PRAGMA user_version migrations.

• Every data-dependent row carries as_of/data_through; retrieval filters before ranking (no leakage).

• Trial rows are inserted before dispatch; retries never double-count.

License

MIT — see pyproject.toml.