Skip to main content
Glama
Runtime Copilot MCP Server

Runtime Copilot MCP Server

by kroq86

Interactive Data Systems Lab

Runnable Python and Rust data-system internals with an MCP-native runtime copilot layer.

This repository combines two things:

  • a local-first engineering lab for PostgreSQL-like and Databricks-like internals,

  • a Runtime Copilot MCP surface for diagnostics, explainability, regression checks, and operational memory.

Core capabilities

  • Storage and planner internals: heap tables, B-tree indexing, selectivity, and plan choice.

  • Persistence and replay: WAL/checkpoint style flows and deterministic state transitions.

  • Workflow and write-path modeling: event-first architecture, idempotency, retry semantics.

  • Explainable runtime operations: traced runs, failure summaries, regression verdicts, baseline compare.

  • MCP access: machine-usable operational interface instead of ad hoc shell scripts.

Related MCP server: Alma Atlas

Who this is for

  • Data engineers learning warehouse and query-engine internals.

  • Platform and infrastructure engineers teaching storage and execution fundamentals.

  • Teams building onboarding labs, workshops, and demo environments.

Quick start

Requirements:

  • Python 3.10+ (tested with Python 3.14)

Setup:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "psycopg[binary]"

Run end-to-end flow:

cargo run --bin e2e_flow

Run core demos:

.venv/bin/python mini_pg_like.py
.venv/bin/python mini_databricks_clone.py
cargo run --bin mini_pg_like
cargo run --bin mini_databricks_clone

What this repository contains

  • mini_pg_like.py: PostgreSQL-like toy engine with heap table, B-tree index, and planner output.

  • mini_databricks_clone.py: Databricks-like toy platform with versioning, partitions, DAGs, and events.

  • src/bin/mini_pg_like.rs: Rust PostgreSQL-like demo.

  • src/bin/mini_databricks_clone.rs: Rust Databricks-like demo.

  • src/lib.rs, src/common.rs, src/pg.rs: shared Rust core modules.

  • mcp_engine_server.py: MCP runtime adapter for diagnostics and regression workflows.

Why this exists

Most internals content stops at diagrams. This project stays runnable and inspectable:

  • compare Python and Rust implementations of the same system ideas,

  • trace write-path behavior with concrete events and state transitions,

  • run explainable regression checks through MCP,

  • turn runtime operations into a discoverable control surface for AI clients.

MCP Adapter Layer

This repo also includes a minimal MCP server that wraps the lab operations:

  • mcp_engine_server.py

  • Cursor config: .cursor/mcp.json

Current MCP tool list for this release (47 tools total):

Engine state and runtime:

  • init_engine

  • insert_row

  • upsert_row

  • create_index

  • explain_customer

  • reindex_project

  • run_e2e_flow

Explainability and demos:

  • explain_run

  • demo_explain_run

  • demo_explain_run_failure

  • demo_explain_semantic_failure

  • demo_explain_idempotency_conflict

  • demo_explain_concurrency_failure_storm

  • explain_regression_suite

Trace and retrieval:

  • record_tool_trace

  • similar_incidents

  • refresh_trace_path

  • refresh_docs_path

  • memory_upsert

  • memory_search

SLO and ROI:

  • health_check

  • benchmark_calls

  • scenario_load_test

  • capture_roi_baseline

  • report_drift_bug

  • decision_gate

Schema evaluation (verdict + report via MCP, no external Postgres):

  • schema_load_tool

  • schema_explain_tool

  • schema_evaluate_tool

  • schema_evaluate_full_tool

Project contract and regression:

  • project_manifest

  • project_capabilities

  • project_tool_catalog

  • project_get_defaults

  • project_run_regression

  • project_capture_baseline

  • project_compare_baseline

Generic project state:

  • project_list_entities

  • project_get_entity

  • project_upsert_entity

  • project_delete_entity

  • project_append_event

  • project_ingest_trace

  • project_explain_run

  • project_export_state

Generic heuristics:

  • project_list_heuristics

  • project_run_heuristic

For machine-readable discovery, prefer:

  • project_tool_catalog

  • project_get_defaults

Current heuristic profiles available through project_run_heuristic:

  • pain_structure

  • naive_bias

  • price_distribution

  • liquidity_signals

  • price_liquidity_matrix

  • cross_category

  • sale_format

  • speed_signals

  • trust_signals

If Cursor MCP auto-discovery is enabled, restart Cursor and connect mini-data-engine. Default MCP runtime data paths are under tests/artifacts/mcp/*.

Cursor approval setup (reduce repeated prompts)

If Cursor keeps asking for MCP or command approval on every call, apply this once:

  1. Enable workspace trust in Cursor user settings:

"security.workspace.trust.enabled": true
  1. In Cursor, open Settings -> Agents -> Auto-Run and set:

    • Auto-run mode: Run in Sandbox

    • MCP Allowlist: add mini-data-engine tools you use often

    • Command Allowlist: add frequently used safe commands

  2. Keep this repo opened as the same trusted workspace and reload the window once.

Notes:

  • MCP server approval and per-tool allowlist behavior are enforced by Cursor security settings.

  • In some Cursor versions, allowlist behavior can be best-effort and still prompt in edge cases.

Fastest way to see the new explainability use case in action through MCP:

demo_explain_run

That single tool call creates a traced run, records step-level events under one run_id, and returns an explanation with:

  • ordered timeline

  • tool path

  • total elapsed time

  • failure summary if anything breaks

You can then replay the same explanation directly with:

explain_run(run_id="...")

Explain Regression Suite

Use explain_regression_suite when you want regression checks to run through MCP and come back as explainable run summaries instead of isolated test output.

The suite drives the current validation surface through the MCP layer, attaches run_id traces, and returns explain output for each check so regressions can be inspected with the same mechanism used for runtime incidents.

It currently runs:

  • Python unit tests via python -m unittest discover

  • Rust tests via cargo test, including the current engine_cli integration tests

  • health_check

  • benchmark_calls

  • scenario_load_test

  • explainability control demos

The explainability demos intentionally include both positive and negative controls:

  • demo_explain_run as expected_success

  • demo_explain_run_failure as expected_failure

  • demo_explain_semantic_failure as expected_failure

  • demo_explain_idempotency_conflict as expected_failure

  • demo_explain_concurrency_failure_storm as expected_failure

That means the suite is not only checking that the happy path stays green. It also checks that the explain layer still classifies and summarizes known failure classes correctly.

The current regression surface covers:

  • happy-path traced execution

  • runtime/path failures

  • semantic data validation failures

  • idempotency conflict failures

  • concurrency and failure-storm control scenarios

  • sampled benchmark and scenario SLO regressions

Fastest MCP call for the full regression bundle:

explain_regression_suite

Use it as the top-level MCP regression entrypoint when you want one answer that includes:

  • which checks passed

  • which failures were expected controls

  • explain summaries for each traced run

  • early signals that a latency or behavior regression appeared

The MCP layer is an access interface, not the core product idea. The core of the repository is the runnable lab itself.

Product note:

Use in Codex:

Run persistent engine CLI (productization path):

# Initialize storage
cargo run --bin engine_cli -- init ./tests/artifacts/engine/data orders

# Insert and upsert (WAL append)
cargo run --bin engine_cli -- insert ./tests/artifacts/engine/data orders 1 4242 50
cargo run --bin engine_cli -- upsert ./tests/artifacts/engine/data orders 1 4242 55

# Build index and explain
cargo run --bin engine_cli -- index ./tests/artifacts/engine/data orders
cargo run --bin engine_cli -- explain ./tests/artifacts/engine/data orders 4242

# Write snapshot and truncate WAL
cargo run --bin engine_cli -- checkpoint ./tests/artifacts/engine/data orders

# Transaction simulation: begin/commit/rollback semantics,
# per-table write lock, snapshot read, and conflict detection
cargo run --bin engine_cli -- tx-demo ./tests/artifacts/engine/data orders

# Crash/restart recovery for transaction journals
cargo run --bin engine_cli -- tx-recovery-list ./tests/artifacts/engine/data orders
cargo run --bin engine_cli -- tx-recovery-commit ./tests/artifacts/engine/data orders <tx_id>
cargo run --bin engine_cli -- tx-recovery-rollback ./tests/artifacts/engine/data orders <tx_id>

What You Should See

  • In mini_pg_like.py: selective predicate switches to Index Scan; non-selective stays Seq Scan.

  • In mini_databricks_clone.py: layer-by-layer demo output, workflow DAG order/metrics, and canonical events count from the single write path.

  • In mini_pg_like (Rust): same planner behavior with shared core modules.

  • In mini_databricks_clone (Rust): same layered demo using shared Rust library code.

  • In engine_cli (Rust): persistent snapshot + WAL replay flow with simple operational commands.

  • In engine_cli tx-demo: explicit transaction scopes, snapshot reads, per-table write lock, and concurrent upsert conflict detection.

  • In engine_cli tx-recovery-*: staged transaction operations survive process restarts via per-transaction journal files and can be committed or rolled back explicitly.

  • In e2e_flow: one command runs write path, checkpoint, bronze->silver transform, planner explain, and DuckDB SQL validation on persisted data.

Technical Design Backbone

TECHNICAL_DESIGN_GENERIC.md captures the architectural discipline behind the code:

  • cross-layer reasoning (Idea -> API -> Runtime -> Storage -> Perf),

  • deterministic state transitions,

  • event-first design,

  • adapter contracts,

  • DAG-driven orchestration,

  • measurable go/no-go criteria.

It is not a separate product claim. It is the review and implementation spine used across the lab.

Docker package

Use the local build (recommended for development)

Build the image from this repo so MCP uses your local code (including schema tools) instead of the GitHub image:

./scripts/docker-build-local.sh

This builds mini-data-engine:local. To drive another project (e.g. threads) with this MCP, set Cursor MCP to use the local image and mount that project as workspace:

  • Copy .cursor/mcp.docker.local.json into your project’s .cursor/mcp.json (or merge the mcpServers entry into your Cursor user config).

  • Open the project you want to drive (e.g. threads). ${workspaceFolder} will be that project; the container gets WORKSPACE_ROOT=/workspace and your project mounted at /workspace, so e.g. schema_path="schema.sql" resolves to that project’s file.

Example local config (uses mini-data-engine:local and mounts current workspace as /workspace):

{
  "mcpServers": {
    "mini-data-engine": {
      "command": "docker",
      "args": [
        "run", "--rm", "-i",
        "-e", "WORKSPACE_ROOT=/workspace",
        "-v", "${workspaceFolder}:/workspace",
        "-v", "${workspaceFolder}/tests/artifacts:/app/tests/artifacts",
        "mini-data-engine:local"
      ]
    }
  }
}

Use the published image (GHCR)

Image is published to GHCR:

  • ghcr.io/kroq86/data-engineering-runtime-lab:latest

Pull:

docker pull ghcr.io/kroq86/data-engineering-runtime-lab:latest

Use in Cursor MCP config (example):

{
  "mcpServers": {
    "mini-data-engine": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "-v",
        "${workspaceFolder}/tests/artifacts:/app/tests/artifacts",
        "ghcr.io/kroq86/data-engineering-runtime-lab:latest"
      ]
    }
  }
}

Loom stack

MCP surface for loom-ops and ops runbooks. Ecosystem: ECOSYSTEM.md

pip install ops-runtime-mcp
ops-runtime-mcp   # stdio MCP; see docs/use-in-codex.md

Available Tools

47 tools
benchmark_callsC

Benchmark MCP operations with SLO-style summary metrics.

Inputs: iterations, root_dir, table, min_success_rate, +1 more. Returns: object

capture_roi_baselineC

Capture baseline KPI snapshot for ROI Phase 0.

Inputs: root_dir, table, benchmark_iterations, scenario_iterations, +1 more. Returns: object

create_indexC

Create customer index for engine table.

Inputs: root_dir, table. Returns: object

decision_gateC

Evaluate migration triggers and return pass/fail gate.

Inputs: trace_db_path, baseline_path, drift_counter_path, need_rust_portfolio, +3 more. Returns: object

demo_explain_concurrency_failure_stormC

Run a traced concurrency conflict plus failure-storm scenario.

Inputs: root_dir, table, trace_db_path, workers. Returns: object

demo_explain_idempotency_conflictB

Run a traced idempotency-conflict flow through WriteCore.

Inputs: trace_db_path. Returns: object

demo_explain_runC

Run a traced demo flow, then explain the run immediately.

Inputs: root_dir, table, customer_id, trace_db_path. Returns: object

demo_explain_run_failureC

Run a traced failing flow, then explain the failed run immediately.

Inputs: root_dir, table, trace_db_path. Returns: object

demo_explain_semantic_failureC

Run a traced semantic-corruption flow and explain the failed validation.

Inputs: root_dir, table, customer_id, trace_db_path. Returns: object

explain_customerC

Run EXPLAIN ANALYZE style output by customer filter.

Inputs: root_dir, table, customer_id. Returns: object

explain_regression_suiteC

Run regression checks and return explain output for each traced run.

Inputs: root_prefix, trace_db_path, benchmark_iterations, scenario_iterations. Returns: object

explain_runC

Explain one recorded run_id from the local trace store.

Inputs: run_id (required), trace_db_path, max_timeline_events. Returns: object

health_checkC

Run a quick MCP smoke flow and summarize status.

Inputs: root_dir, table. Returns: object

init_engineD

Initialize persistent engine storage.

Inputs: root_dir, table. Returns: object

insert_rowC

Insert one row into the persistent engine.

Inputs: root_dir, table, order_id, customer_id, +1 more. Returns: object

memory_upsertC

Upsert one operational memory entry for semantic recall.

Inputs: run_id (required), summary (required), status, tool_name, +9 more. Returns: object

project_append_eventC

Append a generic project event to the local event log.

Inputs: event_type, entity_type, entity_id, payload_json, +3 more. Returns: object

project_capabilitiesB

Return declared runtime capabilities and contract coverage.

Inputs: no named parameters. Returns: object

project_capture_baselineC

Capture a baseline snapshot and return a unified verdict envelope.

Inputs: root_dir, table, benchmark_iterations, scenario_iterations, +1 more. Returns: object

project_compare_baselineC

Compare current benchmark/scenario results against a stored baseline.

Inputs: baseline_path, root_dir, table, benchmark_iterations, +4 more. Returns: object

project_delete_entityC

Delete a declared entity from the generic state store.

Inputs: entity_type, entity_id, root_dir, dry_run. Returns: object

project_explain_runC

Read one run explanation through the generic project explain entrypoint.

Inputs: run_id (required), trace_db_path, max_timeline_events. Returns: object

project_export_stateC

Export generic project state as a JSON snapshot.

Inputs: root_dir, entity_type, output_path, include_events. Returns: object

project_get_defaultsA

Return default workspace, paths, runtime mode, and project metadata.

Inputs: no named parameters. Returns: object

project_get_entityC

Load one declared project entity by identity key.

Inputs: entity_type, entity_id, root_dir. Returns: object

project_ingest_traceC

Append one normalized trace record through the generic project ingest path.

Inputs: run_id (required), tool_name (required), status (required), summary (required), +10 more. Returns: object

project_list_entitiesC

List declared project entities from the generic state store.

Inputs: entity_type, root_dir, limit. Returns: object

project_list_heuristicsA

List declared heuristic profiles available for generic project analysis.

Inputs: no named parameters. Returns: object

project_manifestB

Describe project state roots, schemas, and supported regression primitives.

Inputs: no named parameters. Returns: object

project_run_heuristicC

Run one declared heuristic profile over a source and persist the analysis through project state.

Inputs: heuristic_name (required), source_path (required), source_kind, root_dir, +3 more. Returns: object

project_run_regressionD

Run the explain-first regression bundle and return a unified verdict.

Inputs: root_prefix, trace_db_path, benchmark_iterations, scenario_iterations. Returns: object

project_tool_catalogB

Return the full MCP tool catalog with groups, entrypoints, and summaries.

Inputs: group. Returns: object

project_upsert_entityC

Create or update a declared entity in the generic state store.

Inputs: entity_type, entity_id, payload_json, root_dir, +2 more. Returns: object

record_tool_traceC

Append one MCP tool trace record to local trace store.

Inputs: run_id (required), tool_name (required), status (required), summary (required), +9 more. Returns: object

refresh_docs_pathC

Incrementally ingest project docs, code, and config files.

Inputs: source_dir, trace_db_path, refresh_state_path, scenario_id, +4 more. Returns: object

refresh_trace_pathC

Incrementally ingest new lines from source path into trace store.

Inputs: source_path (required), trace_db_path, refresh_state_path, scenario_id. Returns: object

reindex_projectC

Re-index this project dataset (engine_cli index).

Inputs: root_dir, table. Returns: object

report_drift_bugC

Increment and persist split-logic drift bug counter.

Inputs: note, counter_path. Returns: object

run_e2e_flowC

Execute full MiniPG + MiniDatabricks + DuckDB end-to-end flow.

Inputs: root_dir. Returns: object

scenario_load_testD

Mixed workload load test: insert/upsert/explain/reindex/e2e and summary metrics.

Inputs: iterations, root_dir, table, min_success_rate, +2 more. Returns: object

schema_evaluate_full_toolC

One-shot: load schema → run EXPLAIN (from query_profiles_json/seed_sql_json) → evaluate. Generic: you supply queries and optional seed SQL.

Inputs: schema_path, ddl_text, schema_entity_id, query_profiles_json, +4 more. Returns: object

schema_evaluate_toolB

Build verdict from schema metadata and EXPLAIN outputs; write evaluation_report.json and verdict.md. If query_profile_names empty, discovers explain_*.txt in artifacts.

Inputs: schema_entity_id, query_profile_names, root_dir, artifacts_dir. Returns: object

schema_explain_toolB

Run EXPLAIN for each profile in query_profiles_json (JSON: name -> SQL). Optional seed_sql_json (JSON array of SQL) runs before EXPLAIN. Writes explain_.txt.

Inputs: schema_entity_id, query_profiles_json, seed_sql_json, root_dir, +1 more. Returns: object

schema_load_toolA

Ingest DDL from file or raw text, validate with DuckDB, store metadata in project state (schemas/).

Inputs: schema_path, ddl_text, schema_entity_id, root_dir. Returns: object

similar_incidentsC

Find semantically similar historical incidents.

Inputs: query_text (required), top_k, min_score, status, +5 more. Returns: object

upsert_rowC

Upsert one row by order_id.

Inputs: root_dir, table, order_id, customer_id, +1 more. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    C2.1/5.0

    Scored across 47 tools

    Disambiguation1/5

    Many tools have overlapping purposes, particularly the multiple 'explain' and 'project_*' tools are hard to distinguish. The demo, benchmark, and schema tools also have unclear boundaries, making selection confusing for an agent.

    Naming Consistency2/5

    Naming conventions are inconsistent: some use underscore (benchmark_calls), some use project_ prefix, others schema_ prefix, and demo_explain_*. While groups have internal consistency, the overall pattern is mixed and unpredictable.

    Tool Count2/5

    47 tools is excessive for a single server. Many tools could be consolidated or split into separate servers. The count feels bloated and hard to navigate.

    Completeness2/5

    The tool set appears to cover many areas (benchmarking, demo, schema management, project state) but lacks clear CRUD coverage for key entities and has dead ends (e.g., no delete for most entities). Gaps in lifecycle operations are notable.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      B
      quality
      C
      maintenance
      A read-only MCP server that exposes dbt project artifacts and data quality result tables (BigQuery/Postgres) to LLM clients, enabling deep introspection, run-history analysis, source freshness, test coverage, and lineage walks.
      27
      42 npm
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to query live schema, lineage, and query-context across data warehouses, dbt projects, orchestration systems, and BI tools via MCP tools.
      Apache 2.0
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables MCP clients to ask bilingual data questions, inspect database tables and schemas, and execute validated read-only SQL through exposed tools.
      MIT