Skip to main content
Glama

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
EVOKIT_WORKSPACENoOptional workspace directory. Defaults to the directory where DSH starts.
EVOKIT_SKILL_ROOTNoOptional skill root directory. Defaults to the bundle's own `skills` folder.
EVOKIT_MCP_COMMANDNoOptional command used to run the MCP server. Defaults to `evokit` on PATH.

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

evokit runs measured self-improvement experiments on one scenario: a YAML file naming editable files, tasks and a frozen scorer. Every tool takes config, the scenario YAML path on this host; the first call on a new scenario creates its workspace and freezes the task splits. A version is a SHA-256 snapshot of the editable files; an arm (run) scores one version on the diagnosis or selection split. Typical loop: scenario_describe and version_current; run_launch the base on diagnosis and read failure_cards; edit and version_create; proposal_create; run the base with repeats >= 2 on selection and call noise_estimate; run parent and candidate (run_interleaved); compare_paired; gate_check. Runs execute scenario tasks and may cost money; every other tool is local. Records are append-only. Adoption, confirmation, reverts and final measurement belong to the separate operator profile.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}

Tools

Functions exposed to the LLM to take actions

NameDescription
scenario_describeA

Describe the bound scenario: epoch id, editable file globs, frozen scorer version, task counts per split, primer, budgets, replay support and readiness flags. Call it first to learn what may be edited and how large each split is; use version_current for the current version id and ledger_history for lineage. Read-only; never reveals confirmation or final task ids. Returns one JSON object.

version_currentA

Return the id of the current base version: the latest adopted or reverted-to snapshot, or the initial snapshot before any adoption. Use it to get the parent for version_create, run_launch and noise_estimate; use ledger_history for the whole base chain. Read-only. Returns {"version": ""}.

version_createA

Store a candidate version of the editable files, either by snapshotting a complete scenario checkout or by applying a unified patch to a parent version. Use it after editing and before proposal_create and run_launch; use version_diff to review a stored change. Writes content-addressed snapshot files and appends a ledger event; it never adopts or changes the current base. Rejects edits outside the editable globs, unsafe paths and patches that do not apply. Returns {version, parent, changed}; identical content always gets the same version id.

version_diffA

Show the unified text diff of editable files between two stored versions, after verifying both snapshots' content digests. Use it to review a candidate before proposing or gating; use compare_paired to compare measured outcomes instead. Read-only. Returns {parent, child, changed, diff}; binary files are named, not diffed.

run_launchA

Start or resume one arm: a run of one version over the tasks of the diagnosis or selection split, scored by the scenario. Use it to measure a single version; use run_interleaved to run a parent and candidates side by side. Detached by default: it returns at once while a background worker runs, so poll run_status, then read run_results. Writes run records, results and traces, and costs whatever the scenario's tasks cost (for example model calls). Resuming a run_id with the same parameters only fills missing task slots; changed parameters are refused. Returns the run status object.

run_interleavedA

Run two or more versions as matched arms in rotating task blocks, so each arm sees every task at nearly the same time. Use it for parent-versus-candidate measurements when latency or cost may drift; use run_launch for one arm or a detached run. Blocks until all arms finish. Writes run records, results and traces, and costs whatever the scenario's tasks cost. Repeating the call with identical arms and parameters resumes and skips committed slots; a new plan needs fresh run ids. Returns each arm's run status, keyed by run id.

run_statusA

Report one arm's progress: state (pending, running, completed or interrupted), completed and expected result counts, and execution errors. Use it to poll a detached run_launch; use run_results for the task rows. Read-only. Returns {run_id, version, split, state, completed, expected, errors}.

run_resultsA

List one arm's committed task results: task_id, group, repeat, outcome, reason, cost, seconds, output and archived trace path. Use it for raw evidence; use failure_cards for failures only and compare_paired for parent-versus-child deltas. Read-only; partial while the arm is running. Returns an array in stable order.

compare_pairedA

Compare two complete arms task by task: fixes, breaks, net score change, per-task deltas, per-group tables, and cost and latency ratios. Use it to judge a candidate against its parent; use gate_check to record a decision with noise and budget checks. Both arms must be complete, error-free and match on tasks, split, repeats and anchors. Read-only. Returns the comparison object.

noise_estimateA

Measure the current base's run-to-run noise from one repeated arm and record the estimate that gate_check requires. Use it once per base, on a complete, non-smoke selection arm of the current version with at least two repeats; use compare_paired for candidate effects. Appends a noise record to the ledger. Returns noise_id, per-task flip rates, net spread, threshold and method; pass noise_id to gate_check.

failure_cardsA

List failure cards for one arm: every task with outcome below 1, or, with parent_run and change, the tasks this arm fixed or broke against the parent. Use it to diagnose before proposing; use run_results for all rows, and failures_similar (HB workspaces) to search failures across runs. A card has task_id, group, outcome, reason, payload summary, output and trace pointer unless the scenario renders its own. Also saves the cards to the record store for later search, overwriting identical entries; nothing else changes. Returns an array.

proposal_createA

Record a falsifiable proposal for a candidate version before testing it: targets, mechanism, predicted effect, falsification criterion and prior attempt. Use it after version_create and before running and gating the candidate; use proposal_list to read earlier proposals. Checks that targets are editable and cover every changed file, pins the current base as parent and snapshots the improver playbook. Appends to the ledger; runs and adopts nothing. Returns the stored proposal with its new id.

proposal_listA

List every recorded proposal in creation order, rejected ones included, with id, parent, version, prediction, playbook version and a not_confirmed flag. Use it to find a proposal_id for gate_check or a prior attempt; use ledger_history for adoptions and the base chain. Read-only. Returns an array.

gate_checkA

Judge whether a proposal's candidate may be adopted and record the verdict. It checks frozen anchors, that the proposal's parent is still the current base, an editable-only diff, a scan for task-specific literals, complete non-smoke selection arms of parent and candidate, cost and latency budgets, scenario quality guards and the recorded noise threshold. Use it after compare_paired looks promising; use ledger_adopt (operator profile) to act on a passing verdict, because this tool never changes the current base. Appends one verdict record per call. Returns {id, verdict, reasons, threshold, comparison, secondary_guards} with verdict promote (net above noise), adopt-no-harm (net >= 0 with a rationale) or reject.

ledger_historyA

Read the public lineage: current base, base chain (initial version plus every adoption and revert), queued runner-ups and the append-only public event list. Use it to audit what was adopted and when; use meta_metrics for aggregate rates. Read-only; private confirmation and final events are hidden. Returns {current, base_chain, runner_ups, events}.

meta_metricsA

Report aggregate improvement metrics: proposal and adoption counts, acceptance rate, mean absolute prediction error, gain per cost unit, total experiment cost and confirmation measurement counts. Use it to judge how well proposals predict and pay off; use ledger_history for individual events. Read-only. Returns one JSON object.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

TDQS

A4.4/5.0

Scored across 16 tools

Disambiguation5/5

Each tool targets a distinct resource+action pair, and descriptions explicitly disambiguate near-neighbors (run_launch vs run_interleaved, run_status vs run_results, compare_paired vs noise_estimate vs gate_check, ledger_history vs meta_metrics). The 'use X for… use Y for…' cross-references make selection reliable even in the crowded run/comparison area.

Naming Consistency4/5

Names follow a predictable resource-prefix scheme (version_*, run_*, proposal_*, ledger_*, gate_*, compare_*), which is easy to scan and group. A few singular tools (compare_paired, gate_check, meta_metrics, scenario_describe) break the strict prefix family, but the overall convention remains readable and stable.

Tool Count4/5

At 16 tools the surface is on the heavy side, but the domain genuinely spans scenario inspection, versioning, execution, analysis, proposal/gating and ledger auditing, and each tool maps to a needed step. Nothing looks redundant or like filler.

Completeness4/5

The lifecycle from edit → version_create → run → compare → noise → proposal → gate is well covered, with read-only audit and metrics views. Two gaps: gate_check points to a ledger_adopt tool that is not in this set (adoption is unreachable here), and there is no cancel/abort for a detached run or a general version listing beyond ledger_history.

Maintenance

ActivityMaintained
ResponsivenessOutdated data