super-loop-mcp
Super Loop MCP is a strict AI-agent supervisor that orchestrates self-improving loops — mining past sessions, running phase-gated workflows, enforcing tool-measured benchmarks, and promoting only verified, benchmark-beating changes. The operator is the sole stop condition; the supervisor never self-declares done.
Loop Management
loop_start/request_next_phase/loop_next— stream loops one phase at a time, requiring evidence before advancingloop_register— add custom hash-locked, phase-gated loops or skills; can never overwrite mandated built-in loopsloop_library/skill_fetch— browse all loops/skills or retrieve specific sections by mode
Campaign Initialization & Control
initialize_loop_run— ask-once gate (goal, path, corpus scope, success criteria, hard limit); never asks againrun_campaign— fully autonomous mine→improve→test→reverify→promote cycle until you create a stop-file (requiresSUPER_LOOP_ALLOW_EXEC=1)campaign_status— read-only view of lanes, queues, retirement counts, and risk advisorycontinue_run— logs next lane and first action; obligation persists until real progress runsreport_saturation— auto-pivots lanes (e.g., Strip Miner → Loop-de-loop) without pausingcycle_decision_request— propose supervisor-accepted transitions; stop/completion intents are refused
Benchmarking & Measurement
benchmark_propose— define scorecards with ≥1 value dimension, ≥1 cost dimension, ≥1 concrete casebenchmark_select— freeze an immutable scorecard for the cyclebenchmark_freeze_maker— operator-controlled held-out benchmark, bypasses worker proposalsbenchmark_run— set a tool-measured baseline bar from a recorded artifact; self-reported metrics refused
Artifacts & Observations
artifact_record— persist raw artifacts with sha256 hashes; baseline role is write-onceobservation_record— log lightweight phase evidence
Hypothesis Testing & Promotion
register_hypotheses— register 3–5 frontier challenger hypotheses (haiku/mini/nano/lite/prior-gen routes banned)test_hypothesis— record one full test of 3–5 frontier agents with measurementRefs, aggregated vs. frozen baselineexecute_full_test— supervisor launches frontier workers via execFile (no shell), captures output directly (requiresSUPER_LOOP_ALLOW_EXEC=1)reverify_run— re-hashes every raw artifact to confirm metrics reproduce; promotion blocked until this passespromotion_request— promotes only if tool-measured, reverified, and frontier-moving; never overwrites your canonical loop file
Human Review & Reporting
human_review_request— queue items for Approve/Sludge; model cannot resolve them, dashboard-onlyupdate_dashboard— renders local dashboard with score matrix, phase progress, and stop-condition noticereport_export— reproducible markdown report with baseline lock, frozen benchmark, score matrix, and promotionsexport_trajectories— Hermes-format JSONL of tool actions with supervisor verdict labels (refused for held-out runs)
Host & Environment Detection
host_capability_preflight— filesystem stat only (no execution) to check which frontier CLIs are installed; returns resolved host profilehost_runtime_detect— advisory guess of host runtime from MCP config files;SUPER_LOOP_HOSTenv var is authoritative
Integration with Hermes for self-improving agent campaigns, using its internal loop mechanism to call tools each turn.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@super-loop-mcpMine past sessions and improve workflows until I stop"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Loop Factory
Make AI agents prove they got better.
Loop Factory is a local MCP server and autonomous CLI that continuously finds workflows, proposes improvements, tests the original against the challenger, and sends measured winners to an approval desk.
It is a deterministic slop terminator: models can propose anything, but claims, self-reported scores, and confident summaries do not count as progress.

Related MCP server: Self-Learning MCP
What It Does
mine a real workflow
|
freeze the original and benchmark
|
generate competing revisions
|
test original, challenger, and controls
|
re-open saved outputs and recalculate the result
|
queue measured winners for Approve or Deny
|
continue to the next target until the operator stops itPending reviews do not pause the factory. A model cannot promote itself. The operator is the only stop condition.
Recursive Harness V2
The current build can also improve the mechanism it uses to propose later improvements. It does this conservatively:
one model proposes at most three typed changes to a deterministic mechanism;
five tasks calibrate real candidate movement against an irrelevant sham;
only a qualified calibration unlocks five untouched confirmation tasks;
three replicates per arm produce 60 calls per stage;
the verifier reopens every saved prompt, receipt, candidate, and evaluation;
a passing descendant enters bounded routing, never canonical promotion; and
the next generation receives an immutable receipt explaining the measured mechanism, result, placebo movement, regressions, and evidence hashes.
Context allocation can move only 10% after five valid token observations. Mechanism records are never deleted or summarized for savings; an oversized prompt may use a hash-bound, lossless index while full bytes remain on disk. Related, adjacent, failure-derived, wildcard, and permanent no-memory control slots remain deterministic and replayable.
This implementation is locally tested. It is not presented as scientifically proven until a separately approved live campaign passes the same verifier on real disjoint tasks. See Recursive Harness V2.
What The Final Run Proved
The final Build Week run found a real workflow for deciding whether a revision should move forward, then tested two meaningful rewrites from scratch.
Procedure | Quality | Mean CLI tokens | Quality change | Cost change |
Original |
|
| baseline | baseline |
H1: clear acceptance rules |
|
|
|
|
H2: evidence and recheck rules |
|
|
|
|
The run used:
12real model calls;exact
gpt-5.6-solat high reasoning;zero retries and zero exit failures;
12isolated workspaces;724,453CLI-reported tokens;34/34saved-file hash checks; anda separate verifier that returned
PASS.
Both revisions remain pending in the approval desk. No promotion was recorded. H1 is the measured recommendation, not an automatic decision.
Read the production report or compare the original with the recommended revision.
Verify It In 30 Seconds
Requirements: Node.js 18 or newer.
git clone https://github.com/alexalexalex222/Loop-Factory-mcp-public.git
cd Loop-Factory-mcp-public
npm run verify:submissionExpected top-level result:
{
"status": "PASS"
}This command makes no model call. It:
re-derives a public 16-call controlled proof run from saved transcripts;
checks five paired challenger wins, zero wins by the irrelevant-edit control, and zero regressions;
verifies model selection, retries, isolated workspaces, output formats, token counts, and file hashes; and
integrity-checks the final production evidence.
It exits nonzero if any gate fails.
Current Model Policy
The default policy now uses the current routes exercised by this project:
Role | Default |
Primary worker |
|
Test routes |
|
Drafting and gate checks |
|
Independent judge |
|
Claude workers are launched with an explicit --model flag. Codex workers are
launched with an explicit -m flag. The operator may replace the full policy
when starting a run. Strict proof runs can lock every call to one
exact model, as the final GPT-5.6 Sol production run did.
Optional Improvement Memory
Loop Factory records deterministic improvement receipts and can use them in two
off-by-default modes. shadow writes an auditable ranking packet without
changing execution. active-canary routes only reverified, control-complete
harvest evidence before hypothesis generation, preserves a permanent no-memory
control, compiles executable mechanisms before registration, and binds every
affected hypothesis to the exact route, policy, capsule, treatment, and
interface hashes.
Active routing is not permission to call a result an improvement. Automatic banking remains closed until the supervisor receives sealed paired baseline/routed/sham evidence with zero sham movement, zero control regressions, complete transfer evidence, and every existing promotion gate. The operator API accepts persisted measurement references, never caller-supplied quality numbers. A restart resumes registered pending hypotheses without rerunning the frozen baseline; unused routes require an immutable operator retirement receipt.
The operator can also import a persisted V4 executable-canary pass. The import reruns the independent verifier, pairs only the shared confirmation tasks, and stores a routing-only receipt; it never updates policy, promotion, or canonical loop bytes. Automatic import additionally requires a predeclared sealed config flag. When that flag is present, the executable-canary CLI performs the import before reporting closed-loop success. A causal PASS whose verifier-owned import fails exits nonzero; a valid causal FAIL remains evidence and imports nothing.
The autonomous campaign CLI also persists a private, hash-chained parent scheduler ledger. Queue state, the active target and deterministic child run ID, coverage, counters, deduplication sets, promotion state, and idle/mining epochs survive a cold process restart. The exact campaign config is hash-bound to every checkpoint, completed child receipts are reopened without another worker call, and config or ledger drift fails closed.
The feature is off by default. Enable it in initialize_loop_run:
{
"config": {
"metaLearning": {
"enabled": true,
"mode": "shadow",
"policyId": "meta-policy-v1",
"seed": "run-bound-safe-id"
}
}
}Autonomous campaign configs place the same object under
engineConfig.metaLearning. See
the improvement memory contract for receipt,
partition, fallback, privacy, and claim boundaries, and
the adaptive intelligence contract for the
active-canary safety boundary.
Run The Factory
The autonomous driver is opt-in because it launches real model workers:
SUPER_LOOP_ALLOW_EXEC=1 npm run run-campaign -- \
--config examples/campaign.json \
--stop-file ./STOP \
--dashboard-port 8787Open http://127.0.0.1:8787 for the campaign and approval dashboard.
Create the stop file when you want the factory to stop:
touch STOPThe example campaign uses GPT-5.6 Sol, Fable 5, and GPT-5.6 Terra. Edit the config before running if you want a narrower policy or a different target.
Use It As An MCP Server
Point an MCP-capable host at src/server.mjs:
{
"mcpServers": {
"loop-factory": {
"command": "node",
"args": ["/absolute/path/to/Loop-Factory-mcp-public/src/server.mjs"],
"env": {
"SUPER_LOOP_HOST": "codex"
}
}
}
}Start with initialize_loop_run. Loop Factory asks for the goal, starting
path, benchmark, limits, and model policy once, then persists the campaign.
What It Refuses To Trust
A worker saying its own revision is better.
A score typed by a model instead of derived by the tool.
A challenger tested against a conveniently weak baseline.
A result without both the original output and the parsed result.
A requested model that differs from the sealed plan.
A measured win that has not been checked again from saved evidence.
A model attempting to approve or promote its own work.
Useful Commands
Command | Purpose |
| Re-run the public judge proof with no model call |
| Run the complete test suite |
| Verify the bundled loop hashes |
| Generate a deterministic local demo |
| Start the autonomous factory |
| Recompute a persisted production run |
| Run the optional exact-model enforcement proof |
Evidence
Honest Boundaries
The production evidence publishes privacy-safe results and hashes, not raw provider transcripts containing machine-specific paths.
The public controlled proof run includes transcript-backed evidence and is the fully portable no-model verification path.
The production run proved one mined workflow improved under its frozen benchmark. It does not prove every possible workflow will improve.
Model availability depends on the operator's authenticated Claude Code and Codex installations.
Loop Factory records a winner only after operator approval. It does not overwrite canonical user files.
SUPER_LOOP_HOME is always authoritative. Existing <package>/.super-loop
state remains discoverable and is never moved automatically. A source checkout
keeps that historical default; a fresh packed installation uses the writable
per-user location listed below. The selected path and its source are printed at
server startup.
Platform | Fresh installed state path |
macOS |
|
Linux |
|
Windows |
|
The autonomous driver is opt-in. Without the exact value
SUPER_LOOP_ALLOW_EXEC=1, Loop Factory does not launch workers.
POSIX shell:
SUPER_LOOP_ALLOW_EXEC=1 node scripts/run-campaign.mjs --config examples/campaign.json --stop-file ./STOP
touch ./STOPPowerShell:
$env:SUPER_LOOP_ALLOW_EXEC = "1"
node scripts/run-campaign.mjs --config examples/campaign.json --stop-file .\STOP
New-Item -ItemType File .\STOPWindows Command Prompt:
set "SUPER_LOOP_ALLOW_EXEC=1"
node scripts\run-campaign.mjs --config examples\campaign.json --stop-file .\STOP
type nul > .\STOPStart the local dashboard with node scripts/dashboard-server.mjs; pass
--home "<path with spaces>" when selecting state explicitly. Run
npm run package:smoke to pack, install into a clean path containing spaces,
handshake the installed MCP, verify all tools and loop hashes, and round-trip
isolated state.
On Windows, JSON configuration paths need escaped backslashes (or forward slashes), for example:
{
"command": "node",
"args": ["C:\\Users\\Ace\\Loop Factory\\src\\server.mjs"],
"env": { "SUPER_LOOP_HOST": "codex" }
}Core platform evidence
CI PASS means the flow passes the public repository's
Portability workflow
on GitHub-hosted runners. Provider authentication is a separate boundary.
OS | Core package/install | MCP stdio | Persistence | Dashboard | Autonomous supervisor | Fake executor | Authenticated Claude CLI | Authenticated Codex CLI | Authenticated OpenCode routes |
macOS | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | not retested here | one live executor audit | not verified |
Ubuntu | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | not verified | not verified | not verified |
Windows | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | CI PASS | not verified | not verified | not verified |
The workflow also checks Node 18, 22, and 24 on Ubuntu, matching the existing
node >=18 package declaration. Profiles in
hosts/registry.json remain labeled independently from
core OS support.
Optional live path
Judges with GPT-5.6 Sol access can run:
npm run judge:gpt56-solThis pins gpt-5.6-sol, refuses fallback, and runs three controlled enforcement
fixtures: PHASE_SKIP, MODEL_REPORTED_METRIC, and SELF_PROMOTION. Those are
adversarial regression prompts, not claims of spontaneous model behavior.
What changed on July 18, 2026
Build Week added the exact GPT-5.6 Sol route and invocation receipts, the one-command live judge kit, the Campaign Console, and the controlled adversarial fixtures above. The July 19 causal canary added paired baseline/challenger/sham evidence and fresh-checkout re-verification. The July 20 production frontier run then demonstrated the complete mine-to-measure-to-review workflow on a real mined procedure without weakening operator-owned promotion.
Evidence index
Public causal canary:
submission/evidence/context-isolation-canary-20260719/Production frontier packet:
submission/evidence/production-frontier-20260720/Deterministic verifier:
scripts/verify-submission.mjsTamper regression:
test/submission-verifier.test.mjsJudge guide:
docs/JUDGE_GUIDE.mdControlled enforcement transcript:
proof/build-week/gpt56-sol-live-20260718-final/TRANSCRIPT.mdCampaign Console browser QA:
proof/build-week/campaign-console-20260718-final/qa-summary.json
For developers
Engineering detail: tools, host matrix, trajectory export, layout, and block codes.
Host compatibility
Loop Factory is MCP-first, with the continuous driver chosen per host from hosts/registry.json, and scripts/run-campaign.mjs as the universal fallback.
Host | Driver family | Tier | Continuous driver | Verified |
Codex |
| 1 |
| ✅ |
Claude Code |
| 1 |
| ✅ |
ZCode |
| 1 |
| ⚠︎ |
OpenCode |
| 1 |
| ⚠︎ |
Cursor |
| 2 | none — continuation rules snippet | ⚠︎ |
Kilo Code |
| 2 | none — rules snippet (CLI fork = tier 1 with a goal plugin) | ⚠︎ |
OpenClaw |
| 2 | config ( | ⚠︎ |
Factory Droid |
| 2 | Super Loop runs inside a Mission worker | ⚠︎ |
Hermes |
| 3 | its own loop — call tools each turn | ⚠︎ |
MiniMax Mini-Agent |
| 3 | its own loop, or the CLI fallback | ⚠︎ |
anything else |
| 3 |
| — |
Three tiers (per hosts/registry.json):
Native goal (Codex, Claude Code, ZCode, OpenCode+plugin) — engage
/goalwith an operator-stop objective.MCP + continuation contract (Cursor, Kilo IDE, OpenClaw, Factory Droid) — no reliable continuous slash command; ship the continuation rules snippet.
Internal loop or CLI (Hermes, Mini-Agent,
super-loop-run) — host owns its loop, or drive headless with the CLI.
⚠︎ verified:false entries are modeled from the design — confirm the exact command in your build. host_capability_preflight returns the resolved host profile (tier, driverFamily, setupHint).
Why this exists
Drop a 300+ line loop into a model's context and it may ingest the whole thing, skip the structure, and treat an unverified argument as a test. Loop Factory fixes that with hard mechanics:
Ask-once — starts with a brief explanation plus a few short questions once:
the goal;
the path — improve a loop you already run, discover/find a loop (optionally scouting a public loop library), or mine your whole history (deep);
the loop or domain to start from;
corpus scope — your whole session history or a set number of loops, and best-first vs in-order (asked with an up-front warning that a run can take hours, days, or weeks depending on how deep it mines);
what "better" means (this becomes the frozen benchmark);
any task-specific limit;
which models to use (primary, optional test/builder/judge routes — press enter for defaults, or say
any modelto disable the banlist for this run);and a final deeper-explanation offer, honored in the same response.
You choose the models at init (defaults:
gpt-5.6-solprimary, builders Fable 5 / GPT-5.6 Sol, standard frontier test set). The supervisor still owns measurement, integrity, and promotion — it never asks about promotion mode or benchmark policy. Afterward it does not ask again or mark the campaign complete by itself. A fresh run also carries a cold-start notice: don't resume a prior campaign or assume a path from memory — infer only from this message and the answers (pass arunIdto resume on purpose).Phase-gated streaming — holds the loop inside the MCP and hands you the next section only after the current one has recorded evidence. No 1k-line dump.
Benchmark-first — the baseline is hash-locked and the scorecard is frozen before any challenger. Model self-reported metrics never count.
Hypothesis engine — full tests need 3–5 hypotheses on routes allowed by the run's
modelPolicybanlist. Default banlist rejects haiku/mini/nano/lite/prior-gen (weak models produce noisy campaigns); sayany modelat init to turn the banlist off for that run. One no-improvement run is never "perfect".Promotion gate — promotion requires a tool-measured, deep-reverified result that moves the quality/cost frontier past threshold. Otherwise:
BLOCKED.
Two surfaces share one engine: the reactive MCP (a host calls its tools — the in-conversation hook) and the autonomous driver (super-loop-run CLI / run_campaign tool) that drives the whole campaign itself and only stops on the operator stop-file. The whole point: a model cannot promote, upgrade, or call a loop "perfect" from reasoning alone — every decision is hooked through a tool that demands tool-measured artifacts on disk, and the operator is the only stop condition.
Built fresh, zero dependencies, runs on plain Node ≥18. The full private 345-line Strip Miner and the full private 75-line Loop-de-loop (Loop 2) live inside the supervisor, byte-identical to source and hash-locked, streamed one section at a time.
The bundled loops (hash-locked)
id | file | lines | sha256 | trigger |
|
| 345 |
|
|
|
| 75 |
|
|
These are the local big sources — the operator's full private cross-agent Strip Miner (with the old pause/complete language patched into checkpoint/continue semantics), not the short public miner. The server refuses to start, and the test suite fails, if either file's hash or line count drifts — so the short public miner can never be silently substituted.
Add your own loops (local loop library)
Users add their own loops through a tool, not by hand-editing source:
loop_register { id:"my-loop", title:"My Loop", content:"<full loop text>" } → hash-locked, sectionized, persisted locally
loop_library → lists mandated (hash-locked) + your custom loops
loop_start { loop:"my-loop" } → streams it phase-gated, exactly like the mandated loopsCustom loops are sha256 hash-locked (write-once per version; overwrite:true makes a new version), get a safe id (no path traversal), persist under SUPER_LOOP_HOME/custom-loops/, and cannot collide with or overwrite the mandated Strip Miner / Loop-de-loop. They stream through the same phase gate. Nothing leaves your machine.
Tools (29)
tool | what it enforces |
| autonomous supervisor (opt-in |
| ask-once (brief + a few short Qs: goal, path picker (improve / discover / mine + library scout), the loop/domain, corpus scope + order, what "better" means, a hard limit, which models (enter = defaults, |
| add your own loop to the local MCP: hash-lock, safe id, sectionize, persist locally; never overwrites a mandated loop |
| list mandated (hash-locked) + custom local loops |
| retrieve skill knowledge for the current task — |
| begin phase-gated streaming of any loop (mandated or custom); returns section 0 only |
| next section iff the current one has evidence, else |
| lightweight phase evidence |
| persist a raw artifact + sha256; |
| propose scorecards (≥1 value dim, ≥1 cost dim, ≥1 case, optional deterministic |
| bench-maker only — freeze a scorecard directly with |
| read-only export of recorded tool actions as Hermes JSONL with supervisor verdict labels; refuses gate-partition runs |
| set the tool-computed baseline bar; a caller-reported measurement is rejected |
| Standard mode: 3–5 hypotheses. Strict real-test mode: exactly two substantive, supervisor-ID-bound hypotheses for one finding. Benchmark-first; rejects banned routes and shape-only placeholders. |
| one full test = 3–5 frontier agents, each tool-computed; aggregates vs the bar; reports quality authority |
| opt-in ( |
| re-derive metrics from the sealed raw bytes and confirm they reproduce (a tampered number cannot survive) |
| promote only on measured + reverified frontier movement; a quality win the MCP can't tool-verify routes to the dashboard ( |
| the supervisor hook — a worker proposes a transition packet (promote/advance_phase/change_baseline/change_benchmark/saturate); only a supervisor-accepted transition is progress; completion/stop intents refused |
| mark a lane saturated → supervisor auto-transitions to the next lane (Strip Miner → Loop-de-loop); never pauses/stops |
| read-only lane/target queue, auto-transitions, 30-batch retirement + 10–15 advisory accounting, active |
| records the next lane + first concrete action; it does not clear the obligation until a real progress tool runs |
| queue/list Approve/Sludge items only; model-callable resolve is blocked |
| render the polished always-on local dashboard with the stop-condition notice |
| reproducible markdown campaign report |
| local report of which frontier-agent CLIs are installed on PATH (filesystem stat only, never executes, not SOTA/web research) plus the resolved host profile — |
| advisory guess of which host runtime the agent is in, from which MCP config files exist on disk (per the host registry); read-only existence check — never reads file contents or mutates config; |
Block codes you will see
All 42 codes from src/constants.mjs BLOCK (runtime vocabulary):
NOT_INITIALIZED · UNKNOWN_RUN · NO_ACTIVE_LOOP · NOT_STARTED · PHASE_SKIP · UNKNOWN_LOOP · BASELINE_FIRST · BASELINE_LOCKED · BASELINE_BAR_FIRST · BASELINE_PLACEHOLDER · BASELINE_TOO_SHALLOW · BASELINE_AUTHOR_FORBIDDEN · BENCHMARK_FIRST · BENCHMARK_FROZEN · WEAK_BENCHMARK · HYPOTHESIS_COUNT · BANNED_ROUTE · UNKNOWN_HYPOTHESIS · FULLTEST_AGENTS · MODEL_REPORTED · NO_SCORE_MATRIX · NOT_REVERIFIED · BELOW_THRESHOLD · BELOW_FLOOR · STAGED_TRADEOFF · OPERATOR_IS_STOP · DASHBOARD_ONLY · MEASUREMENT_AUTHORITY · QUALITY_UNVERIFIED · PROMOTION_NEEDS_APPROVAL · PROMOTION_REJECTED · LOOP_EXISTS · LOOP_SOURCE · NO_ACTIVE_LANE (reserved — not currently emitted) · BUILDER_ROUTE · EXEC_DISABLED · EXEC_FAILED · ROUTE_UNSPAWNABLE · MANUAL_PROVENANCE_REQUIRED · INTEGRITY_GATE · TARGET_SATURATED_NEEDS_NEW_TARGET · BAD_INPUT
Live execution + autonomous harness (opt-in)
By default the server never executes commands (audited posture). Set SUPER_LOOP_ALLOW_EXEC=1 to let Loop Factory own benchmark execution end-to-end. Native executables stay on direct, shell-free execFile semantics. An allowlisted Windows .cmd or .bat npm shim alone goes through the dedicated cmd.exe adapter; % expansion syntax is refused before launch. The prompt remains stdin data and never reaches argv or the command string. On a Windows shim timeout, Loop Factory kills the ordinary descendant process tree before returning TIMEOUT; unconfirmed cleanup fails closed. A failed/timed-out/non-allowlisted launch is an invalid batch and does not count toward retirement.
The autonomous driver sits on top of that — the difference between "a supervisor you call" and "a harness that drives itself":
SUPER_LOOP_ALLOW_EXEC=1 node scripts/run-campaign.mjs --config campaign.json --stop-file ./STOPIt runs the whole loop unattended (intake → mine → improve targets → validate every worker → bank Stones → advance/retire → re-mine) and only stops when you create the stop-file. Existing queued targets continue even when reviews are pending. When a mining pass yields no novel candidate, the CLI enters a zero-inference idle state: the dashboard and decision inbox stay live, but no model is called again until new work arrives.
Queue new work atomically at <home>/runs/<runId>/inbox-targets.json:
{
"runId": "your-run-id",
"targets": [
{
"kind": "improve",
"loop": "loop-de-loop",
"baselineContent": "<complete current procedure>"
}
]
}The raw inbox is SHA-256 recorded and archived as applied/rejected/invalid before execution. The same supervisor logic powers the bounded run_campaign MCP call; because an MCP call cannot remain parked forever, that surface returns the nonterminal idle checkpoint for its host to resume.
Workers run on the real CLIs via stdin (claude -p --output-format json, codex exec --json) — the prompt never touches argv (no injection), and the real answer text + token usage are extracted for benchmarking. Benchmark modes: oracle (deterministic → tool-measured, reverified, then queued for mandatory operator Approve — never self-ships) and judge (an independent judge on a trusted builder/gating route from the active modelPolicy — defaults Opus/GLM — scores baseline-vs-challenger real outputs under a rubric → subjective → queues to the dashboard, never auto-promotes; the challenger never scores itself).
Model policy (operator-chosen at init)
Ask-once includes one friendly model question. Press enter / say defaults for today's historical behavior; say any model to set banlist.mode: "off" for that run. The policy is persisted as state.config.modelPolicy and shown on the dashboard + report.
For the Build Week lane, pass modelPreset: "gpt-5.6-sol" to initialize_loop_run or say use the gpt-5.6 sol preset. The preset uses the exact gpt-5.6-sol model ID as the primary and first full-test route while preserving the existing Opus/GLM builder boundary and Opus judge route. See examples/model-policy-gpt-5.6.json.
Field | Default | Notes |
|
| Primary worker route |
| gpt-5.6-sol / claude-fable-5 / gpt-5.6-terra | Full-test agent routes |
| claude-fable-5 / gpt-5.6-sol | Builds + in-loop gating |
|
| Independent judge; fallback is |
|
|
|
|
| Punch holes or add denials per run |
Why a default banlist? Weak / cheap models produce noisy campaigns that look "done" without real frontier movement. That is a default, not a cage — you can disable it per run.
Controlled GPT-5.6 Sol enforcement proof
With an authenticated Codex CLI, run:
SUPER_LOOP_ALLOW_EXEC=1 npm run proof:gpt56-sol -- \
--model gpt-5.6-sol \
--out proof/build-week/gpt56-sol-liveThis launches three short, explicitly adversarial fixtures through the real codex exec -m gpt-5.6-sol --json path in read-only, ephemeral mode. The fixtures ask the worker to propose a phase skip, a self-reported metric, and self-promotion; Loop Factory must reject each proposal with the matching supervisor code. Evidence includes raw JSONL, prompt/output hashes, the exact model argv receipt, token usage when the CLI reports it, persisted verdict events, a dashboard, and a markdown report. These are controlled regression prompts, not claims of spontaneous model behavior. The command refuses to overwrite an existing evidence directory and never falls back to another model.
A full campaign, in order
initialize_loop_run → brief + ask-once (a few Qs) → answer → INITIALIZED
loop_start strip-miner → section 0
observation_record (phase 0) → request_next_phase → section 1 → … (gated)
artifact_record role=baseline → hash-locked
benchmark_propose → benchmark_select → scorecard frozen
artifact_record measurement → benchmark_run arm=baseline → bar set (tool-measured)
register_hypotheses (3–5 frontier)
test_hypothesis (3–5 agents, tool-measured) → MOVED_FRONTIER | NO_IMPROVEMENT
reverify_run → promotion_request → PROMOTE | BLOCKED
update_dashboard / report_export → checkpoint; lanes keep runningTwo distinct thresholds, neither of which stops the campaign:
Risk advisory (10–15, configurable): after ~12 consecutive valid no-improvement full tests the supervisor raises an economic-exhaustion risk advisory and opens dashboard review — it only reports risk, it does not stop.
Branch retirement (30 valid batches): a branch retires only after 30 valid full real test batches (3–5 frontier workers each) with no qualifying improvement, then the supervisor auto-pivots to the next lane. Invalid / fake-metric / early-stopped / summary-only batches are blocked upstream and never count.
If the Strip Miner saturates, the supervisor auto-transitions (Strip Miner → Loop-de-loop, or the next improvement lane) via report_saturation — never a pause/await/stop. Checkpoint/report/dashboard/refused-terminal/saturation/retirement events persist a machine-readable continuation obligation until a real progress tool runs. continue_run records the model's next-lane commitment but deliberately cannot clear the obligation by itself. Only the operator stops the campaign.
Design notes
Zero dependencies on purpose. No SDK, nothing to
npm installthat can fail or time out, nothing phoning home. The MCP transport is ~90 lines of newline-delimited JSON-RPC insrc/server.mjs. There is nothing to install.Tool-computed measurement authority. The MCP derives quality from the frozen oracle and derives an internal deterministic estimate from recorded output bytes. User-facing reports label that estimate
artifactOutputTokenEstimate; executor receipts showcliReportedTotalTokensanddurationMsseparately. In strict mode each counted agent run directly links a raw stdout artifact and extracted final artifact whose SHA-256 values match the invocation receipt. A number the model types iscaller-reportedand is refused by the benchmark/test gates (MEASUREMENT_AUTHORITY).reverify_runre-derives from the sealed bytes, so a tampered number cannot survive. Subjective quality routes to the dashboard for a human and never auto-promotes (QUALITY_UNVERIFIED); deterministic, oracle-scored quality still queues for mandatory operator Approve before it becomes an internal champion.Host capability preflight, no execution.
host_capability_preflightresolves known frontier-agent CLI names againstPATHwith a filesystem stat — it never spawns a command, never probes a model-supplied binary, and is not SOTA/web research. Presence on PATH ≠ working auth, and it says so.Anti-tampering. Baseline and benchmark are write-once within a cycle; changing either needs an explicit new epoch + rationale.
Path hardening.
runIdand artifact ids are validated before touching disk, andsourcePathreads are refused so a model cannot turn the MCP into a local-file reader. Submit artifact bytes throughcontent.Dashboard-only human review, with a real apply path. The model-callable MCP surface can queue/list review items (and may propose a loop adoption carrying improved loop text), but
human_review_request { action:"resolve" }returnsDASHBOARD_ONLY. The served dashboard is the HTTP decision surface: choose Approve/Deny, then explicitly confirm the queue action. The local server binds to 127.0.0.1, requires a loopback Host, a same-origin browser request, a per-server session token, and the SHA-256 binding for the exact reviewed state. Queue acceptance remains visibly distinct from supervisor application, survives a reload, and the supervisor rejects a decision if its evidence binding changed before drain. Headless fallback exports the same hash-bound payload forruns/<runId>/inbox-decisions.jsonornode scripts/apply-decisions.mjs --file <export>. Approving a loop-adoption review installs the improved loop as a new versioned custom loop (the prior version is archived for rollback viaoperator.rollbackLoop), whichloop_startthen streams next cycle. The mandated canonical loops are immutable and never touched. Applying is non-blocking — the campaign never pauses for it, and adoption remains off the model-callabletools/callsurface.Continuation is a host obligation, stated honestly. An MCP cannot force the host agent loop to keep running — only the host can (which is why the agent is told its native continuous command — Claude Code / Codex
/goal, with/loopas Claude's polling alternate, or the per-host driver from the registry — on start). What the MCP can do, and does: every report / dashboard / saturation / no-improvement / refused-terminal event persists a machine-readable continuation obligation with a concrete next tool+lane, andcontinue_runrecords intent without clearing it (only a real progress tool clears it). The MCP makes stopping early visibly incomplete; it does not pretend to be the host scheduler. The operator is the only stop condition.Never overwrites your canonical loop. Promotion records an internal champion; changing the canonical loop file is HUMAN-GATED and left to you.
Standalone by design.
Run-trajectory export
Bench-maker sessions are out-of-lineage: a separate operator-controlled MCP invocation freezes held-out scorecards; the worker being measured never proposes them.
Protocol (ephemeral bench-maker)
Spin up a dedicated MCP host pointed at the same
SUPER_LOOP_HOME(or a copy) with a freshrunIdfor the held-out worker run.Hash-lock baseline on that run:
artifact_record { role:"baseline", content:"..." }.Freeze held-out benchmark via
benchmark_freeze_maker(notbenchmark_propose):{ "runId": "<eval-run>", "benchmark": { "name": "...", "taskValueDimensions": ["..."], "resourceDimensions": ["..."], "cases": [{ "id": "..." }], "oracle": "..." }, "benchPartition": "gate" }This sets
benchSource:"maker"andbenchPartition:"gate". Workerbenchmark_proposeon that run becomes a no-op while the maker scorecard is frozen.Run the worker through the normal phase gate / hypotheses / full tests on the gate benchmark.
Do not export gate runs for reuse —
export_trajectoriesrefusesbenchPartition:"gate"runs (hard firewall against exam-set leakage).Harvest runs (worker-frozen benchmarks with default
benchPartition:"harvest") export via:{ "runId": "<harvest-run>", "outPath": "trajectory.jsonl" }Output is Hermes-format JSONL: one line per recorded tool call with
label.verdict/label.code/label.reasonfrom the sealed gate results already stored on the run (never re-run gates).Terminate the bench-maker host session when done — no persistent bench-maker process is required; access is operational (separate host invocation), not a background daemon.
Layout
loops/ bundled hash-locked loop sources (verified once per process, then cached)
hosts/ host runtime registry (PURE DATA — continuous drivers + tiers)
examples/ campaign configs, improve-driver, MCP host snippets, rules
src/
server.mjs MCP stdio JSON-RPC transport + tool schemas
engine.mjs Loop Factory core — every tool handler + gate
integrity.mjs Integrity Gate — negative control, answer-key/padded echo, solution pressure
loops.mjs registry, hash verify-on-first-load + process cache, sectionizer
measure.mjs tool-computed measurement (derive cost/quality from bytes) + honest boundary
executor.mjs opt-in live worker execution (allowlist, execFile, stdin) — off by default
run-verifier.mjs independent read-only receipt/artifact publication verifier
canary-runner.mjs blinded one-proposal / three-arm executable canary
schemas/ strict Codex final-output JSON schemas
supervisor.mjs autonomous campaign driver (validate → accept/re-enter boundary)
host.mjs host capability preflight + registry loader
models.mjs modelPolicy / banlist (operator-chosen at init; defaults = historical)
scorecard.mjs promotion frontier rule + score matrix
skill-schema.mjs skill frontmatter + section schema (shared frontmatter parser)
skill-match.mjs skill ranking / match against task
store.mjs local atomic JSON persistence (runs + custom-loops + skills)
dashboard.mjs polished dashboard.html + markdown report
constants/util shared facts + helpers
scripts/ demo.mjs, run-campaign.mjs, run-real-test-canary.mjs, verify-run.mjs,
dashboard-server.mjs, apply-decisions.mjs,
verify-sources.mjs, flywheel-harden.mjs, quarantine-addendum.mjs,
tier-test.mjs, trajectory-capture.mjs, verify-trajectory.mjs
test/ node:test suites (sources, ask-once, phase gate, benchmark,
hypotheses, promotion, hook, dashboard, transport, security,
loop library, measurement authority, host preflight, executor,
supervisor, adoption, dashboard-server)License
MIT
Available Tools
29 toolsartifact_recordA
Persist a raw artifact (run log, baseline copy) with a sha256 hash. role:"baseline" hash-locks the baseline (write-once; tampering refused). Pass measurement:{tokenCost,quality} so the artifact can serve as a tool-measured, reverifiable measurementRef. sourcePath reads are disabled; pass explicit content.
| Name | Required | Description | Default |
|---|---|---|---|
| loop | No | ||
| name | No | ||
| role | No | baseline | evidence | runlog | |
| phase | No | ||
| runId | Yes | ||
| content | No | ||
| newEpoch | No | ||
| rationale | No | ||
| sourcePath | No | disabled; pass content instead | |
| measurement | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: hash-locking for role='baseline' (write-once, tamper-proof), disabling sourcePath, and using measurement for tool-measured artifacts. However, it does not cover idempotency, error handling, or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the core action. Each sentence provides essential information without redundancy: first states the primary function, then adds critical constraints on role and measurement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, no output schema, no annotations), the description covers the main purpose and key constraints but lacks details on return values, error cases, and usage of other parameters like loop, phase, and rationale.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (20%), but the description adds meaning to parameters like role (hash-lock behavior), measurement (tool-measured), and sourcePath (disabled). It does not address other parameters (loop, name, phase, content, etc.), leaving some gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Persist a raw artifact (run log, baseline copy) with a sha256 hash.' It specifies the resource (artifact) and action (persist), and distinguishes from siblings by mentioning hash-locking for baselines and measurement capabilities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (e.g., for persisting artifacts with integrity, especially baselines) but does not explicitly state when not to use it or compare with alternatives. Sibling tools like observation_record or run_campaign are not mentioned, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
benchmark_freeze_makerA
Bench-maker only: freeze a benchmark directly (benchSource:maker) without worker benchmark_propose. Defaults benchPartition to gate (held-out). Worker benchmark_propose becomes a no-op while this scorecard is frozen.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| newEpoch | No | ||
| benchmark | Yes | Full scorecard: name, taskValueDimensions, resourceDimensions, cases, oracle, etc. | |
| rationale | No | ||
| benchPartition | No | gate (held-out, default) or harvest |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool freezes a benchmark, defaults benchPartition to gate, and makes worker benchmark_propose a no-op. However, it does not elaborate on side effects, permissions, or error conditions, but the disclosed behavior is sufficient for core understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences, with no wasted words. It front-loads key information (purpose and usage context). While it could benefit from clearer structure, it is efficient and direct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 parameters, nested objects, no output schema), the description leaves gaps. It does not explain what 'freeze' means in terms of lifecycle, return value, or error handling. It covers the main intent but lacks completeness for full contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (40%). The description adds meaning only for benchPartition (defaults to gate). It provides no additional information for parameters like runId, newEpoch, benchmark, or rationale, which are left to the schema descriptions alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'freeze a benchmark directly (benchSource:maker) without worker benchmark_propose'. It specifies the resource (benchmark) and action (freeze), and differentiates from sibling tool benchmark_propose by indicating it is for maker-only direct freezing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly indicates when to use this tool ('Bench-maker only' and 'without worker benchmark_propose') and what happens when it is used ('Worker benchmark_propose becomes a no-op'). It also specifies the default behavior for benchPartition, providing clear context for usage vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
benchmark_proposeA
Propose one or more benchmark scorecards built from real prior uses/failures. Each needs ≥1 task-value dimension, ≥1 resource/cost dimension, and ≥1 concrete case, or it is rejected as a hand-waved benchmark.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| benchmarks | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only mentions the core action and rejection condition, but lacks details on side effects, error handling, permissions, or state changes. For a creation tool, more behavioral context is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main action, and a second sentence adding essential constraints. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (nested object with 7 subfields) and no output schema, the description only covers the high-level requirement. It omits explanations of subfields (oracle, qualityScale, comparisonRule) and what constitutes a 'concrete case', making it incomplete for an agent to correctly formulate inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds some meaning by implying the 'benchmarks' array must contain subfields (task-value dimensions, resource dimensions, cases), but it does not explain individual properties like 'name', 'oracle', 'qualityScale', etc. Many parameters remain unclear to an agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Propose') and the resource ('benchmark scorecards built from real prior uses/failures'). It includes specific constraints (≥1 task-value dimension, ≥1 resource/cost dimension, ≥1 concrete case) that help differentiate it from sibling tools like benchmark_select or benchmark_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong context for when to use (proposing benchmarks with real prior uses/failures) and includes rejection criteria (hand-waved benchmarks). However, it does not explicitly state when not to use it or mention alternatives like benchmark_select.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
benchmark_runA
Record a tool-measured run of an arm through the frozen benchmark. arm:"baseline" sets the bar challengers must beat. Requires a measurementRef → a recorded raw artifact; model self-report never sets the bar.
| Name | Required | Description | Default |
|---|---|---|---|
| arm | Yes | "baseline" or a hypothesis id | |
| runId | Yes | ||
| measurementRef | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It discloses that measurementRef must be from a raw artifact and that model self-report never sets the bar, but does not describe other behavioral traits like whether it is destructive, permissions needed, or what the outcome of recording a run entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences with no wasted words. The first sentence states the purpose, the second adds a critical constraint. Front-loaded and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is adequate for the basic purpose and a key constraint, but lacks details on return values, side effects, or integration with sibling tools. It is minimally complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has low coverage (33%), but the description adds meaning for 'measurementRef' (recorded raw artifact) and 'arm' (baseline sets bar). This compensates for the missing schema descriptions, though 'runId' remains unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the specific action 'Record a tool-measured run of an arm through the frozen benchmark' and distinguishes the arm 'baseline' as a special case that sets the bar. This is a specific verb+resource and differentiates from siblings like 'reverify_run' or 'initialize_loop_run'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context: requires a measurementRef from a recorded raw artifact, not model self-report. While it doesn't explicitly exclude alternatives, it gives enough guidance on prerequisites and the special role of 'baseline'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
benchmark_selectA
Freeze ONE proposed benchmark as the immutable scorecard for this cycle. Requires the baseline to be hash-locked first. Changing a frozen benchmark needs a new epoch + rationale.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| newEpoch | No | ||
| rationale | No | ||
| benchmarkId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the operation freezes the benchmark immutably and requires a precondition, but does not detail side effects, permissions, error conditions, or return values. The description adds value but lacks comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences that efficiently convey purpose, precondition, and change guidance. No unnecessary words; information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given four parameters, no output schema, and no annotations, the description lacks completeness. It does not explain how to use the parameters (especially newEpoch and rationale beyond the change scenario), nor does it describe the return value or error conditions. More detail is needed for full self-containment.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It hints at the roles of 'newEpoch' and 'rationale' (for changing a frozen benchmark), but does not explain the other parameters (runId, benchmarkId) or provide explicit mapping. This is insufficient for four parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'freeze' and the resource 'proposed benchmark as the immutable scorecard for this cycle,' which distinguishes it from sibling tools like benchmark_propose and benchmark_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies a precondition: 'Requires the baseline to be hash-locked first.' It also notes that changing a frozen benchmark requires a new epoch and rationale, providing clear usage context. However, it does not explicitly mention alternative tools or when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
campaign_statusC
Read-only supervisor status: the lane/target queue, auto-transitions, branch-retirement accounting (30 valid no-improvement batches), the 10-15 risk advisory band, and how many dashboard review items are pending. Pending review never blocks the campaign.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares 'Read-only' (non-destructive) and notes that pending review does not block the campaign, adding useful behavioral context. However, with no annotations, it lacks details on authentication, error handling, or side effects like rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence packed with specifics, front-loaded with 'Read-only supervisor status'. Efficient but slightly dense; could be broken into bullets or shorter clauses for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple read-only tool with one parameter: enumerates key data points. However, lacks explanation of output format or how the status items relate, and does not reference capabilities like pagination or limits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter `runId` has 0% schema description coverage and the tool description does not mention it at all. The agent receives no guidance on its format, purpose, or how to obtain it, leaving the parameter semantically opaque.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides 'Read-only supervisor status' and lists specific resources (lane/target queue, auto-transitions, branch-retirement accounting, risk advisory band, pending review items). It distinguishes from sibling tools like loop_next or reverify_run which are action-oriented, but does not explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. any of the 23 siblings. It does not specify prerequisites (e.g., runId must be valid) or contexts where this status check is appropriate before actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
continue_runA
Record the next runnable improvement lane and first concrete action after reports, dashboards, saturation findings, no-improvement advisories, or refused terminal/checkpoint intents. This never asks the user and never marks the campaign complete. It does not clear the continuation obligation by itself; a real progress tool must run next.
| Name | Required | Description | Default |
|---|---|---|---|
| lane | Yes | the next runnable lane/bottleneck being pursued now | |
| runId | Yes | ||
| rationale | No | ||
| firstAction | Yes | the concrete next tool/action the model is about to perform |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool never asks the user, never marks the campaign complete, and does not clear the continuation obligation by itself. This provides key behavioral traits for safe invocation. It does not mention any destructive actions, which is appropriate given it is a recording tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, efficiently conveying purpose and key behavioral constraints without extraneous words. The first sentence front-loads the primary action, and the second sentence clarifies limitations. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, 50% schema coverage, no output schema, and no annotations, the description adequately covers usage context and behavioral traits. It explains when to use and what the tool does not do, which is sufficient for a recording tool. It could optionally mention the return value, but that is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%, with 'lane' and 'firstAction' having descriptions. The description adds context by explaining 'next runnable improvement lane' and 'first concrete action,' aligning with parameter meaning. However, 'runId' (required) and 'rationale' (optional) are not elaborated in the description, leaving a gap for those parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool records the next runnable improvement lane and first concrete action after specific events. It uses a specific verb ('record') and resource ('run', 'lane', 'action'), and distinguishes from sibling tools by noting it never asks the user or marks the campaign complete, which is unique among the listed siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly lists when to use the tool (after reports, dashboards, saturation findings, etc.) and states it does not mark the campaign complete nor clear the continuation obligation, implying it is an intermediate step. It advises that a 'real progress tool must run next,' providing clear guidance on context, though it does not explicitly name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cycle_decision_requestB
The supervisor decision hook. A worker proposes a transition packet; only a supervisor-accepted transition counts as progress. Reasoning alone is never proof. Allowed transition intents: promote | advance_phase | change_baseline | change_benchmark | saturate. Completion/stop-style intents are refused (the operator is the only stop condition).
| Name | Required | Description | Default |
|---|---|---|---|
| loop | No | ||
| runId | Yes | ||
| intent | Yes | ||
| newEpoch | No | ||
| rationale | No | ||
| hypothesisId | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains that the tool is a decision hook where only supervisor-accepted transitions count as progress, and that completion/stop intents are refused. While this gives some behavioral insight, it omits details on success/failure responses, side effects, or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences that front-load the core purpose and constraints. However, it could be better structured by grouping related information (e.g., listing parameters with brief explanations).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations, and 0% schema coverage), the description is incomplete. It fails to explain crucial parameters like runId, rationale, or hypothesisId, and does not describe return values or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description only adds meaning for the 'intent' parameter by listing allowed values. The other five parameters (runId, loop, newEpoch, rationale, hypothesisId) are left completely unexplained, which is insufficient for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a supervisor decision hook for transition intents, listing allowed intents and explicitly stating what is refused. However, it does not directly differentiate from sibling tools like promotion_request or loop_next.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context for when to use the tool (worker proposing a transition requiring supervisor approval) and what intents are refused. However, it lacks explicit guidance on when not to use it or references to alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
execute_full_testA
SUPERVISOR-EXECUTED full test (off by default; opt in with env SUPER_LOOP_ALLOW_EXEC=1). Sling itself LAUNCHES 3-5 allowlisted frontier workers (claude/codex/glm/gemini binaries on PATH) via execFile (never a shell), captures each output, and feeds the tool-captured bytes through the same gate as test_hypothesis — so there is no model-supplied run-log to fabricate. A failed/timed-out/non-allowlisted launch is an invalid batch and does not count toward retirement. Without the opt-in this returns BLOCKED (EXEC_DISABLED) and you record run-logs via artifact_record + test_hypothesis instead.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| prompt | Yes | the loop + task the launched worker should actually run | |
| routes | Yes | 3-5 frontier worker routes to launch (each must map to an allowlisted binary) | |
| timeoutMs | No | per-worker hard timeout (default 600000) | |
| hypothesisId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes exact execution behavior: launches 3-5 allowlisted workers via execFile, captures output, no model-supplied run-log. Also states what happens when opt-in is missing (returns BLOCKED). No annotations provided, so description fully covers behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single dense paragraph but front-loads key information and every sentence adds value. Could benefit from structure but remains clear and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Provides context on execution environment, opt-in, and alternatives, but lacks detail on return format and does not explain all parameters (runId, hypothesisId) thoroughly. Adequate given tool complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60% with descriptions for prompt, routes, and timeoutMs. The description adds context for routes (3-5 frontier worker routes) but does not explain runId or hypothesisId beyond what's in the schema. Adequate but not enhanced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's a supervisor-executed full test that launches frontier workers, distinguishing it from sibling tools like test_hypothesis which record run-logs without execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it's off by default with opt-in (env variable), and specifies alternatives (artifact_record + test_hypothesis) when exec is disabled. Also clarifies that failed launches don't count toward retirement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export_trajectoriesB
Export a run's recorded tool trajectory as Hermes-format JSONL (one assistant/tool_call line per action, with supervisor verdict labels from recorded gate results). Read-only over the store. Refuses gate-partitioned (held-out) runs.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| outPath | Yes | Relative path inside the run dir, or absolute path under SUPER_LOOP_HOME/exports/ |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states read-only behavior and refusal for held-out runs, but lacks details on authentication needs, rate limits, or output behavior (e.g., whether it saves a file or returns content). The information provided is adequate but not extensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences, front-loading the main purpose and adding key details. No superfluous words, and it avoids repeating obvious information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema, the description should explain what the export returns (e.g., file path, success message). It only mentions the export format but omits the outcome. For two simple parameters, it is moderately complete but lacks output clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (only outPath has a description). The tool description does not add meaning beyond the schema: runId is not explained, and outPath's description is already in schema. The description could have clarified the expected format or constraints for runId, but it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool exports a run's recorded tool trajectory in a specific format (Hermes-format JSONL) with details about content (one line per action, supervisor labels). It is specific and distinguishes from siblings by mentioning gate results and read-only behavior, though it doesn't explicitly compare to similar export tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives some usage guidance: it is read-only and refuses gate-partitioned runs. However, it does not provide when to use this tool versus siblings like report_export or observation_record, nor does it mention prerequisites or typical scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
host_capability_preflightA
Local capability report: which known frontier-agent CLIs (claude, codex, gemini, opencode, optional glm) are installed on PATH, PLUS the resolved host profile (driverFamily, tier, setupHint, and the host matrix when SUPER_LOOP_HOST is unknown). Filesystem stat only — NEVER executes a command, NEVER probes arbitrary binaries, and is NOT web/SOTA research. Presence on PATH is not proof of working auth.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No | optional; only used to journal |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: filesystem stat only, no command execution, no arbitrary binary probing, not web/SOTA research, and caveat about auth. This is comprehensive and honest.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise, front-loaded sentences. First sentence states purpose, second sets boundaries, third provides important limitation. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity, no output schema, and full parameter coverage by schema, the description is complete. It covers purpose, constraints, and caveats, leaving no obvious gaps for an AI agent to misuse the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter (runId) is fully described in the schema with 100% coverage; the description adds no further meaning. Baseline 3 is appropriate as the schema already documents it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides a local capability report on installed frontier-agent CLIs and host profile, using specific verb 'report' and resource 'local capability'. It distinguishes itself from sibling tools like host_runtime_detect by focusing on CLI presence and host resolution without execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies safe usage ('NEVER executes a command') but lacks explicit when/when-not guidance or naming of alternatives. It does not contrast with sibling tools like host_runtime_detect or other inspection tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
host_runtime_detectA
Advisory guess of which host runtime the agent is in, from which MCP config files exist on disk (per the host registry). READ-ONLY existence check — never reads file contents, never mutates config. SUPER_LOOP_HOST, if set, is authoritative. Returns a guess, the candidate hosts with evidence, and the CLI fallback; nothing is auto-applied.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No | optional; only used to journal |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden of behavioral disclosure. It explicitly states the tool is read-only, never reads file contents, never mutates config, and returns a guess with evidence without auto-applying anything.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences that front-load the purpose and efficiently cover all critical aspects without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, no output schema), the description is complete. It covers the purpose, behavior, output summary, and a key constraint (SUPER_LOOP_HOST). No gaps are evident.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single optional parameter 'runId', with its description as 'optional; only used to journal'. The tool description does not add further semantics beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: an advisory guess of the host runtime based on MCP config files. It uses specific verbs like 'detect' and 'guess', and distinguishes itself from siblings by emphasizing it does not read file contents or mutate config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that SUPER_LOOP_HOST is authoritative, providing some usage context. However, it does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or when-not scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
human_review_requestA
Queue a change for the operator’s Approve/Sludge dashboard or list pending items. This tool CANNOT resolve human review; approval/sludge is dashboard-only. Never blocks deterministic lanes — the loop keeps running.
| Name | Required | Description | Default |
|---|---|---|---|
| item | No | ||
| notes | No | ignored/refused; human decisions are dashboard-only | |
| runId | Yes | ||
| action | No | add | list (resolve is refused: dashboard-only) | |
| decision | No | ignored/refused; human decisions are dashboard-only | |
| reviewId | No | accepted only for rejected legacy resolve attempts |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behaviors: queuing, listing, no resolution, and non-blocking nature. However, it lacks details on error handling or side effects like ignoring notes/decision fields (though schema covers some of these).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences front-loaded with core purpose, no wasted words, efficiently conveys key guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, and key behavioral constraints. Lacks details on return values or what happens after queuing, but given no output schema and 6 parameters, the description is reasonably complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, so baseline is 3. The description does not elaborate on specific parameters beyond implying 'add' vs 'list' actions; it adds marginal value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool queues changes for human review or lists pending items, with specific verb-resource and differentiation from siblings like 'cannot resolve human review' and 'never blocks deterministic lanes'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (queue/list for dashboard) and when not to (approval/sludge is dashboard-only, does not block lanes), providing clear context for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
initialize_loop_runA
Ask-once gate. Confirms the task before any loop runs. If the task is underspecified, returns one brief explanation plus a few short questions once (goal; PATH — improve an existing loop / discover-or-find a loop, optionally scouting a public loop library / mine your whole history; the loop or domain to start from; corpus scope — whole history or a set number of loops, and best-first vs in-order; what "better" means; any task-specific hard limit; and a deeper-explanation offer); call again with { answers } to begin. It never asks the operator to choose the model, promotion mode, benchmark policy, deterministic-vs-subjective routing, or the standing guarantees — the supervisor decides those from the task. Stores every user message locally with a sha256 hash. After initialization it does not ask again or mark the campaign complete; the operator remains the stop condition and the dashboard stays available.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | what to improve/build | |
| model | No | frontier route; defaults to claude-opus-4-8 | |
| runId | No | reuse to continue a run; omit to create one | |
| config | No | { failurePatience(10-15), comparisonRule, promotion:{...}, mode } | |
| answers | No | answers to the ask-once questions | |
| userMessages | No | verbatim operator messages — stored + hashed for the hook | |
| acceptanceCriteria | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses key behaviors: stores messages with sha256 hash, asks once, never asks again after initialization, operator remains stop condition, dashboard stays available. This is comprehensive but could mention idempotency or side effects more explicitly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description starts with a concise summary ('Ask-once gate') but then expands into a dense paragraph. While it is informative, the length could be slightly reduced by separating key points. Still, it remains focused and structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, nested objects, no output schema, and many siblings, the description provides enough context for usage. It explains the initialization flow, constraints, and operator involvement. Lacks details on return values or state persistence, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 86%, so baseline is 3. The description adds context (e.g., 'answers' are responses to ask-once questions, 'userMessages' stored and hashed), but the schema already covers most parameters. Additional value is moderate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Ask-once gate. Confirms the task before any loop runs.' It explains the interaction flow (returns questions, then call again with answers), which distinguishes it from sibling tools like loop_start or continue_run that handle later stages.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (before a loop runs) and clarifies what the tool will not ask (model, promotion mode, etc.), guiding the agent on what inputs are relevant. While it does not explicitly name alternatives, the context of the workflow is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_libraryA
List every loop and skill available to this local MCP: mandated hash-locked loops, custom loops you registered, and skills (metadata only — id, title, provenance, partition, section count). No full bodies.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | No | optional; only used to journal |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavioral traits. It states that only metadata is returned ('no full bodies'), which implies a read-only, non-destructive operation. However, it does not explicitly confirm that no side effects occur, nor does it mention any permissions or rate limits. The disclosure is partially adequate but could be more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the tool's purpose. Every word serves a function—defining scope, content, and limitations. No redundant or unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and no output schema, the description covers the what and what-not. However, given the large sibling set, the lack of usage guidance leaves it slightly incomplete. The description would benefit from a brief recommendation on when to use this tool over, say, skill_fetch.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single optional parameter 'runId', and the schema already describes it as 'optional; only used to journal'. The description does not add any extra meaning beyond what the schema provides (e.g., when to supply runId). With high schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'List', the resources 'every loop and skill', and the scope 'available to this local MCP'. It specifies what is included (mandated, custom loops, skills with metadata) and what is excluded ('no full bodies'). This effectively distinguishes it from sibling tools like loop_register or loop_start.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs. alternatives. With 27 sibling tools, including many that deal with loops and skills (e.g., loop_register, loop_next, skill_fetch), the description does not provide any comparison or context for selection. The agent receives no information about prerequisites or when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_nextD
Alias of request_next_phase.
| Name | Required | Description | Default |
|---|---|---|---|
| loop | No | ||
| runId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No behavioral traits are disclosed. The description does not mention side effects, permissions, rate limits, or any other behavioral aspects. With no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise but under-specified. It provides only an alias reference without substantive content, sacrificing usefulness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not cover output, behavior, or parameter details. For a tool with two parameters and no output schema, the description is incomplete and does not enable correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning to the input schema. Neither 'loop' nor 'runId' are explained, and schema descriptions are absent. The tool fails to clarify parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description only says 'Alias of request_next_phase,' which does not directly state what the tool does. It relies on the user knowing another tool's purpose, making the purpose vague.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use loop_next vs request_next_phase or other siblings. The description only indicates they are aliases without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_registerA
Add YOUR OWN loop to this machine's local MCP, or register a skill (retrievable knowledge file) with role:"skill". Pass the full text as content; loops are hashed, phase-gated, and streamed; skills are section-addressable with provenance-enforced frontmatter. Cannot overwrite the hash-locked Strip Miner / Loop-de-loop. Nothing leaves your machine.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | safe lowercase id (no slashes/spaces); must not collide with a mandated loop | |
| role | No | artifact type — "custom" (phase-streamed loop) or "skill" (retrievable knowledge file). Default custom. | |
| tags | No | free-form labels for skill keyword pre-filter | |
| runId | No | optional; only used to journal the registration | |
| stack | No | tech stack tokens for skill routing | |
| title | No | ||
| source | No | provenance URI or path description (required when role is "skill") | |
| content | Yes | the full loop or skill markdown (loops: headers/paragraphs become phases; skills: frontmatter + body with _synthesis section) | |
| license | No | SPDX id or LicenseRef-… (required when role is "skill") | |
| trigger | No | e.g. "/loop my-loop" (loops only) | |
| overwrite | No | replace an existing custom loop or skill of the same id with a new local version | |
| source_paths | No | optional auxiliary reference file paths for multi-file skills | |
| anti_patterns | No | outputs the model must not reproduce from the skill | |
| skillPartition | No | partition — "working" (retrievable in normal runs) or "reference" (held-out, opt-in only). Default working. | |
| supports_tasks | No | task intents this skill helps with | |
| token_budget_hint | No | expected total skill size hint for Router 1 planning | |
| synthesis_guidance | No | how to apply fetched skill knowledge to the current task without copying |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It reveals that loops are hashed, phase-gated, and streamed; skills are section-addressable with provenance-enforced frontmatter; overwriting is restricted for certain loops; and data does not leave the machine. This adds significant context beyond the schema, though details on streaming behavior could be richer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, succinct and well-structured. It front-loads the primary action, then details specifics about loops and skills, and ends with constraints. No unnecessary words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 17 parameters and no output schema, the description covers the core behavior and two registration types. It explains hashing, phase-gating, provenance, and data privacy. It lacks details on expected return values or outcomes (e.g., how to invoke the registered loop/skill), but is fairly complete for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 94%, so most parameters are already described. The description adds value by explaining that 'content' holds full text, and clarifies the role distinction. However, it does not elaborate on many parameters like stack, tags, or anti_patterns beyond what's in the schema, so added meaning is moderate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds/registers loops or skills, with specific verb 'Add' / 'register' and resource types. It distinguishes between loops and skills and mentions constraints like 'Cannot overwrite hash-locked Strip Miner / Loop-de-loop'. This makes the purpose distinct from siblings like loop_library or skill_fetch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the difference between loops and skills via the 'role' parameter, but does not explicitly tell when to use this tool versus alternatives (e.g., loop_library to list, or artifact_record for other data). Usage context is implied but lacks direct when-not-to-use guidance or comparison to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_startA
Begin phase-gated streaming of a bundled or custom local loop. Opens/activates the supervisor lane for that loop. Use "strip-miner" (The Strip Miner Loop / cross-agent source miner, 345 lines), "loop-de-loop" (Loop 2, the improvement loop, 75 lines), or any id registered with loop_register. Returns ONLY section 0; the full loop stays inside the supervisor.
| Name | Required | Description | Default |
|---|---|---|---|
| loop | Yes | strip-miner, loop-de-loop, or a custom loop id from loop_library | |
| runId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description describes key behavior: activating the supervisor lane and returning only section 0. It does not detail side effects, idempotency, or state changes, but the disclosed behavior is sufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is five sentences with no wasted words. It front-loads the main action, then provides examples and return behavior. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the main operation and return behavior, it does not explain the role of 'runId' or how this tool fits with siblings like loop_next. Given two required params and no output schema, the description is adequate but has gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: 'loop' has a description which the tool's description reinforces, but 'runId' lacks any description in schema or description. The description adds moderate value for 'loop' but does not fully compensate for the missing runId context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool begins phase-gated streaming and opens/activates the supervisor lane for a loop. It specifies valid loop values like 'strip-miner' and 'loop-de-loop', distinguishing it from siblings like loop_next or loop_register.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides examples of when to use the tool (e.g., 'Use strip-miner...') but does not explicitly compare with alternatives like loop_next or loop_register. It implies usage context but lacks explicit exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
observation_recordA
Record lightweight evidence for the current phase (what you actually did/observed). Attach { loop, phase } to satisfy the phase gate and unlock the next section.
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | ||
| loop | No | ||
| phase | No | ||
| runId | Yes | ||
| summary | Yes | ||
| sourceRef | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the full burden. It reveals that the tool records lightweight evidence and is tied to phase gating, but does not disclose idempotency, side effects (e.g., whether it appends or overwrites), or what happens if called multiple times. Adequate but could be more detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. First sentence states purpose, second gives usage instruction. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple record tool with no output schema and no annotations, the description covers core purpose, key parameters, and the phase gate context. It lacks return value details and behavior on repeated calls, but is mostly complete given the tool's straightforward nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage and 6 parameters, the description explicitly mentions 'loop' and 'phase' and implies 'runId' and 'summary' are needed. It does not explain 'kind' or 'sourceRef', leaving gaps. Adds meaning but incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Record' and resource 'evidence' in the context of the current phase. It also adds outcome ('unlock the next section'), distinguishing it from siblings like 'artifact_record'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains it is for recording evidence to satisfy a phase gate and unlock the next section, providing clear context. It does not explicitly state when not to use or name alternatives, but the context is sufficient for appropriate selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
promotion_requestA
Request promotion of a hypothesis to internal champion. Requires a tool-measured, reverified full test on the frozen benchmark that moves the quality/cost frontier past threshold. Old green unit tests without a score matrix, model-reported metrics, or below-threshold results are BLOCKED. Never overwrites the operator’s canonical loop file.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| hypothesisId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description discloses key behaviors: requires tool-measured reverified full test, blocks non-conforming results, and never overwrites canonical loop file. Could add details on idempotency or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each adding meaningful constraint or action. No wasted words, front-loaded with purpose and key requirements.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers what the tool does and its constraints well, but lacks parameter explanation and return behavior. No output schema; description does not mention what the agent gets back (e.g., success/failure, new champion ID).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% coverage; description does not explain runId or hypothesisId beyond names. With 0% coverage, description should define or provide context for parameters, but it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Request promotion of a hypothesis to internal champion' with specific verb and resource, and distinguishes from sibling tools like 'loop_next' and 'reverify_run' by detailing unique conditions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes when to use (hypothesis meeting strict criteria) and explicitly blocks old green unit tests without required metrics. Does not mention alternatives like 'reverify_run' or 'test_hypothesis' but context implies sequential use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
register_hypothesesA
Register 3–5 challenger hypotheses, each on a frontier route. Requires baseline hash-lock + frozen benchmark + measured baseline bar (benchmark-first). Rejects <3 or >5, and any haiku/mini/nano/lite/prior-gen route.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| hypotheses | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It describes what the tool requires and rejects, but does not mention side effects, idempotency, or whether it is read-only or destructive. Partial disclosure is present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, front-loading the purpose and then adding constraints. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the complexity of the input schema (array of objects with multiple fields), the description omits parameter details, return value, and error handling beyond rejections. It lacks completeness for a tool with many siblings and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It mentions constraints on hypotheses (frontier routes, model exclusions) but does not explain the meaning of fields like 'tradeoff', 'falsifier', or 'operation' within the hypothesis object. The runId parameter is completely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'register', the resource 'challenger hypotheses', and includes constraints on number (3–5) and route type. However, it does not differentiate from sibling tools like 'test_hypothesis' that also involve hypotheses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly lists prerequisites ('baseline hash-lock + frozen benchmark + measured baseline bar') and rejection conditions ('<3 or >5, any haiku/mini/nano/lite/prior-gen route'), providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
report_exportC
Write a reproducible markdown report (baseline lock, frozen benchmark, score matrix, promotions, failure patience, campaign state) to the run dir.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| format | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description must fully disclose behavior. It says 'write' but omits side effects: does it overwrite? require permissions? fail silently? What is the run dir and what happens if it doesn't exist?
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, but the list of components is dense and could be clearer with bullet points or broken into separate sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output schema and annotations; does not explain 'run dir' or the effect of format. For a tool with 24 siblings, this is insufficient for correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and description does not explain either 'runId' or 'format'. The report contents are listed, but how parameters influence output is absent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Write') and resource ('reproducible markdown report'), and lists concrete components (baseline lock, frozen benchmark, etc.), clearly distinguishing it from siblings like 'report_saturation'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. the many siblings. Does not mention prerequisites, exclusions, or context like whether it is for final export vs. intermediate reporting.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
report_saturationA
Tell the supervisor the current lane (e.g. the Strip Miner) has reached evidence-backed saturation. The supervisor AUTO-TRANSITIONS to the next lane (Strip Miner → Loop-de-loop, or the next improvement branch). It never pauses, awaits the operator, or treats "no re-mining warranted" as terminal — saturation is a pivot. The operator is the only stop condition.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| evidence | No | the saturation evidence (batches that changed nothing material) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden of behavioral disclosure. It clearly explains that the supervisor auto-transitions, never pauses, awaits the operator only for stop conditions, and treats saturation as a pivot. This provides good context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, front-loaded with the main purpose. Each sentence adds useful context (auto-transition behavior, no pausing, operator as stop condition). It could be slightly more concise but is not overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the tool's effect and behavior well, but it does not mention return values or side effects (e.g., confirmation of saturation report). Given the tool's complexity and lack of output schema, more detail on what happens after reporting would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (evidence has description, runId does not). The description mentions 'evidence-backed saturation' but does not add specifics about the runId or evidence parameter format or constraints. It does not compensate for the missing runId description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: informing the supervisor that a lane has reached evidence-backed saturation. It uses specific verbs and resources ('Tell the supervisor...') and implicitly distinguishes from siblings like loop_next by focusing on the saturation trigger. However, it does not explicitly differentiate from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (when evidence-backed saturation is reached) and describes behavioral consequences (auto-transition). However, it lacks explicit guidance on when not to use this tool versus alternatives (e.g., reverify_run, loop_next) and does not state prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
request_next_phaseC
Stream the next loop section. BLOCKED (PHASE_SKIP) unless the current section already has recorded evidence. Prevents 300+ lines collapsing into the model before real decisions.
| Name | Required | Description | Default |
|---|---|---|---|
| loop | No | ||
| runId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses a blocking condition and rationale (prevents 300+ lines collapsing), but lacks detail on side effects, required permissions, or return format. The behavioral information is partial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences. It front-loads the action and provides a constraint. No wasted words, though the second sentence could be more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, low schema coverage, and relatively complex behavior, the description is insufficient. It offers a high-level purpose and one constraint, but omits parameter explanations, error conditions, and return values. The agent cannot fully understand tool invocation requirements.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description does not explain either parameter. The agent gets no guidance on what 'runId' or 'loop' mean, how they affect behavior, or valid values. This is a critical gap for a tool with undocumented parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool streams the next loop section and provides a blocking condition. The verb 'stream' is specific, and the mention of 'BLOCKED (PHASE_SKIP)' clarifies the tool's behavior. However, it does not explicitly distinguish from sibling 'loop_next', so it loses a point.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when the tool is blocked (unless current section has recorded evidence). This gives a usage condition but does not explicitly state alternatives or when not to use the tool. The guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reverify_runA
Deep re-verification: re-hash every raw artifact behind a full test and confirm the claimed metrics reproduce. Promotion is blocked until this passes (anti benchmark-gaming / baseline-tampering).
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| testId | No | ||
| hypothesisId | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It describes re-hashing artifacts and metric reproduction, and notes the consequence of blocking promotion. It does not mention authorization, rate limits, or potential side effects beyond verification.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-load the key term 'Deep re-verification' and efficiently convey purpose and consequence without extraneous details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 3 parameters and no output schema, the description fails to explain how parameters relate to the tool's operation. It does not define runId, testId, or hypothesisId, leaving the agent without guidance on what values to provide.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 3 parameters with 0% description coverage, and the description does not explain any parameters. The agent must infer meaning from parameter names only (runId, testId, hypothesisId), which is insufficient for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it performs deep re-verification by re-hashing raw artifacts and confirming metrics reproduce. It distinguishes from siblings like benchmark_run or test_hypothesis by focusing on verification rather than initial execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when promotion is needed after a test, noting that promotion is blocked until this passes. It provides context for anti-benchmark-gaming but does not explicitly exclude when not to use or list alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_campaignA
AUTONOMOUS SUPERVISOR (opt-in: SUPER_LOOP_ALLOW_EXEC=1). One call drives the whole campaign itself — intake → work the target queue (mine → improve) → for each improve target: hash-lock baseline → freeze benchmark → measure the bar on a real worker → FullTestBatches (3-5 frontier workers, each output VALIDATED through the enforcement boundary) → supervisor delta → reverify → promote (bank a Stone) → advance/retire → re-mine — and keeps going until the operator stop-file. Worker output is never trusted: summary-only / early-stop / fake-metric / self-promote / phase-skip / copied-public are rejected and re-entered, and invalid batches do not count toward retirement. maxBatches bounds the in-call MCP run (a safety cap, NOT completion); the standalone super-loop-run CLI runs it until the stop-file. Returns the exact string MISSING_FULL_PRIVATE_LOOPS if a full private loop is absent.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| config | Yes | { task, routes:[3-5 frontier], benchmark:{name,taskValueDimensions,resourceDimensions,cases,oracle}, targets:[{kind:"mine"|"improve", loop?, baselineContent?, benchmark?, routes?}], noImprovePolicy?(default 30), remineOnEmpty? } | |
| stopFile | No | path whose existence stops the campaign — the operator stop signal | |
| maxBatches | No | safety cap on valid FullTestBatches for this in-call run (default 3); not a completion state |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavior: the complete workflow, trust model (worker output never trusted, rejection reasons), safety cap (maxBatches), and a specific return value for missing private loops. This is comprehensive for a complex tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense with information and front-loads the key supervisor role and opt-in requirement. While it could be more structured (e.g., bullet points), it efficiently communicates a complex process in a single paragraph without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's high complexity and no output schema, the description covers workflow, trust model, and a specific error return. However, it lacks detail on successful return format or other possible error states, which is a gap for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 3 of 4 parameters with descriptions. The tool description adds meaningful context beyond schema, e.g., warning that maxBatches is a safety cap not completion, and explaining stopFile as operator stop signal. However, it does not detail runId or the nested config structure further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it drives a full campaign autonomously, listing the detailed pipeline steps. It distinguishes from sibling tools like loop_next or reverify_run, which are individual steps, and explicitly mentions it's an 'AUTONOMOUS SUPERVISOR'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions opt-in environment variable (SUPER_LOOP_ALLOW_EXEC=1) as a prerequisite and contrasts with the standalone CLI. Implicitly suggests use when full automation is desired, but does not explicitly state when not to use or list alternatives beyond the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
skill_fetchA
Retrieve skill knowledge for the current task. Two modes: 'plan' returns the index (titles, purposes, token estimates) of skills matching your query — read these to decide which sections to fetch; never loads full skill bodies. 'section' fetches ONE section body by (skill_id, section_id) so you pull only the knowledge you need. Default partition is 'working'; 'reference' is opt-in only (held-out skills). Pass runId to pin the exact skill version this run used.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | plan (Router 1 index) or section (Router 2 body fetch). Default plan. | |
| topK | No | plan mode: max skills to return (default 5) | |
| query | No | for plan mode — what you are working on | |
| runId | No | optional — pins the fetched skill version to this run for reproducibility | |
| skill_id | No | for section mode — skill to fetch from | |
| partition | No | working (default, retrievable in normal runs) or reference (held-out, opt-in only) | |
| section_id | No | for section mode — one section to return |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behaviors: plan never loads full bodies, section fetches one section, partition defaults, and runId pins versions. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact (under 100 words) and well-structured: starts with a clear verb+resource statement, then explains modes, parameters, and partitions without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, two modes, and no output schema, the description covers key behavioral details (modes, partitions, version pinning) but omits the exact structure of returned data for section mode. Still, it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds significant meaning beyond the schema: it explains the two modes, the purpose of each parameter in context (e.g., query for plan, skill_id for section), and default values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Retrieve skill knowledge for the current task' and distinguishes two modes (plan and section), making the tool's purpose clear and differentiating it from the listed sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use each mode: plan for reading the index to decide which sections to fetch, and section for fetching one section body. It also clarifies default and opt-in partitions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
test_hypothesisB
Record ONE full test of a hypothesis = 3–5 frontier agents that actually ran the loop end-to-end. Every agent run must carry a measurementRef (tool-measured). Aggregates vs the frozen baseline bar; a no-improvement run is NO_IMPROVEMENT, never "perfect", and bumps the failure counter.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| fullTest | Yes | ||
| hypothesisId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral rules: a no-improvement run must be recorded as NO_IMPROVEMENT (not 'perfect'), and failure counter is bumped. It also requires measurementRef. Without annotations, this provides important behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the tool's purpose. It is efficient but could benefit from clearer structure or bullet points for parameter guidelines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema and no annotations, the description partially explains the test recording process but lacks details on return values, error handling, or how to properly structure the fullTest object. It is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not explain the three required parameters (runId, hypothesisId, fullTest) beyond mentioning measurementRef. It adds some context about agentRuns but insufficiently compensates for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool records a 'full test of a hypothesis' involving 3-5 frontier agents. It adds specificity about agent runs requiring measurementRef. However, it doesn't explicitly differentiate from sibling tools like 'execute_full_test' or 'run_campaign', so purpose is clear but not uniquely distinguished.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a full test is completed but does not provide explicit guidance on when to use this versus alternatives like loop_next or execute_full_test. No 'when not to use' or alternative references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_dashboardC
Render the always-available local dashboard.html (score matrix, phase progress, failure patience, Approve/Sludge, and the stop-condition notice). Human review happens only here; deterministic lanes do not wait on it.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes a read-only render operation, but tool name suggests write/update, creating inconsistency. Fails to disclose side effects, auth needs, or rate limits. The conflicting name undermines transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence efficiently conveys the main action and contents, but could be more structured. The list of dashboard components is concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema and one required parameter, the description fails to explain the runId parameter, and the name-action inconsistency adds confusion. The tool definition is insufficient for correct agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter 'runId' has no description in schema (0% coverage), and the description does not mention it at all, providing no guidance on its meaning or how to use it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it renders the local dashboard.html with specific contents like score matrix and phase progress, and implies a distinct role for human review. However, the tool name 'update_dashboard' contradicts the 'Render' verb, causing potential confusion about the tool's action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Indirectly says 'human review happens only here; deterministic lanes do not wait on it', suggesting use when human review is needed, but does not explicitly state when to use this tool versus sibling tools like human_review_request or loop_next, nor mentions when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
29 tool updates
v1.0.0- First observed
artifact_record - First observed
benchmark_freeze_maker - First observed
benchmark_propose - First observed
benchmark_run - First observed
benchmark_select - First observed
campaign_status - First observed
continue_run - First observed
cycle_decision_request - First observed
execute_full_test - First observed
export_trajectories - First observed
host_capability_preflight - First observed
host_runtime_detect - First observed
human_review_request - First observed
initialize_loop_run - First observed
loop_library - First observed
loop_next - First observed
loop_register - First observed
loop_start - First observed
observation_record - First observed
promotion_request - First observed
register_hypotheses - First observed
report_export - First observed
report_saturation - First observed
request_next_phase - First observed
reverify_run - First observed
run_campaign - First observed
skill_fetch - First observed
test_hypothesis - First observed
update_dashboard
TDQS
Scored across 29 tools
Tools have mostly distinct purposes, with detailed descriptions clarifying roles. However, some tools like 'continue_run' and 'request_next_phase' could be confused without careful reading, slightly reducing disambiguation.
Naming is predominantly snake_case but mixes verb-first (e.g., 'artifact_record') and noun-first (e.g., 'campaign_status') patterns. Some compound names are awkward, and there is an alias ('loop_next') that adds redundancy.
29 tools is high for the domain; while each tool has a specific role, the number exceeds the typical 3-15 range and is above 25, making the surface heavy and potentially overwhelming for agents.
The tool set covers the full campaign lifecycle: initialization, benchmarking, hypothesis testing, looping, human review, verification, and reporting. Minor gaps exist, such as no direct artifact listing, but overall it is comprehensive.
Maintenance
Related MCP Connectors
Deterministic AI code review, with an audit record. Governance inside the agent loop.
Verifies AI agent work end to end: real artifacts and outcomes checked, not self-reported success.
Watchdog for unattended AI agents: alerts, evidence checks and a verifiable proof per run.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnforces disciplined programming practices by requiring AI assistants to audit their work and produce verified outputs at each phase of development, following structured workflows for refactoring, feature development, and testing.2035 npm12MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to learn from their work by recording tasks, extracting patterns, detecting mistakes, and proactively surfacing insights, all using the agent's own model through a cooperative intelligence pattern.MIT
- FlicenseBqualityDmaintenanceA referee for self-improving AI agent loops that mines past sessions for working workflows, improves them, and never stops until you stop it.25-
- FlicenseBqualityDmaintenanceA referee for self-improving AI agent loops that mines past sessions, improves workflows, and never stops until told.26-