Skip to main content
Glama

context-harness

A harness for coding agents (Claude Code, Codex, any MCP client) that changes code in a repository:

  • Budgeted check runs. Every test or check goes through run. It keeps a countdown (12 runs by default), stops after the full check is green plus one confirmation run, stops a test that failed 3 times, and refuses an identical run on an unchanged tree. Overrides are allowed, limited and recorded.

  • Shaped output. The full log stays on disk; the agent gets the exit code, failing test names, the first failure and the tail. Long checks print a heartbeat so the agent waits instead of polling.

  • Pre-existing test guard. Test files are snapshotted at task start. guard lists rewritten or deleted assertions, guard --run runs the originals against the new code, and a completed report is refused until each rewrite is justified.

  • Project memory. Verified lessons and pointers to current code, delivered only when they look relevant; empty context is a normal answer.

  • Agent roadmap. A local dashboard shows the orchestrator and every subagent it spawned, with git branch, model, run time and tokens read from Claude Code transcripts and Codex logs.

  • Installer for Claude Code, Codex, Cursor, Claude Desktop and Gemini CLI, plus this plugin marketplace with hooks for Claude Code and Codex.

Python 3.10+, standard library only. The same state and protocol serve the CLI, the skill and the MCP server.

The dashboard's Agents tab for a demo task. The task, agents and token numbers are sample data.

Install

The commands below install from dimqsh/context-harness.

Claude Code plugin

claude plugin marketplace add dimqsh/context-harness
claude plugin install context-harness@context-harness

This installs the skill, the MCP server (started as python3 …/ch.py mcp, so python3 must be on PATH) and the two hooks. Restart Claude Code afterwards.

Codex plugin

codex plugin marketplace add dimqsh/context-harness
codex plugin add context-harness@context-harness

Codex reads the same marketplace file and uses plugins/context-harness/.codex-plugin/plugin.json: the skill, the MCP server (from .codex-mcp.json, because Codex does not expand ${CLAUDE_PLUGIN_ROOT} in MCP arguments) and the hooks. Codex lists plugin hooks as untrusted until you trust them; until then only the skill and the MCP server are active. In non-interactive runs with -a never we saw Codex refuse MCP tool calls that need approval; the skill's CLI path (scripts/ch.py) does not depend on MCP.

Skill only

For agents that read SKILL.md and have a shell. Clone the repository, then preview and apply:

git clone https://github.com/dimqsh/context-harness context-harness
python3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --skill claude-code --skill codex
python3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --skill claude-code --skill codex --write

The installer is a dry run unless --write is given, never replaces an existing install without --force, and keeps a backup of everything it replaces. Copying plugins/context-harness/skills/context-harness into ~/.claude/skills/ or ~/.codex/skills/ by hand works too. A skill alone has no hooks, so its rules are advice the agent may ignore.

MCP server for Cursor, Claude Desktop and others

python3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --mcp cursor --mcp claude-desktop --write

Other targets: claude-code, codex, gemini, claude-project (with --target-project DIR) and json, which prints a generic mcpServers entry for any other client. For GUI clients the installer pins your current PATH so run finds node, npm and the like. A server started as ch.py mcp --read-only exposes only guard, search and checkpoint and cannot run commands or write.

pip

pip install "git+https://github.com/dimqsh/context-harness"

This gives the context-harness CLI and the context-harness-mcp server. The package carries the skill files, so context-harness install --skill codex --mcp cursor --write works from a pip install too; MCP targets then run python -m context_harness mcp with the same interpreter. Local multilingual search is an optional extra (context-harness[semantic]), see semantic-search.md.

Related MCP server: universal-jev-mcp

How an agent uses it

CH below is python3 <skill>/scripts/ch.py --project <absolute project root> --compact. MCP clients call the tools of the same names (start, run, guard, report, search, remember, checkpoint).

CH start --title "Fix invoice rounding" --goal "Totals round half up to cents" \
  --acceptance "1.005 rounds to 1.01" --query "rounding error in invoice totals src/totals.ts"
CH run -- npx vitest run src/totals.test.ts       # a focused test
CH run --full -- npm run check                    # the complete project check
CH guard                                          # did I rewrite old assertions?
CH report --status completed --summary "..." --artifact src/totals.ts \
  --evidence "npm run check: exit 0"

A refused run exits with code 75 and says why. The full protocol, including multi-agent coordination and independent verification, is in protocol.md; budgets and output shaping are in token-budget.md.

How it differs

context-harness bounds how an agent checks its own work and shows what each agent spent on a task. It is not mainly a memory system, a permission guard or a cost tracker, and it runs next to those tools.

Tool

What it does

Relation to context-harness

claude-mem

Captures what the agent does in each session, compresses it and injects relevant context into later sessions.

Overlaps on memory only. Memory here is narrower: verified lessons and pointers to current code, returned above a relevance cutoff. The run budget and the test guard are the main work.

Serena

Semantic code retrieval and editing tools over MCP, backed by language servers.

Complementary: Serena helps the agent find and change code; context-harness bounds how it checks the result.

nah

A context-aware guard that blocks dangerous agent tool calls.

Complementary. Both use a PreToolUse hook on Bash; context-harness only sends test and check commands through run.

ccusage, CodeBurn

Token usage and cost from local transcripts, by day and model (ccusage) or by model, project and task (CodeBurn).

Overlaps on reading transcripts. context-harness puts tokens on each agent in the spawn tree of one task, next to that task's checks and test guard. It does not price tokens.

Why not just CLAUDE.md?

Keep CLAUDE.md or AGENTS.md for your conventions. Three things here are outside what an instruction file can do:

  • Enforce a stop. An instruction is something the agent may follow. run counts runs and refuses with exit code 75 once the budget or the stop rule applies, and the plugin's hooks deny test commands issued outside run.

  • Check old tests. "Do not modify the tests" does not tell you whether assertions changed. guard diffs the snapshot taken at start, guard --run runs the original tests against the new code, and a completed report is refused until each change is justified. In one public experiment that sentence in the prompt made little difference to how often agents gamed tests they could not pass. Those runs special-cased code instead of editing tests, which the guard does not catch.

  • Stay small. CLAUDE.md is loaded in full every session. Memory here returns pointers to current code and verified lessons only when they match the task, and nothing when nothing matches.

Dashboard and agent roadmap

context-harness --project . dashboard TASK_ID        # live, read-only, http://127.0.0.1:<port>
context-harness --project . export-dashboard TASK_ID --output task.html   # one offline HTML file

The dashboard shows the task, its acceptance criteria and status tiles (verification, checks and run budget, test guard, tokens, memory) above four tabs: Overview, Agents, Checks (pass/fail/refused results) and Events (the log, filterable by kind). A replay player at the bottom steps through the recorded events, with markers coloured by outcome; #agents, #checks and #events in the URL open a tab directly. The Agents tab is the agent roadmap:

  • Node graph. The orchestrator and every agent it created as nodes joined by branch edges (who spawned whom), each with what it does, its type, git branch, model, run time, tool calls, checks and tokens. A dashed edge with a branch label marks an agent that forked onto another git branch; hovering a node highlights its path back to the orchestrator.

  • Timeline. When each agent was active.

  • Tokens by agent. Cached input, fresh input and output, with a table view.

Agents enter the roadmap from start (the solver is the orchestrator) and assign (children of the coordinator, or of --parent; --role, --branch and --agent-model describe them). Token usage is never estimated; it is imported from what the agent runtimes already write:

context-harness --project . import-session --transcript ~/.claude/projects/<project>/<session>.jsonl
context-harness --project . import-session --transcript ~/.codex/sessions/.../rollout-*.jsonl --agent codex-1
context-harness --project . usage TASK_ID --agent reviewer --input 120000 --cached 90000 --output 4000
context-harness --project . agents                   # the roadmap as JSON

A Claude Code session import reads the main transcript and every subagent transcript next to it (<session>/subagents/agent-*.jsonl with its .meta.json), so the tree shows each spawned bot with its description and tokens. Codex rollout files (~/.codex/sessions) and codex exec --json output are read the same way. With the plugin installed, the Stop hook imports the stopping session into the active task automatically. Importing the same source again replaces its numbers instead of adding them.

Hooks

Without hooks the protocol is advisory: nothing stops an agent from running npm test directly or editing an old assertion until it passes. The plugin adds two hooks (plugins/context-harness/hooks/):

Hook

Does

PreToolUse on Bash

Denies test and check commands issued outside the harness (npm test, npm run check, npx vitest, pytest, go test, cargo test, python -m unittest, python -m pytest and close variants such as yarn test or npm run test:unit) and tells the agent the exact run command to use instead. Calls of ch.py / context-harness pass.

Stop

Imports the stopping session's agent tree and token usage into the active task (best effort). Then blocks finishing while guard flags rewritten or deleted pre-existing tests that the latest report does not justify with --test-change "PATH: why". After 3 blocks per task it lets the agent stop and shows a warning instead, so it cannot loop forever.

Both hooks are no-ops unless a context-harness task is active for the project: the project state has a current task that is not closed and had activity in the last 12 hours (CONTEXT_HARNESS_HOOK_TTL_HOURS). The PreToolUse rule also ends once the task's agent reports. The hooks find state the same way the CLI does (CONTEXT_HARNESS_STATE, else the state of the git root of the hook's working directory). They fail open: an internal error never blocks the agent. Set CONTEXT_HARNESS_HOOKS=off to disable them. Command detection is a heuristic over the shell line; it does not follow scripts, aliases or variables.

Platform support

  • macOS and Linux, Python 3.10–3.12 (tested in CI).

  • Windows is not supported for run: it needs POSIX process groups and a POSIX shell, and refuses with an error there. Use WSL. The other commands are untested on Windows.

  • git is strongly recommended: the test guard and the "nothing changed" check use it.

What it stores and what it sends

  • State lives outside the repository in ~/.context-harness/projects/<name>-<hash> (CONTEXT_HARNESS_HOME changes the root): an SQLite database, task packets, test snapshots and the full log of every check.

  • No network access by default. Two explicit exceptions: --backend decisions or --backend responses sends search candidates to the OpenAI API with your OPENAI_API_KEY, and prepare_semantic.py downloads pinned public model files when you run it.

  • run executes commands the agent chooses, with your environment. Your client's tool approval applies to it; mcp --read-only removes every tool that runs commands or writes.

  • Memory, logs and retrieved text are treated as untrusted data, never as instructions.

  • import-session (and the plugin's Stop hook) reads local agent transcripts and stores per-agent metadata in the state database: the first prompt or subagent description (up to 600 characters), branch, model, timestamps, tool-call counts and token totals. It stores no transcript content beyond that.

Evidence so far

One task (a paginated-list bug in a private web application, 10 hidden acceptance tests), one run per condition, Codex as the agent. Tokens are logical input + output tokens reported by Codex.

Condition

Hidden tests

Tokens

Time

Plain Codex (historical control)

10/10

426,987

206 s

v0.3 skill (historical)

10/10

570,150

285 s

v0.5 skill

10/10

702,276

370 s

With n = 1 and historical controls this supports no conclusion except that v0.5 did not save tokens on that run. Its trace showed two causes, both changed afterwards and not re-measured: irrelevant memory pointers at start (addressed by a relevance cutoff) and a silent 75-second check that made the agent poll (addressed by a heartbeat). A comparison that could show an effect is specified in docs/benchmark-protocol.md; it has not been run.

Repository layout

Path

What

.claude-plugin/marketplace.json

Marketplace with one plugin, read by Claude Code and Codex

plugins/context-harness/

The plugin: .claude-plugin/ and .codex-plugin/ manifests, .mcp.json (Claude Code), .codex-mcp.json (Codex), hooks/, and skills/context-harness/

plugins/context-harness/skills/context-harness/

The bundled skill. Generated from src/ by scripts/bundle_skill.py and committed, because plugins install straight from git

src/context_harness/

The Python package; skill_data/ holds SKILL.md, references and helper scripts

tests/

Unit and integration tests (standard library unittest)

scripts/ch.py

Run the CLI from a checkout without installing

Development

PYTHONPATH=src python3 -m unittest discover -s tests

After editing anything under src/, regenerate the plugin's skill folder (CI fails if it is stale):

python3 scripts/bundle_skill.py

Validate the manifests with Claude Code's checker:

claude plugin validate --strict .

A release bumps the version in pyproject.toml and src/context_harness/__init__.py (PEP 440, e.g. 0.5.0a1), in both plugin.json files and .claude-plugin/marketplace.json (the same version in semver form, e.g. 0.5.0-alpha.1, which claude plugin tag requires; a test checks that they agree), and adds a CHANGELOG.md entry. claude plugin tag plugins/context-harness --dry-run shows the tag it would create.

License

MIT, see LICENSE.

Available Tools

7 tools
checkpointC

Current task state for resuming: criteria, latest reports, budget.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNo
projectNoAbsolute project root (once per session is enough)

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. The name 'checkpoint' is ambiguous about read vs. write, and the description says nothing about side effects, persistence, permissions, or whether calling it mutates state. It does not resolve the ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short, front-loaded fragment with no wasted words, but it is under-specified rather than genuinely concise, leaving core questions (verb, side effects, scoping) unanswered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and only partial parameter coverage, the description should do more work. It hints at returned fields but omits the operation type, scoping behavior, and any behavioral constraints, leaving the definition inadequate for a 2-param resumable-state tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Two parameters at 50% schema coverage: 'project' is documented in-schema but 'task' is not. The description never mentions either input, instead listing returned content (criteria, reports, budget), so it adds no meaning about how task/project scope the result.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (current task state) and enumerates its contents (criteria, latest reports, budget), so an agent can infer it returns resumable state. However, there is no clear verb (get? write? update?) and nothing distinguishes it from siblings like 'report' or 'remember', which could also surface state.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only usage signal is the phrase 'for resuming', which hints at a scenario but never states when to prefer this over 'report', 'search', or 'remember'. No exclusions, prerequisites, or alternatives are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

guardC

List pre-existing tests whose lines were rewritten or deleted. With run_template (containing {files}) run the original tests against the current code.

ParametersJSON Schema
NameRequiredDescriptionDefault
cwdNo
projectNoAbsolute project root (once per session is enough)
run_templateNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that tests are listed and executed against current code, but says nothing about permissions, side effects, reversibility, or what happens when tests fail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, with the primary listing purpose front-loaded. The second sentence is slightly telegraphic but still earns its place by explaining the run_template placeholder.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, an ambiguous tool name, and incomplete parameter documentation, the description is too thin. It does not explain return values, output shape, or how the listed tests relate to the subsequent run.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is low at 33%, so the description must compensate. It adds useful meaning for run_template by noting it contains a {files} placeholder, but it leaves cwd undocumented and adds little for project beyond the schema's own description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verbs ('List', 'run') on a clear resource ('pre-existing tests' / 'original tests') and explains the run_template mechanism. It does not differentiate this tool from siblings like search or run, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It implies the context of rewritten or deleted lines, but never states when to choose guard over search, run, report, or checkpoint. No prerequisites, exclusions, or alternative conditions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rememberB

Propose a project lesson that is not visible in the code (stored unverified).

ParametersJSON Schema
NameRequiredDescriptionDefault
titleYes
domainNo
sourceYes
contentYes
projectNoAbsolute project root (once per session is enough)

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does add one meaningful behavioral fact: the entry is stored "unverified" and framed as a proposal, so the agent knows this is a tentative write rather than an authoritative one. It still omits permission requirements, whether repeated calls deduplicate, and what the stored result looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the key constraint (not visible in code / unverified) arrives immediately. It is efficient, though one more clause on what belongs in title vs content would not have hurt.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a write tool with no annotations and no output schema, the description does convey the essential behavioral point (unverified proposal). It falls short on parameter meaning, which matters given three required fields and near-zero schema coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 20% (just the project field), and the description explains none of the five parameters. Title, content, source and domain — three of which are required — have no semantics in either the description or the schema, so an agent must guess what belongs in each.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description pairs a specific verb ("Propose") with a concrete resource ("a project lesson") and adds a scope qualifier ("not visible in the code"), so an agent can tell broadly what the tool captures. It does not, however, differentiate itself from any of the terse siblings (report, checkpoint, guard), which are plausible places to record information.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"not visible in the code" implies a usage condition for what belongs here, which is a reasonable implied trigger. But there is no explicit guidance on when to prefer this over siblings like report or checkpoint, and no stated prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reportB

Finish the task: status, summary, changed files and evidence. Rewritten pre-existing tests need test_changes entries 'PATH: why'.

ParametersJSON Schema
NameRequiredDescriptionDefault
statusYes
projectNoAbsolute project root (once per session is enough)
summaryYes
evidenceNo
artifactsNo
test_changesNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and it does disclose one genuinely non-schema behavioral rule: rewritten pre-existing tests require test_changes entries formatted 'PATH: why'. Beyond that it is silent on whether calling this terminates the session, whether repeated calls are allowed, or what side effects occur.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler, and the primary directive is front-loaded ahead of the special-case rule. Minor ambiguity in 'changed files' (which does not map cleanly to a parameter name) is the only cost.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description could explain what reporting produces, but it does not. For a 6-parameter terminal tool with 17% schema coverage, leaving artifacts and the project scope largely unexplained is a meaningful gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 17% (just 'project'), so the description must compensate. It names status, summary, and evidence, and clarifies the test_changes format, but it never explains 'artifacts' or disambiguates 'changed files' from the artifacts/test_changes parameters, leaving key parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific action ('Finish the task') plus the content to supply (status, summary, changed files, evidence), so the agent knows this is the terminal reporting step. It is clear on its own, but it never names or contrasts itself against plausible alternatives like checkpoint, remember, or guard, which are the tools it could be confused with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Finish the task' implies this is called at the end, which is useful implied timing. However, there is no explicit statement of when-not to use it (e.g., vs. checkpoint for intermediate progress) and no named alternative, so the routing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runA

Run a test/check command in the project under the task budget. Returns exit code, failing tests, first failure and the log path (full log stays on disk). full=true for the complete check. Wait for the result; a refusal means: stop and report.

ParametersJSON Schema
NameRequiredDescriptionDefault
cwdNoFolder inside the project
fullNo
commandYes
projectNoAbsolute project root (once per session is enough)
timeoutNo
overrideNoNew hypothesis; limited, recorded

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses the return shape (exit code, failing tests, first failure, log path with full log on disk), the meaning of full=true, synchronous waiting behavior, and the refusal-stop-and-report failure protocol. It omits permission/auth needs, the nature of the 'task budget', and any side-effect profile for arbitrary commands.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Terse, front-loaded, and free of filler; the core action and its return contract come first, then the full flag and the refusal protocol. The telegraphic phrasing ('a refusal means: stop and report') is efficient but slightly compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter command-execution tool with no annotations and no output schema, the description covers return values and control flow but leaves the command contract, timeout behavior, and budget semantics unexplained. Adequate but with clear gaps given the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%: cwd, project, and override are documented in-schema, while command, full, and timeout are bare types. The description partially compensates by explaining full=true ('complete check') but adds nothing for command or timeout, leaving half the surface undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Run a test/check command in the project', which is a clear, concrete action. It does not explicitly name a sibling to contrast against, but none of the siblings (start, report, search, remember, checkpoint, guard) overlap with test execution, so an agent can still place it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives operational guidance ('Wait for the result; a refusal means: stop and report') and context ('under the task budget'), which implies when this tool is appropriate. However, it never states when to use it versus alternatives or when not to run it, leaving usage inference to the reader.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

startB

Start a coding task: snapshot pre-existing tests, return memory pointers/lessons and a run budget. Call once per task.

ParametersJSON Schema
NameRequiredDescriptionDefault
goalYes
queryNoSymptom in English plus known file/identifier names
titleYes
domainNo
projectNoAbsolute project root (once per session is enough)
max_runsNoTest/check runs allowed (default 12)
acceptanceYes
allow_pathsNo
max_after_greenNoRuns after the full check is green (default 1)
max_fail_streakNoFailures of one test before runs stop (default 3)

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose real behavior beyond the name: it snapshots pre-existing tests and returns memory pointers/lessons plus a run budget, which hints at session lifecycle and side effects. It stops short of stating whether state is persisted, what preconditions must hold, or what happens if called twice.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences lead with the action and add only operational facts, with no filler. The phrase "memory pointers/lessons" is slightly opaque and mildly compressed, keeping it just below a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, only 50% schema coverage, no annotations, and no output schema, the description should do more. It never explains the required goal/acceptance contract or what the returned "run budget" means, leaving an agent to infer how to shape the call at the start of every task session.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%, the baseline case. "Run budget" loosely maps to max_runs/max_after_green/max_fail_streak, but the description leaves title, goal, acceptance, domain, allow_paths, and query unexplained and adds no syntax or format detail beyond the schema. Adequate but not compensating for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb+resource ("Start a coding task") and adds the mechanics: snapshotting pre-existing tests, returning memory pointers/lessons, and a run budget. It is clear what the tool does, though it never names a sibling such as run or checkpoint, so differentiation relies on inference from "Call once per task."

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Call once per task" gives a usage constraint that implicitly separates it from the per-run sibling 'run', which is useful. However, there is no explicit when-not guidance, no stated prerequisites (e.g. that project root is only needed once per session), and no routing advice toward report, checkpoint, or guard.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 7 tool updatesv0.1.0
    • First observedcheckpoint
    • First observedguard
    • First observedremember
    • First observedreport
    • First observedrun
    • First observedsearch
    • First observedstart

TDQS

B3.2/5.0

Scored across 7 tools

Disambiguation4/5

Each tool targets a distinct phase of the task lifecycle: start (begin), run (execute tests), report (finish), checkpoint (resume state), search/remember (read/write memory), guard (test-integrity check). The only mild overlap is start vs checkpoint, since both surface task/budget state, but their descriptions differentiate begin-task from resume-task.

Naming Consistency4/5

All names are single lowercase words with a consistent terse style (start, run, report, search, remember, guard). The one deviation is checkpoint, a noun rather than a verb, but it still fits the minimal naming convention.

Tool Count5/5

Seven tools is well-scoped for a coding-task harness: three cover the task flow (start/run/report), two cover memory, and two cover resumption and test integrity. Each tool earns its place with no redundancy.

Completeness4/5

The surface covers the full task lifecycle plus memory and integrity checks, which is strong. Minor gaps exist around memory management (no listing/deleting or verifying lessons) and checkpoint has no explicit restore counterpart, but agents can work around these.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI coding agents to enforce spec-driven development and verify code before it is marked done, using six tools that catch invented APIs, scan for hallucinated content, check plugin conformance, sandbox-run tests, validate schemas, and record audit evidence.
    771 npm
    8
    PolyForm Noncommercial 1.0.0
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables autonomous agents to enforce financial guardrails by tracking real-time token burn and cost velocity, automatically throttling or freezing execution when budgets are exceeded. It also blocks destructive shell commands, prompt injections, and runaway planning loops while sanitizing outputs before they are returned.
    7
    MIT