context-harness
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@context-harnessrun the test suite, stop if it fails 3 times, and summarize the failures"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
context-harness
A harness for coding agents (Claude Code, Codex, any MCP client) that changes code in a repository:
Budgeted check runs. Every test or check goes through
run. It keeps a countdown (12 runs by default), stops after the full check is green plus one confirmation run, stops a test that failed 3 times, and refuses an identical run on an unchanged tree. Overrides are allowed, limited and recorded.Shaped output. The full log stays on disk; the agent gets the exit code, failing test names, the first failure and the tail. Long checks print a heartbeat so the agent waits instead of polling.
Pre-existing test guard. Test files are snapshotted at task start.
guardlists rewritten or deleted assertions,guard --runruns the originals against the new code, and acompletedreport is refused until each rewrite is justified.Project memory. Verified lessons and pointers to current code, delivered only when they look relevant; empty context is a normal answer.
Agent roadmap. A local dashboard shows the orchestrator and every subagent it spawned, with git branch, model, run time and tokens read from Claude Code transcripts and Codex logs.
Installer for Claude Code, Codex, Cursor, Claude Desktop and Gemini CLI, plus this plugin marketplace with hooks for Claude Code and Codex.
Python 3.10+, standard library only. The same state and protocol serve the CLI, the skill and the MCP server.
The dashboard's Agents tab for a demo task. The task, agents and token numbers are sample data.
Install
The commands below install from dimqsh/context-harness.
Claude Code plugin
claude plugin marketplace add dimqsh/context-harnessclaude plugin install context-harness@context-harnessThis installs the skill, the MCP server (started as python3 …/ch.py mcp, so python3 must be on
PATH) and the two hooks. Restart Claude Code afterwards.
Codex plugin
codex plugin marketplace add dimqsh/context-harnesscodex plugin add context-harness@context-harnessCodex reads the same marketplace file and uses plugins/context-harness/.codex-plugin/plugin.json: the
skill, the MCP server (from .codex-mcp.json, because Codex does not expand ${CLAUDE_PLUGIN_ROOT} in MCP
arguments) and the hooks. Codex lists plugin hooks as untrusted until you trust them; until then only the
skill and the MCP server are active. In non-interactive runs with -a never we saw Codex refuse MCP tool
calls that need approval; the skill's CLI path (scripts/ch.py) does not depend on MCP.
Skill only
For agents that read SKILL.md and have a shell. Clone the repository, then preview and apply:
git clone https://github.com/dimqsh/context-harness context-harnesspython3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --skill claude-code --skill codexpython3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --skill claude-code --skill codex --writeThe installer is a dry run unless --write is given, never replaces an existing install without
--force, and keeps a backup of everything it replaces. Copying
plugins/context-harness/skills/context-harness into ~/.claude/skills/ or ~/.codex/skills/ by hand
works too. A skill alone has no hooks, so its rules are advice the agent may ignore.
MCP server for Cursor, Claude Desktop and others
python3 context-harness/plugins/context-harness/skills/context-harness/scripts/ch.py install --mcp cursor --mcp claude-desktop --writeOther targets: claude-code, codex, gemini, claude-project (with --target-project DIR) and
json, which prints a generic mcpServers entry for any other client. For GUI clients the installer pins
your current PATH so run finds node, npm and the like. A server started as ch.py mcp --read-only
exposes only guard, search and checkpoint and cannot run commands or write.
pip
pip install "git+https://github.com/dimqsh/context-harness"This gives the context-harness CLI and the context-harness-mcp server. The package carries the skill
files, so context-harness install --skill codex --mcp cursor --write works from a pip install too; MCP
targets then run python -m context_harness mcp with the same interpreter. Local multilingual search is
an optional extra (context-harness[semantic]), see
semantic-search.md.
Related MCP server: universal-jev-mcp
How an agent uses it
CH below is python3 <skill>/scripts/ch.py --project <absolute project root> --compact. MCP clients call
the tools of the same names (start, run, guard, report, search, remember, checkpoint).
CH start --title "Fix invoice rounding" --goal "Totals round half up to cents" \
--acceptance "1.005 rounds to 1.01" --query "rounding error in invoice totals src/totals.ts"
CH run -- npx vitest run src/totals.test.ts # a focused test
CH run --full -- npm run check # the complete project check
CH guard # did I rewrite old assertions?
CH report --status completed --summary "..." --artifact src/totals.ts \
--evidence "npm run check: exit 0"A refused run exits with code 75 and says why. The full protocol, including multi-agent coordination and independent verification, is in protocol.md; budgets and output shaping are in token-budget.md.
How it differs
context-harness bounds how an agent checks its own work and shows what each agent spent on a task. It is not mainly a memory system, a permission guard or a cost tracker, and it runs next to those tools.
Tool | What it does | Relation to context-harness |
Captures what the agent does in each session, compresses it and injects relevant context into later sessions. | Overlaps on memory only. Memory here is narrower: verified lessons and pointers to current code, returned above a relevance cutoff. The run budget and the test guard are the main work. | |
Semantic code retrieval and editing tools over MCP, backed by language servers. | Complementary: Serena helps the agent find and change code; context-harness bounds how it checks the result. | |
A context-aware guard that blocks dangerous agent tool calls. | Complementary. Both use a | |
Token usage and cost from local transcripts, by day and model (ccusage) or by model, project and task (CodeBurn). | Overlaps on reading transcripts. context-harness puts tokens on each agent in the spawn tree of one task, next to that task's checks and test guard. It does not price tokens. |
Why not just CLAUDE.md?
Keep CLAUDE.md or AGENTS.md for your conventions. Three things here are outside what an instruction file can do:
Enforce a stop. An instruction is something the agent may follow.
runcounts runs and refuses with exit code 75 once the budget or the stop rule applies, and the plugin's hooks deny test commands issued outsiderun.Check old tests. "Do not modify the tests" does not tell you whether assertions changed.
guarddiffs the snapshot taken atstart,guard --runruns the original tests against the new code, and acompletedreport is refused until each change is justified. In one public experiment that sentence in the prompt made little difference to how often agents gamed tests they could not pass. Those runs special-cased code instead of editing tests, which the guard does not catch.Stay small. CLAUDE.md is loaded in full every session. Memory here returns pointers to current code and verified lessons only when they match the task, and nothing when nothing matches.
Dashboard and agent roadmap
context-harness --project . dashboard TASK_ID # live, read-only, http://127.0.0.1:<port>
context-harness --project . export-dashboard TASK_ID --output task.html # one offline HTML fileThe dashboard shows the task, its acceptance criteria and status tiles (verification, checks and run budget,
test guard, tokens, memory) above four tabs: Overview, Agents, Checks (pass/fail/refused results) and Events
(the log, filterable by kind). A replay player at the bottom steps through the recorded events, with markers
coloured by outcome; #agents, #checks and #events in the URL open a tab directly. The Agents tab is the
agent roadmap:
Node graph. The orchestrator and every agent it created as nodes joined by branch edges (who spawned whom), each with what it does, its type, git branch, model, run time, tool calls, checks and tokens. A dashed edge with a branch label marks an agent that forked onto another git branch; hovering a node highlights its path back to the orchestrator.
Timeline. When each agent was active.
Tokens by agent. Cached input, fresh input and output, with a table view.
Agents enter the roadmap from start (the solver is the orchestrator) and assign (children of the
coordinator, or of --parent; --role, --branch and --agent-model describe them). Token usage is never
estimated; it is imported from what the agent runtimes already write:
context-harness --project . import-session --transcript ~/.claude/projects/<project>/<session>.jsonl
context-harness --project . import-session --transcript ~/.codex/sessions/.../rollout-*.jsonl --agent codex-1
context-harness --project . usage TASK_ID --agent reviewer --input 120000 --cached 90000 --output 4000
context-harness --project . agents # the roadmap as JSONA Claude Code session import reads the main transcript and every subagent transcript next to it
(<session>/subagents/agent-*.jsonl with its .meta.json), so the tree shows each spawned bot with its
description and tokens. Codex rollout files (~/.codex/sessions) and codex exec --json output are read
the same way. With the plugin installed, the Stop hook imports the stopping session into the active
task automatically. Importing the same source again replaces its numbers instead of adding them.
Hooks
Without hooks the protocol is advisory: nothing stops an agent from running npm test directly or
editing an old assertion until it passes. The plugin adds two hooks (plugins/context-harness/hooks/):
Hook | Does |
| Denies test and check commands issued outside the harness ( |
| Imports the stopping session's agent tree and token usage into the active task (best effort). Then blocks finishing while |
Both hooks are no-ops unless a context-harness task is active for the project: the project state has
a current task that is not closed and had activity in the last 12 hours
(CONTEXT_HARNESS_HOOK_TTL_HOURS). The PreToolUse rule also ends once the task's agent reports. The
hooks find state the same way the CLI does (CONTEXT_HARNESS_STATE, else the state of the git root of the
hook's working directory). They fail open: an internal error never blocks the agent. Set
CONTEXT_HARNESS_HOOKS=off to disable them. Command detection is a heuristic over the shell line; it
does not follow scripts, aliases or variables.
Platform support
macOS and Linux, Python 3.10–3.12 (tested in CI).
Windows is not supported for
run: it needs POSIX process groups and a POSIX shell, and refuses with an error there. Use WSL. The other commands are untested on Windows.gitis strongly recommended: the test guard and the "nothing changed" check use it.
What it stores and what it sends
State lives outside the repository in
~/.context-harness/projects/<name>-<hash>(CONTEXT_HARNESS_HOMEchanges the root): an SQLite database, task packets, test snapshots and the full log of every check.No network access by default. Two explicit exceptions:
--backend decisionsor--backend responsessends search candidates to the OpenAI API with yourOPENAI_API_KEY, andprepare_semantic.pydownloads pinned public model files when you run it.runexecutes commands the agent chooses, with your environment. Your client's tool approval applies to it;mcp --read-onlyremoves every tool that runs commands or writes.Memory, logs and retrieved text are treated as untrusted data, never as instructions.
import-session(and the plugin'sStophook) reads local agent transcripts and stores per-agent metadata in the state database: the first prompt or subagent description (up to 600 characters), branch, model, timestamps, tool-call counts and token totals. It stores no transcript content beyond that.
Evidence so far
One task (a paginated-list bug in a private web application, 10 hidden acceptance tests), one run per condition, Codex as the agent. Tokens are logical input + output tokens reported by Codex.
Condition | Hidden tests | Tokens | Time |
Plain Codex (historical control) | 10/10 | 426,987 | 206 s |
v0.3 skill (historical) | 10/10 | 570,150 | 285 s |
v0.5 skill | 10/10 | 702,276 | 370 s |
With n = 1 and historical controls this supports no conclusion except that v0.5 did not save tokens on
that run. Its trace showed two causes, both changed afterwards and not re-measured: irrelevant memory
pointers at start (addressed by a relevance cutoff) and a silent 75-second check that made the agent poll
(addressed by a heartbeat). A comparison that could show an effect is specified in
docs/benchmark-protocol.md; it has not been run.
Repository layout
Path | What |
| Marketplace with one plugin, read by Claude Code and Codex |
| The plugin: |
| The bundled skill. Generated from |
| The Python package; |
| Unit and integration tests (standard library |
| Run the CLI from a checkout without installing |
Development
PYTHONPATH=src python3 -m unittest discover -s testsAfter editing anything under src/, regenerate the plugin's skill folder (CI fails if it is stale):
python3 scripts/bundle_skill.pyValidate the manifests with Claude Code's checker:
claude plugin validate --strict .A release bumps the version in pyproject.toml and src/context_harness/__init__.py (PEP 440, e.g.
0.5.0a1), in both plugin.json files and .claude-plugin/marketplace.json (the same version in semver
form, e.g. 0.5.0-alpha.1, which claude plugin tag requires; a test checks that they agree), and adds a
CHANGELOG.md entry. claude plugin tag plugins/context-harness --dry-run shows the tag it would create.
License
MIT, see LICENSE.
Available Tools
7 toolscheckpointC
Current task state for resuming: criteria, latest reports, budget.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | ||
| project | No | Absolute project root (once per session is enough) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. The name 'checkpoint' is ambiguous about read vs. write, and the description says nothing about side effects, persistence, permissions, or whether calling it mutates state. It does not resolve the ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short, front-loaded fragment with no wasted words, but it is under-specified rather than genuinely concise, leaving core questions (verb, side effects, scoping) unanswered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, and only partial parameter coverage, the description should do more work. It hints at returned fields but omits the operation type, scoping behavior, and any behavioral constraints, leaving the definition inadequate for a 2-param resumable-state tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Two parameters at 50% schema coverage: 'project' is documented in-schema but 'task' is not. The description never mentions either input, instead listing returned content (criteria, reports, budget), so it adds no meaning about how task/project scope the result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the resource (current task state) and enumerates its contents (criteria, latest reports, budget), so an agent can infer it returns resumable state. However, there is no clear verb (get? write? update?) and nothing distinguishes it from siblings like 'report' or 'remember', which could also surface state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The only usage signal is the phrase 'for resuming', which hints at a scenario but never states when to prefer this over 'report', 'search', or 'remember'. No exclusions, prerequisites, or alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
guardC
List pre-existing tests whose lines were rewritten or deleted. With run_template (containing {files}) run the original tests against the current code.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | ||
| project | No | Absolute project root (once per session is enough) | |
| run_template | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that tests are listed and executed against current code, but says nothing about permissions, side effects, reversibility, or what happens when tests fail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, with the primary listing purpose front-loaded. The second sentence is slightly telegraphic but still earns its place by explaining the run_template placeholder.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations, no output schema, an ambiguous tool name, and incomplete parameter documentation, the description is too thin. It does not explain return values, output shape, or how the listed tests relate to the subsequent run.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is low at 33%, so the description must compensate. It adds useful meaning for run_template by noting it contains a {files} placeholder, but it leaves cwd undocumented and adds little for project beyond the schema's own description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verbs ('List', 'run') on a clear resource ('pre-existing tests' / 'original tests') and explains the run_template mechanism. It does not differentiate this tool from siblings like search or run, so it falls short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies the context of rewritten or deleted lines, but never states when to choose guard over search, run, report, or checkpoint. No prerequisites, exclusions, or alternative conditions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rememberB
Propose a project lesson that is not visible in the code (stored unverified).
| Name | Required | Description | Default |
|---|---|---|---|
| title | Yes | ||
| domain | No | ||
| source | Yes | ||
| content | Yes | ||
| project | No | Absolute project root (once per session is enough) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does add one meaningful behavioral fact: the entry is stored "unverified" and framed as a proposal, so the agent knows this is a tentative write rather than an authoritative one. It still omits permission requirements, whether repeated calls deduplicate, and what the stored result looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the key constraint (not visible in code / unverified) arrives immediately. It is efficient, though one more clause on what belongs in title vs content would not have hurt.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a write tool with no annotations and no output schema, the description does convey the essential behavioral point (unverified proposal). It falls short on parameter meaning, which matters given three required fields and near-zero schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 20% (just the project field), and the description explains none of the five parameters. Title, content, source and domain — three of which are required — have no semantics in either the description or the schema, so an agent must guess what belongs in each.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description pairs a specific verb ("Propose") with a concrete resource ("a project lesson") and adds a scope qualifier ("not visible in the code"), so an agent can tell broadly what the tool captures. It does not, however, differentiate itself from any of the terse siblings (report, checkpoint, guard), which are plausible places to record information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"not visible in the code" implies a usage condition for what belongs here, which is a reasonable implied trigger. But there is no explicit guidance on when to prefer this over siblings like report or checkpoint, and no stated prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reportB
Finish the task: status, summary, changed files and evidence. Rewritten pre-existing tests need test_changes entries 'PATH: why'.
| Name | Required | Description | Default |
|---|---|---|---|
| status | Yes | ||
| project | No | Absolute project root (once per session is enough) | |
| summary | Yes | ||
| evidence | No | ||
| artifacts | No | ||
| test_changes | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and it does disclose one genuinely non-schema behavioral rule: rewritten pre-existing tests require test_changes entries formatted 'PATH: why'. Beyond that it is silent on whether calling this terminates the session, whether repeated calls are allowed, or what side effects occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no filler, and the primary directive is front-loaded ahead of the special-case rule. Minor ambiguity in 'changed files' (which does not map cleanly to a parameter name) is the only cost.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description could explain what reporting produces, but it does not. For a 6-parameter terminal tool with 17% schema coverage, leaving artifacts and the project scope largely unexplained is a meaningful gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 17% (just 'project'), so the description must compensate. It names status, summary, and evidence, and clarifies the test_changes format, but it never explains 'artifacts' or disambiguates 'changed files' from the artifacts/test_changes parameters, leaving key parameters undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific action ('Finish the task') plus the content to supply (status, summary, changed files, evidence), so the agent knows this is the terminal reporting step. It is clear on its own, but it never names or contrasts itself against plausible alternatives like checkpoint, remember, or guard, which are the tools it could be confused with.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Finish the task' implies this is called at the end, which is useful implied timing. However, there is no explicit statement of when-not to use it (e.g., vs. checkpoint for intermediate progress) and no named alternative, so the routing is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
runA
Run a test/check command in the project under the task budget. Returns exit code, failing tests, first failure and the log path (full log stays on disk). full=true for the complete check. Wait for the result; a refusal means: stop and report.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Folder inside the project | |
| full | No | ||
| command | Yes | ||
| project | No | Absolute project root (once per session is enough) | |
| timeout | No | ||
| override | No | New hypothesis; limited, recorded |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does well: it discloses the return shape (exit code, failing tests, first failure, log path with full log on disk), the meaning of full=true, synchronous waiting behavior, and the refusal-stop-and-report failure protocol. It omits permission/auth needs, the nature of the 'task budget', and any side-effect profile for arbitrary commands.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Terse, front-loaded, and free of filler; the core action and its return contract come first, then the full flag and the refusal protocol. The telegraphic phrasing ('a refusal means: stop and report') is efficient but slightly compressed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter command-execution tool with no annotations and no output schema, the description covers return values and control flow but leaves the command contract, timeout behavior, and budget semantics unexplained. Adequate but with clear gaps given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: cwd, project, and override are documented in-schema, while command, full, and timeout are bare types. The description partially compensates by explaining full=true ('complete check') but adds nothing for command or timeout, leaving half the surface undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Run a test/check command in the project', which is a clear, concrete action. It does not explicitly name a sibling to contrast against, but none of the siblings (start, report, search, remember, checkpoint, guard) overlap with test execution, so an agent can still place it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives operational guidance ('Wait for the result; a refusal means: stop and report') and context ('under the task budget'), which implies when this tool is appropriate. However, it never states when to use it versus alternatives or when not to run it, leaving usage inference to the reader.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
searchC
Search project memory: verified lessons/facts in full, code as file pointers.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| query | Yes | ||
| domain | No | ||
| project | No | Absolute project root (once per session is enough) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose one genuinely useful behavior: verified lessons/facts are returned in full while code is returned only as file pointers. That is real result-shaping context. It says nothing, however, about ranking, limits, permissions, or what happens on an empty match.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no padding; the scope and result shape come first. It is slightly compressed to the point of being cryptic ('code as file pointers' takes a moment to parse), but nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with no annotations, no output schema, and 25% parameter coverage, this description is too thin. It partially covers return semantics but omits parameter meaning and usage context that the agent needs to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 25% (just the project root), so the description should compensate for query, limit, and domain — and it does not. None of those three parameters are explained anywhere, leaving their semantics (e.g. limit default, domain values) undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Search) and resource (project memory) and even characterizes the result shape: verified lessons/facts in full, code as file pointers. It is clearly distinguishable from siblings like remember/checkpoint/guard, which are write-oriented, though the name 'search' alone would be ambiguous without the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of alternatives. The agent is not told when to search memory versus using remember, checkpoint, or report, nor what a good query looks like.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
startB
Start a coding task: snapshot pre-existing tests, return memory pointers/lessons and a run budget. Call once per task.
| Name | Required | Description | Default |
|---|---|---|---|
| goal | Yes | ||
| query | No | Symptom in English plus known file/identifier names | |
| title | Yes | ||
| domain | No | ||
| project | No | Absolute project root (once per session is enough) | |
| max_runs | No | Test/check runs allowed (default 12) | |
| acceptance | Yes | ||
| allow_paths | No | ||
| max_after_green | No | Runs after the full check is green (default 1) | |
| max_fail_streak | No | Failures of one test before runs stop (default 3) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It does disclose real behavior beyond the name: it snapshots pre-existing tests and returns memory pointers/lessons plus a run budget, which hints at session lifecycle and side effects. It stops short of stating whether state is persisted, what preconditions must hold, or what happens if called twice.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences lead with the action and add only operational facts, with no filler. The phrase "memory pointers/lessons" is slightly opaque and mildly compressed, keeping it just below a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 parameters, only 50% schema coverage, no annotations, and no output schema, the description should do more. It never explains the required goal/acceptance contract or what the returned "run budget" means, leaving an agent to infer how to shape the call at the start of every task session.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%, the baseline case. "Run budget" loosely maps to max_runs/max_after_green/max_fail_streak, but the description leaves title, goal, acceptance, domain, allow_paths, and query unexplained and adds no syntax or format detail beyond the schema. Adequate but not compensating for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a concrete verb+resource ("Start a coding task") and adds the mechanics: snapshotting pre-existing tests, returning memory pointers/lessons, and a run budget. It is clear what the tool does, though it never names a sibling such as run or checkpoint, so differentiation relies on inference from "Call once per task."
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Call once per task" gives a usage constraint that implicitly separates it from the per-run sibling 'run', which is useful. However, there is no explicit when-not guidance, no stated prerequisites (e.g. that project root is only needed once per session), and no routing advice toward report, checkpoint, or guard.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v0.1.0- First observed
checkpoint - First observed
guard - First observed
remember - First observed
report - First observed
run - First observed
search - First observed
start
TDQS
Scored across 7 tools
Each tool targets a distinct phase of the task lifecycle: start (begin), run (execute tests), report (finish), checkpoint (resume state), search/remember (read/write memory), guard (test-integrity check). The only mild overlap is start vs checkpoint, since both surface task/budget state, but their descriptions differentiate begin-task from resume-task.
All names are single lowercase words with a consistent terse style (start, run, report, search, remember, guard). The one deviation is checkpoint, a noun rather than a verb, but it still fits the minimal naming convention.
Seven tools is well-scoped for a coding-task harness: three cover the task flow (start/run/report), two cover memory, and two cover resumption and test integrity. Each tool earns its place with no redundancy.
The surface covers the full task lifecycle plus memory and integrity checks, which is strong. Minor gaps exist around memory management (no listing/deleting or verifying lessons) and checkpoint has no explicit restore counterpart, but agents can work around these.
Maintenance
Related MCP Connectors
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Enforces architectural invariants and code quality rules inside your AI agent's edit loop.
QA platform for agents: coverage signals, in-repo test plans, verified tests and release governance.
Agent routing, safety preflight, execution receipts, best execution and verified outcomes.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceEnables AI coding agents to enforce spec-driven development and verify code before it is marked done, using six tools that catch invented APIs, scan for hallucinated content, check plugin conformance, sandbox-run tests, validate schemas, and record audit evidence.771 npm8PolyForm Noncommercial 1.0.0
- AlicenseAqualityBmaintenanceEnables coding agents to compact conversation contexts verbatim, make fast decisions through choice, boolean, and rubric scoring, and enforce command safety guardrails.6MIT
- AlicenseAqualityAmaintenanceZero-dependency token optimizer, test-failure triage gate, and System 1.5 semantic guardrail for AI coding agents. Intercepts compiler errors, missing dependencies, and doom loops in < 500µs.6124 PyPI11MIT
- AlicenseNot gradedqualityBmaintenanceEnables autonomous agents to enforce financial guardrails by tracking real-time token burn and cost velocity, automatically throttling or freezing execution when budgets are exceeded. It also blocks destructive shell commands, prompt injections, and runaway planning loops while sanitizing outputs before they are returned.7MIT