Skip to main content
Glama

reqtrace-mcp

Make Claude Code prove its work. An MCP server and Claude Code plugin that traces every change to a requirement and every requirement to a test that actually exercises it. It adds an independent reviewer that can't edit code and a sign-off that only a human can give. A benchmark measures whether any of this helps.

Writing software is getting cheap; trusting it isn't.

Built with Claude Code. Inspired by public safety-critical practice (DO-178C-style traceability, independence and structural coverage). It is not a qualified tool and makes no certification claim. All examples are synthetic.

Status: M0–M4 complete: library, server, plugin and a 60-run benchmark. The headline result is honest and mixed; see Results. Languages: Python (pytest) and C/C++ (GoogleTest or plain test executables, clang coverage with MC/DC), tested on a real library: google/crc32c.


What it does

requirements/*.yaml ──► # implements: REQ-001 ──► @pytest.mark.verifies("REQ-001") ──► run under coverage
                                                                                          │
         human sign-off ◄── evidence report ◄── independent verifier ◄── gaps ◄──────────┘

Status

Meaning

NOT_IMPLEMENTED

No code says it implements the requirement

NOT_VERIFIED

No test says it verifies it

FAILING

A linked test fails

WEAKLY_VERIFIED

Linked tests pass but never execute the implementing code (per-test coverage)

VERIFIED

Linked tests pass and execute the implementing code

REVIEWED

Verified, and a human approved this exact content (a sign-off goes stale if the code changes)

Gap analysis also finds:

  • code no requirement asked for;

  • implementing functions that no linked test calls (for example one overload of four);

  • dangling links;

  • uncovered branches, and in C/C++ MC/DC conditions never shown to matter on their own;

  • stale reviews, and results that are stale since the last test run. Change impact maps a git diff to the affected requirements, the tests to rerun, and the reviews it invalidates. Requirement text edits and deleted functions are included.

Why the design choices matter

  • Human-only sign-off, enforced by protocol. record_review is marked anthropic/requiresUserInteraction, so Claude Code prompts on every call, in every permission mode. The reviewer's name and rationale come from the human by MCP elicitation. They are never tool arguments, so the model can't fill them in.

  • Separation of duties. The independent-verifier sub-agent has no Edit, Write or Bash tools, and judges code against requirement text only.

  • Deterministic server. It never calls an LLM. Every output can be reproduced and audited.

  • Safe by construction. The server only starts the repo's own tests (coverage run -m pytest, or the C/C++ build and test binaries named in .reqtrace.yaml), as argv lists built from config with no shell. No tool accepts a command. Paths are confined to the repo.

See docs/DECISIONS.md for the trade-offs behind each choice.

Related MCP server: Metsuke

Quickstart

Needs Python 3.10+ and uv. For Python repos: pytest and coverage in the target repo's environment. For C/C++ repos: clang and LLVM tools.

As a Claude Code plugin (server, skill, sub-agents and commit gate):

claude plugin marketplace add hishanparry/reqtrace-mcp   # once published on GitHub
claude plugin install reqtrace@reqtrace-mcp

Or, without installing, from a clone: claude --plugin-dir /path/to/reqtrace-mcp.

As a plain MCP server for any MCP client: run uvx --from /path/to/reqtrace-mcp reqtrace-mcp over stdio.

As a CLI, for humans and CI:

pip install -e .
cd examples/altitude-alerter
reqtrace verify && reqtrace matrix && reqtrace gaps
reqtrace review ALT-001 --approve --reviewer "Your Name" --rationale "Checked boundaries"
reqtrace report            # .reqtrace/evidence/evidence.md
reqtrace gate --verify     # exit 1 if gaps exceed .reqtrace.yaml thresholds

For C or C++, set language: cpp in .reqtrace.yaml and give the build commands and test binaries. reqtrace builds with clang coverage flags itself, in its own build directory:

language: cpp
source_dirs: [src, include]
build:
  commands:
    - [cmake, -S, ., -B, "{build_dir}", -G, Ninja, -DBUILD_TESTING=ON]
    - [cmake, --build, "{build_dir}"]
tests:
  - {binary: "{build_dir}/my_tests", kind: gtest}                        # one process per test
  - {binary: "{build_dir}/c_api_test", kind: exe, source: tests/c_api_test.c}  # exit code

Annotate with comments: // implements: REQ-001 above a function, and // verifies: REQ-001 above a TEST(...) or a test main. This needs clang and LLVM tools (llvm-profdata, llvm-cov); MC/DC needs clang 18 or later.

The Python example repo seeds one of each problem. Without changing anything you get one VERIFIED, one WEAKLY_VERIFIED (a tagged test that only checks a constant), one untested and one unimplemented requirement, plus one unrequired function.

MCP surface

Tools: list_requirements, get_requirement, trace_matrix, find_gaps, impact_of_change, run_verification, coverage_report, lint_requirements, flag_requirement_issue, propose_links (stored as unconfirmed), record_review (human-only) and evidence_report. All return structured output; read-only tools are annotated.

Resources: reqtrace://requirements, reqtrace://requirements/{id}, reqtrace://matrix, reqtrace://evidence/latest.

Prompts: implement_traceably, independent_review.

Built on the official Python SDK 2.x and the stateless 2026-07-28 protocol. Tested with both 2026 and 2025-era clients.

The benchmark

evals/ asks whether Claude Code produces more trustworthy work with reqtrace than without. Both arms get the same repo, prompt, model, tools and a CONTRIBUTING.md describing the annotation convention. The only difference is the plugin.

  • 10 tasks in four invented domains: battery controller, robot joint limits, thruster limiter and ROV depth alarm.

    • 4 implement tasks.

    • 2 requirement-change tasks.

    • 3 review tasks: one boundary defect, one missing requirement plus an unrequired "maintenance mode" that silences the alarm, and one clean control.

    • 1 task with conflicting requirements.

  • Deterministic graders. Hidden acceptance tests. Verification strength: the agent's tests are run against a reference implementation and against one mutant per requirement, and a requirement counts only if a test linked to it catches its mutant. Link precision and recall. Unrequired code. Seeded-defect detection and false positives. Whether the conflict was flagged. Cost.

  • Validated before use. python -m evals check proves that each reference passes, that each mutant is caught by its own requirement's hidden tests, and that the starting code fails. The grader has its own tests.

python -m evals check
python -m evals run --runs 3 --out evals/results/mvp
python -m evals report evals/results/mvp

Results (10 tasks × 2 arms × 3 runs, Claude Sonnet 5.5)

results

On this benchmark, reqtrace did not measurably improve the work, and it cost 23% more per run (54% more turns). Sonnet 5.5 alone solved every task in every run:

  • all hidden tests passed;

  • every seeded defect was found, with no false positives;

  • the requirement conflict was flagged.

That left nothing for a tool to improve.

The one difference is behaviour. With reqtrace the agent ran its own tests in 16/30 runs; without it, 0/30. In one task, all three baseline runs shipped a test that fails against correct code. Part of that gap comes from the strict permission setup, so it is not a clean win.

The plugin was also never used in review tasks (0/9 runs), and the independent verifier was never chosen unprompted.

Method, caveats, failure analysis and next steps: RESULTS.md.

Real codebase: google/crc32c

The C/C++ support was tested on google/crc32c, a CRC-32C library with portable, SSE4.2 and ARM64 implementations, GoogleTest typed tests and a C test program. I wrote 8 requirements from RFC 3720 and the public API, and added 34 comment lines of annotations to upstream files. Build files were unchanged.

On the upstream tests (22 passing), reqtrace found:

  • no test of empty input;

  • one of four convenience overloads never called;

  • the two widest SSE4.2 loops never executed: they need 4,080+ bytes, and the largest test buffer is 2,048;

  • the portable fallback and the CPU-feature probe unreachable on an SSE4.2 machine (MC/DC 0%);

  • ARM64 code not compiled on x86-64.

Claude Code with the plugin (one run, no general shell) then:

  • added six tests;

  • left the hardware-dependent gaps open, with the right reasons;

  • found an untested acceptance criterion that reqtrace can't see.

A one-bit error in either SSE4.2 lookup table passes all 22 upstream tests and fails the agent's new test. That is a test gap, not a bug: the tables are correct.

Full write-up, reproduction script and raw output: examples/crc32c-case-study.

Limitations, honestly

  • Python (pytest) and C/C++ (GoogleTest or plain executables) only. Links are at function level, not per acceptance criterion: an untested criterion inside a tested requirement is invisible to the tool.

  • C/C++:

    • clang only (no GCC/gcov, no MSVC);

    • coverage only covers code compiled on the host;

    • each test runs in its own process, which is slow for very large suites;

    • per-test coverage includes start-up code;

    • plain test executables report one result for everything in them;

    • the scanner doesn't run the preprocessor, so macro-generated functions can be missed;

    • clang 18 measures MC/DC only for decisions of up to 6 conditions.

  • Annotations are claims. reqtrace checks them with per-test coverage, but execution is not assertion: a test can run code and check nothing. The benchmark measures that with mutants. Mutation testing inside the tool is future work.

  • requiresUserInteraction is a Claude Code hint. Other MCP clients may not honour it, though elicitation still keeps the reviewer's identity away from the model.

  • The commit gate only sees commits Claude makes. reqtrace gate --verify works as a normal git pre-commit hook for humans.

  • Benchmark: small synthetic tasks, and the model hit the ceiling. Showing a benefit on correctness needs harder tasks and a permission ablation. Both are planned in RESULTS.md.

  • The skill doesn't yet trigger on review or requirement-change prompts.

Repo layout

src/reqtrace_mcp/   core library (no MCP dependency), CLI, MCP server, pytest plugin,
                    C/C++ scanner (tree-sitter) and runner (clang coverage)
skills/ agents/ hooks/ .claude-plugin/   Claude Code plugin
examples/altitude-alerter/   demo repo with one of each gap
examples/crc32c-case-study/  C++ case study on google/crc32c (setup script, results, agent run)
evals/              benchmark: domains, tasks, harness, graders, results
docs/               SPEC, DECISIONS (ADRs), DEVLOG

Licence: Apache-2.0.

Available Tools

12 tools
coverage_reportB
Read-only

Statement and branch coverage (plus MC/DC for C/C++ with clang >= 18) of each requirement's implementing code from the latest run, with uncovered lines, branches and conditions, and implementing code that no linked test executes.

ParametersJSON Schema
NameRequiredDescriptionDefault
req_idNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
run_idYes
requirementsYes

TDQS

B3.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, openWorldHint=false, destructiveHint=false), so the burden is lower. The description nonetheless adds real behavioral context: results come 'from the latest run' (temporal scope), MC/DC is conditional on C/C++ with clang >= 18 (capability gating), and it includes code no linked test executes (untested-code reporting).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence that front-loads the core coverage types before qualifications. No filler, though the clause stacking makes it slightly heavy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value shape needn't be explained, and coverage content is well described. The gap is the sole input parameter and when to choose this tool over siblings, leaving an agent unsure how to scope the call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single req_id parameter, so the description carries the compensation burden and does not meet it. It never mentions req_id, its optionality, or whether omitting it reports all requirements versus a single one.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (coverage of each requirement's implementing code) and enumerates its contents: statement/branch coverage, MC/DC, uncovered lines/branches/conditions, and untested implementing code. This is clearly distinguishable from siblings like evidence_report or trace_matrix. It never explicitly contrasts itself with an alternative, which keeps it out of 5 territory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use, when-not-to-use, or alternative tools are mentioned. The agent must infer that this is the coverage-metric tool from the content alone, without routing guidance among the eleven siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evidence_reportA

Write the evidence pack (Markdown + matrix JSON) to .reqtrace/evidence/ for a human to review. Run run_verification first so results are current.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
pathYes
countsYes
open_gapsYes
unrequired_codeYes

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, and idempotentHint=false, so the agent knows this is a non-read, non-destructive write. The description usefully adds the output artifacts and the ordering dependency on run_verification, but it does not disclose what happens on re-run (overwrite vs. append) despite idempotentHint=false, which is the key behavioral unknown for a write tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, no filler: the action and destination come first, and the prerequisite follows. Every clause carries information the agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is unnecessary, and the description covers the action, artifact, location, and ordering. The one material omission is re-run/overwrite semantics, which the non-idempotent annotation implies is relevant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a parameterless tool is 4. No gap exists here, but the description also contributes nothing beyond that baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Write) and a concrete resource (the evidence pack: Markdown + matrix JSON) with an explicit destination path (.reqtrace/evidence/). This is clearly distinguishable from siblings like run_verification, coverage_report, or trace_matrix, which produce or inspect results rather than persisting an artifact.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the prerequisite and sibling: 'Run run_verification first so results are current,' with the reason supplied. Usage context (for human review, after verification) is clear, but there is no explicit when-not guidance, e.g. whether it should be avoided mid-review or re-run speculatively.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_gapsC
Read-only

Everything standing between this repo and verified, signed-off requirements, each with a suggested next action. clean is true when no blocking gaps remain.

ParametersJSON Schema
NameRequiredDescriptionDefault
kindsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
gapsYes
cleanYesTrue when no blocking gaps remain
countsYes

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds the notion of a per-gap "suggested next action" and the `clean` flag semantics, but since an output schema exists this is largely redundant return-value context rather than new behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with the outcome stated first and the `clean` flag second; nothing is padded. The first sentence is somewhat florid ("everything standing between this repo and...") but still front-loads the payoff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return format need not be explained, but the sole input parameter is undocumented everywhere and, among eleven siblings with overlapping reporting/linting purposes, no routing context is offered. For a tool in this dense a family, that leaves real gaps for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter `kinds` has 0% schema description coverage and is never mentioned in the description, so the agent cannot tell what values are valid or what filtering it performs. With low coverage and no compensating text, the description fails to carry parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description conveys that this tool surfaces outstanding gaps blocking verified/signed-off requirements, which is a recognizable resource. But the framing is metaphorical ("everything standing between this repo and...") rather than a crisp verb+resource statement, and it gives no differentiation from closely related siblings such as coverage_report, lint_requirements, or evidence_report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no when-not-to-use, and no alternative tool named. The only implicit signal is "no blocking gaps remain," which hints at a pre-sign-off check but leaves the agent to guess when this is preferred over coverage_report or lint_requirements.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flag_requirement_issueA

Record a problem with a requirement for a human to resolve. Use this instead of guessing when a requirement is ambiguous, conflicts with another, or can't be tested.

ParametersJSON Schema
NameRequiredDescriptionDefault
kindYes
req_idYes
descriptionYes
related_req_idsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYes
kindYes
req_idYes
relatedNo
raised_byYes
timestampYes
descriptionYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already disclose this is a non-read-only, non-idempotent, non-destructive write, so the safety profile is covered elsewhere. The description usefully adds that the record is escalated to a human rather than auto-resolved, but says nothing about duplicate flags, whether it mutates the requirement itself, or what the created artifact looks like. With annotations carrying the load, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero filler; the action and its escalation target come first, and the substitution rule follows immediately. Nothing could be removed without losing routing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return format needn't be described, and the annotations cover the write semantics. The remaining hole is input-level: with 0% schema coverage the description leaves `description` and `related_req_ids` unexplained, which is a real gap for a 4-parameter mutation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it only partly does: the enumerated problem conditions imply the `kind` values, and 'a requirement' implies `req_id`. The required `description` field and the optional `related_req_ids` parameter are never explained, leaving two of four parameters undocumented in both places.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb+resource (record a problem with a requirement) and states the outcome (for a human to resolve), which is more precise than the tool name alone. It does not, however, explicitly differentiate itself from nearby workflow siblings like record_review or find_gaps, which could also be seen as ways to surface requirement problems.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use triggers that map directly to the `kind` enum values (ambiguous, conflicts, can't be tested) and frames it as the alternative to guessing. What's missing is a name-check of the sibling that would otherwise be tempting (find_gaps, lint_requirements) or a note on when *not* to flag.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_requirementB
Read-only

One requirement with its implementing code, linked tests, latest results, coverage, review state and number of open issues.

ParametersJSON Schema
NameRequiredDescriptionDefault
req_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
reviewNoLatest review decision, if current
statusYes
coverageNo
open_issuesNo
requirementYes
verified_byYes
test_resultsYesnodeid -> outcome, latest run
implemented_byYes
unexercised_codeNoImplementing items no passing linked test executes (when others are)
tests_exercising_codeNoLinked tests that executed implementing code

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds what the response aggregates (code links, tests, coverage, review state, open issue count), which is useful context, but says nothing about behavior on a missing or invalid req_id.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tight sentence with the resource front-loaded and no filler. It is a noun phrase rather than a sentence, which costs a little clarity but wastes nothing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the returned fields need not be re-explained, and the annotations carry the safety profile. The remaining gap is the undocumented required parameter and no note on failure behavior, which leaves a minimum-viable but incomplete definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single required req_id parameter, and the description never mentions it, leaving its expected format or identifier source unexplained. With a real parameter at zero coverage, the description fails to compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The fragment 'One requirement with its implementing code, linked tests, latest results, coverage, review state and number of open issues' clearly identifies the resource (a single requirement) and the depth of detail returned. It is distinguishable from the plural sibling list_requirements, though no explicit verb is used and no sibling is named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The singular 'One requirement' implicitly signals a per-requirement detail fetch versus list_requirements, but there is no explicit when-to-use, when-not-to-use, or named alternative. Usage must be inferred from the name and the payload list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

impact_of_changeA
Read-only

Requirements and tests affected by changes since base_ref (a git ref), including requirement text edits and deleted functions. Lists tests to re-run and reviews that the change invalidates.

ParametersJSON Schema
NameRequiredDescriptionDefault
base_refNoHEAD
include_uncommittedNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
base_refYes
changed_filesYes
changed_itemsYesFunctions/classes/tests touched by the diff
tests_to_rerunYes
include_uncommittedYes
reviews_invalidatedYes
affected_requirementsYesreq id -> reasons

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a safe, read-only, closed-world operation, so the safety burden is covered. The description adds real behavioral detail beyond that (it detects requirement text edits and deleted functions, and reports invalidated reviews), but says nothing about defaults, scope, or cost despite the two documented defaults in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the core purpose front-loaded and the output content appended. The parenthetical about base_ref is well placed, though the second sentence slightly overlaps the first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, and annotations cover the safety profile. What remains thin is orientation: no guidance on when this beats trace_matrix or coverage_report, and no note on the effect of the uncommitted-changes flag, for a tool that is clearly a multi-input repository analysis.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry parameter meaning. It clarifies base_ref as 'a git ref' and pins the comparison window ('since base_ref'), which is genuine added value, but include_uncommitted is entirely unexplained in both schema and description, and the HEAD default is not addressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific analysis verb and its object: requirements and tests affected by changes since a git ref, including text edits and deleted functions. It is concrete and unambiguous on its own, but it never names or distinguishes itself from overlapping siblings such as trace_matrix or coverage_report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the phrase 'Lists tests to re-run and reviews that the change invalidates' signals this is a post-change impact analysis. There is no explicit when-to-use, no prerequisites (e.g., must be run in a repo), and no routing to or away from the sibling analysis tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_requirementsA
Read-only

Deterministic requirement quality checks (missing acceptance criteria, vague words, compound requirements). Judging ambiguity and conflicts is your job: use flag_requirement_issue.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the burden is low. The description adds genuinely useful behavioral context by declaring the checks are deterministic, which tells the agent results are reproducible and rule-based rather than judged. It does not discuss result volume or ordering, but that is minor here.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the tool's function, followed immediately by the routing guidance. No filler and no repetition of the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and the safety profile is covered by annotations. With zero parameters and a clearly stated scope plus an explicit hand-off to flag_requirement_issue, nothing an agent needs to select or invoke this tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so per the baseline this scores 4. The description adds nothing parameter-related because there is nothing to document, and the empty schema is unambiguous (it presumably operates over the current requirement set).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('requirement quality checks') and enumerates the concrete check classes (missing acceptance criteria, vague words, compound requirements). It also explicitly distinguishes itself from flag_requirement_issue, so an agent can separate the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit boundary routing: this tool covers deterministic checks, while 'Judging ambiguity and conflicts is your job: use flag_requirement_issue.' This tells the agent both when to use this tool and which sibling to use instead for the excluded case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_requirementsB
Read-only

List requirements with their current verification status.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagNo
statusNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
countsYes
requirementsYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered without the description. The description adds one piece of genuine context — that verification status is returned — but says nothing about filtering behavior, result size, or pagination.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no wasted words. It is arguably too terse for the 2-parameter filtering surface it hides, but as pure structure it is efficient and well-ordered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be enumerated, and both parameters are optional so invocation risk is low. However, the filter semantics of the two optional parameters are undocumented in both schema and description, leaving a real gap for a tool whose only knobs are those filters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and neither parameter has a schema description, so the description carries the full burden. It mentions 'verification status', which loosely corresponds to the status enum parameter, but never explains that tag or status act as filters, nor does it clarify that status accepts one of the defined Status values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (requirements) plus the included payload (current verification status). It reads clearly against the singular sibling get_requirement, but never names alternatives such as coverage_report or find_gaps, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no when-to-use guidance, no conditions, and no exclusions. With 11 sibling tools (get_requirement, coverage_report, find_gaps, trace_matrix), an agent gets no signal about why it would choose this over them; usage is only implied by the verb 'List'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_reviewA

Ask a HUMAN to sign off requirements. The human is prompted for their name and rationale; you cannot supply them. If no human answers, nothing is recorded.

ParametersJSON Schema
NameRequiredDescriptionDefault
req_idsYes
decisionYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
detailYes
recordedYes

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the mutation/interactivity profile (readOnly=false, idempotent=false, closed world). The description adds genuinely non-derivable behavior: a human is prompted for name and rationale, the agent cannot supply them, and a non-answer records nothing. That human-in-the-loop and failure-mode detail is exactly the context annotations can't carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action, with the human-prompt constraint and the failure mode each getting one clause. Nothing redundant; every sentence carries distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and the annotations cover the safety profile; the description covers the crucial human-in-the-loop semantics and the no-op case. The only real gap is the undocumented parameters, which is a schema-side deficiency more than a description one.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and there are 2 required parameters, so the description must compensate — yet neither req_ids nor decision is mentioned, and the approve/reject boundary of decision is left to the schema. 'Requirements' and 'sign off' only loosely gesture at the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action (record a human sign-off on requirements) with the key distinguishing trait that a HUMAN, not the agent, makes the call. That differentiates it from automated siblings like run_verification or flag_requirement_issue, though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the sign-off semantics ('ask a human to sign off requirements'), and it notes the no-response outcome, but it never states when to prefer this over flag_requirement_issue or how it relates to run_verification/evidence_report. No explicit when-not or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_verificationA

Run the repo's tests under coverage and map results to requirements. Python: pytest with per-test contexts. C/C++: builds with clang coverage (and MC/DC) using the build commands in .reqtrace.yaml, then runs each test in its own process. With req_ids, runs only tests linked to those requirements. The commands come from the repo's config, never from arguments.

ParametersJSON Schema
NameRequiredDescriptionDefault
req_idsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
run_idYes
outcomesYes
timed_outYes
duration_sYes
output_tailYesTail of test output, for diagnosing errors
failed_testsYes
requirement_statusYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=false and openWorldHint=false, so the bar is lower. The description still adds real value: it discloses that commands are sourced from .reqtrace.yaml and 'never from arguments', and that C/C++ tests each run in their own process, conveying an expensive build+instrument+execute operation. It does not say what artifacts are written, which is the only notable gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the core action and then progressively adding language-specific detail. No filler. Slightly dense with implementation detail (per-test contexts, MC/DC) that could be trimmed, but each clause is informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is unnecessary, and the description covers invocation mechanics, language branching, and the config-sourced command constraint. The remaining omission is runtime/artifact side effects, which matters for a non-read-only tool but is not critical to correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single req_ids parameter, so the description must carry the load — and it does ('With req_ids, runs only tests linked to those requirements'), clarifying that it is a filter rather than a selector of tests to execute. It stops short of saying whether values are requirement IDs in the same format used by get_requirement/list_requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb+resource ('Run the repo's tests under coverage and map results to requirements') and specifies per-language mechanics. It is clearly distinct from read-only siblings like coverage_report or trace_matrix, but it never names an alternative, so the differentiation is left implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The req_ids clause gives one usable condition ('runs only tests linked to those requirements'), which implies the scoped vs unscoped workflow. However, there is no explicit when-not-to-use guidance and no pointer to sibling tools such as coverage_report when the agent only wants existing data rather than a fresh run.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

trace_matrixB
Read-only

Full trace matrix: requirement -> code -> tests -> latest results, plus code that no requirement asked for and annotations that name unknown requirements.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
run_idYes
generated_atYes
requirementsYes
dangling_linksYes
unrequired_codeYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds genuinely useful behavioral content by naming the two unexpected payload categories (unrequested code and annotations naming unknown requirements), which tells the agent to expect edge-case rows. It stops short of saying whether the matrix is full-system or scoped, which matters for a tool with no filter parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with the primary deliverable front-loaded and the edge-case content trailing. No filler, though the arrow chain and the 'plus' clause make it slightly harder to parse than a plain statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and zero parameters, the description need not document return fields; it correctly spends its words on scope instead. It is complete enough to call, but a reader cannot tell how this differs from the sibling reporting tools, which is the one remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; baseline 4 applies. No parameter claims are made that could mislead.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete deliverable: a full traceability matrix spanning requirement -> code -> tests -> latest results, plus orphan code and annotations referencing unknown requirements. That is a specific resource and scope, not a tautology. It does not, however, distinguish itself from siblings like coverage_report, evidence_report, or find_gaps, which on their face produce overlapping views.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance and no sibling is named. An agent must infer from the content that this is the broad end-to-end traceability view rather than a filtered report, which is exactly the decision it needs help with among eleven siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 12 tool updatesv0.1.0
    • First observedcoverage_report
    • First observedevidence_report
    • First observedfind_gaps
    • First observedflag_requirement_issue
    • First observedget_requirement
    • First observedimpact_of_change
    • First observedlint_requirements
    • First observedlist_requirements
    • First observedpropose_links
    • First observedrecord_review
    • First observedrun_verification
    • First observedtrace_matrix

TDQS

A3.5/5.0

Scored across 12 tools

Disambiguation4/5

Most tools target distinct actions: get_requirement vs list_requirements (one vs all), run_verification vs coverage_report (execution vs metrics), find_gaps vs lint_requirements (blocking gaps vs quality checks). There is mild overlap between trace_matrix, coverage_report, and get_requirement since all surface mapping/coverage data, but the descriptions give enough framing to distinguish them.

Naming Consistency4/5

All names use consistent snake_case with a mostly verb_noun pattern (get_requirement, list_requirements, run_verification, lint_requirements, flag_requirement_issue, propose_links, record_review). A few are noun phrases (trace_matrix, coverage_report, evidence_report, impact_of_change), which is a minor deviation but still predictable and readable.

Tool Count5/5

Twelve tools sit squarely in the well-scoped range for a requirements-traceability and verification server. Each tool maps to a distinct lifecycle activity (inspect, verify, report, flag, propose, sign off), so no tool feels redundant or missing.

Completeness4/5

The surface covers the full traceability loop: listing/getting requirements, running verification, coverage, gap detection, quality linting, issue flagging, link proposals, human sign-off, and evidence export. The only gap is no direct tool to author or edit requirements, but this appears intentional given the annotation-based, human-in-the-loop design.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    B
    maintenance
    Integrates authoritative security compliance frameworks (ISO 27001, NIST 800-53, OWASP ASVS, NIST SSDF) into AI-assisted development, offering control lookups, cross-framework mappings, build-time guardrails, and automated audit evidence generation.
    141 npm
    3
    MIT
  • A
    license
    Not graded
    quality
    F
    maintenance
    Provides SDLC compliance verification as tools that AI agents can invoke, continuously monitoring and evaluating development processes.
    MIT