OrangePro MCP is a local, privacy-preserving test-evidence server that maps code, links tests, ranks coverage gaps, proves whether tests catch breaks, and helps generate grounded tests.
Analyze a local repo or PR and build an evidence graph, then write detailed and summary HTML reports, tests.csv, rtm.md, and metadata.
Find unproven code/behaviors ranked by OrangePro Risk Score.
Run mutation-style targeted proof (orangepro_prove / orangepro_prove_loop) to mark functions Proven or reveal weak tests.
Generate 1–5 grounded tests using BYOK OpenAI, Ollama, or Anthropic; deterministic mode returns placeholders only.
Stage or apply optional AI candidate links and flows as weak suggestions that never affect proven coverage.
Inspect changed-file/PR impact and diff-scoped test opportunities.
Query workspace status, graph readiness score, gap-fill stats, next-evidence recommendations, and traceability matrix.
Record agent test-writing outcomes as static diagnostics.
Export metadata-only evidence packs and optionally a self-contained offline evidence-graph explorer.
Keep analysis local: no code uploads, telemetry, source storage, or source-file edits.
Supports generating and running tests with the Jest framework.
Supports generating and running tests with the Mocha framework.
Enables local test generation with Ollama models.
Enables test generation using OpenAI's models.
Supports generating and running tests with the pytest framework.
Supports generating and running tests with the Vitest framework.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@OrangePro MCP Check my code for untested behaviors and generate tests"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
OrangePro maps every public function in your repository, links each one to the tests that actually call it, and ranks what's left by what it can break. For a test you care about, it breaks the function in an isolated copy and checks that the test fails. It runs on your machine, uses no AI model for scoring, and sends nothing to OrangePro. Mapping, gap ranking and proof work with no model key. Drafting a runnable test needs a provider. orangepro_generate_tests with provider set to deterministic runs with no key and returns placeholders only: the body is expect(true).toBe(true); // placeholder — replace with a concrete assertion, runnable is false, and there is no suggested_path, run_command or prove_run. That mode smoke-tests the pipeline.
npx -y @orangepro/mcp-server@latest start .Related MCP server: mcp-codebase-oracle
Every run writes two reports and a spreadsheet of tests to .orangepro/:
File | For | What's in it |
| Leads, managers, anyone in a hurry | One page: what to do with the 20 highest-priority functions, the top five with the reason and the tests drafted for each, what changed since the last report, and how to reproduce the run. |
| Developers | Worklist: every ranked function with what to do, filters, and a side panel with the evidence, why it ranks, two drafted tests and the proof command. Map: where tests reach, by area. Run details: inputs, counts, proofs, fingerprints and settings. |
| QA, test managers | One row per test slot (preconditions, steps, test data, expected result, priority, folder), ready for Excel, Google Sheets, TestRail, Jira Xray or Zephyr Scale. Export tests (CSV) in the detailed report downloads it, and its Run details show how the columns map (Importing tests.csv). |
Every function gets one of four things to do:
What it means | What to do | |
Done | A test fails when this code breaks. | Nothing. It moves back to Prove if the code changes. |
Prove | A test runs it, or a coverage run executes it, but no test is proven to catch a break. | Run the proof command. |
Check | A test file or test name points here, but no call was traced. | Read that test file first; prove it if it drives the function, otherwise write one. |
Write | No test catches a break here yet. | Write a test (two are drafted for the top 20), then prove it. |
Both from OrangePro run on its own repository at commit 64a5213, without a model key, so no tests were drafted. With a key, each of the top 20 shows two drafted tests, and the side panel shows their steps and code.
It tells you what a test actually checks, not just what it runs. A test that calls a function isn't necessarily checking it. opro prove replaces the function body with a fixed return value in an isolated copy and reruns your own test, unchanged. If the test still passes, it wasn't protecting that function.
It puts consequences first. One filter shows the code paths that reach a delete or purge (cache evictions don't count); others show code that changes often, has many callers or runs on a schedule.
It's honest about evidence. A function is "linked" only when a test calls that exact function. A test with a similar name is a lead, never coverage. Tests that only check mocks don't count.
It's repeatable. Same commit, full git history, same config and same version give the same ranking. Each report records fingerprints, so you can tell when a change in results came from the code and when it came from the tool.
Your coding agent can drive it. As an MCP server it gives Claude Code, Cursor, Copilot, Codex and others the ranked gaps, the suggested test location and the proof step in one loop.
It's free, local and open source (MIT). No account, no API key for analysis, and no upload.
cd /path/to/your/repo
# install the repo's own dependencies first (npm ci, uv sync, go mod download, ...)
npx -y @orangepro/mcp-server@latest start .
open .orangepro/short_behavior-coverage.htmlMapping, gap ranking and proof work with no model key. Drafting a runnable test needs a provider (see Model setup). Deterministic mode returns placeholders only, as described above.
Tips for the most accurate run:
Use a full clone, not a shallow one. Change history drives part of the ranking, and the report says when history was partial.
Exclude what isn't product code with rank_exclude_paths (see Configuration).
Rerun after a change. Each report says what changed since the last one. When nothing changed (same commit, files, settings, coverage and proofs), start reuses the last reports at once and says so; --fresh runs in full anyway.
What's written:
.orangepro/
├── short_behavior-coverage.html ← one-page summary
├── behavior-coverage.html ← detailed report: worklist, map, run details
├── tests.csv ← drafted tests, one row per test
├── report-data.json ← the data both reports show (used to compare and reuse runs)
├── run-key.json, last-start.json ← what the last run saw, so an unchanged repo is not analysed again
├── graph.json ← the evidence graph (deterministic)
├── ledger.json ← proof certificates
├── rtm.md ← traceability matrix
└── config.json ← optional per-repo settings
orangepro_generated/ ← generated tests (a runnable draft needs a provider; deterministic mode returns placeholders only); your files are never editedEvery function gets exactly one evidence label. The four things to do above are read from it.
Evidence | What it means | To do |
Proven | A test passed on the original code and failed at its own assertion when this function was broken in an isolated copy. | Done |
Runs under tests | A coverage tool you ran executed this code. | Prove |
Linked | A test calls this exact function. That's a structural link, not proof. | Prove |
Unconfirmed | A test file or a test with a similar name points here, but no call was traced. That's a lead, not evidence. | Check |
No test found | Nothing links a test to it. | Write |
A proof that ran and did not catch the break turns the function into Write; a proof made before the code changed turns it back into Prove.
A successful orangepro_prove sets the oracle field proven to true and writes the ledger status reproven. That status is used the first time too. The report's Proven count is how many behaviors still match one of those certificates. The traceability row says Reproven (this run). That label does not mean the function was proved earlier.
What OrangePro doesn't count, so linked numbers are a floor, not a coverage percentage:
tests that only assert on mocks;
calls made over HTTP or from end-to-end suites;
in Python, objects that reach a test only through a fixture.
Check the tests before writing new ones for a flagged path.
"Proven 0" is normal on a first run. Proof runs your tests, so it happens only for the functions you choose, or within the attempt budget of
opro start.
Each unproven function gets an OrangePro Risk Score, ORS = P × I × D:
P: how likely it is to change. Change history, fan-out, new code and size.
I: what it can break. Incoming references, entry-point position, data sensitivity, and a floor for paths that reach a delete or purge within two calls.
D: how hard a break would be to notice. Evidence tier (a linked test or a coverage run makes a break easier to notice than a name match or nothing), and whether it runs unattended (jobs and schedulers).
The score sets the order of work. It is not a defect probability. The weights are fixed and no AI model is involved. The report shows the inputs behind every row.
# Python: the mutation value is derived from the function (return annotation or its observed result)
opro prove-loop --target-symbol 'sym:app/billing/invoices.py#void_invoice' \
--test 'tests/test_invoices.py::test_void_marks_invoice_void' --replacement sentinel
# TypeScript / JavaScript: give the inert body to substitute
opro prove-loop --target-symbol 'sym:src/orders.service.ts#OrdersService.cancel' \
--test src/orders.service.spec.ts --replacement 'return null;'Proven: the test passes on the original code and fails at its own assertion on the broken copy. The oracle then says proven: true. The ledger stores that attempt as reproven, including the first time, and the traceability row says Reproven (this run).
Not proven: the test still passes on the broken copy, so it doesn't protect that function. That's a finding too.
Unrunnable: setup failed. This is never counted either way.
orangepro_proveFor a JavaScript or TypeScript function the call needs:
target_symbol: sym:<file>#<Symbol>. A bare name is refused.
test_path: the test file, relative to the repo.
replacement: empty, or a single return of a literal, such as return null;. A throw, a call, or any other statement is refused.
replacement_mode: return-json for a normal function (the default). promise-json when the function is async, so the literal is returned as a resolved promise.
runner: jest, vitest, mocha, or auto. Use the runner the repo uses.
Worked example for an async run in index.js, tested with Jest:
{
"target_symbol": "sym:index.js#run",
"test_path": "index.test.js",
"replacement": "return null;",
"replacement_mode": "promise-json",
"runner": "jest"
}Go and Java do not take replacement. They take test_run (^TestName$ for Go, TestClass#testMethod for Java).
Find weak tests without a model key:
opro roast . # passing tests whose targeted mutant still survivesOrangePro runs as an MCP server. Add it to your client's config:
{
"mcpServers": {
"orangepro-local": {
"command": "npx",
"args": ["-y", "@orangepro/mcp-server@latest", "mcp"]
}
}
}Client | Where to put it |
Claude Code |
|
Cursor |
|
VS Code / Copilot | MCP settings |
Codex / OpenCode / Windsurf |
|
A prompt that runs the whole loop:
"Use
orangepro_start, thenorangepro_find_test_gaps. For the top gap, write a test at the suggested path, run it, then callorangepro_prove_loopand tell me whether it was proven."
Optional. Put it in .orangepro/config.json in the repository, or in ~/.orangepro/config.json for defaults across repositories. Every setting that changes the ranking is shown in the report.
{
"classification": {
"rank_exclude_paths": ["ui/**", "docs/**", "scripts/**"],
"test_support_paths": ["internal/testutil/**"],
"destructive_sinks": ["archive*"]
},
"tuning": { "churn_window_days": 180 },
"proof": { "python_runner": "auto", "attempt_limit": 20 },
"overrides": [
{ "symbol": "sym:src/legacy/shim.ts#shim", "action": "suppress", "reason": "generated shim, not product code" }
]
}rank_exclude_paths removes paths from the ranking. They are still counted as behaviors.
destructive_sinks adds delete-like calls for your codebase.
Overrides (suppress, pin, reclassify) each require a reason, and the report lists them.
Weights and tiers can't be configured.
Language | Map functions | Link tests that call them | Draft tests | Mutation proof |
TypeScript / JavaScript | ✓ | ✓ | ✓ Jest, Vitest, Mocha | ✓ Jest, Vitest, Mocha |
Python | ✓ | ✓ | ✓ pytest | ✓ pytest |
Go | ✓ | ✓ | ✓ | ✓ |
Java | ✓ | ✓ | ✓ JUnit 4 and 5 | ✓ Maven with JUnit 5 only |
Kotlin, Rust, PHP, C#, Ruby, Swift, C, C++ | ✓ | name match only | not yet | not yet |
"Name match only" means a test file or test name that looks like the function makes it Unconfirmed (Check). No call is traced, so these languages never reach Linked or Proven today.
TypeScript and JavaScript are parsed with the TypeScript compiler; the other languages with tree-sitter.
Python proof runs pytest through the repository's own environment: an existing .venv or venv, then uv or poetry when their lock files are present, else python -m pytest. Set proof.python_runner to use your own command.
Java proof needs Maven (Surefire) and JUnit 5. Gradle and JUnit 4 projects are mapped and linked, but not proven yet.
Not traced yet, so tested code can show as Unconfirmed: calls through Go test-suite methods and helper constructors, dependency injection that is wired at runtime, reflection, and calls made over HTTP or RPC.
A patched Node.js: 22.23.4 or newer 22.x, 24.22.1 or newer 24.x, or 26.12.1 or newer, on macOS or Linux. Windows isn't tested: proofs use POSIX paths and symlinks.
Use a patched Node.js. The October 21, 2026 Node.js security releases (22.23.4, 24.22.1, 26.12.1) fix several high-severity issues, and Node.js 20 is end-of-life and gets no fix. Odd-numbered lines (23, 25) have no patched release. The engines field in package.json records this, but npm only warns about it unless you set engine-strict=true. Check your version with node -v. See the Node.js security release notice.
Git, for change history. Without it, or in a shallow clone, the ranking uses structure only and the report says so. A folder that isn't a git checkout always runs in full.
Install the repository's dependencies first (npm ci, uv sync, go mod download, mvn -q install, …). Proof runs your own tests in an isolated copy and needs them to pass there.
What's read: source and test files, skipping anything in .gitignore or .orangeproignore (negations with ! aren't supported), the usual build and dependency folders (node_modules, vendor, dist, build, target, coverage, .venv, …), files over 1 MB, and generated code (files marked Code generated … DO NOT EDIT). Up to 100,000 files; ORANGEPRO_MAX_FILES lowers that and ORANGEPRO_MAX_ANALYZE_MS sets a time limit for the scan.
Test drafting covers the 20 highest-priority functions, two tests each, in the order the worklist shows, and the reports show those 20. Drafts are kept until the function's code changes.
What counts as "nothing changed" for reusing a run: the commit, every modified or untracked file that git doesn't ignore, OrangePro's settings and ignore files, coverage files, the proof ledger, the start options, whether a model key is set, and the analysis version. A run whose test drafting failed on a provider error (out of credits, rate limit) runs again in full.
The CSV column mapping follows each tool's CSV importer (see Importing tests.csv). Check the import preview the first time.
Nothing about your code or your runs is sent anywhere. There is no usage telemetry.
Source is read in-process and never stored or uploaded. Reports and the evidence graph contain metadata, not your source.
Your source files are never edited. Proofs run in an isolated copy.
Model calls happen only if you configure a key, and they go directly from your machine to the provider you chose. Keys, the provider, its endpoint and the model are read only from your shell environment, or from the env block of your MCP client's config. OrangePro reads no .env file, so a repository you scan cannot choose where your key is sent, and OrangePro never writes keys anywhere. opro setup saves only the provider and model name.
With a key, the reports and tests.csv include the tests your model drafted, and the prompts include the function names, files and evidence they're based on.
Reports load nothing from the network. The only outbound links are ones you click.
Reports include a Give feedback link and a This looks wrong link on each finding. Once in a while, after the findings, they also ask whether the report helped.
The links carry nothing about your project. The form sends only what you review and submit.
You stay anonymous unless you leave an email.
opro feedback # print the link and your settings
opro feedback off # stop the "Did this help?" question (links stay)
ORANGEPRO_FEEDBACK_URL=off # hide every feedback linkThe question appears at most once every 14 days (ORANGEPRO_FEEDBACK_COOLDOWN_DAYS). That preference is stored only on your machine. MCP clients get the link as optional result metadata (_meta["ai.orangepro/feedback_url"]), and your agent is never asked to prompt you for feedback.
opro # same as opro start .
opro start . --no-ai --no-auto # analyze + reports, no model calls, no proof attempts
opro start . --fresh # run in full even when nothing changed since the last run
opro start --base main # scope to a branch diff
opro analyze # build the evidence graph and reports
opro gaps --limit 10 # top unproven behaviors, ranked
opro prove-loop ... # mutation proof for one function (see above)
opro roast . # find tests whose mutant survives (no key needed)
opro doctor --proof # why top targets aren't proven yet
opro generate --base main # runnable tests for this branch (needs a provider; --provider deterministic returns placeholders only)
opro rtm # traceability matrix
opro export # metadata-only evidence pack
opro coverage # find or generate runtime coverage artifacts
opro feedback [on|off] # feedback link and invitation setting
opro mcp # run as an MCP server (stdio)Add --json to any read command for machine output. Run opro help for every flag.
Tool | What it does |
| Analyze, write both reports and |
| Build or refresh the evidence graph |
| Unproven behaviors ranked by ORS |
| Setup + mutation proof + report refresh for one behavior |
| Mutation proof only |
| Grounded tests. A runnable draft needs a provider. |
| What a diff touches |
| Workspace state without running anything |
| What evidence to add next |
| Graph readiness (0–100) |
| Traceability matrix |
| Aggregate statistics |
| Record a test run result |
| Why a test was generated |
| Metadata-only evidence pack |
| Incremental graph update |
| Weak behavior→code suggestions (optional AI, never evidence) |
| Candidate flows (optional AI, never evidence) |
Run your own unit and integration coverage first, then opro start. OrangePro ingests Go coverprofiles, lcov, coverage.py XML and JaCoCo XML, and keeps unit, integration and unclassified coverage separate. To label artifacts, add .orangepro/coverage-suites.json:
{
"artifacts": {
".orangepro/coverage/unit.coverprofile": { "suite": "unit", "command": "make unit-test-coverage" },
".orangepro/coverage/integration.coverprofile": { "suite": "integration", "command": "make integration-test-coverage" }
}
}Provider | Environment variable |
OpenAI-compatible |
|
Anthropic |
|
Ollama (local, no key) |
|
Auto-detect order: OpenAI → Ollama → Anthropic. Override with --provider and --model, or run opro setup.
Model output never changes an evidence tier. Only the mutation proof can mark a behavior Proven.
Generated tests land in orangepro_generated/ with the files and existing tests they're grounded on, and where and how to run them.
Drafts whose code fails the compile check are kept as plans ("Plan only"), with the reason named. Without a key, each of the top 20 shows two empty slots with a prompt you can copy into your coding agent.
One row per test, UTF-8. The first 12 columns are for import; the rest keep the priority, evidence and proof command with each test. Priority is High for the five highest-ranked functions and Medium for the rest.
Column in | TestRail | Jira Xray | Zephyr Scale |
ID | Test Case Identifier | (leave Key empty) | |
Title | Title | Summary | Name |
Section ( | Sections Hierarchy | ||
Folder ( | Test Repository Path | Folder | |
Priority | Priority | Priority | Priority (map Medium to Normal) |
Test type | Type | Labels | Labels |
Preconditions | Preconditions | (separate issues in Xray; add to Description if you want them) | Precondition |
Steps | Manual Test Step: Action | ||
Test data | Manual Test Step: Data | ||
Steps with data | Steps | Test Script (Steps) - Step | |
Expected result | Expected Result | Manual Test Step: Expected Result | Test Script (Steps) - Expected Result |
References | References | Description | Objective |
TestRail: Import from CSV with the Test Case (Text) template, one row per test case.
Jira Xray: Test Case Importer; each test imports as one manual step.
Zephyr Scale: it has no test-data field, so Steps with data carries it.
Check the importer's preview before you confirm.
opro generate --base main # tests for what this branch changed (read-only git diff)
opro generate --changed # current branch vs its base
opro generate --pr 1234 # checks out PR #1234 (asks first; refuses on a dirty tree)This repository is the free local tool. The OrangePro platform adds:
a persistent graph across PRs and repositories;
CI gates on evidence and risk changes;
requirement and incident correlation;
team dashboards.
git clone https://github.com/OrangeproAI/orangepro-mcp.git
cd orangepro-mcp && npm ci && npm run build
npm testPRs welcome. Please open an issue first for large changes.
Opt-in AI lane: propose candidate behavior-flow chains over existing deterministic entry and CodeSymbol ids. Generate writes .orangepro/flows.json only; apply=true stores survivors under analysis.candidate_flows. Candidate flows are a verify-these worklist and never affect Proven, deterministic flow counts, tiers, or coverage.
| Name | Required | Description | Default |
|---|---|---|---|
| apply | No | Apply staged flows into analysis.candidate_flows. Default false stages flows only. | |
| model | No | Model name override. | |
| provider | No | BYOK provider override. | |
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only say readOnlyHint=false, destructiveHint=false, openWorldHint=true; the description goes well beyond that by disclosing both write targets (.orangepro/flows.json on generate, analysis.candidate_flows on apply=true) and an explicit safety guarantee that candidates never affect Proven counts, tiers, or coverage. The remaining gap is that openWorldHint=true is never explained — the BYOK provider/model overrides imply remote LLM calls with auth and cost implications that are not surfaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences with the key framing ('Opt-in AI lane') front-loaded and the safety guarantee placed last, which is the right order. The jargon ('CodeSymbol ids', 'survivors', 'Proven') makes it slightly heavier than necessary, but no sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should say what comes back — a list of proposed chains with ids to verify? — and it doesn't. For an open-world tool that also writes files, the absence of any note on provider/API-key prerequisites or expected return shape leaves real gaps, though the write semantics and non-impact guarantee cover the most important behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all four parameters are already documented in structured data, and the description mostly restates the apply behavior that the schema already spells out. It adds no guidance on model/provider selection, defaults, or workspace scoping beyond what is in the schema, so baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource — proposing candidate behavior-flow chains over existing deterministic entries and CodeSymbol ids — and clearly marks this as an opt-in AI lane distinct from the deterministic one. It is clear and distinguishable in substance, but it never names the closest sibling (orangepro_ai_links) so the agent must infer the boundary from the phrase 'behavior-flow chains' vs. links.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Opt-in AI lane' plus 'Candidate flows are a verify-these worklist' implies when this tool is appropriate: an exploratory, non-authoritative pass. However, with 17 siblings including orangepro_ai_links, orangepro_prove, and orangepro_update_graph, there is no explicit statement of when to choose this over those alternatives or when not to use it at all.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Opt-in AI lane: propose weak MAY_RELATE_TO candidate links between existing deterministic behavior and CodeSymbol nodes. Generate writes .orangepro/ai/links.json only; apply=true explicitly merges survivors into candidate_edges. These links never affect proven coverage.
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | Link across all behavior nodes instead of the default gaps-only scope. | |
| apply | No | Apply staged links into candidate_edges. Default false stages links only. | |
| model | No | Model name override. | |
| provider | No | BYOK provider override. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| max_behaviors | No | Maximum behavior targets to process in this AI linking run. | |
| max_prompt_tokens | No | Approximate per-batch prompt token ceiling. | |
| symbols_per_behavior | No | Deterministic CodeSymbol shortlist size per behavior. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only say readOnlyHint=false, destructiveHint=false, openWorldHint=true; the description adds materially by naming the exact write target (.orangepro/ai/links.json) and the downstream mutation of candidate_edges under apply=true. It also reassures that links 'never affect proven coverage', which clarifies blast radius. It omits cost/credential implications of the model call (BYOK) and whether staging is idempotent or overwrites prior staged files.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no redundancy: opt-in framing first, the default behavior second, the escape hatch (apply=true) third, then the safety caveat. Every clause carries distinct information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter, no-output-schema tool, the description covers the critical ambiguity (staging vs. applying) and the boundary condition (does not touch proven coverage). It leaves the effect on previously staged links and the shape of any run summary unstated, a minor gap given the rich schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description goes further than the schema by explaining the default-vs-apply contract ('Generate writes ... only; apply=true explicitly merges survivors'), which is procedural meaning the field description alone does not convey. It says nothing about 'all', model, provider, or the token/behavior caps, leaving those to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('propose weak MAY_RELATE_TO candidate links between existing deterministic behavior and CodeSymbol nodes') and frames the tool as an 'opt-in AI lane', which separates it from the deterministic prove/coverage siblings. An agent can tell what this produces and what it deliberately is not.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It lays out the operational flow clearly: generate stages to .orangepro/ai/links.json, and apply=true merges survivors. The 'opt-in' framing signals this is not part of the default path, but it does not name a sibling alternative or state when to skip this lane entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Build or refresh the local OrangePro evidence graph from a local checkout/path plus optional CSV/Markdown enrichers. Metadata-only by default — reads source in-process but never stores or uploads code. Optionally stages/applies AI candidate flows and refreshes the report when ai_flows=true.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | Persistence privacy mode. Only metadata_only is supported. | |
| model | No | Model name override for ai_flows. | |
| paths | No | Extra template/doc files (.csv/.md) to enrich the graph. | |
| ai_flows | No | Generate and apply AI-suggested candidate flows after analyze, then refresh the behavior report. Requires a BYOK provider; skips without failing analyze if none is configured. | |
| provider | No | BYOK provider override for ai_flows. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| include_markdown | No | Enrich requirements-like Markdown docs. Default true. | |
| include_source_snippets | No | Deferred: source snippets are never persisted in the first slice. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose non-readOnly, idempotent, non-destructive. The description adds genuinely new context: it reads source in-process, never stores or uploads code, and ai_flows skips without failing when no BYOK provider is configured — useful operational behavior beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three front-loaded sentences: purpose first, then the privacy guarantee, then the optional AI behavior. No filler and nothing buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no required params and no output schema, the description covers the key behavioral concerns: privacy mode, in-process reading, and the conditional ai_flows path. It stops short of saying what the built/refreshed graph result looks like or how long a scan may take, which would help an agent set expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters including the mode enum and BYOK provider list. The description only restates the purpose of paths and ai_flows in prose, adding little syntax or format detail the schema lacks.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource ('build or refresh the local OrangePro evidence graph') and names the inputs (local checkout/path, CSV/Markdown enrichers). It does not explicitly distinguish itself from close siblings like orangepro_update_graph or orangepro_ai_flows, so an agent must infer the boundary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'build or refresh' and the ai_flows clause, which explains that AI flows are only generated when ai_flows=true. There is no explicit statement of when to call this versus orangepro_update_graph or orangepro_ai_flows, nor any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
DIFF/PR TOOL — only for real code-review situations. Summarizes what changed vs a base git ref and which local graph behaviors/tests are affected (basic changed-file impact; no deep route/API shape analysis). Requires a git repo and a real diff: returns status 'no_diff' / 'no_code_changes' (diff was docs-only) / 'missing_base_ref' / 'not_a_git_repo' with guidance instead of fabricated impact when there is nothing to analyze. For baseline (no-PR) test opportunities use orangepro_find_test_gaps.
| Name | Required | Description | Default |
|---|---|---|---|
| base_ref | No | Base git ref to diff against. Default 'main'. | |
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the readOnlyHint/destructiveHint annotations by enumerating the non-success outcomes ('no_diff', 'no_code_changes', 'missing_base_ref', 'not_a_git_repo') and promising guidance rather than fabricated impact. It also discloses the analytical ceiling (basic changed-file impact only), which is exactly the kind of honesty an agent needs to avoid over-trusting results. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the tool's identity and gating condition, and every sentence carries information rather than filler. The middle sentence is dense with parentheticals and slash-separated status codes, which costs a little readability but not correctness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still tells the agent what comes back (changed set vs base ref plus affected behaviors/tests) and what the failure statuses are. Combined with the annotation-confirmed read-only nature, an agent has enough to decide and call correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters already carry descriptions including defaults, so the schema does the heavy lifting. The description reinforces the semantics of base_ref ('vs a base git ref') but adds no format, resolution, or workspace-scoping detail beyond the structured fields. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Uses a specific verb and resource ('Summarizes what changed vs a base git ref and which local graph behaviors/tests are affected') and immediately scopes itself to DIFF/PR review. It also delimits what it is not ('no deep route/API shape analysis'), so an agent can distinguish it from graph-scoring or test-generation siblings without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the selection condition ('only for real code-review situations') and names the alternative for the other case ('For baseline (no-PR) test opportunities use orangepro_find_test_gaps'). Prerequisites (git repo, real diff) are front-loaded, so the agent knows when the call is valid at all.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recommend the smallest next source that would most improve generated-test quality, with expected score impact. Teaches what data improves output.
| Name | Required | Description | Default |
|---|---|---|---|
| goal | No | Optimization goal hint, e.g. 'better_tests'. | |
| proof | No | Proof-focused mode: explain why top targets are not Dynamically Proven (deduped blockers + smallest next steps). Read-only; mints nothing. | |
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, and the description adds that this is an advisory recommendation with expected score impact and an educational explanation of what data improves output. It does not mention permissions, rate limits, or clarifying details about what exactly is returned beyond the phrase 'expected score impact'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences and front-loads the core action: recommend the smallest next source with expected impact. 'Teaches what data improves output' is slightly redundant with the quality-improvement framing but still adds a useful orientation for the agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only recommendation tool with no output schema, the description gives the gist of the output ('expected score impact') but does not describe the return shape. It also omits when to choose this tool over sibling diagnostic tools, leaving a meaningful gap for selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents goal, proof, and workspace. The description does not add parameter-level detail beyond the schema, which is acceptable given the high coverage baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (recommend) and a specific resource (the smallest next source to improve generated-test quality), plus that it includes expected score impact. It is clearly distinct from analyze_sources or find_test_gaps, but it never names those siblings to reinforce the distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: an agent can infer this is for deciding what data to add when generated-test quality needs improvement. There is no explicit when-to-use, when-not-to-use, or comparison against alternatives such as analyze_sources or find_test_gaps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Explain why a generated test exists: which graph evidence anchors support it, source refs, and whether weak/candidate evidence was used. Exposes trust artifacts only, not internal generation logic.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. | |
| generated_test_id | Yes | The generated test id (or title) from orangepro_generate_tests. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=true and destructiveHint=false already declaring a safe read operation, the description adds meaningful behavioral context by disclosing exactly what is exposed: graph evidence anchors, source refs, and weak/candidate evidence status. It also clarifies that internal generation logic is intentionally not exposed, which prevents over-expectation about the response.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the explanatory purpose comes first, followed by the concrete artifacts exposed and the boundary against internal logic. Every clause contributes useful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the return-value burden and does so by naming the key contents: evidence anchors, source refs, and weak/candidate evidence usage. For a read-only explanation tool with full parameter schema coverage, this is sufficiently complete for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents both parameters. The description adds some semantic value by tying generated_test_id to the sibling orangepro_generate_tests tool, but it does not add syntax or format details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb-plus-resource purpose: explaining why a generated test exists by exposing graph evidence anchors, source refs, and weak/candidate evidence usage. It distinguishes this trust-explanation tool from the generation-oriented sibling orangepro_generate_tests by explicitly scoping it to trust artifacts rather than internal generation logic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is clearly implied: call this after generation when you need to audit or understand the evidence behind a generated test. It usefully sets a boundary against internal generation logic, but it does not explicitly name alternatives or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Export a portable evidence pack (JSON) plus a human-readable Markdown summary for review or later hosted promotion. Includes facts, provenance, generated outputs, and high-level score metadata — never prompts, weights, traces, or raw source. Validates against the local schema.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. | |
| graph_html | No | Also write a self-contained offline evidence-graph explorer (metadata only, no network). | |
| output_path | No | Output JSON path. Default 'orangepro-evidence-pack.json'. | |
| include_generated_bodies | No | Embed generated test bodies in the pack. Default false: bodies stay in the local workspace so the exported pack is metadata-only. Raw source snippets never cross the boundary regardless of this flag. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations (readOnlyHint=false, idempotent, non-destructive) by disclosing the exact artifact set written, the data-boundary guarantee ('never prompts, weights, traces, or raw source'), and that output validates against the local schema. For a file-writing export with security implications, these are the details an agent needs before invoking it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: what is produced, what is excluded, how it is validated. Front-loaded with the deliverable and free of filler; every clause carries information an agent would otherwise have to guess.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With annotations covering the mutation profile and a fully-described 4-parameter schema, the remaining need is clarity on what the tool writes and what it refuses to include — both explicitly covered. No output schema exists, and the description compensates by naming the returned artifacts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all four parameters are already documented, making 3 the baseline. The description reinforces the include_generated_bodies boundary ('raw source snippets never cross the boundary regardless of this flag'), adding modest value, but it does not explain workspace, output_path, or graph_html behavior beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Export a portable evidence pack') and immediately enumerates the artifacts produced (JSON + Markdown summary, optional graph HTML). No sibling tool in the list overlaps with export/packaging, so an agent can route here unambiguously.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an implied purpose ('for review or later hosted promotion') but never states when to prefer this over siblings such as orangepro_rtm or orangepro_stats, nor any precondition (e.g. must analysis/score be run first). Usage must be inferred from the goal phrasing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List requirements/flows with weak or missing test evidence, plus top_risk_gaps for unproven code symbols ranked by the OrangePro Risk Score (Probability × Impact × DetectionDifficulty). Risk is prioritization only; it never changes coverage/proof status.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max gaps to return. Default 10. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| min_priority | No | Minimum priority to include (low|medium|high|critical). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare a safe read (readOnlyHint=true, destructiveHint=false), so the description's added value is the risk model definition (Probability × Impact × DetectionDifficulty) and the explicit invariant that risk never alters coverage/proof status — non-obvious behavior an agent could otherwise misread as a status mutation. It does not mention result ordering or pagination behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with what is returned, then the ranking rule and the invariant. Every clause carries information and nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does the necessary work by naming both return sections and the ranking basis, which is enough to interpret results. Minor gaps remain: it does not describe the shape of an individual gap entry or whether requirement gaps are also ranked.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so limit, workspace, and min_priority are already documented with defaults and ranges. The description adds no further meaning about the parameters, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and two concrete resources: requirements/flows with weak or missing test evidence, plus 'top_risk_gaps' for unproven code symbols. The named output bucket and the risk-ranking criterion make it clearly distinct from siblings like orangepro_rtm or orangepro_graph_score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the description tells you what you get, from which an agent can infer this is the gap-discovery/prioritization step, but it never says when to call it versus orangepro_rtm, orangepro_stats, or orangepro_generate_tests. The closing caveat about risk not changing proof status is a helpful semantic guardrail rather than routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a small capped set (1-5) of grounded tests from local evidence using your own model key (BYOK: OpenAI-compatible, Ollama, or Anthropic). Tests are diversified across lightweight LOCAL scenario buckets (happy_path, validation_error, edge_case, integration_flow, security_privacy, regression) chosen from the evidence — unjustified buckets are skipped, never padded. By default it focuses the top-gap behavior and returns up to limit bucket-diverse tests for it; pass multiple target_ids to split the budget across them. Each test reports its bucket plus VALIDATED grounding evidence (every cited entity resolved against the local graph with its kind, evidence strength, and source_ref) and discloses weak/candidate evidence; evidence_summary reports proof coverage and any broken citations, so the grounding is verifiable rather than asserted. If evidence is too thin, returns missing-evidence guidance instead of generic tests. Returns each test's code plus run_hints (a suggested file path + run command) and an agent_workflow: YOU (the agent) write each file and run it with your shell tools, then report pass/fail + stack traces — OrangePro never writes to or runs anything in the repo. Requires the repo's test framework/deps to already be installed.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Total tests to generate (1-5), spread across local scenario buckets. Default 3. | |
| model | No | Model name override. | |
| compare | No | Run BOTH arms — prompt-only baseline vs Local KG, same model + system prompt — and return side-by-side scores (completeness, context awareness, accuracy, domain specificity) plus both test suites. Non-persisting testing view. | |
| base_ref | No | PR-scoped generation: restrict to the behaviors the diff vs this ref touches (e.g. 'main') — for branch/PR review, target only the changed code. Requires a git repo + a real diff; returns structured guidance (status + guidance) instead of fabricating when there is none. Overrides target_ids. | |
| provider | No | BYOK provider override (auto-detected from env if omitted). Use 'deterministic' for an offline stand-in; without any provider configured, generation returns setup guidance instead of degrading. | |
| framework | No | Framework hint, e.g. 'playwright', 'vitest', 'pytest'. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| target_ids | No | Behavior/requirement external ids to target. Default: the top-gap behavior. Pass several to split the budget across them (each gets a test when the budget allows, in priority order). | |
| write_files | No | Deferred/premium: repo file writing is disabled in the first slice; always treated as false. | |
| prompt_version | No | Prompt strategy. Defaults to v2; v5 is opt-in until corpus validation clears it as default. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=false, destructiveHint=false, openWorldHint=true; the description adds the crucial contract that OrangePro never writes to or runs anything in the repo — the agent must write files and execute them itself. It also discloses the BYOK prerequisite, the no-provider fallback to setup guidance, and that evidence is validated against the local graph with broken citations surfaced. This is behavior an agent cannot infer from the schema or annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the verb, cap, and BYOK requirement before any detail, and each subsequent clause adds distinct information (buckets, evidence validation, thin-evidence fallback, agent workflow, prerequisites). It is a single dense paragraph rather than structured bullets, which makes the long evidence/run_hints passage harder to scan, but there is little true filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 parameters, no output schema, and openWorldHint=true, the description carries the burden well: it describes the returned fields (per-test bucket, validated evidence with kind/strength/source_ref, evidence_summary, code, run_hints, agent_workflow), the failure mode, and the environment prerequisite. Nothing essential for a correct first call appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description goes beyond it by explaining limit as a bucket-diverse budget, target_ids budget-splitting in priority order, and base_ref overriding target_ids. Provider auto-detection and the 'deterministic' offline stand-in are also given operational meaning. Only compare's full side-by-side scoring detail and framework/prompt_version nuances are left entirely to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource plus scope: 'Generate a small capped set (1-5) of grounded tests from local evidence.' It further pins the mechanism (BYOK providers) and output character (bucket-diversified, evidence-grounded), so an agent knows exactly what it gets. It stops short of naming a sibling (e.g. find_test_gaps vs this generator) to route between them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete operating guidance: default targets the top-gap behavior, pass multiple target_ids to split budget, use base_ref for PR/branch-scoped generation (and that it overrides target_ids), use compare for a prompt-only vs KG baseline. It also states the thin-evidence path returns guidance rather than generic tests. It never states when NOT to use this tool or names the sibling to pick instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the local graph readiness score (0-100), band, per-dimension breakdown, and plain-language missing evidence. Readiness signal, not a proof of test lift.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description adds valuable interpretive framing (a signal, not proof) and discloses the return contents, but says nothing about prerequisites such as whether the graph must first be built via orangepro_update_graph.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, and the core output contents are front-loaded before the caveat. Every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully enumerates the return values and the interpretive caveat. Slight gap: it doesn't mention any preconditions (e.g., needing an analyzed/built graph) or the local-only scope implications for an agent deciding timing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single workspace parameter with 100% schema description coverage, so the schema already documents the default-to-current-directory behavior. The description adds no further meaning beyond the schema, which is the expected baseline here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb/resource ('Return the local graph readiness score') and enumerates exactly what comes back: 0-100 score, band, per-dimension breakdown, and missing evidence. This is clearly distinguishable from siblings like orangepro_prove or orangepro_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Readiness signal, not a proof of test lift' implicitly steers the agent away from using this as a proof tool (i.e., toward orangepro_prove), giving real selection context. It doesn't explicitly say when in a workflow to call it or name the alternative tool, so it falls short of the top score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run the dynamic targeted-proof oracle for a CodeSymbol and append a metadata-only ledger certificate. Public Proven closes only when baseline green plus a valid sentinel mutation of the credited symbol fails the same test at an assertion. Static record_run is diagnostics only.
| Name | Required | Description | Default |
|---|---|---|---|
| method | No | Optional consistency check. If provided, must match the member derived from target_symbol. | |
| run_id | No | Optional caller-supplied run id. Defaults to a local monotonic id. | |
| runner | No | Test runner override. Default auto. | |
| source | No | Source checkout containing the test and target files. Defaults to workspace. | |
| test_env | No | Explicit non-secret KEY=value env entries for the test run. | |
| test_run | No | Go: fully-anchored test name for `go test -run`, e.g. '^TestCompute$' (required for Go targets). Java: 'TestClass#testMethod', e.g. 'CalculatorTest#addsTwoNumbers' (required for Java targets). | |
| target_id | No | Optional graph target id that must resolve to exactly one hard-linked CodeSymbol. | |
| test_path | No | TS/JS: repo-relative test file to run (required for TS/JS targets). | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| timeout_ms | No | Per baseline/mutant test timeout. | |
| jest_config | No | Repo-relative Jest config path. | |
| replacement | No | TS/JS: inert sentinel body, e.g. 'return null;' or 'return {"ok":false};'. Must be JSON-only per the oracle (required for TS/JS targets). Go derives its own sentinel. | |
| target_file | No | Optional consistency check. If provided, must match the file derived from target_symbol. | |
| target_symbol | No | Exact CodeSymbol external id, e.g. sym:src/service.ts#OrderService.createOrder. Preferred. | |
| vitest_config | No | Repo-relative Vitest config path. | |
| replacement_mode | No | Sentinel wrapping mode. Default return-json. | |
| link_node_modules | No | Trusted-repo speed mode: symlink node_modules into temp copies. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-read-only, non-idempotent, open-world, and non-destructive behavior. The description adds useful behavioral context beyond those hints: it appends a metadata-only ledger certificate, and it specifies the success condition involving baseline green plus a sentinel mutation failing at an assertion. It does not describe response shape or failure modes in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three dense sentences and is reasonably front-loaded with the core action. Every sentence contributes to purpose, success condition, or sibling contrast, though the jargon-heavy phrasing could be clearer for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 17-parameter tool with no output schema, the description explains the conceptual proof procedure and the ledger-certificate side effect. However, it does not explain what the tool returns, how failures are reported, or which runner-specific inputs are required, leaving important invocation context to the schema alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 17 parameters are already documented in the input schema. The description mentions the CodeSymbol target concept but does not add syntax, format, or per-runner parameter guidance beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: running the dynamic targeted-proof oracle for a CodeSymbol and appending a metadata-only ledger certificate. It distinguishes itself from the sibling orangepro_record_run by labeling that tool as diagnostics only. The domain jargon is dense but specific, not tautological.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when this tool is appropriate by explaining that Public Proven requires a baseline green plus a valid sentinel mutation, and it contrasts static record_run as diagnostics only. However, it does not explicitly say when to use this versus orangepro_prove_loop or other verification-related siblings, and it gives no explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Product wrapper around orangepro_prove: run trusted-local setup_commands in the source checkout, then the UNCHANGED dynamic targeted-proof oracle, then refresh the behavior report. Setup is non-secret repo prep only (build/install/shim); a setup failure returns unrunnable, does not run the oracle, and leaves the ledger untouched (never Proven). Public Proven still closes only through the unchanged certificate.
| Name | Required | Description | Default |
|---|---|---|---|
| method | No | Optional consistency check. If provided, must match the member derived from target_symbol. | |
| run_id | No | Optional caller-supplied run id. Defaults to a local monotonic id. | |
| runner | No | Test runner override. Default auto. | |
| source | No | Source checkout containing the test and target files. Setup runs here. Defaults to workspace. | |
| test_env | No | Explicit non-secret KEY=value env entries for the test run. | |
| test_run | No | Go: fully-anchored test name for `go test -run`, e.g. '^TestCompute$' (required for Go targets). Java: 'TestClass#testMethod', e.g. 'CalculatorTest#addsTwoNumbers' (required for Java targets). | |
| target_id | No | Optional graph target id that must resolve to exactly one hard-linked CodeSymbol. | |
| test_path | No | TS/JS: repo-relative test file to run (required for TS/JS targets). | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| timeout_ms | No | Per baseline/mutant test timeout. | |
| jest_config | No | Repo-relative Jest config path. | |
| replacement | No | TS/JS: inert sentinel body, e.g. 'return null;' or 'return {"ok":false};'. Must be JSON-only per the oracle (required for TS/JS targets). Go derives its own sentinel. | |
| target_file | No | Optional consistency check. If provided, must match the file derived from target_symbol. | |
| target_symbol | No | Exact CodeSymbol external id, e.g. sym:src/service.ts#OrderService.createOrder. Preferred. | |
| vitest_config | No | Repo-relative Vitest config path. | |
| setup_commands | No | Trusted-local repo prep run in the source checkout before the oracle. Non-secret only (build/install/shim). First non-zero exit returns unrunnable. | |
| replacement_mode | No | Sentinel wrapping mode. Default return-json. | |
| setup_timeout_ms | No | Default per-setup-command timeout (ms). Default 30000. | |
| link_node_modules | No | Trusted-repo speed mode: symlink node_modules into temp copies. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-readonly, open-world, non-idempotent, and non-destructive behavior. The description adds important semantics: setup commands are non-secret repo prep only, setup failure returns unrunnable and skips the oracle while leaving the ledger untouched, and Proven status still requires the unchanged certificate. It does not cover permissions, rate limits, or every mutation effect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences are front-loaded with the workflow, safety constraint, and failure behavior. There is no filler, and every sentence carries useful information for invoking the tool correctly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the workflow and key failure states, and the schema fully documents all 19 parameters. Since there is no output schema, normal success return details and report-refresh output shape are not fully described, but the critical safety and proof semantics are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already explains test_path, test_run, replacement, setup_commands, and the other parameters in detail. The description reinforces setup_commands semantics but adds little parameter meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
It names the specific workflow sequence: setup_commands in the source checkout, then the unchanged dynamic targeted-proof oracle, then refresh the behavior report. It also explicitly identifies itself as a wrapper around orangepro_prove, making it distinguishable from that sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The use case is clear: this tool is for when trusted-local repo prep is needed before running the proof oracle and refreshing the report. It names orangepro_prove as the underlying alternative but does not explicitly state when to call that sibling directly instead of this wrapper.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Record an agent test-writing attempt as static diagnostics. When test_path is provided for TS/JS, OrangePro first tries scoped deterministic re-prove; otherwise it re-analyzes the repo. Static hard COVERS edges render as Associated signal; public Proven requires orangepro_prove.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Model name used by the agent/generator, if any. | |
| run_id | No | Optional caller-supplied run id. Defaults to a local monotonic id. | |
| source | No | Source checkout to re-analyze. Defaults to workspace. | |
| vacuous | No | Whether the agent judged the test vacuous/non-assertive. Advisory only. | |
| provider | No | Model provider used by the agent/generator, if any. | |
| target_id | No | Optional graph target id that must resolve to exactly one hard-linked CodeSymbol. | |
| test_path | No | Workspace-relative test file the agent just wrote. Enables scoped deterministic re-prove for TS/JS. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| agent_pass | No | Whether the agent-reported test command passed. Advisory only. | |
| evidence_ids | No | Generated-test/evidence ids tied to this attempt. Metadata only. | |
| target_symbol | No | Exact CodeSymbol external id, e.g. sym:svc/add.go#Add. Preferred. | |
| prompt_version | No | Prompt or workflow version used, if any. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that this is non-read-only, non-idempotent, and non-destructive. The description adds meaningful behavior beyond that: with test_path for TS/JS it tries scoped deterministic re-prove, otherwise it re-analyzes the repo, and it explains how static COVERS edges render as Associated signal. It does not cover auth, failure modes, or return format, but it is substantially more informative than the annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loads the core action before explaining conditional behavior. It is dense but not repetitive; the main cost is that some terminology is opaque rather than verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter, non-idempotent recording tool with no output schema, the description explains the core recording behavior and one alternative path but omits what the recorded result looks like and how it relates to the broader workflow. It is minimally adequate given the rich schema descriptions, but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameter meaning is already well documented. The description adds some context for test_path by explaining the scoped deterministic re-prove path, but this largely repeats the schema description and does not clarify the roles of the other eleven parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource: recording an agent test-writing attempt as static diagnostics. It also distinguishes this from orangepro_prove by stating that public Proven requires that separate tool. The domain jargon ('static hard COVERS edges') keeps it from being fully transparent, but an expert can identify the operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'Record an agent test-writing attempt', but there is no explicit guidance on when to use this versus siblings such as orangepro_generate_tests, orangepro_find_test_gaps, or orangepro_prove_loop. The only routing guidance is the note that public Proven requires orangepro_prove.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Write and return a deterministic Requirements Traceability Matrix. Status comes only from the local graph and gap-fill ledger: Proven, Runtime-covered, Associated signal, No integration signal, Reproven, or Generated-unverifiable. No LLM calls.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Optional max rows to emit. Markdown defaults to 500 rows; JSON/CSV are uncapped unless set. | |
| format | No | Output format. Default md. | |
| status | No | Optional status filter, e.g. ['no-link','runtime','associated']. | |
| base_ref | No | Optional PR/diff scope. Restricts rows to behaviors touched vs this ref. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| output_path | No | Output path. Default .orangepro/rtm.<format>. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is not read-only, is idempotent, and is not destructive; the description aligns by saying it writes and returns a matrix. It adds meaningful behavioral context by specifying that statuses come only from the local graph and gap-fill ledger and that no LLM calls occur, though it does not discuss file overwrite behavior or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core action, and avoids repetition. The status enumeration is compact and relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
It explains the tool's purpose and status provenance, but with no output schema it does not fully describe the returned matrix structure or how writing to output_path interacts with the return value. The six optional parameters are documented in the schema, so the main missing context is output and sibling-routing behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema carries the parameter documentation burden. The description does not clarify limit, format, workspace, output_path, or base_ref, and its status labels do not exactly match the status-filter examples in the schema, potentially creating some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Write and return a deterministic Requirements Traceability Matrix.' It distinguishes the tool as deterministic and non-LLM, but does not explicitly contrast it with siblings such as orangepro_export_evidence_pack or orangepro_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the phrase 'Write and return a deterministic Requirements Traceability Matrix' and the 'No LLM calls' constraint. However, there is no explicit guidance on when to choose this tool over related reporting or export siblings, and no exclusions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
One-command local setup for a repo or PR: analyze sources, auto-apply weak AI candidate links and candidate flows when a real BYOK provider is configured, write behavior-coverage.html (detailed report), short_behavior-coverage.html (summary), tests.csv + rtm.md, summarize changed/gap targets, and return agent next actions. When nothing changed since the last run it returns at once with unchanged_since and reuses the reports (fresh=true forces a full run). AI lanes stay separate and never affect Proven coverage.
| Name | Required | Description | Default |
|---|---|---|---|
| fresh | No | Run in full even when nothing changed since the last run. By default an unchanged repository returns at once with unchanged_since and the reports of the last run. | |
| model | No | Model name override. | |
| no_ai | No | Disable automatic AI weak-link and candidate-flow passes even if a provider key is configured. | |
| ai_all | No | Run AI weak-linking over all deterministic behavior nodes instead of gaps-only. | |
| source | No | Source path to analyze. Defaults to workspace. | |
| base_ref | No | Optional PR/diff base ref. Default auto-detects main/master when possible. | |
| provider | No | BYOK provider override for the AI passes. | |
| workspace | No | Local workspace root. Defaults to the current directory. | |
| no_ai_flows | No | Disable automatic AI candidate-flow discovery while keeping weak AI links enabled. | |
| prompt_version | No | Opt-in generation strategy for the auto-prove generation lane. Default v2/deterministic; v5 uses batched two-phase generation. The prove/mint gate is unchanged either way. | |
| include_markdown | No | Enrich requirements-like Markdown docs. Default true. | |
| generate_coverage | No | Run safe local coverage generation before analyze where supported. Default false. | |
| coverage_timeout_ms | No | Per-command coverage-generation timeout. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare it not read-only, open-world, idempotent, and non-destructive; the description adds real context beyond that: it writes specific files, reuses prior reports on no-change runs, and explains that AI lanes never affect Proven coverage. It stops short of stating permissions/rate-limit or BYOK failure behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, purpose front-loaded, no filler. However the first sentence is a dense run-on list of six-plus artifacts and behaviors that is harder to scan than it needs to be for such a complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 13-parameter tool with no output schema, the description does cover the key behavioral contract: what files are written, the no-change short-circuit, and that 'agent next actions' are returned. Return value details (shapes of the next-actions payload) remain unspecified, which is a minor gap given the absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so every one of the 13 parameters is already documented in structured fields. The description restates the fresh/unchanged_since behavior and the AI-lane separation but adds little parameter-level meaning the schema doesn't already carry, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('One-command local setup for a repo or PR') and enumerates the concrete artifacts it produces (behavior-coverage.html, tests.csv, rtm.md). It's clear what it does, but it never names which sibling it replaces or how it relates to orangepro_analyze_sources or orangepro_generate_tests, which it appears to subsume.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Useful guidance is given for one path only: unchanged repos short-circuit unless fresh=true forces a full run. There is no explicit statement of when to reach for this umbrella tool versus the granular siblings (analyze_sources, prove, ai_links). Usage is implied rather than directed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize the local gap-fill ledger: attempted, dynamic-reproven, unproven, legacy static-associated, generated-but-unverifiable, and kept rate. Metadata-only.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description does add real context beyond them: 'local' (reads a workspace-local ledger, no network), 'Metadata-only' (does not parse source bodies, so it is cheap). It says nothing about behavior on an empty/absent ledger or what the kept rate is computed against, which is the practical gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with verb+resource, then the metric list which is dense but each item is the actual return content. No filler. The metric names are jargon-heavy but that mirrors the domain vocabulary used by siblings.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so enumerating the reported metrics in the description is the right compensating move and it is done. Missing only the edge cases (empty ledger, zero-denominator for the rate) and a pointer to the sibling that lists the underlying gaps individually.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
A single optional parameter with 100% schema description coverage, so the schema already explains that 'workspace' defaults to the current directory. The description adds no further meaning, such as how the ledger is located relative to the workspace root. Baseline 3 applies because the schema carries this burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Summarize) and resource (local gap-fill ledger) and enumerates the exact metrics returned: attempted, dynamic-reproven, unproven, legacy static-associated, generated-but-unverifiable, kept rate. This separates it from siblings like orangepro_find_test_gaps and orangepro_rtm. It stops short of an explicit contrast with those siblings, so 5 is not warranted.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to call this versus orangepro_find_test_gaps, orangepro_status, or orangepro_rtm, all of which an agent could plausibly pick for gap reporting. Usage is only implied by the word 'Summarize' and by the metric list; no exclusions or prerequisites are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Show local workspace state without generating anything: whether the graph is fresh/stale/missing, source mix, score, and privacy settings (always local, no upload).
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds genuinely new behavioral context: it performs no generation, operates entirely locally, and never uploads data, which matters for privacy-sensitive users. It stops short of describing cost or latency, but it is well above the annotation baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with the verb and subject front-loaded, followed by a compact colon-delimited list of what is reported. No filler, no redundancy, and the most important scope qualifier ("without generating anything") appears before the detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully serves as a stand-in by enumerating the reported signals (freshness, source mix, score, privacy settings). For a single-optional-parameter read-only tool with annotations, that is nearly sufficient; only the boundary against the other diagnostic siblings is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one optional parameter with 100% schema description coverage ("Local workspace root. Defaults to the current directory"). The description never mentions the workspace argument, so it adds nothing beyond the schema, which puts it at the baseline of 3 for fully documented parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ("Show") and resource ("local workspace state") and enumerates what that state includes: freshness (fresh/stale/missing), source mix, score, privacy settings. The phrase "without generating anything" separates it from generation siblings like orangepro_start or orangepro_update_graph. It does not, however, distinguish itself from adjacent read-only siblings such as orangepro_doctor, orangepro_stats, or orangepro_graph_score, so it falls short of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Without generating anything" implies the usage context (a pre-flight/read-only check before generation), but usage is only implied. No explicit when-to-use statement and no alternatives named, even though orangepro_doctor, orangepro_graph_score and orangepro_stats plausibly overlap with the reported fields.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Refresh the local graph from changed files. Incremental and non-destructive by default — preserves evidence and marks affected generated tests stale.
| Name | Required | Description | Default |
|---|---|---|---|
| workspace | No | Local workspace root. Defaults to the current directory. | |
| force_full_rebuild | No | Force a full rebuild instead of an incremental update. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (readOnly=false, idempotent=true, destructive=false), the description discloses real side effects: evidence is preserved and affected generated tests are marked stale. That is genuinely useful context that the annotations alone would not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with zero filler; the core action comes first and the side-effect caveat follows immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-optional-parameter mutation tool with no output schema, the description covers action, default mode, and side effects. It leaves open whether a prior graph must exist and what the caller gets back, but the essentials are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are already documented. The description only echoes the default behavior ("incremental and non-destructive by default") without adding format or interaction details beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Refresh the local graph from changed files" gives a specific verb (refresh) and resource (local graph) with scope (changed files only). It is clearly distinct from analyze_sources or graph_score, though it never names a sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase "from changed files" implies this is the post-edit refresh step, but no explicit when-to-use guidance, prerequisites, or alternatives (e.g., when to prefer a full rebuild or run start first) are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
orangepro_ai_flows1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_ai_links4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / max_behaviors / maximumAdded value: +9007199254740991Input schema / properties / max_prompt_tokens / maximumAdded value: +9007199254740991Input schema / properties / symbols_per_behavior / maximumAdded value: +9007199254740991orangepro_analyze_sources1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_changed_impact1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_doctor1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_explain_test1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_export_evidence_pack1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_find_test_gaps1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_generate_tests1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_graph_score1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_prove1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_prove_loop3 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / setup_commands / items / additionalPropertiesRemoved value: -falseInput schema / properties / setup_commands / items / properties / timeout_ms / maximumAdded value: +9007199254740991orangepro_record_run1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_rtm1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_start1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_stats1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_status1 field changedInput schema / additionalPropertiesRemoved value: -falseorangepro_update_graph1 field changedInput schema / additionalPropertiesRemoved value: -falsev0.2.59analyze_pr_riskanalyze_release_readinessbuild_regression_packconvert_bug_to_testsexplain_quality_riskgenerate_missing_coveragegenerate_test_scriptsget_coverage_gapsorangepro_ai_flowsorangepro_ai_linksorangepro_analyze_sourcesorangepro_changed_impactorangepro_doctororangepro_explain_testorangepro_export_evidence_packorangepro_find_test_gapsorangepro_generate_testsorangepro_get_agentorangepro_get_agent_healthorangepro_get_agent_logsorangepro_graph_scoreorangepro_list_agent_runsorangepro_list_agentsorangepro_proveorangepro_prove_looporangepro_record_runorangepro_resolve_storyorangepro_rtmorangepro_run_agentorangepro_startorangepro_statsorangepro_statusorangepro_update_graphv0.2.0analyze_pr_riskanalyze_release_readinessbuild_regression_packconvert_bug_to_testsexplain_quality_riskgenerate_missing_coveragegenerate_test_scriptsget_coverage_gapsorangepro_get_agentorangepro_get_agent_healthorangepro_get_agent_logsorangepro_list_agent_runsorangepro_list_agentsorangepro_resolve_storyorangepro_run_agentScored across 18 tools
Most tools target distinct operations, but there is real overlap: orangepro_start orchestrates much of what analyze_sources, find_test_gaps, and rtm do individually; orangepro_analyze_sources and orangepro_update_graph both (re)build the graph; orangepro_status and orangepro_graph_score both surface score/state; and prove vs prove_loop (explicitly a 'wrapper') can confuse selection. Descriptions do a good job disambiguating (e.g., changed_impact explicitly redirects baseline work to find_test_gaps), which keeps this from scoring lower.
All 18 tools use snake_case with a uniform 'orangepro_' prefix, which is strongly consistent. However, the naming convention is mixed: some are verb_noun (find_test_gaps, generate_tests, update_graph, record_run) while others are bare nouns or noun phrases (status, doctor, stats, rtm, graph_score, changed_impact), so the pattern is not predictably verb-first throughout.
18 tools is on the heavy side but justified by a genuinely broad domain (graph analysis, gap detection, test generation, dynamic proof, AI lanes, reporting, export). Most tools earn their place, though a few (start vs analyze_sources, prove vs prove_loop, status vs graph_score) could arguably be consolidated, pushing it slightly past the ideal 3-15 range.
Coverage is strong across the full lifecycle: setup/analysis (start, analyze_sources, update_graph, status, doctor, graph_score), gaps (find_test_gaps), generation (generate_tests, explain_test), proof (prove, prove_loop, record_run), ledger/reporting (stats, rtm), AI lanes (ai_links, ai_flows), diff/PR (changed_impact), and export (export_evidence_pack). Minor gaps exist—no config/privacy settings tool despite status reporting privacy settings, and no standalone report-regeneration or ledger-cleanup/reset—but these are workaroundable.
Generates unit tests for Python code with coverage before/after reports and concrete edge cases.
Discover software problems, analyze evidence, and create implementation-ready Build plans.
Deterministic context layer for your codebase: change impact, blast radius, answers with receipts.
AI Agent with Architectural Memory. Impact analysis (free), tests and code from the graph (pro).
