Skip to main content
Glama

Record gap-fill outcome

orangepro_record_run

Records an agent test-writing attempt as static diagnostics and triggers scoped TS/JS re-prove or repo re-analysis to show coverage signals and whether public proof is required.

Instructions

Record an agent test-writing attempt as static diagnostics. When test_path is provided for TS/JS, OrangePro first tries scoped deterministic re-prove; otherwise it re-analyzes the repo. Static hard COVERS edges render as Associated signal; public Proven requires orangepro_prove.

Input Schema

NameRequiredDescriptionDefault
modelNoModel name used by the agent/generator, if any.
run_idNoOptional caller-supplied run id. Defaults to a local monotonic id.
sourceNoSource checkout to re-analyze. Defaults to workspace.
vacuousNoWhether the agent judged the test vacuous/non-assertive. Advisory only.
providerNoModel provider used by the agent/generator, if any.
target_idNoOptional graph target id that must resolve to exactly one hard-linked CodeSymbol.
test_pathNoWorkspace-relative test file the agent just wrote. Enables scoped deterministic re-prove for TS/JS.
workspaceNoLocal workspace root. Defaults to the current directory.
agent_passNoWhether the agent-reported test command passed. Advisory only.
evidence_idsNoGenerated-test/evidence ids tied to this attempt. Metadata only.
target_symbolNoExact CodeSymbol external id, e.g. sym:svc/add.go#Add. Preferred.
prompt_versionNoPrompt or workflow version used, if any.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • removedInput schema / additionalProperties
      Removed value: -false
  2. Addedv0.2.59

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is non-read-only, non-idempotent, and non-destructive. The description adds meaningful behavior beyond that: with test_path for TS/JS it tries scoped deterministic re-prove, otherwise it re-analyzes the repo, and it explains how static COVERS edges render as Associated signal. It does not cover auth, failure modes, or return format, but it is substantially more informative than the annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the core action before explaining conditional behavior. It is dense but not repetitive; the main cost is that some terminology is opaque rather than verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter, non-idempotent recording tool with no output schema, the description explains the core recording behavior and one alternative path but omits what the recorded result looks like and how it relates to the broader workflow. It is minimally adequate given the rich schema descriptions, but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameter meaning is already well documented. The description adds some context for test_path by explaining the scoped deterministic re-prove path, but this largely repeats the schema description and does not clarify the roles of the other eleven parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: recording an agent test-writing attempt as static diagnostics. It also distinguishes this from orangepro_prove by stating that public Proven requires that separate tool. The domain jargon ('static hard COVERS edges') keeps it from being fully transparent, but an expert can identify the operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'Record an agent test-writing attempt', but there is no explicit guidance on when to use this versus siblings such as orangepro_generate_tests, orangepro_find_test_gaps, or orangepro_prove_loop. The only routing guidance is the note that public Proven requires orangepro_prove.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.