Server Quality Checklist

Tool Scores

• forgehand_resultA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal readOnly, idempotent, and non-destructive behavior, and the description's 'Return' is consistent with those. It adds the nuance of 'compact' receipt but does not disclose non-obvious behaviors such as error handling, whether missing IDs return null/error, or any polling semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is an eight-word single sentence that states the core operation immediately. There is no filler, redundant restatement of the tool name, or optional background that could be dropped.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read with rich annotations and an output schema, this is minimally viable. However, it stope short of explaining how the task_id relates to forgehand_delegate or forgehand_tasks, and whether a receipt exists immediately after delegation or after completion, leaving contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description shoulders the burden of explaining task_id, but 'by ID' adds little beyond the parameter's own name and title 'Task Id'. It does not describe the ID format, where to obtain it, or any constraints, so parameter semantics remain under-specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Return') and noun phrase ('one compact task receipt') with a clear qualifier ('by ID'), which plainly identifies what the tool does. It distinguishes itself from the sibling tools like forgehand_tasks (likely listing) and forgehand_health by focusing on a single item lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'by ID' implies it should be used when the caller already has a task_id and wants a single receipt, but the description does not explicitly say when to use this over forgehand_tasks or forgehand_delegate. No alternatives or exclusion criteria are given, leaving usage to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgehand_tasksA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds useful behavioral context by stating it returns compact receipts and deliberately avoids loading raw logs or diffs, clarifying the tool's lightweight nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise, front-loaded sentence that communicates the action, resource, and key behavioral qualifier without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter list tool, the description, annotations, and output schema provide enough context for correct invocation. The main gap is explicit parameter semantics, but overall completeness is solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single 'limit' parameter has no schema description, and schema description coverage is 0%. The tool description does not explain what 'limit' controls or how it affects results, so it fails to compensate for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'List compact receipts for recent tasks'. The qualifier 'without loading raw logs or diffs' clarifies the output format and distinguishes it from heavier task-detail tooling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys a clear context: use this for a lightweight overview of recent tasks, and not when raw logs or diffs are needed. It does not explicitly name sibling alternatives or exclusion conditions, but the intended use is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgehand_healthA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds value by specifying the exact information returned (active worker, safety limits, registered repositories, paths), providing concrete context beyond the annotations. It does not contradict any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action, and lists all key output categories with no filler or redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only health check with an output schema present, the description is fully sufficient. It names all the key information an agent would need to evaluate the tool's relevance and expected result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides no parameter semantics to enrich. The description correctly focuses on the output rather than inputs, and the baseline for zero-parameter tools is 4. No additional parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a clear resource: active worker, safety limits, registered repositories, and paths. This strongly differentiates it from siblings like forgehand_delegate, forgehand_tasks, and forgehand_result, which appear to be action-oriented operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a health/status-check use case but does not explicitly state when to use this tool versus the sibling tools, nor does it mention any alternatives or exclusions. The purpose is inferable from the name and return value list, but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgehand_delegateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond annotations by explaining that commands inherit host permissions and network, risk acknowledgement is required when commands are present, required command IDs must all exit zero for success, and the worker is bound to the scope. Annotations carry destructiveHint=false and readOnlyHint=false, and the description adds important context about the command execution model without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the primary purpose in the first sentence, and every sentence adds operational detail without redundancy. It covers command execution semantics, safety, and success conditions in about 60 words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (14 parameters, command execution, security considerations) and the presence of an output schema, the description covers the most critical operational context. It doesn't explain every parameter but the essentials for safe and correct invocation — registration prerequisite, scope limitations, command execution model, and success criteria — are present. An output schema exists, so return values need not be explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameter semantics. It explains the significance of scope, commands, risk acknowledgement, and required command IDs, which maps to key parameters. Not every parameter is individually explained (edit_mode, base_revision, etc.), but the core ones an agent needs to invoke correctly are covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs 'one complete implementation task' with bounded files and commands, distinguishing it from sibling tools like forgehand_health, forgehand_tasks, and forgehand_result which are inspection/retrieval tools. The verb 'Run' plus the resource and scope constraint makes the purpose explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for use: repository must be registered first, scope bounds what the worker may access, and commands are supervisor-authored. It doesn't explicitly name sibling alternatives or when not to use this tool, but the distinction from the health/tasks/result siblings is implied by the delegation purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.