Server Quality Checklist

Tool Scores

• forge_statusA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that the tool shows phase and progress and that it auto-selects the active run when runId is omitted, which is helpful. However, it does not explain what happens when runId is omitted and there are zero or multiple active runs, nor any error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, purposeful sentences. The main purpose is front-loaded, and the conditional behavior is stated efficiently without filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status tool with one optional parameter and no output schema, the description adequately covers what the tool reports and how runId selection works. It could mention edge-case behavior when no runId is provided and the active-run condition is not met, but overall it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents runId and the optional condition with 100% coverage. The description mostly restates this, adding only the phrase 'uses that one' to clarify the auto-selection behavior, so it adds little beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Shows') and resource ('which phase a run is in and its progress'). It is easy to tell this is a status/read tool, though it does not explicitly name or contrast sibling tools like forge_next or forge_tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you need a run's phase and progress. It also gives a useful fallback rule for when runId can be omitted, but it provides no explicit guidance about when to prefer this tool over its siblings or what alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_skillA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It clearly indicates a read-only retrieval behavior by saying 'Returns... content', but it does not disclose behavior for missing skills, error handling, or whether applying the skill has side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, purposeful sentence with no filler. It front-loads the return value and immediately states the intended usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one well-documented parameter, the description provides enough context to call it correctly: it names the input, states the output, and explains why the agent would use it. It does not describe output format, but for SKILL.md content this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'name' is fully described in the schema as 'Skill name (a folder in the arsenal)', giving 100% coverage. The description adds no additional parameter details, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Returns'), a specific resource ('SKILL.md content of a skill'), and the intended purpose ('load and apply it in the current phase'). This clearly distinguishes it from sibling tools like forge_skills, which likely lists skills, and forge_complete_phase, which advances phases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: when you need a skill's SKILL.md content to apply it in the current phase. However, it gives no explicit guidance on when not to use it or how it relates to alternatives such as forge_skills for finding skills.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_complete_phaseA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does a good job: it discloses side effects (closes current phase, advances to next, persists summary) and warns about required structured evidence. It could additionally explain reversibility or what happens to the current phase after closure, but it is already genuinely informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The main action is front-loaded, and the evidence caveat logically follows immediately. Every clause earns its place, including the guardrail against closing uncompleted phases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with nested objects, no annotations, and no output schema, the description covers the essential invocation context: what the tool does, what is persisted, when evidence is required, and how invalid evidence is surfaced. It does not describe the return value, but that is not critical for correct invocation here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds some framing—'summary is persisted' and 'evidence matching their contract'—but the input schema already provides detailed evidence shapes for gates, qa, design, plan, user confirmation, and optional phases. runId is left to the schema, which adequately covers it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb/resource/effect: 'Closes the CURRENT phase with a summary...' and 'advances to the next one.' It also conveys the tool's core contract around persisted summaries and structured evidence. However, it does not explicitly contrast itself with sibling tools like forge_next, so it misses the full sibling-differentiation benchmark.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit when-not: 'Do not close a phase you have not actually done.' It also specifies when evidence is required (completionSchema phases, user-confirmation phases, optional phases) and points to the rejection message for invalid evidence. It does not explicitly name alternatives or say 'use forge_next instead', so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_skillsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It clearly indicates a read-only listing behavior, explains the conditional filtering, and names the data source (SKILL_MAP). It does not discuss error cases or return structure, but for a simple list operation the disclosed behavior is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero wasted words. The main purpose is front-loaded, and the conditional variations are presented in a logical order that an agent can parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: one optional parameter, no required inputs, and no output schema. The description covers the purpose, the valid parameter values, and the behavioral difference between the two call forms. It leaves minor ambiguity about the exact return shape, but nothing essential for invoking the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the phase parameter at 100% coverage, establishing a baseline of 3. The description adds meaningful value beyond the schema by enumerating the valid phase keys and explaining how the parameter changes the result set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Lists available skills.' It further clarifies two modes of operation (with or without a phase key), making the purpose unambiguous. It does not explicitly differentiate itself from sibling tools like forge_skill, though the behavior is specific enough to infer the distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance on when to use each mode: provide a phase key to get phase-specific skills, omit it to list everything. It does not mention alternatives or when not to use the tool, but the conditional usage context is explicit and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_nextA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It clarifies that despite the 'next' name, the tool does not advance anything—it 'returns' the current phase and its goal, which implies a read-only, non-mutating operation. It stops short of explicitly stating side effects or prerequisites, but the wording is sufficiently indicative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the core behavior and then provides the direct usage condition, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description covers what the tool returns and when to call it. It could additionally note behavior when no run is active or when multiple runs exist, but the schema already hints at the single-active-run condition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the optional runId parameter is fully documented in the schema. The description adds no additional parameter meaning, which places it at the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it 'Returns the CURRENT phase with its detailed goal.' It clarifies exactly what the tool provides and frames it as guidance for 'what to do now,' which distinguishes it from siblings like forge_status and forge_complete_phase.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Call this when you are unsure what comes next.' It clearly communicates when the tool should be used, though it does not explicitly mention when not to use it or name alternative tools for exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_startA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden, and it does disclose the key behavior: starting a run and returning the first phase, with an imperative not to skip phases. It doesn't mention permissions, persistence, or cancellation, but for a start-tool the main behavioral contract is clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, with the core purpose first, then return behavior, then workflow guidance. Every sentence carries information needed for correct use, and there is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter start tool with no output schema, the description provides the essential return info (first phase and goal), the required inputs are fully documented in the schema, and the follow-up tool sequence is explicit. An agent has enough context to invoke it correctly and continue the pipeline.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters clearly ('Project directory where the pipeline runs' and 'The user's request, verbatim'). The description restates the request concept but adds no new meaning or formatting details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action ('Start'), a specific resource ('a new forge pipeline run'), and a concrete entry point ('first phase (classify) and its goal'). This clearly differentiates it from sibling tools like forge_next and forge_complete_phase, which handle later phases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states exactly when to use the tool (at the beginning) and explicitly routes the agent through the subsequent flow: 'follow the flow phase by phase using forge_next and forge_complete_phase — do not skip phases.' This both names alternatives and gives an exclusion (no skipping).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forge_tasksA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure. It does so by stating that it shows 'id, request and current phase', giving the agent an understanding of what the tool returns. It also implicitly signals a read-only operation ('Lists'). While it doesn't mention ordering or limits, the behavior is sufficiently disclosed for a zero-parameter listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, both informative. The main action and purpose are front-loaded ('Lists the active pipeline runs'), and the second sentence adds output details without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with zero parameters and no output schema, the description is complete: it states what is listed, why it is used, and what fields are returned. An agent has everything needed to call this tool correctly and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so the baseline is 4. There are no parameters to document, and the description correctly does not invent any. Nothing more is needed here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Lists the active pipeline runs'. It also clarifies the purpose with 'to resume from any session', which distinguishes it from siblings like forge_start or forge_status that start or check status of runs. The scope is clear and non-tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use context ('to resume from any session'), indicating when this tool is appropriate. However, it does not explicitly mention alternatives or when not to use it. Still, the context is enough for an agent to recognize this as the listing/resume discovery tool among the siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.