Server Quality Checklist

Tool Scores

• get_manifestB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Return the manifest' – it does not mention potential errors, authentication requirements, rate limits, or what the manifest contains. The read-only nature is implied but not explicitly stated beyond the verb.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no wasted words. It is appropriately sized for a simple zero-parameter tool, though it could arguably add a bit more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description is the only source of information. It does not explain what the manifest contains, how it is structured, or what the agent should expect as a return value. This is a significant gap for an agent deciding whether and how to use the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so there is nothing to explain. The description adds no parameter information, but none is needed. The baseline of 4 applies because the schema already fully covers the (empty) parameter set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a clear resource ('Trust Ledger OS manifest'). It distinguishes itself from sibling tools (get_phase, list_packages, etc.) by naming a distinct entity, though it does not explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description gives no context about scenarios, prerequisites, or why an agent might choose get_manifest over siblings. The agent must infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_phasesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of indicating behavior. The verb 'List' implies a read-only retrieval operation, but the description does not explicitly state that it has no side effects, nor does it mention output format or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no unnecessary words or redundant details. It is well-structured and immediately comprehensible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete enough for a trivial parameterless tool, but it does not describe what the returned list contains or how it is formatted. Since there is no output schema, some additional detail about the return value would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters and full coverage, so there are no parameter semantics to document. The description correctly aligns with the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('List') and object ('roadmap phases'), making its primary purpose obvious. It does not explicitly contrast with sibling tools like get_phase, but the plural 'phases' and list verb sufficiently distinguish it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus siblings such as get_phase, list_packages, or list_routes. No context or selection criteria are offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_phaseA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Fetch' indicates a read operation, but it does not disclose what happens when the id is not found (e.g., error vs null), nor the return format. For a simple get, this is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with zero waste. It immediately states the verb, resource, and method. Perfectly concise for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter get, the description is minimally sufficient: it indicates the operation and target. However, with no output schema and no annotations, it does not explain the return value structure or error behavior, leaving some ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for the lack of parameter documentation. It does not add any meaning to the 'id' parameter beyond the schema's basic type and required flag. The description merely says 'by id' without explaining what id represents or its format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (fetch), the resource (roadmap phase), and the method (by id). It distinguishes itself from list_phases by implying a single record retrieval. The verb is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly signals usage: you use this when you have a specific id and want one phase, as opposed to list_phases for all phases. However, it does not explicitly state when not to use it or name alternatives, so it misses the explicit guidance level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_routesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  'List' implies a read-only operation, but the description does not explicitly disclose side effects, return format, or error behavior. With no annotations present, the description carries the burden but only partially fulfills it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundancy or irrelevant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless listing tool, the description is mostly complete. It clearly identifies the object being listed, though it does not specify the output structure or clarify what 'live' means in this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and the input schema fully covers this. The baseline of 3 applies since there is no parameter information to add beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and the specific target ('live app routes'), making the tool's purpose immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus sibling tools such as list_packages or list_phases. The intended use case must be inferred from the name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_packagesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not disclose side effects, permissions, or return format. While listing is likely read-only, this is not stated and no annotations are present to clarify behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is well-structured and directly states the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters, the description is adequate. It could be slightly more explicit about what 'package surfaces' entails, but the meaning is reasonably inferable from the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is fully covered by an empty object. There is no parameter meaning to add beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and the subject (npm, PyPI, and MCP package surfaces). It is distinct from sibling tools like list_phases and list_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of context, prerequisites, or conditions that would make this the preferred choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.