Server Quality Checklist

Tool Scores

• get_capability_manifestB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. The verb 'Report' implies a read-only operation, but the description does not explicitly state that, nor does it elaborate on any side effects, authentication needs, or what 'trust boundaries' entails. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads its core function. It is efficient with no waste, earning the full score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no parameters and an output schema, the description leaves key context unexplained. The 'four-tool diagnostic surface' is neither defined nor related to the sibling tools, and there is no guidance on when this tool is appropriate. The cryptic phrasing significantly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero properties, so there are no parameters to explain. Since 0 params baselines at 4, the description doesn't need to add anything, and it doesn't detract from the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Report' and identifies the resource as 'the exact four-tool diagnostic surface and its trust boundaries,' which aligns with the tool name. However, the phrase 'four-tool diagnostic surface' is ambiguous without listing which four tools, so it doesn't fully distinguish from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives like get_msty_status or get_model_manifest. It doesn't mention exclusions, prerequisites, or any context that would help an agent decide between this tool and its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_compatibility_checkA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description does not disclose potential side effects, error behavior, or prerequisites; 'validate' implies a read-only check but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no parameters and an output schema, the description omits context about what triggers a compatibility failure or how results should be interpreted, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the description need not compensate; baseline of 4 applies since schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Validate') and targets two resources ('installed Studio version', 'local model endpoint schemas'), distinguishing it from siblings that retrieve status or manifests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to run this tool versus siblings; it only implies a validation purpose without explicit conditions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_model_manifestA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only describes what is listed, not how the tool behaves (e.g., read-only nature, potential errors, or output format). There is no indication of side effects, permissions, or performance characteristics, leaving significant ambiguity for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no redundant wording. It is well-front-loaded and contains exactly one piece of information, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description is largely complete for a simple list operation. However, it omits any context about what 'Msty model services' are or what 'facts' might include, and lacks behavioral details. Still, the low complexity and presence of an output schema keep this from being severely incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline for this dimension is 4. The description adds no parameter semantics, but none are needed. The empty schema is fully understood without additional explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and clearly identifies the resource as 'facts advertised by fixed, loopback-only Msty model services.' This distinguishes it from sibling tools like get_capability_manifest and get_msty_status, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives. The phrase 'fixed, loopback-only' hints at a specific context, but it does not state when this tool is appropriate or mention any alternative tools. No 'when' or 'when not' guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_msty_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of transparency. It explicitly discloses that it does not read Msty data, which is a strong safety guarantee. However, it does not mention any other behavioral traits like whether it makes network calls or has side effects, though 'Report' implies read-only behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose. It contains zero filler words and every word contributes meaning, making it highly concise and effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description does not need to explain return values. The description covers the core function and even adds a safety disclaimer. It is slightly brief for a status tool, but still adequately complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to explain parameter semantics. The empty schema is fully covered by the description, and the baseline for no parameters is 4. There is no additional meaning needed beyond what the description already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Report') and clearly specifies the resource ('installation, local-service, and readiness state'). It distinguishes itself from sibling tools like get_capability_manifest and run_compatibility_check by focusing on status rather than manifests or checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without reading Msty data' implies a safe read-only use case, but it does not explicitly state when to use this tool versus the sibling tools. No alternative tools are mentioned, though the context of 'status' suggests it is the go-to for health checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.