Server Quality Checklist

Tool Scores

• read_resourceC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only says 'Read', implying a non-destructive operation, but provides no details about error behavior, authentication requirements, or resource existence handling. The description adds minimal value beyond what the name already suggests.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, achieving high conciseness. However, it is so brief that it borders on under-specification. For the purpose of being concise, it scores well, though it lacks substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple schema and lack of output schema or annotations, the description is quite incomplete. It does not explain what happens when a resource is not found, whether the URI must be fully qualified, or any other context. This makes the tool minimally viable but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the parameter is just a required 'uri' string. The description mentions 'by its URI', which is minimally helpful but does not explain what constitutes a valid URI, any formats, or examples. It adds little beyond the schema's parameter name and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and resource ('resource'), and clarifies the identifier type ('URI'). While it doesn't explicitly differentiate from sibling tools like list_resources, the name and action imply a single-resource read operation, which is reasonably clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. It merely states what it does without any context about appropriate use cases, limitations, or exclusions. Sibling tools like list_resources could overlap, but no differentiation or selection criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_resourcesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the tool lists resources and returns URIs and descriptions, but does not mention whether there is pagination, rate limiting, or any side effects. As a read-only listing, this is somewhat implicit but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the purpose and output without any unnecessary words or redundancy. It is perfectly concise for the simplicity of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description adequately states what it does and what it returns. It could be slightly more complete by clarifying what constitutes a 'resource' in this context, but it is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so there are no parameter semantics to clarify. Per the baseline for zero parameters, a score of 4 is appropriate; the description doesn't need to explain any inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the action as 'list' with a clear resource type ('all available resources') and the included details (URIs and descriptions). It is distinct from sibling tools like read_resource, which implies a targeted access, while this tool provides an overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus alternatives. It does not mention that this is for discovery before using read_resource, nor does it state any exclusions or conditions. The usage context is only implied by the word 'list'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_sop_feedbackA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite having no annotations, the description precisely discloses behavioral details: feedback is appended as a JSON line to a .feedback.jsonl file in the SOP's folder, including version, timestamp, and text. This goes beyond a generic statement and gives clear expectations for the side effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that front-load the purpose and then add an efficient behavioral detail. No wasted words; every clause adds necessary context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and full schema coverage, the description covers purpose, mechanics, and side effects. It could optionally mention validation or error conditions, but these are not required for a basic feedback submission tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully documented in the schema with clear descriptions (sop_name and feedback). The description does not add significant additional parameter-level semantics beyond what the schema already provides, so it remains at the schema-corrected baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Submit' with a resource 'improvement feedback for a specific SOP', clearly differentiating it from sibling tools like list_resources, run_sop, and publish_sop. It states exactly what the tool does without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when one has improvement feedback for an SOP but does not explicitly mention alternatives or when-not-to-use scenarios. It provides context that feedback is for future review, but lacks a direct comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sopA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently explains that each call returns one step, that current_step must be incremented to advance, and that all actions in the step must be executed (not just read). It also mandates supplying step_output when continuing. This is rich, actionable behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the primary purpose. It uses short, direct sentences and scannable formatting (IMPORTANT warning) to highlight critical execution requirements. Every sentence provides necessary operational guidance without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core usage loop, prerequisites, and required outputs, which is strong for a stateful tool without an output schema. However, it does not explain how to determine when the SOP is complete or what the returned step content looks like structurally (e.g., text vs. structured actions). This is a minor gap but not severe given the detailed procedural guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the parameters are already well-documented. The description adds no new meaning beyond what is in the schema, merely restating the same information about current_step and step_output. It reinforces the stateful usage but does not deepen parameter understanding, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Start or advance a Standard Operating Procedure step by step.' It also specifies the resource (SOP) and the step-wise execution model. The usage of 'discover available SOPs' via list_resources helps distinguish it from sibling tools like read_resource or publish_sop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to use list_resources to discover SOPs before calling this tool, providing a clear prerequisite. It also explains the step-by-step advancement process and when step_output is required. However, it does not explicitly contrast with alternatives like read_resource or submit_sop_feedback, though the workflow is strong enough to imply appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_sopA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It thoroughly discloses server-side version computation, stage argument precedence, content overwriting, lint errors blocking publish, and warning field behavior. This is exceptional transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a one-sentence purpose, then uses bold-labeled sections to organize complex details. Every sentence earns its place, including the example and lint guidance. Despite length, it is well-structured and not padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and no annotations, the description covers all critical aspects: required input format, versioning and stage precedence, error/warning behavior, and local linting workflow. It is practically complete for correct invocation and outcome understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions are minimal, but the description adds extensive semantics: required YAML frontmatter fields with types and rules, versioning behavior, stage mismatch handling, and a concrete example call. This far exceeds the schema's surface-level descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the action ('Publish') and the resource ('a new or updated Standard Operating Procedure document'). It distinguishes from sibling tools like submit_sop_feedback and read_resource, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: prepare content with required frontmatter and iterate with sop-lint locally before calling to avoid latency. However, it does not explicitly compare with sibling alternatives or state when not to use this tool, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.