Server Quality Checklist

Tool Scores

• craft_get_itemsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it 'gets items' but doesn't disclose behavioral traits such as whether it's read-only, requires authentication, has rate limits, pagination behavior, or error handling. The description is minimal and lacks critical operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. It avoids unnecessary words, though it could be slightly more structured (e.g., by explicitly listing item types). Every part earns its place, but it's borderline due to potential under-specification.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a tool that retrieves multiple items (implying potential complexity like pagination or filtering), the description is incomplete. It lacks details on return format, error cases, authentication needs, or how it interacts with sibling tools, leaving significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters ('fields' and 'limit'). The description adds no additional meaning beyond what the schema provides, such as examples of field names or implications of the limit. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get items') and resource ('from the craft.io workspace'), specifying the types of items (products, features, etc.). It distinguishes from 'craft_get_item' (singular) by implying retrieval of multiple items, but doesn't explicitly differentiate from 'craft_get_workspace' or other siblings beyond scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'craft_get_item' or 'craft_get_workspace'. The description implies it's for retrieving multiple items but doesn't specify contexts, prerequisites, or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• craft_pingB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool tests connectivity but doesn't explain what that entails—whether it makes a network call, returns success/failure, requires authentication, or has side effects. This leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core purpose and efficiently communicates the essential function without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimal but incomplete. It lacks behavioral details (e.g., what 'test connectivity' means in practice) and usage context, making it insufficient for full agent understanding despite the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there's no need for parameter explanation in the description. The baseline for this scenario is 4, as the description appropriately doesn't discuss parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as testing connectivity to the craft.io API, which is a specific action (test connectivity) on a specific resource (craft.io API). However, it doesn't differentiate from sibling tools like craft_get_item, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, typical use cases, or relationships to sibling tools like craft_get_item, leaving the agent with no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• craft_get_workspaceB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'Get details,' implying a read-only operation, but doesn't specify what details are returned (e.g., name, settings, permissions), whether it requires authentication, or any rate limits. This is a significant gap for a tool with zero annotation coverage, making it minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence: 'Get details about the configured craft.io workspace.' It's front-loaded with the core purpose, has zero waste, and is appropriately sized for a simple tool. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is adequate but has clear gaps. It states what the tool does but lacks context on return values, authentication needs, or error handling. Without an output schema, the description should ideally hint at what 'details' include, but it doesn't, making it minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there's no need for parameter details in the description. The description appropriately doesn't discuss parameters, which is efficient. Baseline for 0 parameters is 4, as it avoids unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get details about the configured craft.io workspace.' It uses a specific verb ('Get details') and identifies the resource ('configured craft.io workspace'), which distinguishes it from siblings like 'craft_get_item' or 'craft_get_items' that target individual items or collections. However, it doesn't explicitly differentiate from 'craft_ping' (which likely checks connectivity), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios like retrieving workspace metadata, checking configuration, or prerequisites. With siblings like 'craft_get_item' and 'craft_get_items', there's no indication of when workspace details are needed over item details, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• craft_get_itemA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves an item, implying a read operation, but does not cover aspects like error handling (e.g., what happens if the ID is invalid), authentication needs, rate limits, or response format. This leaves significant gaps for a tool with no structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get a specific item by ID from craft.io') and includes a helpful example without unnecessary details. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, 100% schema coverage, no output schema), the description is adequate for basic understanding. However, without annotations or an output schema, it lacks details on behavioral traits (e.g., error cases, response format) and does not fully guide usage relative to siblings, leaving room for improvement in contextual completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema fully documents the 'itemId' parameter. The description adds minimal value by reinforcing the parameter's purpose with an example ('e.g., UPS-1234'), but does not provide additional semantics beyond what the schema already states. Since there is only one parameter, a baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get') and resource ('a specific item by ID from craft.io'), with an example ('e.g., UPS-1234') that helps distinguish it from sibling tools like 'craft_get_items' (which likely retrieves multiple items). It avoids tautology by not merely restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing a single item by ID, but it does not explicitly state when to use this tool versus alternatives like 'craft_get_items' (for multiple items) or 'craft_get_workspace' (for workspace data). No exclusions or prerequisites are mentioned, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.