Server Quality Checklist

Tool Scores

• fetch-elements-of-categoryC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'fetch' which implies a read operation, but doesn't disclose critical traits like whether it's safe, what permissions are needed, rate limits, pagination, or what happens on errors. For a tool with complex nested parameters and no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, stating the core purpose without unnecessary elaboration. Every word earns its place, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (nested objects, no output schema, no annotations), the description is incomplete. It doesn't explain what 'elements' are, the return format, or how the config affects results. For a tool with rich input schema but no output schema, more context is needed to understand its behavior and usage fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('category' and 'config') well-documented in the schema. The description adds no additional meaning beyond the schema, such as explaining the purpose of the config object or providing examples beyond the basic 'e.g. IFCWALL' in the schema. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Fetch elements of a specified category' clearly states the action (fetch) and target (elements of a category), but it's vague about what 'elements' are and doesn't distinguish from sibling tools like 'convert-ifc-to-frag' or 'load-frag'. It provides a basic purpose without specific context about the domain (e.g., IFC/BIM data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites, when this tool is appropriate, or any exclusions. With sibling tools like 'convert-ifc-to-frag' and 'load-frag', there's no indication of how this tool relates to them or when to choose one over another.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert-ifc-to-fragC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'Needs file-system server' which adds useful context about infrastructure requirements, but it doesn't describe other behavioral traits like whether the conversion is destructive, what happens if files already exist, error conditions, or performance characteristics. For a file conversion tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that each serve a purpose: the first states the core functionality, the second provides important context about server requirements. There's no unnecessary verbiage, and the information is front-loaded with the primary purpose stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a file conversion tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the .frag format is, what the conversion entails, potential limitations, or what happens upon completion. The 'Needs file-system server' hint helps but doesn't compensate for the lack of behavioral and output information that would help an agent use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters (inputPath and outputPath) clearly documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the schema descriptions. According to the rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: converting an IFC file to a .frag file. It specifies both the input format (IFC) and output format (.frag), making the verb+resource combination explicit. However, it doesn't differentiate from sibling tools like 'load-frag' which might handle .frag files differently, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance with 'Needs file-system server' indicating a prerequisite, but it doesn't explain when to use this tool versus alternatives like 'load-frag' or 'fetch-elements-of-category'. There's no explicit when/when-not guidance or comparison with sibling tools, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load-fragC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool loads a file and requires a file-system server, but it doesn't disclose critical traits like whether this is a read-only operation, potential side effects (e.g., memory usage), error handling, or performance considerations. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two short sentences that directly address the tool's function and requirement. Every word earns its place, and it is front-loaded with the core action, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of file loading operations, the lack of annotations and output schema, and the description's minimal detail, this is incomplete. It doesn't explain what 'load' means in terms of output (e.g., data structure, success/failure indicators) or behavioral nuances, which are crucial for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'filePath' parameter well-documented in the schema. The description does not add any meaning beyond what the schema provides (e.g., it doesn't clarify file format expectations or path validation rules). According to the rules, with high schema coverage, the baseline is 3 even without param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Load') and the resource ('.frag file'), making the purpose understandable. It distinguishes from sibling tools like 'convert-ifc-to-frag' and 'fetch-elements-of-category' by focusing on loading rather than conversion or fetching. However, it doesn't specify what 'load' entails (e.g., reading, parsing, or processing), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Needs file-system server', which provides some context about prerequisites, but it lacks explicit guidance on when to use this tool versus alternatives. There is no mention of when-not-to-use scenarios or comparisons with sibling tools, leaving usage decisions ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.