Server Quality Checklist

Tool Scores

• execute_scriptA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description bears full responsibility. It discloses key behaviors: auto-truncation of large lists, locale-dependent number formatting, file access restrictions, case sensitivity of element types, and available modules.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is excessively long, containing a full API manual. While the purpose is stated upfront, the sheer volume of information may overwhelm agents and hinder quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool and lack of annotations/parameter descriptions, the description is extremely thorough, covering API commands, element types, filters, file access, and common patterns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% coverage (no descriptions). The description adds meaning for all three parameters: port (linked to list_instances), script (Python code), and timeout_seconds (optional, default no timeout).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute Python script with full Archicad API access.' It distinguishes from sibling tools like list_instances, get_docs, and get_properties by focusing on arbitrary script execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes common patterns and examples, but lacks explicit guidance on when to use this tool versus alternatives. Agents can infer usage from examples, but no direct comparison with siblings is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_docsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so description carries full burden. It describes behavior: returns categories, commands, or full schemas, and includes search features (typo tolerance, partial match). It does not mention rate limits or authentication, but as a read-only documentation tool, these are less critical. Could be more precise about output structure, but overall transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with sections for usage, discovery workflow, search features, args, and examples. Front-loaded purpose. Slightly long but every section adds value; minor redundancy in examples.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers discovery workflow, search features, and all parameter use cases. Has output schema so no need to explain return format in detail. Given complexity, description is complete and self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description compensates fully: explains each parameter's purpose, provides multiple examples, and details search features (typo tolerance, multi-word). Adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it provides documentation for Archicad commands. The description distinguishes it from sibling tools (execute_script, get_properties, list_instances) by focusing on documentation retrieval rather than execution or property listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage patterns (with and without arguments) and a discovery workflow. It also describes search features. However, it does not explicitly state when not to use this tool (e.g., when needing to execute commands instead of reading docs), which would further enhance clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_instancesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully carries the burden. It discloses the scanning behavior over a specific port range and that it returns instance info. It does not mention auth or rate limits, but those are less relevant for a discovery tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: three sentences, front-loaded with the purpose, no wasted words. Every sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, an output schema exists, and the description covers the return fields (port, project name, version). The tool is simple and well-documented for its purpose. Siblings are distinct.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters. Per guidelines, baseline is 4 since no parameter information is needed. The description adds value by explaining what the tool does without parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'find all running Archicad instances', specifies the method (scans ports 19723-19744), and lists returned info (port, project name, version). It distinguishes from siblings like execute_script and get_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using the 'port' value in other tools to target a specific instance, providing clear follow-up guidance. It implies usage for discovery but does not explicitly state when to use versus alternatives or exclude cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_propertiesA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It describes the search behavior, filtering modes, and return value (GUIDs for use with GetPropertyValuesOfElements). It does not mention any destructive or side effects, but it is clear that this is a read-only search operation. Missing a note about sorting or pagination beyond limit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (WHEN TO USE, MODES, Args, Returns, NOTE). It is concise, with no redundant sentences. Each part adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and an output schema, the description covers usage comprehensively. It explains the purpose of parameters, the return format, and typical use cases. The note about get_docs prevents confusion. It is complete for an AI agent to use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema coverage, the description adds meaning to every parameter. It explains search, group, property_type, measure_type, property, and limit with examples and default values. It also shows how to use them in combination.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching and discovering Archicad element properties. It uses specific verbs like 'search and discover' and explicitly distinguishes from the sibling tool 'get_docs' in the final note.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'WHEN TO USE' examples and 'MODES' that cover various parameter combinations. It also tells when not to use (for command documentation, use get_docs).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.