Server Quality Checklist

Tool Scores

• read_skillsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description adds useful context about reading sub-paths and reference files within a skill. It does not contradict annotations and provides behavioral details beyond the structured hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, starting with the core purpose, then providing usage prerequisites, context, and examples. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter, an output schema, and strong annotations, the description fully covers usage, prerequisites, and examples. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the 'skills' parameter with 100% coverage, and the description adds meaning by showing example formats (e.g., ['staad-analysis'] or sub-paths) and explaining that sub-paths access reference files. This goes beyond the schema's basic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Read' with the resource 'one or more skills by name', and distinguishes itself by explaining that it can also read sub-paths/reference files within skills. This differentiates it from sibling tools like discover_api (listing skills) and execute_code (running code).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use discover_api first to list available skills, and provides concrete examples of valid inputs. It does not explicitly mention when not to use this tool or compare with execute_code, but the usage context is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_codeA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes far beyond the annotations (destructiveHint=true, etc.): describes sandbox restrictions (blocked import, dir, getattr), pre-injected variables (staad, input_data, json, math), return value semantics (last expression or result assignment), file writing behavior, path constraints, and overwrite behavior. This provides comprehensive context for safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but each sentence carries operational weight—sandbox rules, return behavior, file I/O, and path security. It is well-structured in paragraphs and front-loaded with the core purpose. It could be slightly more compact, but the density is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (code execution, sandbox, file I/O, multiple parameters), the description covers all critical aspects: prerequisites, environment setup, return conventions, output data structures, filesystem constraints, and even a Claude Desktop workaround. With a strong schema and these descriptions, an agent is fully equipped to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema: explains pre-injected variables for the 'code' parameter, details the result variable format for output_data_path (list-of-lists or dict of sheets), and clarifies the 'overwrite' flag usage. This elevation to 4 reflects the added semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Execute Python code in a sandbox against the OpenSTAAD API'. This is a specific verb+resource that distinguishes it from sibling tools (discover_api, read_skills, list_instances, get_status), which are for discovery/status, not code execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage context: instructs to call discover_api and read_skills for API guidance, suggests using output_data_path to avoid flooding context, and notes path restrictions. Does not explicitly contrast with alternative tools (none exist for execution), but effectively guides when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description adds no contradictory info. It discloses useful context about return values and the instance-selection behavior, going beyond the annotation's safety hint. It stops short of explaining error cases or connection failure behavior, but that is not critical for a status check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences cover purpose, parameter usage, and return values. The information is front-loaded and there is no redundant or vague wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one optional parameter) and the description covers the core actions, parameter handling, and output. The existing annotations plus description are sufficient for an agent to invoke it correctly without further assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the only parameter, but the description fully explains it: it is an alias from list_instances and can be omitted when only one instance runs. This completely compensates for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and scope: 'Check the connection to a STAAD.Pro instance.' It also specifies what the tool returns (connection state, STAAD version, model path), which distinguishes it from siblings like list_instances.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage instructions for the instance parameter: pass an alias from list_instances, or omit it when only one instance is running. This implies a workflow but does not explicitly contrast the tool with alternatives, leaving a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_apiA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint and idempotentHint. The description adds useful behavioral context about its role as a discovery tool and the follow-up step with read_skills, but does not detail return structure or pagination. Given the annotations, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: one stating purpose and one providing usage flow. It is front-loaded, efficient, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, output schema present, annotations covering safety), the description fully covers its role and relationship to sibling tools. It is complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the description does not need to explain any. The baseline is 4 per the rubric, and no additional detail is required since the schema is empty and no inputs exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: 'Discover available API guidance and skills.' It distinguishes itself from sibling tools by positioning as the entry point, with read_skills for loading specific guidance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Call this FIRST before using other openstaad-mcp tools. Then use read_skills with one or more specific skill names to load full guidance.' This clearly instructs when and how to use the tool relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_instancesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the safe-read nature is covered. The description adds valuable behavioral context beyond annotations: the alias stability across the server session, the version warning condition for versions below 25.0.1, and the specific fields returned. This goes beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at three sentences, each earning its place. The first sentence states the core purpose, the second covers output fields and usage, and the third handles an edge case warning. Information is front-loaded and no fluff exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters, a simple output schema, and clear annotations, the description covers all necessary behavioral aspects. It explains when to use the tool, what it returns, and the version-notice edge case. The existence of an output schema means the description needn't enumerate return fields in detail, making this sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the input schema confirms this with an empty properties object. Since there are no parameters to explain, the description's silence on parameters is appropriate. The baseline for 0 params is 4, and the description does not need to add any parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'List all running STAAD.Pro instances,' which is a specific verb+resource statement that clearly identifies what the tool does. It also distinguishes itself from siblings like execute_code by specifying that it returns instance details such as alias, process ID, open file path, and version.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit usage directive: 'Call this before execute_code when multiple STAAD instances may be running so you can pick the right one.' This tells the agent when to use the tool and why, while implicitly excluding other contexts. It also explains that the alias is stable, which is critical for subsequent tool calls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.