Server Quality Checklist

Tool Scores

• openvsp.run_vspaeroC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool runs edits and VSPAero, implying a mutation operation, but doesn't detail critical behaviors such as whether it modifies the input file in-place, creates new files, requires specific permissions, handles errors, or has performance implications like runtime or resource usage. The description is too sparse for a tool that likely involves complex computational tasks.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just one sentence—with no wasted words. It front-loads the core action ('Run OpenVSP edits followed by VSPAero') and includes essential input hints. This efficiency is appropriate, though it may sacrifice detail for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of running geometry edits and aerodynamic analysis, the description is incomplete. No annotations exist to clarify safety or behavior, and while an output schema is present (which might describe results), the description doesn't hint at what the tool returns or its operational context. For a tool with nested parameters and likely significant computational impact, more guidance on usage, outcomes, and constraints is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'geometry commands and case_name,' which loosely maps to the 'set_commands' and 'case_name' parameters in the schema. However, with 0% schema description coverage, the schema provides no descriptions for parameters, and the description doesn't fully compensate by explaining all parameters (e.g., 'geometry_file' and 'run_vspaero' are not addressed). It adds minimal semantic value beyond the parameter names, meeting the baseline for partial coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run OpenVSP edits followed by VSPAero.' It specifies the verb 'run' and the resources 'OpenVSP edits' and 'VSPAero,' which is specific and actionable. However, it doesn't explicitly distinguish this tool from its siblings (openvsp.inspect and openvsp.modify), which likely involve inspection or modification without the VSPAero execution step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions providing 'geometry commands and case_name,' but doesn't explain scenarios where this tool is preferred over openvsp.inspect or openvsp.modify, nor does it outline prerequisites or exclusions. This lack of contextual direction leaves the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openvsp.modifyB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that the tool 'returns the generated script path,' which adds some behavioral context about the output. However, it lacks details on permissions needed, whether edits are destructive or reversible, error handling, or how the script interacts with the model. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences that are front-loaded with key information: the tool's purpose and a usage hint. There's no wasted text, though it could be slightly more structured to separate purpose from guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with 1 parameter but nested objects), no annotations, and an output schema (which reduces the need to describe returns), the description is partially complete. It covers the main action and output but misses critical behavioral details like safety, prerequisites, and parameter explanations, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It references 'set_commands' to adjust geometry, which aligns with one parameter, but doesn't explain other parameters like 'geometry_file' or 'run_vspaero' (which seems contradictory to the description's 'without running VSPAero'). The description adds minimal value beyond the schema, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Apply scripted parameter edits to an OpenVSP model without running VSPAero.' It specifies the action (apply edits), resource (OpenVSP model), and constraint (without VSPAero). However, it doesn't explicitly differentiate from sibling tools like 'openvsp.run_vspaero' beyond mentioning the VSPAero exclusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Use set_commands to adjust geometry' and distinguishes it from VSPAero execution. However, it doesn't explicitly state when to use this tool versus alternatives like 'openvsp.inspect' or 'openvsp.run_vspaero', nor does it mention prerequisites or exclusions beyond the VSPAero aspect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openvsp.inspectA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool is non-modifying and returns 'component IDs and raw info,' which covers basic behavior. However, it lacks details on error handling, performance, or any constraints like file size limits, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded and concise, consisting of two sentences that efficiently convey the tool's purpose and output. Every sentence earns its place by adding value without redundancy, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter) and the presence of an output schema, the description is mostly complete. It covers the non-modifying nature and output type, but could benefit from more behavioral context, such as error cases or usage tips, to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information beyond what the input schema provides, as schema description coverage is 0%. However, with only one parameter, the baseline is high. The description's mention of 'OpenVSP geometry' and '.vsp3 file' in the schema provides adequate context, compensating for the lack of explicit param details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Describe') and resource ('OpenVSP geometry'), and distinguishes it from siblings by emphasizing it doesn't modify the geometry. However, it doesn't explicitly name the sibling tools for comparison, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by stating it 'without modifying it,' suggesting it's for inspection rather than modification. However, it doesn't provide explicit guidance on when to choose this over alternatives like 'openvsp.modify' or 'openvsp.run_vspaero,' nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.