Server Quality Checklist

Tool Scores

• setC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'mutating/management commands' and examples like 'restart' and 'save', implying destructive or state-changing operations, but fails to detail critical aspects such as required permissions, side effects, error handling, or rate limits, which are essential for safe tool invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with a clear purpose statement followed by relevant examples. Each example serves to illustrate usage without redundancy, making it efficient, though the lack of explicit guidelines slightly reduces its overall effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutating operations with 2 parameters, 0% schema coverage, and no annotations) but with an output schema present, the description is moderately complete. It covers the tool's purpose and provides usage examples, but lacks details on behavioral traits and parameter semantics, which are crucial for a mutation tool without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It lists example commands that imply 'command' and 'args' parameters (e.g., 'set log level debug' uses both), but doesn't explain their semantics, valid values, or formatting rules beyond the examples, leaving significant gaps in parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool runs 'mutating/management commands' with examples like 'set/change/restart/save/etc.', which clarifies it performs various administrative actions. However, it doesn't explicitly distinguish this from sibling tools like 'get' (likely read-only) or 'run' (possibly execution-focused), leaving the differentiation implied rather than explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get' or 'run'. It lists example commands but doesn't specify contexts, prerequisites, or exclusions, such as whether it's for system-level changes or user-specific settings, leaving the agent to infer usage from examples alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the read-only behavioral trait, which is helpful. However, it doesn't mention authentication needs, rate limits, error handling, or output format. The examples hint at command patterns but lack full behavioral context like what 'monitor report' entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with a clear opening statement followed by examples. Every sentence earns its place by providing essential information and practical guidance. However, the examples could be more tightly integrated with the main description for better flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description doesn't need to explain return values. However, with no annotations, 0% schema coverage, and 2 parameters, the description is incomplete—it lacks details on parameter usage, system context, and behavioral constraints. The examples help but don't fully compensate for the gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides no explicit information about the 'command' or 'args' parameters beyond the examples. The examples imply possible values but don't explain parameter roles, formats, or constraints. This leaves significant gaps in understanding how to use the parameters effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool runs 'read-only/info/show style commands', which gives a general purpose but lacks specificity about what resource or system it operates on. It distinguishes from siblings 'run' and 'set' by emphasizing read-only nature, but doesn't explicitly name what it retrieves. The examples provide concrete patterns but don't define scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates this is for read-only operations through the phrase 'read-only/info/show style command', which implicitly suggests when to use it versus write operations. However, it doesn't explicitly name alternatives like 'run' or 'set' or provide exclusion criteria. The examples offer practical guidance but no explicit when-not-to-use statements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• runA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool runs 'raw OpenSimulator console commands,' which implies it could execute any command, including potentially destructive or administrative ones, but doesn't disclose specific behavioral traits like safety risks, permission requirements, or rate limits. The description adds minimal context beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the purpose, and the second provides usage guidelines. It is front-loaded and appropriately sized, with every sentence earning its place by adding critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (running arbitrary console commands) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers purpose and usage well, but lacks behavioral details like safety warnings or execution context, which are important for such a powerful tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, with one parameter 'command' undocumented in the schema. The description adds meaning by specifying it's for 'raw OpenSimulator console commands,' giving context to the parameter's purpose. However, it doesn't provide details on command syntax, examples, or constraints, so it partially compensates but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run any raw OpenSimulator console command') and distinguishes it from siblings by mentioning 'commands not covered by get/set categorization.' It provides a verb+resource combination that is precise and differentiates from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('for commands not covered by get/set categorization') and implies when not to use it (for commands covered by get/set). It directly references sibling tools 'get' and 'set' as alternatives, providing clear contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.