Server Quality Checklist

Tool Scores

• octave_validateB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides some behavioral details like optional repairs, diff output, and debugging info, but does not disclose side effects, permissions, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the main purpose. It could be more structured but is not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fails to explain return format or behavior in detail (e.g., what 'canonical form' looks like, error handling), despite having no output schema and 9 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds token-saving notes for diff_only and compact, and explains profiles, but does not add significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Schema check + repair suggestions for OCTAVE content' and specifies focus on I3 and I5 constraints, distinguishing it from sibling tools like octave_compile_grammar and octave_write.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like octave_write or when to avoid it. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octave_ejectA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses template generation when content is null, flexible section matching, and silent omission of non-existent sections. However, it does not state whether the tool modifies any state (e.g., it appears to be a pure transformation), nor does it describe error handling, rate limits, or authorization requirements. The behavioral disclosures are adequate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences clearly convey the core functionality. The first sentence is front-loaded with the primary action. The second adds key details (modes, formats, template generation). No redundant or extraneous information. Very concise while still informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, 2 with enums, no output schema), the description covers the main behaviors: projection modes, output formats, template generation, and special section handling. It does not specify validation behavior or side effects, but the core use case for an eject/export tool is well-covered. The omission of return value format is partly mitigated by the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already documented. The description adds meaning by explaining the effect of each mode (e.g., executive yields STATUS,RISKS,DECISIONS), and the flexible section input (e.g., '§3', '3', '§3::CAPABILITIES'). This goes beyond the schema's enum labels and type descriptions, providing actionable context for choosing parameter values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('eject OCTAVE content'), the resource ('OCTAVE content'), and the modalities ('projection modes', 'canonical, authoring, executive, and developer views'). It also lists output formats and template generation capability. The tool is distinct from sibling tools (compile, validate, write) because it focuses on exporting/transforming content in different views and formats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings (e.g., octave_validate, octave_write). The description does not mention prerequisites, typical use cases, or conditions that would make this tool preferable. An agent would have to infer usage from the tool name and behavior alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octave_compile_grammarA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It mentions input exclusivity and output formats but does not detail failure modes, side effects, or authentication needs, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences, front-loading the main purpose and essential details, with no unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a medium-complexity tool with 3 parameters and no output schema, the description adequately covers inputs and output formats. However, missing details about output structure or error conditions slightly reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds value by clarifying mutual exclusivity of 'schema' and 'content', and specifying the default for 'format'. This goes beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: compile OCTAVE schema or contract to constraint grammar, specifying supported output formats and input options. This distinguishes it from siblings like octave_eject, octave_validate, octave_write.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on when to use the tool (compilation to grammar) and mentions output formats, but lacks explicit exclusions or alternative tool references, which would make it more helpful for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octave_writeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, description fully explains behavioral traits: error conditions (E_OP_TARGET_MISMATCH, E_UNRESOLVABLE_PATH, E_INVALID_OP_DESCRIPTOR), format options with deprecation warnings, and normalization behavior. Exceeds minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long but every sentence earns its place. Could be slightly restructured for readability (e.g., bullet points for changes ops), but front-loads key info. Not overly verbose given complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 13 parameters, no output schema, and nested objects, the description covers all aspects: purpose, usage modes, parameter details, error handling, validation, format options, and deprecation. Leaves no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds substantial meaning: details about $op descriptors for changes, deprecation for format_style=null, explanation of schema validation, and dry run aliasing. Goes far beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Unified entry point for writing OCTAVE files' and covers creation/modification. Distinguishes from legacy tools (octave_create, octave_amend). Specific verb 'write' and resource 'OCTAVE files'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when to use content vs changes vs omit both for normalization. Also mentions it replaces other tools. However, no explicit guidance on when not to use or when to use sibling tools like octave_validate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.