Server Quality Checklist

Tool Scores

• notation_validateA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden of disclosing behavior. It mentions it returns errors with fix suggestions and is cheaper, but does not explicitly state that the tool is read-only, idempotent, or free of side effects—common expectations for a validation tool but not confirmed. More explicit behavioral context would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first sentence states purpose and output; the second gives usage guidance. No repetition or filler. Essential information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description covers purpose and usage but omits detail on error types, fix suggestion format, input limitations, or edge cases. It provides a minimal but functional level of completeness, with room for more context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 6 parameters with 0% description coverage; the description adds no parameter-specific meaning. While parameter names (title, composer, tempo, etc.) are self-explanatory, the description fails to clarify constraints, relationships, or how parameters influence validation. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool validates an input shape without rendering, distinguishing it from the sibling notation_render. It uses specific verbs ('validate') and identifies the resource ('input shape'), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance: 'Cheaper than notation_render — use this when iterating on input shape.' It tells the agent when to use (during iteration) and implies an alternative (notation_render for actual rendering).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notation_examplesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full burden. It discloses that the response should be cached client-side and that the shape is stable, which is valuable behavioral context for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded content. The first sentence lists examples clearly, and the second adds caching and stability info. No redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters or output schema, the description is sufficiently complete. It tells what the tool fetches and describes response characteristics, covering all necessary information for a simple fetch operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4. The description adds meaning by enumerating example categories, going beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches canonical example inputs and lists specific examples like single melody and chord progression. It distinguishes from siblings such as knowledge_search, notation_render, notation_schema, and notation_validate by focusing on examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by listing examples, but the description lacks explicit guidance on when to use this tool versus other notation tools. No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notation_schemaA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so description carries full burden. Discloses stable API result and suggests client-side caching, adding value. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Front-loaded with main action. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a zero-parameter tool. Describes purpose and behavior. Could mention return format, but not essential given simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, so baseline is 4. Description adds no parameter info, but none needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches the JSON Schema for notation_render input shape, specifying verb and resource. It distinguishes from siblings like notation_render (rendering) and notation_validate (validation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage context (fetch schema for notation_render) and advises caching due to stability. Does not explicitly exclude alternatives but given sibling tools, purpose is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notation_renderA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden of behavioral disclosure. It explains that bar lines are inferred from time signature and notes crossing bar lines are split and tied automatically. It also describes the pitch and duration format expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph that efficiently conveys purpose, output, input formats, behavior, and usage advice. It is front-loaded with the main action and each sentence adds value, though it could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the lack of an output schema, the description provides good coverage of input formats and behavior. However, it does not explain all parameters (e.g., title, composer, tempo) in detail, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33%, but the description adds significant meaning: it explains scientific pitch notation ('C4', 'F#5'), duration codes (w, h, q, etc.), and the structure of notes (shortcut strings vs. objects). However, parameters like title, composer, and tempo are not elaborated beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Render music notation from a JSON score.' It specifies the output formats (SVG, MusicXML, MIDI) and distinguishes itself from sibling tools like notation_validate by advising to validate first.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells users when to use notation_validate instead ('if you are unsure your input is well-formed — validate is cheaper than render'). It also explains that bar lines are inferred and notes are automatically split, providing clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• knowledge_searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite lacking annotations, the description thoroughly discloses behavioral traits: output JSON structure, sourceType enums, error on empty inputs, empty chunk behavior, and typical latency (200-700 ms). It covers all necessary operational aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-structured with labeled sections (WHEN TO USE, WHEN NOT TO USE, INPUT, OUTPUT, EXAMPLE INPUT, TYPICAL LATENCY). Every sentence adds value, and the purpose is front-loaded. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description fully explains the output format, including chunk objects with fields, sourceType list, empty chunks behavior, and attribution. It leaves no critical gaps for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant context beyond the schema. It explains topics as kebab-case tags with semantic matching via Voyage, step as a fallback, and provides usage guidance for limit and maxTokens defaults (8 and 1500) with rationale.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches the Gradus music-theory knowledge base for authoritative source material, listing specific corpus contents. It also distinguishes itself from sibling tools by directly referencing notation_render and notation_examples for score notation, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides 'WHEN TO USE' and 'WHEN NOT TO USE' sections, detailing scenarios such as looking up theory facts before generating notation, and excluding generic music vocabulary, non-theory queries, and score notation. It also suggests alternative tools for out-of-scope tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.