Server Quality Checklist

Tool Scores

• catalog_queryC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, covering safety. Description adds the AND-combined filter behavior, which is useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, front-loaded with action. Could be slightly longer to include more detail, but remains concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 optional parameters and an output schema, the description is too minimal. It does not explain the role of each filter or the query's scope, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only mentions 'optional AND-combined filters' without elaborating on any parameter names, types, or meanings. Fails to compensate for undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Query' and resource 'stored local document metadata', and adds specificity with 'AND-combined filters'. However, it does not distinguish from sibling tools like catalog_scan, missing differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings or alternatives. No 'when-not' or context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• catalog_record_generatedB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the behavioral constraint 'never return contents', which goes beyond the annotations. Annotations already indicate readOnlyHint=false (write operation) and destructiveHint=false. However, the description does not disclose side effects (e.g., what exactly 'record' does, whether it overwrites), nor does it explain behavior for non-existent files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, focus-packed sentences. The first states the primary purpose, the second adds a critical behavioral exclusion. No filler or redundancy. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema exists), the description covers the core: recording metadata for an existing local PDF and not returning contents. However, it omits details like whether recording is idempotent, how the file is located, or any error handling. The output schema likely explains returned data, but preconditions and side effects are unclear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description must compensate. It adds meaning by specifying that the file_path must point to 'one existing generated local PDF', implying constraints on existence and type. While beneficial, it could be more precise (e.g., absolute/relative path, allowed file extensions).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Record metadata') and the resource ('one existing generated local PDF'). It distinguishes from potential content-returning tools by adding 'never return contents'. However, the verb 'Record' is somewhat vague, not fully specifying whether it creates, updates, or stores a catalog entry.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings like catalog_record_printed or catalog_query. The phrase 'never return contents' hints at one differentiator, but there is no when-to-use or when-not-to-use instruction. The description relies on tool names and implicit understanding.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• catalog_record_printedB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=false and destructiveHint=false. Description adds 'existing local PDF' context but does not disclose additional behavioral traits beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single concise sentence of 12 words, front-loaded with verb and resource. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low schema coverage, the description provides basic context (existing local PDF) but lacks details about what metadata is recorded or parameter constraints beyond the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to compensate. The description only mentions 'optional print job ID', which is already hinted by the parameter name. No additional meaning is added for file_path or the full scope of metadata.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Record' and the resource 'metadata and optional print job ID for one existing local PDF'. It distinguishes from siblings like catalog_record_generated by specifying 'existing local PDF'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description implies usage for recording an existing PDF but does not provide when-to-use or when-not-to-use information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• catalog_scanA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false, so the tool is neither read-only nor destructive. The description adds that it 'stores metadata only', clarifying it writes, but does not elaborate on potential side effects, performance impact, or configuration requirements. This adds some value beyond annotations but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and free of unnecessary words. It effectively communicates the core functionality without excess.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, clear purpose), the description is mostly complete. It explains what is scanned and the outcome (metadata storage). However, it could mention if the scan is incremental or full, and how repositories are configured. Still, it meets the needs for a low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (empty). The description does not need to provide parameter-level details, and it appropriately avoids missing information. Baseline score for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans configured local repositories and the journal, storing only metadata. The verb 'Scan' and resource 'configured local repositories and journal' are specific. It distinguishes from sibling tools like catalog_query (querying) and catalog_record_generated/printed (record creation), which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., catalog_query). The description lacks explicit context for when a scan is appropriate or when to prefer other operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.