Server Quality Checklist

Tool Scores

• doc_contextC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It fails to mention whether the operation is read-only, idempotent, or has side effects. Minimal information beyond the return action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence is concise and front-loaded with the primary function. No waste, but could benefit from a bit more structure (e.g., listing parameter roles).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no output schema, and no annotations, the description is grossly insufficient. It does not explain return format, default behavior, or error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, leaving all 7 parameters without explanations. The tool description does not add any meaning beyond parameter names (e.g., anchorId vs query, how before/after interact with limit).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and resource ('blocks') with clear context ('nearby blocks around an anchor' or matching a query). It distinguishes itself from sibling tools like doc_read (reads whole doc) and doc_search (searches tokens).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not state when not to use it, prerequisites, or which sibling to prefer in different scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doc_searchC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that it returns excerpts but does not mention read-only nature, auth needs, or details like case sensitivity or anchor mechanics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with verb and resource, but lacks parameter descriptions. Could be expanded without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters, 0% schema description coverage, no output schema, and no annotations, the description does not provide sufficient context for correct invocation. Agent needs more detail on constructing queries and interpreting anchors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and description provides no explanation for any of the 6 parameters (indexId, path, query, limit, contextChars, caseSensitive). Agent must rely on schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Search' and resource 'document blocks', and mentions returning 'excerpts tied to stable anchors', which clearly distinguishes from siblings like doc_read (full document) or doc_analyze (analysis).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives (e.g., doc_read for full retrieval, doc_context for contextual info). Does not mention prerequisites or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doc_list_formatsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for transparency. It only states what the tool lists, but does not disclose behavioral traits such as read-only nature, return format, or any side effects. The description is insufficient for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that clearly states the tool's purpose without any verbose or redundant phrasing. Every word adds value, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should explain the return value format (e.g., whether it returns a list of strings or objects). It fails to do so, leaving the agent unsure of what to expect from the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, and schema description coverage is 100% (trivially). According to guidelines, baseline is 3 when coverage is high. The description adds no parameter information, which is acceptable since there are no parameters to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource (Flyfish File Viewer known formats and document-engine extractors). It distinguishes the tool from its siblings (doc_analyze, doc_context, doc_read, doc_search) as the only one focused on listing formats and extractors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any prerequisites, conditions, or scenarios where another sibling tool would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doc_readB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description states read-only operation from cache but omits behavioral details like idempotency, side effects, or prerequisites (e.g., index must exist from doc_analyze).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence conveys core purpose efficiently. Slightly front-loaded but could benefit from additional context without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description is insufficient given six parameters, no output schema, and no annotations. Lacks explanation of how to combine parameters (e.g., indexId vs path) or what the return value includes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is low (33%), and description mentions only general element types (blocks, chunks, anchors) without clarifying parameter relationships or usage logic for the five non-described parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool reads blocks, chunks, or anchors from a cached index, distinguishing it from source document inspection. Verb 'Read' and specific resource 'cached index' are explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage when a cached index is available ('without inspecting source internals'), but does not provide explicit when-to-use or when-not-to-use guidance compared to siblings like doc_analyze or doc_search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doc_analyzeB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It mentions the output includes a 'read-only profile', which implies the operation is read-only, but it does not explicitly state that the tool does not modify files, nor does it mention any authentication or performance considerations. The behavioral transparency is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 20-word sentence that front-loads the action and lists the main outputs. Every word adds value; there is no fluff or repetition. It is highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters and no output schema, yet the description does not explain the output structure or how parameters like chunkSize affect the result. Given the complexity and the presence of sibling tools, the description is too brief to be fully complete. An agent would likely need additional context to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters have descriptions in the input schema (100% coverage), so the schema already explains them. The description adds no additional meaning or context for parameters beyond what is in the schema. Thus, baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Parse' and the resource 'local document', and lists the specific output components: read-only profile, anchors, outline, blocks, and chunks. This provides a detailed and distinct purpose compared to sibling tools like doc_read or doc_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives like doc_read or doc_search. It lacks any 'when to use' or 'when not to use' information, leaving the agent to infer context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.