Server Quality Checklist

Tool Scores

• get_fantasy_entityB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so safety profile is clear. The description adds that the tool also returns 'directly related cards', which is behavioral context beyond annotations. No contradictions. Could be improved by mentioning whether results are paginated or what happens on missing key.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words. Front-loaded with action 'Reads' and resource. Structure is optimal for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, no output schema, and annotations present, the description is minimally adequate. It states the tool reads an entity card and related cards, but omits details like what 'directly related cards' means, error behavior, or data count limits. Given the complexity (related cards), the description feels slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'by stable key' but does not explain what a 'stable key' is, its format, or constraints (beyond schema's type and length). No examples or clarification of how the key relates to the entity. The description adds minimal semantic value over the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'reads' and the resource 'one canonical fantasy entity card', and mentions it also returns 'directly related cards'. This distinguishes it from siblings like search_fantasy_catalog (search) and trace_fantasy_relations (tracing relations). However, it does not explicitly clarify the role of the 'key' parameter beyond 'stable key'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention any prerequisites, when not to use it, or how it compares to sibling tools like read_web_source or search_fantasy_catalog. The agent is left to infer usage from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_github_sourceB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint. The description adds context about using the GitHub contents API and token requirements, which is helpful. It could mention potential truncation or pagination via offset parameter, but overall adds value beyond annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no waste. The first sentence gives purpose and API, the second gives token guidance. Could be improved by structuring parameter listing, but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters (3 required), no output schema, and sibling tools, the description is incomplete. It does not explain key parameters like owner, repo, path, or the truncation behavior via offset. Missing details that would help an agent handle large files or failures.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% (descriptions for offset and maxChars). The description adds no parameter details; it only mentions GITHUB_TOKEN which is not a schema parameter. The remaining parameters (owner, repo, path, ref) are left unexplained, failing to compensate for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Reads one file from a GitHub repository using the GitHub contents API', specifying the verb and resource. It implicitly distinguishes from sibling tools like read_web_source or read_local_source by targeting GitHub, but does not explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides token guidance (optional for public, required for private), giving some usage context. However, it does not indicate when to use alternatives or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_fantasy_catalogA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds that results include 'facts, relations, and provenance' and explicitly states it never modifies the catalog, reinforcing the read-only nature. However, it does not mention pagination, ordering, or performance details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the core purpose, the second adds behavioral context. No verbose or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and 0% parameter coverage, the description should compensate by detailing parameter usage and result structure. It does neither, leaving significant gaps for a search tool with multiple parameters and sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%. The description does not explain any of the three parameters (query, kinds, maxResults). Without parameter semantics, the agent cannot effectively use the input schema to formulate correct queries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Searches' and specifies the resource as 'canonical fantasy entity cards stored in the managed catalog'. It clearly differentiates from sibling tools like 'get_fantasy_entity' (single entity retrieval) and 'trace_fantasy_relations' (relation tracing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool is for searching and returning multiple results, but it does not explicitly state when to use this tool over alternatives like 'get_fantasy_entity' or 'trace_fantasy_relations'. There is no guidance on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trace_fantasy_relationsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds 'bounded relation graph' context, but does not disclose specific traversal behavior or output format. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no wasted words. It efficiently states the tool's purpose and key constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should clarify what the tool returns (e.g., list of relations). It does not explain output, the bounded nature beyond 'maxDepth', or how to interpret results. Missing critical context for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It hints that 'key' refers to a canonical entity card, but does not explain 'maxDepth' or provide syntax/format details. The parameter semantics are only partially addressed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'traverses' and identifies the resource 'bounded relation graph around one canonical fantasy entity card' for the purpose 'multi-entity reasoning'. It clearly distinguishes from siblings like 'get_fantasy_entity' and 'search_fantasy_catalog'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for multi-entity reasoning but does not explicitly state when to use this tool versus alternatives like 'search_fantasy_catalog' or 'get_fantasy_entity'. No when-not or alternative names are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_local_sourcesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds value by disclosing that build outputs, dependency folders, and Git metadata are skipped, which is important behavioral context beyond the annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the purpose and then provides key exclusions. Every word is relevant and there is no fluff. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters and no output schema, the description should provide more complete context. It does not mention return format, pagination, ordering, or how to use parameters effectively. The exclusions are useful but insufficient for a search tool that benefits from clarity on result structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate. It does not explain any of the four parameters (root, query, maxFiles, maxResults). The parameter names are self-explanatory, but the description adds no additional meaning about format, behavior, or constraints, leaving the agent to rely solely on schema names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches filenames and bounded UTF-8 files inside SOURCE_ROOTS. This distinguishes it from siblings like read_local_source (reads a specific file) and search_fantasy_catalog (searches fantasy entities). The verb 'searches' and resource 'local sources' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for local file search by specifying what is skipped (build outputs, dependency folders, Git metadata). However, it does not explicitly state when to use this tool over alternatives like read_local_source or search_fantasy_catalog, nor does it provide 'when not to use' or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_local_sourceA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable behavioral context: symbolic-link escapes are rejected, and it returns canonical provenance. This goes beyond the annotations, though it does not cover error handling or file-not-found behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that is front-loaded with the action and resource. Every word adds value, and there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with three parameters and no output schema, the description covers the essential: what is read, where from, security constraint, and return type. It is missing error behavior, but given the annotations, it is reasonably complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67% (offset and maxChars have descriptions). The description itself does not add parameter-specific semantics but provides context that path must be within SOURCE_ROOTS, which indirectly aids understanding. It does not fully compensate for the missing path description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'reads', the resource (UTF-8 text file), the constraint (inside SOURCE_ROOTS), and the output (canonical provenance). It distinguishes itself from sibling tools that read from web, GitHub, or images.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives like search_local_sources or other read tools. The usage context is implied by the name and the mention of SOURCE_ROOTS, but no direct guidance or exclusion criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_web_sourceA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false. The description adds behavioral details: it returns a bounded excerpt, blocks private addresses, and provides retrieval time and fingerprint. This adds value beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. Every sentence provides essential information: what the tool does, what it returns, and a key constraint (no private addresses).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's core behavior, return values, and a significant constraint. It does not explain pagination via offset or the maxChars limit in detail, but those are covered in the schema. For a read tool with no output schema, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (offset and maxChars have descriptions; url has format uri). The description does not add parameter-specific semantics beyond the schema, but the overall context of reading HTTP(S) as text is clear. Baseline 3 is appropriate for moderate coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a public HTTP(S) page as text and returns specific metadata (URL, retrieval time, SHA-256 fingerprint, excerpt). The name and description together distinguish it from sibling tools like read_github_source or read_local_source.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for public web pages and notes that private network addresses are blocked, but does not explicitly guide when to use this tool versus alternatives like read_github_source or read_local_source. There are no when-not-to-use instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_source_policyA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds specific items that are listed, providing context beyond annotations. No behavioral traits like pagination or format are mentioned, but the tool is simple.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that directly states what the tool does with no unnecessary words. Information is front-loaded and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, clear annotations), the description fully covers what the agent needs to know to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is 100%. The description does not add parameter details since none exist. Baseline for 0 parameters is 4, and the description adequately describes what the tool outputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Lists' and names the resource 'source policy' with concrete components (read-only roots, host allowlist, source limits, provenance guarantees). It clearly distinguishes from sibling tools that read or search specific sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. While the purpose is clear, there is no mention of preconditions, context, or cases where another tool would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fantasy_catalog_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, destructiveHint. The description adds that it returns aggregate data, confirming safe read-only behavior without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with zero waste, front-loaded with the core action and scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description specifies return type (aggregate readiness and entity counts), which is sufficient for a simple tool with comprehensive annotations and sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100%. The description need not add param info, meeting the baseline of 4 for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('returns') and resource ('aggregate readiness and entity counts for the managed fantasy catalog') and distinguishes the tool from siblings like get_fantasy_entity (single entity) and search_fantasy_catalog (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for high-level summary vs. detailed entity retrieval but does not explicitly state when not to use or name alternatives. Context from sibling tools supports differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_image_sourceA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds beyond annotations by specifying supported image formats and that provenance is included. Annotations already indicate read-only, open-world, idempotent, non-destructive behavior; description complements with format and provenance details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundant words. Core function in first sentence, critical usage condition in second. Perfectly front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, description covers purpose, parameter, formats, provenance, and prerequisite. Sibling tools exist but could be contrasted more. Still fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Describes source as 'allowed local path or public URL', which adds meaning beyond the schema's plain string definition. However, 'allowed' is vague and no examples are given. With 0% schema coverage, the description compensates partially.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool returns an image (PNG, JPEG, GIF, WebP) from a local path or public URL with provenance. This distinguishes it from siblings like read_web_source which likely return non-image content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions that the AI must support image content to interpret the result, providing a usage condition. Does not explicitly exclude alternatives or mention when not to use, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.