Server Quality Checklist

Tool Scores

• verify_claimC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'attach exact spans', hinting at an output behavior, but does not disclose whether the operation is read-only, whether external systems are involved, or what happens when claims are unsupported (e.g., are they ignored or flagged?). This is insufficient for a tool that likely performs an analysis task.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler words. It is front-loaded with the main verb and noun. However, its brevity contributes to under-specification; while structurally clean, it omits crucial detail that would make the content value-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return value structure is covered, but the description still lacks operational context. It does not explain what 'supported' means, how claims are identified, whether multiple claims are handled, or the relationship with 'lookup'. Given the tool's potential complexity (verification and span attachment), the description is incomplete for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes 'text' only by type and length constraints, and schema description coverage is 0%. The description does not mention the parameter at all, leaving the meaning of 'text' completely ambiguous. The description should explain what kind of text is expected (e.g., a document, a sentence, a claim list) and how it is used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('verify') and a specific target ('claims') with a distinctive outcome ('attach exact spans'). It distinguishes from the sibling 'lookup' by focusing on verification rather than retrieval, though 'claims' and 'supported one' remain somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus the sibling 'lookup'. There is no mention of prerequisites, typical input cases, or exclusions. The description implies a verification/annotation use-case but does not explicitly state it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookupC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the retrieval method (BM25+dense reciprocal-rank fusion) but does not state whether the operation is read-only, what side effects or requirements exist (e.g., permissions, rate limits), or the nature of the output. The word 'Retrieve' implies read-only, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core action. It avoids unnecessary filler, and each word contributes to conveying the purpose, though the technical detail might be considered slightly niche.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters) and presence of an output schema, the description technically covers the basic retrieval action. However, it lacks crucial context such as usage guidance relative to the sibling tool and explicit safety disclosure (since no annotations exist), making it incomplete for an agent to fully understand the tool's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no explanation of the parameters ('topic' and 'limit'). While the schema defines types and constraints, the description does not clarify what 'topic' means in the context of CV evidence or how 'limit' affects results, failing to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Retrieve CV evidence') and identifies the resource (evidence). It distinguishes from the sibling tool 'verify_claim' by focusing on retrieval rather than verification, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus 'verify_claim' or any other alternative. It does not state prerequisites, exclusions, or typical use cases beyond the basic action of retrieval, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.