Server Quality Checklist

Tool Scores

• check_conditionsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context about the output format ('pass/fail/uncertain with grounded evidence') and decision-support nature. Since annotations already declare readOnlyHint: true, the description does not need to restate safety, but it does not reveal additional behavioral details like authentication or reversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundant wording, and the key information is front-loaded. Every phrase earns its place: the conditions, the output nature, and the decision-support scope are all stated efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only one input parameter and an output schema, so the description does not need to explain return values. It names all four conditions and specifies the pass/fail/uncertain plus evidence format. This is complete for a decision-support tool, though it does not mention edge cases like empty input or handling of ambiguous documents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the single parameter case_text is self-explanatory as the document text. The description refers to 'the document' providing a light mapping, though it does not explicitly say the parameter represents the document. This is adequate for a simple string parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Evaluate') and resource ('four conditions precedent') against the document, and names the four conditions (notice timing, notice validity, rent/no-arrears, vacant possession). It clearly distinguishes itself from siblings like extract_break_clause, find_citation, and assess_validity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when evaluating conditions precedent, and states 'Decision-support only' which hints at its advisory role. However, it does not explicitly mention when to use this tool over alternatives, nor does it provide any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_validityA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, but the description adds valuable behavioral context: it discloses abstention to AMBIGUOUS rather than guessing, mandatory human-verify gates, and decision-support-only intent. These go beyond the annotations and help the agent understand safety and verification requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, starting with the core purpose and verdict types, then adding key behaviors (abstention, human-verify gates, decision-support). Every sentence contributes value without redundancy or waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's orchestrated nature, output components (per-condition results, conditions forcing verdict, calibration note), abstention behavior, and human-verify requirement. It is largely complete given the schema and output schema exist, though it omits any mention of the input parameter and when to use it, which is partially covered by other dimensions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the single parameter (case_text). It does not explicitly explain what case_text should contain, its format, or any requirements. While the parameter name is self-explanatory, the description provides no direct semantic guidance, leaving a gap for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Orchestrated assessment') and resource (the case text), and defines the output verdicts (VALID/INVALID/AMBIGUOUS). It distinguishes itself from siblings (extract_break_clause, check_conditions, find_citation) by being an orchestrated, holistic assessment rather than a targeted extraction or check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('Orchestrated assessment', 'Decision-support only') but does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or when-not-to-use guidance. The abstention behavior is a partial guideline, but there is no direct comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_break_clauseA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations (readOnlyHint: true, openWorldHint: false) already declare safe read-only behavior, and the description adds the behavioral detail of returning verbatim text and a source span, or found=false when absent. It also includes 'Decision-support only,' which signals its role as an analytical aid. The description is consistent with the annotations, providing extra context without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that packs essential information: extraction action, input reference, output details, and fallback behavior. There is no wasted wording, and each clause adds value. It is appropriately concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one string parameter) and presence of an output schema, the description is sufficient to convey the tool's function and return behavior. It covers the core purpose, the input, and the failure case (found=false), which is important. It does not need to elaborate on output format since the output schema exists, and the description is complete for this level of complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, lease_text, is described implicitly through 'lease document,' indicating it should contain the lease text. However, the schema description coverage is 0%, and the description does not elaborate on input format, length, or preconditions. The parameter name itself is fairly self-explanatory, but the description adds minimal meaning beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: extracting the tenant break clause from a lease document, with a specific verb ('Extract') and resource ('tenant break clause'). It also specifies the return format (verbatim text and source span, or found=false), making the purpose unambiguous. While it doesn't explicitly compare to sibling tools, the unique extraction task distinguishes it from check_conditions, find_citation, and assess_validity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for extracting a specific clause but provides no explicit guidance on when to use this tool versus alternatives. 'Decision-support only' is a caveat about usage context, not a comparison with sibling tools. No exclusions or alternative tool references are given, so the usage guidance is limited to what's self-evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_citationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, so the description adds value by disclosing deterministic behavior, the constraint of never inventing text, and the explicit NOT_FOUND return convention. This goes beyond the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences front-loaded with the core action and output behavior. Every clause adds value (verbatim, NOT_FOUND, never invents, deterministic, decision-support). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the essential return values and constraints. The presence of an output schema (not shown) likely documents return structure; description explains the high-level behavior. Missing edge-case details but acceptable for the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the parameter names (source_text, claim) are self-explanatory. The description refers to 'source' and 'claim' without adding format or constraints beyond what the schema provides, leaving room for more detail but not critical.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: returning exact verbatim text from a source that supports a claim, or NOT_FOUND. This is a specific verb-resource combination that distinguishes it from siblings like extract_break_clause or check_conditions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you need supporting evidence for a claim) but does not explicitly state when to prefer this over alternatives or provide exclusions. The phrase 'Decision-support only' hints at appropriate contexts but lacks explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.