Server Quality Checklist

Tool Scores

• get_filing_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the behavioral burden. It usefully discloses that only coarse progress is returned and that no personal information is included, but it does not mention authentication needs, invalid document handling, or whether this operation is purely read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence that front-loads the main purpose, lists the possible status values, and adds a useful privacy guarantee. Every part serves a purpose with no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple shape (one required string parameter, no output schema), the description is nearly sufficient: it communicates the coarseness and enumerates possible values. It only lacks a note about missing or invalid documents, and the relationship to 'start_filing' is implicit rather than explicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain 'docId' at all. The name 'docId' is suggestive but not amplified by the tool description; the parameter semantics are left mostly to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('get') and resource ('filing status'), names the coarse statuses ('awaiting_signature, awaiting_payment, filed, delivered'), and clearly differentiates itself from the sibling tools 'check_property_tax_savings' and 'start_filing'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for retrieving a filing's status after starting a filing, but it does not explicitly say when to use it versus alternatives or any workflow order. It provides clear context but not explicit when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_filingA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the signing link is never returned to the caller and that a flat $29 fee applies, which are important side-effects. It does not mention potential errors or auth, but for a simple mutation it is adequately transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences that front-load the action and prerequisites, then key output behavior, then cost. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity, no output schema, and no annotations, the description covers the essential points: action, prerequisites, outcome (link emailed, not returned), and cost. It does not mention error handling or what the caller receives as a response, but for a simple tool this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains that continueToken comes from a qualifying check, adding meaning beyond the schema. However, ownerName, ownerPhone, and ownerEmail are only listed without formats or constraints, providing minimal additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (start a filing) and the target resource (a qualifying property). It distinguishes from siblings by focusing on the initiation step, whereas check_property_tax_savings evaluates eligibility and get_filing_status retrieves status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the prerequisite (continueToken from a qualifying check) and describes the workflow (link emailed to owner, payment). It implies the use case (after a qualifying check) but does not explicitly mention alternatives or when not to use it, though the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_property_tax_savingsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool is free, returns qualification status, estimated savings, assessment, county, deadline, and a continueToken. It does not explicitly state whether it makes any modifications, but as a 'check' tool, it is reasonably transparent about its read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the primary purpose and lists all key outputs concisely. Every phrase adds value, with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given only two simple parameters and no output schema, the description is complete. It enumerates all return values and mentions the continueToken, which links to sibling tools for filing, providing enough context for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both unit and address are described). The description adds minimal additional meaning beyond the schema, only implicitly requiring the address to include city. Baseline 3 is appropriate when the schema fully documents parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks California home qualification for Proposition 8 decline-in-value property tax reduction, using a specific verb and resource. It distinguishes from siblings (start_filing, get_filing_status) by focusing on the pre-filing check and mentioning a continueToken for qualified properties.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by noting it returns a continueToken when the property qualifies, suggesting it is a prerequisite for filing. However, it does not explicitly state when to use it versus alternatives or include when-not guidance, so a slight deduction is warranted.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.