Server Quality Checklist

Tool Scores

• search_businessesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states that the tool searches filings, but does not mention any side effects, required permissions, rate limits, or output characteristics (despite an output schema existing). The lack of read-only indication or result format is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the key purpose. It contains no redundant words. However, it could be more structured by explaining parameters explicitly, but given its brevity, it earns a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters and no annotations, the description is incomplete. It fails to explain the state parameter's options, the default limit, and the structure of output (even though an output schema exists). Essential details for correct tool usage are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema description coverage is 0%, the description must compensate. It adds partial meaning for the 'query' parameter (search by name) and implies the 'state' parameter through 'across NY, FL, and CO', but does not explain that state can be null for all states or specify valid values. The 'limit' parameter is completely undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching for businesses by name within filings from three specific states (NY, FL, CO). While sibling tools have distinct names (check_usage, get_business_detail, get_new_formations), the description does not explicitly differentiate from them, but the purpose is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lacks guidance on when to use this tool versus alternatives. It does not mention any conditions, prerequisites, or exclusions. For example, it does not suggest using get_business_detail after finding a business, nor does it explain when to prefer the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_business_detailB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It only states output contents but does not disclose any side effects, permissions, rate limits, caching, or whether it is read-only. Minimal disclosure beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single, front-loaded sentence with no filler. Every word adds value, stating purpose and key output components.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has output schema and no annotations. Description covers output partially but omits input parameters entirely. For a 2-param tool with 0% schema coverage, this is incomplete; the agent would need to guess parameter meaning.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds no meaning for the two parameters (state, entity_id). It does not explain required nature, format, or examples. Completely fails to compensate for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full details for a specific business, listing example details (officers, addresses, registered agent). It distinguishes itself from siblings like search_businesses (which lists businesses) and get_new_formations (new formations).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage: use after obtaining an entity_id and state. No explicit guidance on when to prefer this over check_usage or search_businesses, nor any exclusions or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_new_formationsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description lacks disclosure of behavior like pagination, error handling, or default limit effects. Only mentions date format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and usage context. No redundant or unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite an output schema, description is too brief for a 3-parameter tool. Doesn't explain what the returned data contains or how parameters interact.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning for 'since' parameter (YYYY-MM-DD format) but does not explain 'state' (unclear filter) or 'limit' (default 100). Incomplete coverage for 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Name and description align: 'Get businesses formed since a given date' clearly indicates retrieving new formations. Useful for sales prospecting. Distinct from siblings like search_businesses or get_business_detail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Only mentions 'useful for sales prospecting,' implying context but no explicit when-to-use or when-not-to-use compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_usageA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It implies a read-only operation without side effects, but does not explicitly state this or mention any rate limits or authentication requirements. Adequate for a simple query tool but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the core action and resource. Every word is useful, and there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description is complete enough to understand the purpose. It could briefly note that it is a read-only operation, but overall it provides sufficient context for a simple usage check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100%. The description adds no parameter information, which is acceptable since there are none. Baseline for zero parameters is 4, and the description meets this without needing to elaborate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('check') and the resource ('FreshFilings API key usage and remaining calls'). It distinguishes from siblings (e.g., get_business_detail, get_new_formations) by focusing on usage tracking rather than data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to check API usage and remaining calls. It does not explicitly exclude use cases or mention alternatives, but the purpose is straightforward enough that no further guidance is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.