Server Quality Checklist

Tool Scores

• get_requirement_detailA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates a retrieval operation (not destructive), but does not disclose rate limits, authentication needs, or potential absence of results. However, the description is clear about its read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and to the point. It front-loads the purpose. Could potentially be slightly expanded without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 parameter, no output schema, no annotations), the description covers the essential information. It explains what the tool does and the key parameter. It does not discuss return format, but that may be inferred.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter. The description adds value by giving an example format for requirement_id ('austin_mobile_food_vendor_permit'), which clarifies the expected pattern beyond the schema's generic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'detailed information about a specific permit or requirement by its ID'. It distinguishes from siblings: list_locations and lookup_requirements are broader, while this tool targets a single item by ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool should be used when you have a specific requirement ID, but does not explicitly state when to use it versus siblings or exclude cases. With only one sibling (lookup_requirements), more guidance on when to use which would be helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_requirementsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description mentions the tool returns detailed requirements including fees, agencies, etc., which is good. However, it does not state if the tool modifies data, requires authentication, or has rate limits. Since there are no annotations, a score of 3 is reasonable as the description adds practical context but lacks some behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence of moderate length. It front-loads the core purpose and immediately follows with what is returned. Every word is informative and necessary, with no filler. This is highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters (2 required, 1 optional) and no output schema, the description is fairly complete. It explains what the tool returns and the optional filter by category. However, it does not specify the return format (e.g., array of objects) or pagination behavior, which may be important. With no annotations and no output schema, a score of 3 indicates adequate but not comprehensive coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for each parameter. The description does not add much beyond the schema: it lists the enum values for activity and location implicitly but does not explain their meaning or format. The category parameter is described in the schema with its purpose. Since the schema already documents parameters well, the description adds minimal extra value, earning a baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up permits, licenses, and regulatory requirements for a business activity in a specific location. It specifies the return includes fees, agencies, processing times, prerequisites, and gotchas. The verb 'look up' and resource 'requirements' are specific and the tool is easily distinguished from siblings like 'get_requirement_detail' which retrieves details for a specific requirement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when determining what regulations apply to a business activity in a given location. It does not explicitly state when not to use it or mention alternatives, but the sibling 'get_requirement_detail' suggests this tool is for initial lookup and the other for viewing details of a specific requirement. The context is clear enough for an agent to choose appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_locationsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly states that it lists cities currently supported, implying a read-only operation. No contradictions; it accurately describes the behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that conveys the essential information without any wasted words. It is front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters, no annotations, and no output schema, the description is complete enough. It explains what the tool does and what it returns (list of supported cities). A minor gap is not specifying the output format, but it is reasonable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so there are no parameters to describe. The description adds value by explaining the purpose and return content (locations/cities), which is sufficient given no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific action 'List all locations (cities)' and the resource 'currently supported by CanIDo for regulatory lookups', making the purpose clear and distinguishable from siblings that likely deal with requirements or details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used to get supported cities before performing regulatory lookups. It provides clear context for when to use it, but does not explicitly mention when not to use it or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.