Server Quality Checklist

Tool Scores

• clv_list_supported_statesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, covering safety and idempotency. The description adds valuable context about what data is returned (portal URLs, health status, trades) and the scope ('currently supported'), which helps the agent understand the tool's behavior beyond basic annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and includes all necessary details without redundancy. Every word earns its place, making it appropriately sized for this simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema), rich annotations (readOnly, non-destructive, idempotent), and clear purpose, the description is nearly complete. It could slightly improve by mentioning the default response format or when to choose markdown vs. json, but it adequately covers the tool's function and scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'response_format' fully documented in the schema (enum: markdown/json, default: markdown). The description doesn't add any parameter-specific information beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all US states'), resource ('supported for contractor license verification'), and scope ('including portal URLs, health status, and available trades'). It distinguishes itself from sibling tools like clv_batch_verify, clv_search_by_name, and clv_verify_license by focusing on metadata listing rather than verification operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying what information is returned (portal URLs, health status, trades), suggesting it's for discovering available verification options. However, it doesn't explicitly state when to use this tool versus alternatives or mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clv_batch_verifyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, covering safety and idempotency. The description adds valuable behavioral context beyond annotations: the 25-item maximum limit and partial failure handling where individual failures don't block others. This provides important operational details not captured in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and front-loaded: two sentences that efficiently convey the core functionality (batch verification with limit) and key behavioral trait (partial failure handling). Every word earns its place with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (batch operation with partial failures), rich annotations (read-only, non-destructive, idempotent), and 100% schema coverage, the description is mostly complete. The main gap is lack of output schema, so the description doesn't explain return values or format details, but it does mention response format as a parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are fully documented in the schema. The description doesn't add any parameter-specific semantics beyond what's in the schema (e.g., it doesn't explain license number formats or trade types). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Verify multiple contractor licenses in a single request' with a specific maximum (25). It distinguishes from sibling tools by emphasizing batch verification versus single verification (clv_verify_license) or search by name (clv_search_by_name).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (batch verification of up to 25 licenses) and mentions partial failure handling, which is useful guidance. However, it doesn't explicitly state when NOT to use it or directly compare it to alternatives like clv_verify_license for single licenses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clv_search_by_nameA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, indicating a safe, non-mutating, repeatable operation. The description adds valuable context beyond this by specifying the database source ('state licensing database') and detailing the return content ('matching contractors with license numbers, status, and confidence scores'), which helps the agent understand the tool's behavior and output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose and efficiently lists the return values. There is no wasted verbiage, and every part of the sentence contributes essential information, making it highly concise and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a search operation with 5 parameters), rich annotations (covering safety and idempotency), and 100% schema coverage, the description is largely complete. It specifies the database source and return content, though without an output schema, it could benefit from more detail on result structure (e.g., pagination or error handling). However, it provides sufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description mentions 'business or individual name' and 'state licensing database', which aligns with the schema but doesn't add significant semantic detail beyond it. The baseline score of 3 is appropriate as the schema handles parameter documentation effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for contractors'), the resource ('in a state licensing database'), and the scope ('by business or individual name'). It distinguishes from siblings like clv_verify_license (which verifies specific licenses) and clv_list_supported_states (which lists states), making the purpose unambiguous and well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'in a state licensing database' and listing what it returns, but it doesn't explicitly state when to use this tool versus alternatives like clv_batch_verify or clv_verify_license. The input schema references clv_list_supported_states for valid values, providing some guidance, but the description itself lacks explicit when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clv_verify_licenseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, indicating a safe, non-mutating, repeatable operation. The description adds valuable context beyond annotations: it specifies the data source ('state licensing board portal'), mentions caching behavior ('bypass cache' via force_refresh parameter), and lists the specific return fields (e.g., disciplinary actions), enhancing transparency about behavior and output without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that front-loads the core action and resource, followed by a concise list of return values. Every word earns its place, with no redundancy or fluff, making it highly efficient and well-structured for quick comprehension by an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 parameters, no output schema) and rich annotations (read-only, non-destructive, idempotent), the description is largely complete: it covers purpose, data source, return fields, and caching behavior. However, it lacks details on error handling (e.g., invalid license numbers), rate limits, or authentication requirements, which could be useful for an agent despite the safe annotations. The absence of an output schema means the description must fully explain returns, which it does adequately but not exhaustively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter well-documented in the input schema (e.g., state codes, license number format, trade defaults, force_refresh purpose, response_format options). The description does not add significant semantic details beyond the schema, such as explaining parameter interactions or trade-specific nuances. With high schema coverage, the baseline score of 3 is appropriate, as the description relies on the schema for parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('verify a contractor's license'), resource ('contractor's license'), and scope ('by checking the state licensing board portal'), distinguishing it from sibling tools like clv_batch_verify (batch operations) and clv_search_by_name (search by name). It explicitly mentions the return data (validity, holder name, status, expiration, disciplinary actions), making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (verifying individual licenses via state portals) and references sibling tools (clv_list_supported_states for valid inputs), providing clear guidance on prerequisites. However, it does not explicitly state when to use this tool versus alternatives like clv_batch_verify (e.g., for single vs. multiple licenses) or clv_search_by_name (e.g., when license number is unknown), missing explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.