Server Quality Checklist

Tool Scores

• call_endpointB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden. It mentions 'Validate params' but does not disclose error behavior, output format, side effects, permissions, or rate limits. The idempotency_key parameter hints at idempotency but is not explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, first states purpose, second adds a key usage instruction. No fluff, efficient, and front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 4 parameters, and no output schema, the description is lacking. It omits error handling, return structure, whether the tool is destructive, and authentication requirements. The toolexecutes an external API call and should disclose more behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25% (idempotency_key has a description). The description adds value for the 'params' parameter by specifying naming convention from get_endpoint_schema, but does not elaborate on endpoint_id, max_items, or idempotency_key beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Validate params and call a JustSerpAPI endpoint by endpoint_id', specifying the verb (validate and call) and the resource (JustSerpAPI endpoint). This distinguishes it from siblings like get_endpoint_schema (retrieving schema) and search_endpoints (searching).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating 'Params should use snake_case names from get_endpoint_schema', suggesting a prerequisite but does not explicitly state when to use the tool versus alternatives or provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_catalogB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose all behavioral traits. It mentions 'Admin-only' and 'Refresh', but does not explain effects on existing data, whether the operation is destructive, or the behavior of the 'force' parameter. The schema description partially covers the parameter's behavior, but the tool description lacks sufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two short sentences with no filler. It front-loads the critical admin constraint and clearly states the action. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and one parameter, the description is incomplete. It lacks details on return values, side effects, or when the refresh actually writes. Despite simplicity, more context is needed for full clarity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'force' parameter. The tool description adds no additional parameter info, meeting the baseline for this dimension.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Refresh', the resource 'endpoint catalog', and the source 'JustSerpAPI OpenAPI documents'. It also notes 'Admin-only' as a constraint. This distinguishes it from sibling tools like call_endpoint or search_endpoints, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to refresh the catalog and are an admin, but it does not explicitly state when to use versus alternatives. No guidance on prerequisites or when not to use is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_endpointsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only states that it returns endpoint_id candidates. It does not disclose behavioral traits such as rate limits, authentication needs, or that it is a read-only query. More detail is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the main purpose, and contains no filler. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters and no output schema, the description is minimal but somewhat adequate because the workflow (call get_endpoint_schema next) partly compensates. However, details about return format, pagination, or search behavior are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no information about parameters beyond what the schema already provides. Although schema coverage is 50%, the tool has 6 parameters and the description does not compensate with any usage hints or format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Find' and resource 'JustSerpAPI endpoint candidates', distinguishing from siblings like get_endpoint_schema (mentioned as next step) and call_endpoint. It is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (searching endpoints from natural language) and provides a workflow hint to call get_endpoint_schema next. However, it does not explicitly state when not to use it or compare alternatives like list_services.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It only states the action and output type; missing details about authentication, rate limits, caching, or whether the listing is real-time. For a simple read-only tool, this is minimal but not sufficient for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is extremely concise and to the point. Every word adds value, with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is adequate for a basic listing. However, it could be more complete by hinting at the response format or any prerequisites, which would help the agent make better use of the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the baseline is 4. The description does not need to add parameter details, and it correctly avoids introducing confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'list' and the resource 'supported JustSerpAPI services' along with what is included ('endpoint counts'). It effectively distinguishes this tool from siblings like 'call_endpoint' or 'search_endpoints' which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovering available services but lacks explicit guidance on when to use this tool versus alternatives like 'search_endpoints' or 'refresh_catalog'. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_schemaA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates a read operation ('Get') and specifies the output as 'full schema and parameter contract'. With no annotations provided, the description adequately conveys the tool's behavior without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, directly stating the tool's purpose without unnecessary words. It is appropriately concise and front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description fully covers what the tool does, the source of the parameter, and what it returns. No additional details are needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes the endpoint_id parameter as 'Endpoint id returned by search_endpoints.' The tool description echoes this information, adding no new semantic meaning beyond the schema. With 100% schema coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'full schema and parameter contract', and specifies the source of the parameter (from search_endpoints). It distinguishes itself from siblings like call_endpoint, list_services, refresh_catalog, and search_endpoints, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the endpoint_id must first be obtained via search_endpoints, providing clear context for use. However, it does not explicitly state when to use this tool over alternatives or provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.