Server Quality Checklist

Tool Scores

• yapi_get_categoriesB
  3.1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description contradicts the input schema: it states the project_id parameter can be omitted (using a default from token), but the schema marks it as required. This inconsistency undermines agent trust and reliability. No annotations are provided, so the description fails to disclose behavioral traits accurately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, brief sentence that efficiently conveys the purpose and contents. No unnecessary words; it is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema, so the description is generally sufficient. However, the contradiction between description and input schema makes it incomplete and potentially confusing for the agent. Additionally, it does not explain the return format beyond a vague 'summary'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (baseline 3), but the description adds misleading information about the parameter being optional when it is required per schema. This contradiction reduces the semantic value below baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb '获取' (get) and the resource '接口分类列表' (interface category list) for a specified project, specifying what is returned (category ID, name, and interface summary). It distinguishes from sibling tools like yapi_get_api_details (gets API details) and yapi_list_projects (lists projects).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving categories of a project, but does not explicitly state when to use this tool versus alternatives, nor does it provide exclusions or prerequisites. It lacks guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• yapi_get_api_detailsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description lists what data the tool returns (params, headers, body, response), which is useful behavioral context. However, with no annotations, it fails to disclose whether the operation is read-only, requires authentication, or has any side effects. A more explicit statement would improve clarity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that effectively conveys the tool's purpose without unnecessary words. It is front-loaded with the key action and details, though it could benefit from slight restructuring (e.g., listing items) for even better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple single-parameter tool and no output schema, the description provides adequate context by listing the types of details returned. However, it uses '等' (etc.) implying incompleteness and does not specify the response format or any limitations, leaving some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with a clear description of the api_id parameter. The tool description does not add additional meaning beyond the schema, meeting the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed API information including request parameters, headers, body structure, and response. It uses a specific verb ('获取...详细信息') and resource ('接口'), and distinguishes from sibling tools like search_apis (search) and get_categories (list categories).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The parameter description for api_id explicitly mentions it can be obtained from search_apis or get_categories, providing a clear usage context. However, the main description does not include explicit when-to-use or when-not-to-use guidance, but the parameter hint suffices.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• yapi_save_apiA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses only the create/update behavior and required parameters, but lacks details on side effects, authentication, error handling, or return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise Chinese sentence that front-loads the main purpose and required parameters. There is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters and no output schema, the description lacks guidance on return values, error handling, and conditional parameter usage (e.g., path/method required for creation). It is insufficiently complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds key value by explaining the conditional create/update logic tied to api_id, which is not evident from the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates or updates a YApi interface, with specific conditions based on api_id. It distinguishes itself from sibling tools like yapi_get_api_details or yapi_search_apis by being the only mutative tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use create vs update based on api_id, and it identifies required parameters (project_id, catid). However, it does not explicitly exclude use cases or compare with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• yapi_search_apisA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses case-insensitive substring matching on title/path and optional method filter. But lacks details like pagination, result limits, or response structure (no output schema). Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with action, minimal and efficient. Every word contributes to describing the tool's purpose and key options.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters and no output schema, the description adequately explains input behavior. Lacks output details but the return of a 'list of APIs' is mentioned. Could be more complete with pagination info, but still serves its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description adds context that search is 'within a project' (scope), but otherwise the parameter descriptions in schema already cover the search functionality. No significant additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches APIs by keyword within a project, matching title/path with optional method filter. Distinguishes from sibling tools (e.g., yapi_get_api_details for details, yapi_save_api for saving) by specifying its unique search function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage context: when needing to find APIs by search criteria. However, does not explicitly state when not to use or mention alternatives, but the context is clear enough given sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• yapi_list_projectsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It mentions the token-based authentication and that no parameters are required, indicating a read-only list operation. However, it does not disclose potential pagination, rate limits, or error cases, which would make it fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently communicates the action, scope, and field list. It is front-loaded with the key verb and resource, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description provides sufficient context: it lists the returned fields and the authentication context. It could be improved by mentioning the response format (e.g., array of objects), but it is largely complete for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (0 params, 100% schema coverage). The description explicitly confirms 'no parameters,' which aligns with the schema. For zero-parameter tools, the baseline is 4, and the description adds the confirmation value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists YApi project information for the current token, specifying the fields returned (ID, name, description). This action is distinct from sibling tools like yapi_get_api_details or yapi_save_api, which handle specific APIs or searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a project list is needed but lacks explicit guidance on when to prefer this tool over alternatives. No exclusion criteria or recommended contexts are provided, relying on the user to infer from tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.