Server Quality Checklist

Tool Scores

• add_apiC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It implies a write operation ('Add a new API configuration') but doesn't specify whether this requires authentication, what happens on duplicate names, if the configuration is persisted, or any error conditions. This leaves significant behavioral gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point without any unnecessary words. It's appropriately sized for the tool's complexity and front-loads the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after adding (e.g., success confirmation, error handling, or how to verify with 'list_saved_apis'), nor does it address authentication requirements or potential side effects, leaving the agent with incomplete operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so all parameters are documented in the input schema. The description mentions 'name, URL and optional description' which aligns with three of the four parameters but omits 'headers'. Since the schema already provides full documentation, the description adds minimal value beyond confirming some parameter purposes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add a new API configuration') and the resources involved ('with name, URL and optional description'), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'remove_api' or explain how it relates to 'list_saved_apis' for context about existing configurations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'remove_api' or 'list_saved_apis', nor does it mention prerequisites such as checking for existing APIs before adding. It simply states what the tool does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_api_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but provides no information about authentication requirements, rate limits, error conditions, response format, or whether this is a read-only operation. The description is too minimal for a tool that presumably interacts with external APIs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise at just 6 words, front-loading the essential information with zero wasted words. It follows the principle that every sentence earns its place, though this brevity comes at the cost of completeness in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the context of API information retrieval with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'general information' includes, how the information is structured, whether authentication is required, or what happens with invalid API identifiers. For a tool that likely returns structured data about APIs, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, with the single parameter 'api' documented as 'API name or direct URL'. The description adds no additional parameter context beyond what's in the schema, so it meets the baseline for high schema coverage but doesn't enhance understanding of what constitutes valid API identifiers or how they're resolved.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('general information about an API'), making the purpose immediately understandable. However, it doesn't distinguish this tool from similar siblings like 'get_endpoint_details' or 'get_model_schema' which also retrieve API-related information, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_endpoint_details' (for specific endpoints), 'get_model_schema' (for data models), and 'list_saved_apis' (for listing APIs), there's no indication whether this tool returns metadata, configuration, or summary data about an API.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'Get' operation, implying it's likely read-only, but doesn't confirm safety aspects like whether it requires authentication, has rate limits, or what format the detailed information returns. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that retrieves 'detailed information'. It doesn't specify what details are included, the response format, or error conditions. With 4 parameters and complex sibling tools, more context is needed for the agent to use this effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds no additional meaning about parameters beyond what's in the schema (e.g., it doesn't explain relationships between 'api', 'path', and 'method', or provide examples). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed information about a specific endpoint'), making the purpose unambiguous. However, it doesn't distinguish this tool from sibling tools like 'get_api_info' or 'search_endpoints' that might also retrieve endpoint-related information, so it doesn't reach the highest clarity level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_api_info', 'list_endpoints', or 'search_endpoints'. It mentions no prerequisites, exclusions, or specific contexts, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_model_schemaC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Get' implies a read operation, the description doesn't specify whether this requires authentication, has rate limits, returns structured data, or has any side effects. For a tool with zero annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose with zero wasted words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'detailed schema' means in terms of return format, whether it includes field types, validation rules, or example values. For a schema retrieval tool with no structured output documentation, the description should provide more context about what information is returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with both parameters clearly documented in the schema itself. The description adds no additional parameter information beyond what's already in the schema, so it meets the baseline score for high schema coverage without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detailed schema') and resource ('for a specific model'), providing a specific verb+resource combination. However, it doesn't explicitly distinguish this tool from its siblings like 'get_api_info' or 'get_endpoint_details' which might also provide schema-related information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'list_models', 'get_api_info', and 'get_endpoint_details', there's no indication of when this specific schema retrieval is appropriate versus other information-gathering tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_endpointsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states 'List all endpoints' but doesn't mention pagination behavior (implied by parameters), rate limits, authentication requirements, or what 'all' means in practice (e.g., completeness guarantees). For a tool with 8 parameters and no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a listing tool and front-loads the core functionality. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain the return format, pagination behavior, or how filtering parameters interact. The agent must rely entirely on the input schema for operational details, which is insufficient for contextual understanding of this complex listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 8 parameters. The description adds no parameter-specific information beyond the generic 'List all endpoints in an API' statement. According to guidelines, baseline is 3 when schema does the heavy lifting, even without additional param details in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('endpoints in an API'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'search_endpoints' or 'get_endpoint_details', which would require more specificity about scope or filtering capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'search_endpoints' and 'get_endpoint_details' available, there's no indication whether this is for comprehensive listing, filtered queries, or detailed endpoint information. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('List all data models') but doesn't disclose critical behavioral traits such as whether this is a read-only operation, potential rate limits, authentication needs, pagination behavior (implied by parameters but not described), or what the output looks like. For a tool with 10 parameters and no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, with zero waste. Every word earns its place by conveying the core action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no output schema, no annotations), the description is incomplete. It doesn't address behavioral aspects like pagination, filtering logic, or output format, which are crucial for a listing tool with many options. The schema covers parameters well, but the description fails to provide context on how to use the tool effectively or what to expect in return.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with each parameter well-documented (e.g., 'API name or direct URL', 'Page number (1-based)'). The description doesn't add any meaning beyond what the schema provides—it doesn't explain parameter interactions, default behaviors, or usage examples. With high schema coverage, the baseline is 3, as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all data models in an API'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_model_schema' or 'list_endpoints', which might list related resources. The purpose is clear but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_model_schema' (for specific model details) or 'list_endpoints' (for listing endpoints instead of models), nor does it specify prerequisites or exclusions. Usage is implied by the name but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_apiC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool performs a removal operation, implying it's destructive, but doesn't clarify if the action is reversible, what permissions are required, or what happens on success/failure. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a destructive mutation tool with no annotations and no output schema, the description is insufficiently complete. It doesn't address behavioral aspects like safety, permissions, or response format, which are critical for an agent to use the tool correctly in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'name' parameter fully documented in the schema itself. The description doesn't add any additional meaning or context about the parameter beyond what's in the schema, so it meets the baseline score when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Remove') and the target resource ('a saved API configuration'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'add_api' or 'list_saved_apis' beyond the obvious verb difference, so it doesn't reach the highest level of sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., that the API must exist), exclusions, or suggest related tools like 'list_saved_apis' for verification, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_endpointsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions search behavior but lacks details on pagination (implied by parameters), rate limits, authentication needs, or output format. For a search tool with 9 parameters and no annotations, this is a significant gap in behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It front-loads the core purpose ('Search endpoints') and specifies key aspects (query targets) without unnecessary elaboration. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters with full schema coverage but no annotations or output schema, the description is adequate for a search tool but lacks depth. It covers the basic action but misses behavioral context (e.g., pagination, authentication) and output details, leaving gaps for an AI agent to infer usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds minimal value by hinting at searchable fields (path, description, tags), which partially maps to the 'query' parameter but doesn't elaborate on others. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search') and target resource ('endpoints'), specifying searchable fields (path, description, tags). It distinguishes from siblings like 'list_endpoints' by emphasizing search functionality. However, it doesn't explicitly contrast with 'list_endpoints' which might also list endpoints without querying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'list_endpoints' or 'get_endpoint_details'. The description implies usage for searching, but lacks explicit context, prerequisites, or exclusions. Sibling tools are not referenced to clarify differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_apisB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states a read operation ('List'), implying it's likely safe and non-destructive, but doesn't specify aspects like pagination, rate limits, or what 'saved' means in terms of permissions or data scope, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste—it directly states the tool's purpose without fluff or redundancy. It's front-loaded and appropriately sized for a simple list operation, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on behavioral traits like return format or usage context, which could be helpful for an agent despite the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any, earning a high score for not introducing unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('saved API configurations'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'list_endpoints' or 'list_models' which also list resources, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_endpoints' or 'get_api_info'. It lacks context about prerequisites, such as whether APIs must be saved first, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.