Server Quality Checklist

Tool Scores

• check_usageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool checks usage and remaining calls, implying a read-only operation, but doesn't cover critical aspects like authentication requirements (beyond the api_key parameter), rate limits, error handling, or what the output contains. This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the core purpose stated first in a single sentence. The additional 'Args' section is brief and relevant. There's no wasted text, though it could be more structured (e.g., separating usage guidelines).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter) and the presence of an output schema (which handles return values), the description is minimally complete. However, with no annotations and incomplete parameter guidance, it lacks details on authentication, rate limits, and error scenarios, which are important for usage checking tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds minimal semantics by mentioning 'api_key: Your API key', which clarifies the parameter's purpose beyond the schema's title 'Api Key'. However, it doesn't explain format, validation, or sourcing, leaving gaps. With one parameter and low coverage, this earns a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check your current usage and remaining calls.' It specifies the verb ('check') and the resource ('usage and remaining calls'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'estimate_agent_cost' or 'buy_credits', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'estimate_agent_cost' or 'buy_credits', nor does it specify prerequisites or contexts for usage. This leaves the agent without clear direction on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_skillsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the API key is optional and 'shows usage if provided,' which hints at authentication and potential usage tracking. However, it lacks details on rate limits, pagination, error handling, or what 'shows usage' entails. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the core purpose stated first. The additional note about the API key is brief and relevant. However, the structure could be improved by integrating the parameter note more seamlessly or adding a line break for clarity, but overall, it avoids unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter) and the presence of an output schema (which likely covers return values), the description is minimally adequate. It states the purpose and hints at authentication behavior. However, with no annotations and incomplete parameter semantics, it doesn't fully address potential agent needs, such as error cases or performance considerations, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantics beyond the input schema. It notes that the API key is 'optional' and 'shows usage if provided,' which provides some context not in the schema (which only lists it as a string with a default). However, with 0% schema description coverage and 1 parameter, the description doesn't fully compensate—it doesn't explain what 'usage' means or how the key affects output. The baseline is 3 due to the single parameter, but the added value is limited.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all available skill files with descriptions.' It specifies the verb ('List') and resource ('skill files'), and mentions the inclusion of descriptions. However, it doesn't explicitly differentiate this from sibling tools like 'get_skill' (which presumably retrieves a specific skill), leaving some ambiguity about when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_skill' (for retrieving a specific skill) or 'register' (which might involve skills), nor does it specify prerequisites or contexts for usage. The only contextual hint is the optional API key parameter, but this doesn't constitute usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_manifest_lintA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'get a pass/fail report with fixes' and 'No API key needed,' which adds useful context about output format and authentication. However, it lacks details on rate limits, error handling, or performance characteristics, leaving gaps for a mutation-like analysis tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first. The additional sentences ('The only MCP linter...' and 'No API key needed.') add value without redundancy. The 'Args:' section is structured but slightly informal; overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (analysis of JSON definitions), no annotations, and an output schema present, the description is reasonably complete. It covers purpose, parameter semantics, and output format ('pass/fail report with fixes'). However, it could benefit from more behavioral details like error cases or limitations, but the output schema mitigates this gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates well by explaining the single parameter: 'tools_json: Your MCP tool definitions as a JSON array or single object.' This adds clear meaning beyond the schema's basic type definition. Since there's only one parameter, the description effectively covers its semantics, though it could specify JSON structure examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Lint your MCP tool definitions for anti-patterns and missing fields.' It specifies the verb ('lint') and resource ('MCP tool definitions') with scope ('anti-patterns and missing fields'). However, it doesn't explicitly differentiate from sibling tools like 'agent_health_check' or 'check_usage', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by stating 'The only MCP linter that exists' and 'No API key needed,' which implies when to use this tool. However, it doesn't explicitly guide when to use this vs. alternatives like 'check_usage' or 'agent_health_check,' nor does it mention prerequisites or exclusions beyond the JSON input requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• registerA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds context about the free tier and its limitations, which is useful. However, it does not cover other behavioral aspects such as authentication requirements, rate limits beyond the daily retrieval count, or what happens upon registration (e.g., confirmation process, key delivery method).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Register for an API key') and adds essential context ('Free tier: 10 skill retrievals per day') without any wasted words. Every part of the sentence contributes meaning, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, but with an output schema), the description is reasonably complete. It explains the purpose and key limitation, and since an output schema exists, it need not detail return values. However, it could be more comprehensive by addressing potential prerequisites or post-registration steps, slightly reducing completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description does not add parameter details, which is appropriate here since there are no parameters to explain. This earns a baseline score of 4, as it avoids redundancy and fits the tool's simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Register for an API key') and specifies the resource ('API key'), making the purpose explicit. However, it does not differentiate this tool from potential siblings like 'buy_credits' or 'check_usage', which might involve similar account-related actions, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning the free tier limit ('10 skill retrievals per day'), suggesting this is for initial setup or free access. However, it lacks explicit guidance on when to use this tool versus alternatives like 'buy_credits' for paid plans or 'check_usage' for monitoring, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_skillA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses authentication needs ('Requires a valid API key') and return content ('curated context package', 'auditable provenance'), which adds value. However, it lacks details on rate limits, error handling, or response format, leaving behavioral gaps for a tool with authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose, followed by key details. The Args section is structured but slightly verbose; overall, most sentences earn their place, though minor trimming could improve conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (authentication required, 2 parameters), no annotations, and an output schema exists, the description is fairly complete. It covers purpose, parameters, and prerequisites, but lacks behavioral details like error cases or output specifics. The output schema mitigates some gaps, but more context would help, scoring 4.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for both parameters: skill_name is explained as 'The skill ID (use list_skills to see options)', and api_key as 'Your API key from register()'. This clarifies usage beyond the bare schema, though it doesn't detail formats or constraints, warranting a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieve' and resource 'expert skill file', and explains it returns a 'curated context package' for task improvement. It distinguishes from list_skills by specifying retrieval vs. listing. However, it doesn't explicitly differentiate from other siblings like check_usage or buy_credits, keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'use list_skills to see options' for skill_name, and 'Requires a valid API key' with reference to register() for api_key. It implicitly suggests using list_skills first, but doesn't explicitly state when not to use this tool or compare to alternatives like agent_health_check, so it scores 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_health_checkA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's function (scoring configuration, providing diagnostic reports) and mentions 'No API key needed' which is useful context about authentication requirements. However, it doesn't disclose other behavioral traits like rate limits, whether the operation is read-only, or what happens with the submitted system prompt.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise. The first sentence states the core purpose, the second explains what you get, and the third provides important context about API keys. The Args section clearly documents the single parameter. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there's an output schema (which presumably describes the diagnostic report format), the description is reasonably complete. It explains what the tool does, what input it needs, and what output to expect. For a single-parameter diagnostic tool with output schema, this provides adequate context, though it could benefit from more behavioral details given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only 1 parameter, the description provides excellent parameter semantics. It clearly explains what the 'system_prompt' parameter should contain ('Your agent's system prompt or configuration text'), adding meaningful context beyond what the bare schema provides with just a title and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Score', 'get a detailed diagnostic report') and resource ('agent's configuration on governance and best practices'). It distinguishes itself from sibling tools like check_usage or estimate_agent_cost by focusing on configuration evaluation rather than usage monitoring or cost estimation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you want to evaluate your agent's configuration for governance and best practices. It mentions 'No API key needed' as a prerequisite, but doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_agent_costA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: returns a comparison table with costs per call/run/day, includes optimization tips and pricing guidance, and specifies that no API key is needed. This covers output format and authentication requirements well, though it could mention rate limits or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: first sentence states the purpose, next describes the output, then notes key features, and finally details parameters. Every sentence adds value with zero waste, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (cost estimation across models) and the presence of an output schema, the description is complete. It covers purpose, output format, key features (no API key needed), and all parameters. With an output schema handling return values, no additional explanation of outputs is needed in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds significant value by explaining all 5 parameters in the Args section, providing examples (e.g., 'claude-sonnet-4') and clarifying their purposes (e.g., 'Estimated input tokens per call'). This goes well beyond the bare schema, though it doesn't detail constraints like token ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Estimate the cost of running an agent task across all major models.' It specifies the verb 'estimate' and resource 'cost,' and distinguishes itself from siblings like check_usage or buy_credits by focusing on cost estimation rather than usage monitoring or purchasing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for estimating agent task costs across models. It mentions 'No API key needed,' which is helpful guidance. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings like check_usage for actual usage data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buy_creditsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively explains key behaviors: that it returns a Stripe checkout URL, requires human intervention for completion, and that credits are added automatically after payment. It doesn't mention rate limits, authentication details beyond the api_key parameter, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise. It begins with the core purpose, explains the return value and usage flow, then provides parameter semantics in a clean 'Args:' section. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, 0% schema coverage, but the presence of an output schema, the description is remarkably complete. It covers purpose, behavior, parameters, and workflow. The output schema presumably handles return value documentation, so the description appropriately focuses on the purchase process and parameter meanings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic meaning for both parameters. It explains that 'api_key' is the user's API key and that 'tier' represents credit tiers with specific values ('50' or '250') and their corresponding prices. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get a checkout link to purchase more credits') and identifies the resource (credits). It distinguishes itself from sibling tools like 'check_usage' or 'estimate_agent_cost' by focusing on purchasing rather than monitoring or estimating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('to purchase more credits') and specifies that the resulting URL should be given to a human operator. However, it doesn't explicitly state when NOT to use it or mention alternatives like whether credits can be obtained through other means.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.