Server Quality Checklist

Tool Scores

• check_serverA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. 'Look up' implies a read-only operation, and the mention of 'full trust grade, dimensions, and findings' indicates the kind of data returned. However, it does not disclose error behavior, authentication requirements, or any limitations, which would be valuable for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the essential information without wasted words. Every word earns its place, and the structure is immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool with an output schema, the description is adequate. It specifies the input (slug) and the nature of the output (trust grade, dimensions, findings). The output schema presumably details return fields, so the description needn't repeat them. Minor gaps like prerequisites (e.g., slug must exist in list_servers) are implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by stating 'by slug', clarifying that the 'slug' parameter is the identifier for selecting a single server. This adds meaning beyond the bare schema property name, though it doesn't specify slug format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb 'look up' and identifies the exact resource: 'full trust grade, dimensions, and findings for one server by slug'. This clearly distinguishes it from list_servers, which presumably lists all servers, though it does not explicitly name siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you need detailed information for a specific server identified by slug, but it does not explicitly state when not to use it or mention alternatives like list_servers. Context signals provide sibling tool names but the description itself lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_methodologyA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only states the purpose ('Explain...') without disclosing any behavioral traits such as read-only nature, authentication requirements, potential side effects, or return behavior. It carries the full burden but offers no transparency beyond the core function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that directly conveys the tool's purpose without any redundant words or extra detail. It is perfectly sized for a tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters) and the presence of an output schema, the description is sufficient. It states what the tool explains, and since an output schema exists, the description does not need to describe return values. The description fully covers the necessary context for an agent to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema covers 100% by default. The baseline is 4 for zero-parameter tools, and the description adds no parameter-related information because none is needed. No additional meaning is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Explain how trust grades are computed and the honesty model behind them.' This distinguishes it from sibling tools list_servers and check_server, which focus on server information rather than methodology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a user needs to understand trust grade computation or the honesty model, but it does not explicitly mention when not to use it or reference alternative tools. Sibling tools have different purposes, so context is clear but no explicit exclusions or alternatives are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_serversA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return format ('Returns JSON') and the key fields (scanned_at, scan_age_days, stale flag, grade), giving the agent insight into what to expect. It does not mention side effects, but the verb 'List' implies a read-only operation, and the output description adds meaningful context beyond the empty input schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the main purpose. The second sentence adds specific output details without redundancy. No word is wasted, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (no params, no annotations) and the description fully covers what the tool does and what it returns. The output schema exists, so the description doesn't need to exhaustively list return values, but it already provides key fields. For a list tool, this is complete enough for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to explain parameter usage. The schema is empty and coverage is 100%, meeting the baseline for no-parameter tools. The description appropriately focuses on the output rather than inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource: 'every graded MCP server.' It also states what the output contains (trust grade and scan freshness), distinguishing it from sibling tools like check_server (which likely checks a specific server) and get_methodology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'List every' clearly indicates this is for retrieving an overview of all servers. It doesn't explicitly name alternatives or exclusions, but the scope is unambiguous. The context of siblings (check_server, get_methodology) implies usage, though no explicit when-to-use guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.