Server Quality Checklist

Tool Scores

• get_tier_policyC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description does not contradict them. However, it adds no behavioral detail about the tool itself, such as output format, edge cases, or what 'AML' means; the safety profile is covered, but the policy content is unexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short and free of filler, but the terseness sacrifices comprehensibility. The key facts are not front-loaded in a recognizable way, and the cryptic phrasing reduces the value of the brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description should explain what the caller receives or can infer from the policy. Instead, it offers ambiguous fragments and undefined acronyms, leaving an agent to guess the response semantics and the meaning of 'AML'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so no parameter documentation is required. The description correctly avoids inventing parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description never states a verb or object; it reads as cryptic policy fragments ('Tier 0 vs 1+', 'Convert closed at 0'). The name and title imply retrieval of tier policy, but the body does not explicitly say what the tool returns or does. This borders on restating the title without adding a clear operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no when-to-use guidance and no mention of alternatives. The intended use is only implied by the tool name, and the description does not clarify when this policy lookup is appropriate versus the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_docs_urlsB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already mark the operation as read-only, idempotent, and non-destructive. The description adds a loose content scope, mentioning several categories, but does not state return format, whether the list is exhaustive, or how results are ordered. It does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very brief and has no filler, but it is a fragment, not a structured sentence. It front-loads keywords, yet the lack of a verb reduces clarity, making it less effective than a concise complete sentence would be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter read-only tool, annotations and schema cover safety and invocation constraints. However, the description leaves ambiguity about what is actually returned: documentation URLs, endpoints, or both. The listed categories are helpful but not sufficient to fully predict the tool's output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, so the schema fully specifies invocation. No further parameter documentation is needed, and the baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lacks a verb and reads as a list of content categories rather than a clear statement. It implies that the tool returns URLs for HowTo, use cases, tiers, discovery, OpenAPI, and MCP endpoints, but the purpose is only clear from the title. It does not meaningfully distinguish the tool from siblings beyond the title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings like get_public_url, get_agent_facts, or get_tier_policy. There are no exclusions, alternatives, or context cues that would help an agent choose among the listed tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_catalog_placementA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool read-only, idempotent, and non-destructive. The description adds useful domain context by clarifying the catalog placement logic (docs vs spend, OpenAI docs-only), but it does not describe the exact return shape or any behavioral caveats. With annotations carrying the safety profile, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short, front-loaded sentence with no filler. The additional fact 'OpenAI is docs-only' is compact and directly useful. Slightly telegraphic style, but every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only, annotated tool, the description communicates the core output. However, there is no output schema, and the description does not specify whether the response is a list, mapping, or config snippet, or how complete the coverage is. This is a minor gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so per the rubric the baseline is 4. The description still adds meaningful semantics about what kind of mapping the tool reports, even though there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's specific output: which directories should list the docs MCP versus the spend MCP, with the OpenAI docs-only exception. It is clearly distinct from siblings focused on facts, tier policy, URLs, and docs links. It lacks an explicit verb like 'returns' or 'lists,' but the phrasing is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: an agent would call this when it needs to decide where to place each MCP in a catalog. However, the description gives no explicit direction about when to choose this tool over alternatives or whether any context/preconditions apply.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_agent_use_casesA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds meaningful context by explicitly stating 'Does not move money' and by instructing that the content should be cited as-is. This clarifies that the tool is informational and has a specific usage expectation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and information-dense, front-loading the word 'Playbook' and immediately providing substantive examples. Each clause contributes value: scope, examples, tool association, citation rule, and safety caveat. It is slightly telegraphic, but it wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only informational tool with no output schema, the description sufficiently conveys the content and scope. The safety note and citation instruction add important behavioral context. It does not specify the exact output format, but given the simplicity of the tool and strong annotations, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty, so there are no parameter semantics to document. The description adds useful context about what kind of content will be returned, which helps compensate for the absence of any parameter-driven behavior. The 0-parameter baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The name and title clearly indicate this tool returns agent purchasing use cases, and the description lists concrete examples such as Visa prepaid cards, travel eSIM, gift cards, and KYT address screens. It distinguishes itself from sibling tools by framing the content as a buying playbook rather than generic agent facts or policy. It lacks an explicit verb like 'returns' or 'lists,' but the intent is still clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit when-to-use guidance or comparison with sibling tools like get_agent_facts, get_tier_policy, or get_catalog_placement. 'Cite as-is' is a post-retrieval instruction, not a tool-selection guideline. The intended usage is only implied by the title and playbook framing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_public_urlA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already convey readOnlyHint, idempotentHint, and non-destructive behavior. The description adds concrete detail: it performs an HTTP GET, targets only allowlisted public docs, and explicitly disavows state-changing operations. This enriches the safety profile beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with the core action and constraints front-loaded. No filler or redundancy; the negative assurance about register/deposit/buy earns its place by reinforcing the tool's non-mutating nature.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter, read-only tool with full schema coverage and safety annotations, the description is nearly complete. It lacks only explicit error or return behavior for non-allowlisted URLs, which is not critical for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description adds little beyond the schema's own explanation of the url parameter. The mention of public docs and allowlist is already in the parameter description, so no extra semantic value is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States the mechanism (HTTP GET) and resource (an allowlisted public docs URL), with concrete categories like HowTo, use cases, discovery JSON, and llms-agent. This is specific enough to separate it from the sibling getters and from mutating operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only usage guidance is implicit: the URL must be allowlisted public docs and the tool never calls register/deposit/buy. It does not explicitly say when to choose this over siblings like get_agent_use_cases or list_docs_urls, nor give positive selection criteria beyond the allowlist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_agent_factsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful behavioral context beyond this: 'Cite as-is' instructs exact quoting, and 'Does not move money' reinforces the no-side-effect nature of the tool in plain language.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and every phrase carries weight: the scope comes first, followed by a concrete topic list, then two short usage directives. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only facts tool without an output schema, the description is largely sufficient: it names the domain, lists covered topics, and gives quoting guidance. It could be slightly stronger by distinguishing itself from overlapping siblings like get_tier_policy, but nothing essential is missing for invoking it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100%, so there is nothing to document. The description instead clarifies the content scope of the returned facts, which is the only semantic information an agent needs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a canonical prepaid-agent policy and catalog reference and enumerates the exact fact domains it covers (Visa prepaid, eSIM, gift cards, KYT, USDT TRC-20, rails, tiers). This is specific and informative, though it does not explicitly differentiate from siblings such as get_tier_policy or get_catalog_placement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Canonical... policy and catalog' provides clear context that this is the authoritative reference source, and 'Cite as-is' tells the agent to quote content verbatim. It also excludes transactional use with 'Does not move money.' However, it does not explicitly name alternatives or state when a sibling would be preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.