Server Quality Checklist

Tool Scores

• broker_describe_toolA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It mentions the tool returns 'exact description, input schema, upstream owner, transport, and mutating metadata,' which is transparent about the output. However, it does not discuss error handling, permissions, or rate limits, leaving some gaps for a bare description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the action, the second provides workflow context. Every word is purposeful with no redundancy, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema) and the presence of sibling tools, the description is fairly complete. It explains the workflow position and return contents, though it could mention error cases or that the name must come from search results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter schema coverage is 100% with a thorough description and examples. The tool description adds 'by broker-qualified name' but largely restates schema info, so it provides minimal additional value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Describe one profile-visible upstream MCP tool by broker-qualified name.' It also distinguishes from siblings by specifying its place in the workflow, making the purpose highly specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use this tool: 'Use this after broker_search_tools and before broker_call_tool.' It also explains the benefit—avoiding guesswork—which helps the agent decide correctly among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• broker_call_toolA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses the broker's mutating-tool policy is gated by the profile's allowlist, and hints at server-side projection to cut tokens and latency. Missing details on error handling or response format, but acceptable given no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states purpose, the second provides a critical usage guideline. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers invocation, policy, and prerequisite steps. Lacks explicit statement about return format (dynamic), but the projection parameter description partially compensates. Enough for a proxy tool with variable output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context that names must come from broker_search_tools or broker_describe_tool, but does not significantly enhance understanding beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool invokes an upstream MCP tool via the broker using a qualified name and arguments. It distinguishes from sibling tools (broker_describe_tool, broker_search_tools, broker_status) by focusing on execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises calling broker_describe_tool first and passing only accepted arguments, providing clear when-to-use guidance and key steps for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• broker_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses return fields and the key behavioral detail of not starting hidden/denied upstreams. Since no annotations, the description covers the needed transparency for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no unnecessary words. Efficient and complete.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates by listing return fields and behavior. Could add a note about interpretation, but overall complete for a simple status tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters to document. Schema already covers with description 'No user arguments accepted'. Description adds no param semantics, but that's appropriate given zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource. Explicitly states what it does and what it does not do (no starting hidden/denied upstreams). Distinguishes from sibling tools by specifying health-check usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit use cases provided (health checks, support tickets, client validation). Does not list when not to use, but context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• broker_search_toolsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it does not advertise every tool to the client, and the input schema's description adds details about profile visibility, mutating-upstream policy, and return behavior including skipped upstreams on failure, fully compensating for missing annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main description is two focused sentences, and the schema description is also concise. Every sentence adds value, no fluff, and the purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully covers the tool's behavior: searching, policy application, return of metadata, and links to sibling tools for next steps. It is complete for a search tool with simple parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with good parameter descriptions (limit's usage hint to keep small, query's examples). The main description adds no extra parameter context but the schema descriptions are thorough and provide guidance beyond basic definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the profile-visible upstream MCP tool catalog, distinguishing it from siblings by specifying it is used first to get candidate tool names. The verb 'Search' and resource are specific and exact.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this first when the task mentions a capability, service, keyword, or upstream prefix' and mentions alternatives like broker_describe_tool for exact schema details, providing clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.