Server Quality Checklist

Tool Scores

• recommend_improvementsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits, but it only adds that the tool is 'based on real failure data' (a data source) without mentioning read-only behavior, side effects, auth requirements, or return format. This is a minimal disclosure beyond the bare purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) and front-loaded, but it is under-specified. It prioritizes conciseness over necessary detail, leaving out essential usage and parameter context, so it is not appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is incomplete. It does not explain what the recommendations look like, how 'hours' affects results, or how this tool fits into the broader workflow alongside siblings. The agent would need more information to use it confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description provides no meaning for the 'agent_id' and 'hours' parameters. The tool's behavior cannot be correctly invoked without understanding what these fields do, and the description does not compensate at all.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Give') and a concrete resource ('actionable recommendations to improve reliability'), clearly distinguishing it from siblings like analyze_failures (analysis) and get_reliability_score (scoring).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like analyze_failures or generate_audit_report. The phrase 'Based on real failure data' implies a prerequisite but does not state it clearly or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_audit_reportC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full burden of behavioral disclosure. It only states a high-level purpose and does not mention what data the report covers, whether it is read-only, how it handles the hours parameter, or any side effects. This is a significant gap for a tool that could potentially access a large dataset.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose without wasted words. It is exceedingly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and two parameters that are not explained, the description is severely incomplete. An agent cannot determine what 'full audit-style report' entails, what hours means, why agent_id is required, or what the output format will be. This is inadequate for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention either parameter (agent_id, hours). The agent must infer that the report is tied to an agent and a time window, but this is not stated. The description provides no added meaning over the bare schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('Generate') and specific resource ('full audit-style report') with stated audiences (investors, compliance, internal reviews). It does not explicitly differentiate from sibling tools like analyze_failures or get_reliability_score, but the report scope is distinct enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives context on when the report is useful (investors, compliance, internal reviews) but provides no explicit guidance on when to choose this tool over siblings or any exclusions. Usage context is implied rather than directly contrasted with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• end_sessionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It states the action and output but fails to mention important side effects, such as whether the session becomes invalid, whether data is saved, or what the 'quick summary' contains. This is a significant transparency gap for an action that terminates something.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, succinct sentence that front-loads the action and outcome. There is no filler or redundant information, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description should cover return values, error scenarios, and behavioral context. It only mentions a 'quick summary' without specifying its contents, and does not address invalid session IDs or whether the action is reversible. For a tool with one parameter and no other metadata, this is inadequate for full understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only lists session_id as a required string, and the description provides no additional context about this parameter. Since the schema description coverage is 0%, the description should have elaborated on what session_id refers to or how to obtain it, but it does not. The parameter is left entirely to interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('End') and the specific resource ('a tracking session'), and also indicates the output ('get a quick summary'). This distinguishes it from sibling tools like start_session and record_interaction, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this when you want to end a tracking session. However, it does not provide explicit guidance on when to choose this over alternatives, nor does it mention any conditions or prerequisites (e.g., an active session). The intent is clear but not fully elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_interactionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only says 'log', implying a write, but does not specify persistence, failure modes, error handling, or required permissions. The behavior is under-disclosed for a mutation-like tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the primary action and usage instruction. No wasted words or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, no annotations, and no output schema, the description is incomplete. It covers the purpose and timing but omits parameter meanings, return values, and behavioral consequences, leaving significant gaps for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description provides no parameter guidance. For 7 parameters, the agent must rely solely on parameter names, which is insufficient for fields like metadata, latency_ms, and error_message.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (log), the object (one tool-call result), and distinguishes this tool from sibling reliability/audit tools. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to call this tool after every important tool use, providing clear context. It does not mention alternatives or exclusions, but the instruction is direct and sufficient for typical usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_failuresA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose that the tool returns failure counts and sample error messages, implying a read-only operation, but it does not detail any permissions, rate limits, or potential side effects. The behavioral description is minimal but non-contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the primary action, and the second specifies the return content. Every word is purposeful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (3 params, no output schema), and the description covers the core purpose and high-level output. However, it does not explain the semantics of the optional parameters (hours, limit) or the exact structure of the returned data, leaving gaps for an agent trying to use it correctly. Given the lack of annotations and schema descriptions, this feels adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only clarifies 'agent_id' implicitly through 'for an agent', but 'hours' and 'limit' are unexplained. The parameter meanings are not conveyed beyond the default values in the schema, so the description adds minimal value here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Find' with a clear resource ('the most common failure patterns for an agent') and explicitly states the output ('tools that fail most often and sample error messages'). This clearly distinguishes it from sibling tools like get_reliability_score which suggests a numeric rating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for analyzing agent failures but provides no explicit guidance on when to prefer it over related tools such as get_reliability_score or recommend_improvements. There are no mentioned alternatives or exclusions, so the usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_mcp_healthA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It mentions 'Simple health check' and 'HTTP transport,' implying a read-only operation, but it doesn't state what happens on success/failure, timeouts, or any side effects. Key behavioral details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, both concise and purposeful. The first sentence states the action and transport, the second gives usage context. No unnecessary words or repetitions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one optional parameter, no output schema, no annotations), but the description omits return value or error behavior. It gives enough to understand the general purpose, but an agent might need more details about what the health check actually returns (e.g., status code, boolean). It is minimally sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. The description mentions 'another MCP server' but never explicitly explains that the server_url parameter is the target URL. While the parameter name is self-explanatory, no additional semantic value is added beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('check') and resource ('another MCP server'), immediately distinguishing it from sibling tools like start_session or generate_audit_report. It explicitly mentions HTTP transport, further clarifying the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'Useful for multi-agent systems that depend on other MCP servers.' This implies when to use it, though it doesn't explicitly list alternatives or exclusions. The sibling tools are sufficiently different that no confusion arises.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_sessionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It states that a session is started and returns a session_id to pass to later tools, which is useful. However, it does not disclose side effects, idempotency, or any permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each with a distinct purpose: what the tool does, when to call it, and what it returns. The description is front-loaded and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers purpose, usage timing, and return value adequately. Minor gaps like lifecycle closure or invalid agent_id behavior are not critical at this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for agent_id, but the tool description's 'for an agent' provides minimal mapping to the required parameter. It adds a little meaning beyond the schema but lacks details on agent_id format or how to obtain it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb 'Start' with a clear resource 'new tracking session for an agent'. It distinguishes itself from sibling tools like end_session by explicitly focusing on initialization, and it mentions the returned session_id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Call this at the beginning of a multi-step agent run', giving a clear when-to-use context. It does not provide explicit when-not-to-use guidance or name alternatives, but the sibling tool list implies the session lifecycle.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reliability_scoreA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It clearly defines the score's meaning and formula, and the verb 'Get' implies a non-mutating read operation. It doesn't discuss error handling or permissions, but for a simple retrieval tool, this is sufficient context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the purpose and define the metric. No wasted words; every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description adequately covers what the tool returns (0-100 score) and how it's computed. Sibling tool names provide surrounding context, and the description is complete enough for a low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining 'hours' as 'last N hours' and implicitly the meaning of the score. 'agent_id' is self-explanatory from the name. The description adds meaning beyond the schema's bare property definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb 'Get' and resource 'reliability score for an agent', with a precise definition (0-100, percentage of successful tool calls). This clearly differentiates from siblings like analyze_failures or generate_audit_report, which focus on analysis and reporting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is a read-only query tool for reliability metrics, but it doesn't explicitly state when to use it over alternatives like analyze_failures or generate_audit_report. The context of siblings suggests it's for direct score retrieval, but no explicit guidance or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.