Server Quality Checklist

Tool Scores

• brave-searchC
  2.1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It omits key behaviors such as whether the search is read-only, API rate limits, authentication needs, result format, or side effects. The description adds no behavioral context beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with no filler words, but it is under-specified. The 'Tools:' prefix suggests it might be a list entry, and the fragment-like structure does not provide enough detail to justify its brevity. It is not a complete standalone description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists, which reduces the need to explain return values, the description is still incomplete. The generic parameters, lack of annotations, and missing usage instructions mean the agent cannot confidently determine how to call the tool or what behavior to expect. The description fails to cover even basic invocation details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema exposes two generic parameters, 'tool' and 'arguments,' both with null defaults and no descriptions (0% coverage). The description does not explain what these parameters mean or how to populate them, leaving the agent without the necessary information to invoke the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Web search using Brave Search API,' which clearly conveys the tool's function of performing web searches via the Brave API. It distinguishes from sibling tools, none of which are search-related. However, the phrasing is a noun phrase with a 'Tools:' prefix, making it read like a list label rather than a proper imperative description, which slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no indication of when to use this tool, no mention of alternatives, no prerequisites, and no exclusions. It is purely definitional, leaving an agent without any guidance on choosing this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• filesystemC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits but only gives a high-level summary. It does not warn about destructive operations, permission requirements, scope, or the nature of the 'tool' and 'arguments' parameters, leaving significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core purpose. However, it is so brief that it sacrifices substance, bordering on under-specification rather than effective brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's dispatcher-like schema (tool + arguments) and lack of annotations or parameter explanations, the description is grossly incomplete. It leaves the agent without enough information to correctly invoke the tool or understand what arguments are valid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters ('tool' and 'arguments') with zero schema description coverage. The description provides no explanation of what these parameters mean or how they map to filesystem operations, so it does not compensate for the missing schema detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides 'Local filesystem access' with operations 'read, write, and manage files', giving a specific verb+resource combination. It is distinct from the listed sibling tools (e.g., brave-search, manage_servers), though it lacks detail on what 'manage' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description only implies usage for local filesystem operations but does not mention any exclusions, prerequisites, or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_serversB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits, but it only lists actions without describing side effects, destructive nature (e.g., 'remove'), permissions required, or impact of test/retry operations. The examples show invocation but not consequences, leaving significant transparency gaps for a management tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the purpose, and uses examples effectively to illustrate usage. The example list is longer than strictly necessary but each line earns its place by clarifying action-parameter combinations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, 6 actions) and no annotations, the description is not fully complete: it lacks per-action parameter requirements and edge cases. An output schema exists, so return values are not needed, but the action-parameter mapping remains underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description partly compensates by demonstrating usage of name, command, and description in examples. However, it does not explain parameters like args, base_url, or transport, nor how parameters vary across actions, so it adds only partial meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it 'Manage[s] ToolMux backend MCP servers' and enumerates concrete actions (list, add, remove, validate, test, retry), making the tool's purpose clear. It distinguishes from siblings that focus on tool schema/count/list or file operations, though 'manage' is somewhat broad and relies on the action list for specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through examples such as adding or removing a server with specific parameters, but there is no explicit guidance on when to use this tool versus alternatives like list_all_tools or when not to use it. No exclusions or alternative tool references are provided, so the guidance is present but implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_schemaA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It communicates a read-only 'Get' operation but does not disclose details such as error handling, case sensitivity, or whether the operation has side effects. It adds minimal behavioral context beyond the literal function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently states the action and target. It is appropriately concise for a simple tool, though it could include slightly more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and an output schema, the description covers the core purpose but lacks usage guidance. It does not mention how to get a valid tool name or when to use this tool instead of list_all_tools. The output schema likely covers return values, but the description alone is only moderately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required parameter 'name' with no description, and schema coverage is 0%. The description adds that the schema is for 'a specific tool,' which clarifies that 'name' refers to a tool name, providing a small semantic layer. However, it does not specify the expected format or how to obtain valid names, such as via list_all_tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' to indicate retrieval, and specifies the resource as 'full description and inputSchema for a specific tool.' This clearly distinguishes it from sibling tools like list_all_tools (which lists all tools) and get_tool_count (which counts tools), as it targets a single tool's schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a specific tool' implies the tool is used when you need schema details for one tool, but it does not explicitly state when to use it versus alternatives. It lacks any mention of list_all_tools as a way to discover valid tool names or any contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_countA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits itself. 'Get count' implies a read-only operation, but it does not explain whether it requires authentication, how it groups by server, or if there are any limitations. The description is adequate but minimal for a simple count tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that uses clear noun-verb structure. No wasted words or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema present, the description sufficiently conveys the purpose. It could add detail about the output format, but the output schema likely covers that. The behavior is simple and the description is complete enough for the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema fully covers this with an empty properties object. The description correctly implies no input is needed, so there is no missing parameter information to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get count' and the resource 'available tools by server', which distinguishes it from sibling tools like list_all_tools (which likely returns full details) and get_tool_schema (which returns schemas).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool instead of alternatives like list_all_tools or get_tool_schema. It is left to the agent to infer that this is for a lightweight count rather than full details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_all_toolsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden of behavioral disclosure. It clearly states the output behavior (list names and descriptions grouped by server) and that filtering is optional. No hidden side effects or additional traits need disclosure for such a simple listing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: the first states the core function, the second explains the optional filter. Every word earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-optional-parameter tool with an output schema, the description is complete. It covers what is listed, how it is grouped, and the only parameter's purpose. There is no missing information that would prevent an agent from using it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the schema provides no parameter descriptions (0% coverage), the description explains the 'server' parameter as an optional filter by server name. This compensates fully for the schema's lack of detail and adds clear meaning to the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('List all tool names and descriptions') and the grouping key ('grouped by server'). It also distinguishes itself from sibling tools like get_tool_schema (which retrieves a single tool's schema) and get_tool_count (which counts tools) by focusing on listing all tools in a grouped format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for obtaining an overview of available tools, optionally filtered by server. However, it does not explicitly compare with siblings like get_tool_schema or get_tool_count, leaving the choice of when to use this versus those tools to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimize_descriptionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the three actions (status, generate, save), the workflow, and the important consequence that a restart is required after saving. It does not detail all side effects or return behaviors, but the output schema presumably covers return values. This is more transparent than a simple 'update' tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a brief purpose statement, a categorized action list, and a numbered workflow. Every sentence provides useful information—no filler or redundancy. It is concise enough despite covering multiple actions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has moderate complexity (three actions, output schema exists). The description includes purpose, actions, workflow, and the restart requirement. It does not describe the exact return format, but that is likely in the output schema. Overall, it is sufficiently complete for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by explaining each parameter in the workflow. It explicitly shows action values ('generate', 'save', 'status'), the server parameter ('server="server_name"'), and the descriptions object format ({"tool1": "desc1", ...}). This gives complete semantics beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb + object: 'Optimize ToolMux tool descriptions'. It distinguishes itself from sibling tools like list_all_tools or get_tool_schema by focusing on improving descriptions rather than retrieving them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit numbered workflow: generate → review → save → restart. It explains when to use the tool by framing it as producing higher-quality descriptions than algorithmic versions. It does not explicitly mention when not to use it or alternatives, but the workflow gives clear context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.