Server Quality Checklist

Tool Scores

• web_fetchA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals that the tool does NOT summarize or modify content, returns original text with specific elements, and explains that different engines (llm vs tavily/firecrawl) have different capabilities and dependencies (e.g., anti-bot handling, API keys). This provides solid insight into behavior beyond the basic fetch action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately structured with sections for Returns and Notes, and each sentence provides useful information. It is slightly longer than many tool descriptions, but given the need to explain engine options and output format, it is not bloated. The main action is stated upfront.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The absence of an output schema means the description must explain return values, and it does so thoroughly with a detailed Returns list (metadata, table of contents, content structure). It also covers engine behaviors and limitations. For a tool with two parameters and moderate complexity, all essential context is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaningful semantics: URL must be valid and accessible, and the fetch_engine parameter is explained in depth (default via FETCH_ENGINE env, llm model-based, tavily/firecrawl dedicated crawl with API key requirements). This goes beyond the schema's brief descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetches and extracts the complete content from a specified URL and returns it as a structured Markdown document.' This uses a specific verb and resource, and the output format is explicit. However, it does not explicitly distinguish itself from sibling tools like 'web_search', so it misses the top score for sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by explaining what kind of URL is acceptable (valid HTTP/HTTPS, not behind auth) and mentions engine choices, but it does not explicitly state when to prefer this tool over alternatives like web_search. There are no direct 'when to use' or 'when not to use' instructions, so it stops at implied usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It explains the return format (JSON string with url, title, summary) and general search behavior, but lacks information about error handling, rate limits, potential side effects, or what happens with no results. The return structure is helpful, but overall transparency is moderate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a brief opening, parameter guidance, and a clear Returns section. It is slightly longer than strictly necessary but every section earns its place, providing useful examples and output details without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, four parameters, and no output schema, the description covers all aspects: query formulation, platform, min/max results, and return format. It could mention edge cases like empty results or errors, but for a search tool the provided information is largely sufficient. The absence of an explicit distinction from web_fetch is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds substantial meaning beyond the schema. It explains that query should be a clear, self-contained natural-language query with optional constraints, provides concrete examples for platform (Twitter, GitHub, Reddit), and clarifies min_results/max_results as bounds on the result count. This goes well beyond the bare schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a third-party web search and returns results as a JSON string. The verb 'performs' plus resource 'web search' is specific, and the tool name aligns. It implicitly distinguishes from sibling web_fetch by focusing on search rather than fetching a specific page.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides detailed guidance on how to formulate queries and use platform/min_results/max_results parameters, but it never explicitly says when to use this tool versus alternatives like web_fetch. Usage context is implied by the tool name and behaviors, but no clear 'use this for X, not for Y' guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• toggle_builtin_toolsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden of explaining behavior. It clearly defines the meaning of each action ('on' blocks built-in tools, 'off' allows them) and states the return format (JSON with current status and deny list). It does not disclose the scope (e.g., session vs persistent), but for a simple toggle, this is a reasonable disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences with no filler. It leads with the core purpose, then unpacks the parameter and return value in a structured manner. Each sentence serves a distinct informative role.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool, the description covers the purpose, the parameter values and their behavior, and the return structure. Although it doesn't mention the scope of the toggle, the overall picture is sufficiently complete for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes 'action' as 'Action type' with an enum, but the description enriches each enum value with semantics: 'on' blocks, 'off' allows, 'status' checks. This goes beyond the schema's minimal description, so the description adds meaningful parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool toggles Claude Code's built-in WebSearch and WebFetch tools on/off, using the specific verb 'toggle' and naming the precise resources. This distinguishes it from sibling tools like web_search/web_fetch (which are the targets) and get_config_info/switch_model (which handle other settings).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the action parameter (on/off/status) and thus implies when to use the tool, but it does not explicitly contrast it with alternatives or state when not to use it. It does not mention that to actually perform a search, one should use web_search instead. Hence, usage guidance is only implicit, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_config_infoA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavior: it automatically tests API connectivity to /models endpoint, masks the API key, includes response time, and only includes available_models on success. This covers side effects, security, and output nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening sentence, bulleted use cases, a 'Returns' section detailing fields, and 'Notes'. It is detailed but every part earns its place, and the front-loaded purpose makes it scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description thoroughly explains the return value structure and behavior. It also covers configuration details, connection testing, and security measures, making it complete for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters, the input schema is empty and the baseline is 4. The description additionally notes that no parameters are required, which adds clarity. No further parameter explanations are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Returns the current OpenAI Search MCP server configuration information and tests the connection.' This distinguishes it from sibling tools like web_search or switch_model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases ('Verifying that environment variables are correctly configured', 'Testing API connectivity', 'Debugging configuration issues'), which gives clear context. However, it does not explicitly mention alternatives or when not to use the tool, though sibling tools are clearly different in purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• switch_modelA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even without annotations, the description discloses key behavioral traits: persistence to a specific config file path, that the setting applies to all future operations, and a detailed return JSON with status, previous/current model, and config file location. This fully compensates for missing annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Purpose, Parameters, Returns, Notes). Each section adds useful information without redundancy, and the main purpose is stated upfront.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter configuration tool, the description covers all relevant context: side effects (persistence), config file location, return format, and a pointer to a related tool for verification. It is fully sufficient for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a description ('Model ID') with 100% coverage, and the description adds examples and clarifies the parameter is the model to switch to. This goes slightly beyond the schema's baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: switching and persisting the default AI model for search and fetch operations. It uses a specific verb ('switches') and distinguishes itself from sibling tools like web_search and get_config_info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It lists concrete use cases (changing the model, testing models, persisting preferences) and references get_config_info for verifying available models. However, it does not explicitly state when not to use the tool or compare against all siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.