Server Quality Checklist

Tool Scores

• search_docsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=true, covering safety and idempotency. The description adds useful context about what gets returned (doc page links with snippets and matching example projects), which isn't covered by annotations. However, it doesn't mention rate limits, authentication needs, or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence establishes purpose and scope, while the second describes the return format. No wasted words, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, 3 with enums) and rich annotations, the description is mostly complete. It covers purpose, usage context, and return format. However, without an output schema, it could benefit from more detail about the structure of returned results (e.g., pagination, error handling).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters well-documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the schema, so it meets the baseline of 3. The description does imply the tool supports filtering (via platform, chain, category), but this is already explicit in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches MetaMask Embedded Wallets documentation and examples, specifying the exact resource (documentation and examples) and purpose (SDK discovery, feature lookup, finding examples). It distinguishes from siblings like get_doc (single document), get_example (single example), get_sdk_reference (reference material), and search_community (community content).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (SDK discovery, feature lookup, finding examples), but doesn't explicitly state when NOT to use it or name specific alternatives. The sibling tools suggest different use cases, but the description doesn't contrast them directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_docA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond what annotations provide. While annotations already indicate read-only, idempotent, and open-world characteristics, the description reveals the multi-source fallback strategy ('Tries Algolia, then llms.txt, then GitHub raw MDX as fallbacks'), which helps the agent understand reliability and performance implications. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three tightly focused sentences. The first states the core purpose, the second provides usage guidance, and the third reveals implementation behavior. Every sentence earns its place with no wasted words, and the information is front-loaded effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter, read-only operation with fallback behavior) and comprehensive annotations, the description provides good contextual coverage. It explains the purpose, usage sequencing, and implementation strategy. The main gap is the lack of output information (no output schema), but the description compensates somewhat by indicating it fetches 'full content.'

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already fully documents the single 'url' parameter. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline expectation but doesn't provide additional semantic context about the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Fetch the full content') and resource ('MetaMask Embedded Wallets documentation page'), distinguishing it from sibling tools like search_docs (which finds pages) and get_sdk_reference/get_example (which target different content types). It precisely defines what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Use after search_docs to read the actual doc'), providing clear sequencing guidance. It also distinguishes it from search_docs by indicating this is for reading full content after searching, establishing a clear workflow relationship.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exampleA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=true, covering safety and idempotency. The description adds valuable context beyond this: it specifies that the tool fetches from GitHub and returns all source files, and emphasizes the importance of examples as primary references. There is no contradiction with annotations, and the description enhances understanding of the tool's behavior without repeating structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence clearly states the tool's purpose, followed by a sentence on return value, and a final sentence providing usage guidance. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema) and rich annotations (readOnlyHint, idempotentHint, openWorldHint), the description is largely complete. It covers purpose, usage guidelines, and behavioral context. However, it doesn't detail the return format (e.g., file structure or pagination), which could be useful since there's no output schema, leaving a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 5 parameters with descriptions and enums. The description does not add any parameter-specific information beyond what the schema provides, such as syntax or format details. However, it implies filtering capabilities through 'Fetch the complete source code of a Web3Auth integration example,' which aligns with the schema but doesn't offer additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('fetch the complete source code'), resource ('Web3Auth integration example from GitHub'), and scope ('all source files needed to understand how the integration works'). It explicitly distinguishes this tool from siblings by stating 'Examples are the PRIMARY reference for integration patterns — always prefer example code over raw SDK source,' which differentiates it from get_sdk_reference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool versus alternatives: 'always prefer example code over raw SDK source' directly contrasts with get_sdk_reference. It also implies context by positioning examples as the primary reference for integration patterns, though it doesn't explicitly mention other siblings like get_doc or search_community.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sdk_referenceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and openWorldHint=true, indicating safe, repeatable operations. The description adds valuable context beyond this by specifying the tool's purpose for reference/debugging only, warning about internal/legacy SDK options, and recommending get_example for integration patterns. It doesn't contradict annotations, and the added context helps the agent understand behavioral nuances not covered by structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and usage guidelines in the first sentence, followed by specific instructions and warnings. Every sentence adds value—clarifying the tool's scope, restrictions, and alternatives—with no redundant or unnecessary information. It efficiently conveys critical information in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (fetching SDK source code with multiple parameters) and rich annotations (readOnlyHint, idempotentHint, openWorldHint), the description is largely complete. It covers purpose, usage guidelines, and behavioral context effectively. However, without an output schema, it doesn't describe return values or format, which is a minor gap. The annotations and schema compensate well, but the description could slightly enhance completeness by hinting at output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context by mentioning 'type definitions, interfaces, hooks, and error types,' which aligns with the 'focus' parameter options, but doesn't provide additional syntax or format details beyond what the schema specifies. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Fetch SDK source code') and resource ('from the open-source Web3Auth SDK repos'), distinguishing it from sibling tools like get_doc or get_example. It explicitly mentions fetching type definitions, interfaces, hooks, and error types, providing a precise scope that differentiates it from other tools that might return documentation or examples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('for REFERENCE and DEBUGGING only') and when not to use it ('Do NOT use this to discover features'). It also names an alternative tool ('Always use get_example first for integration patterns'), clearly differentiating usage contexts from siblings like get_example, get_doc, search_community, and search_docs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_communityA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate this is a read-only, idempotent, and open-world operation, which the description doesn't contradict. The description adds valuable context beyond annotations by specifying the forum source (builder.metamask.io) and the two distinct behavioral modes (search vs. fetch), though it doesn't mention rate limits or authentication needs. With annotations covering core safety traits, this additional context earns a strong score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by usage context and parameter guidance in just two sentences. Every sentence earns its place by providing essential information without redundancy, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters, 100% schema coverage, and annotations covering key behavioral traits, the description provides sufficient context by clarifying the forum source and use cases. The lack of an output schema is a minor gap, but the description compensates by explaining what the tool returns (search results or full discussions). It's nearly complete for this complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters (query and topic_id). The description adds minimal semantic value beyond the schema by mentioning the two modes ('search or fetch'), but doesn't provide additional details like query syntax examples beyond what's in the schema. Given the high schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search or fetch posts') and target resource ('MetaMask Embedded Wallets community forum'), distinguishing it from sibling tools like search_docs by specifying the forum context. It provides concrete examples of use cases (troubleshooting, finding workarounds, checking known issues), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('for troubleshooting real user issues, finding workarounds, and checking if an issue is known') and provides clear alternatives for the two parameter modes ('Provide a query to search or a topic_id to read the full discussion'). This gives the agent precise guidance on selecting between query-based search and topic fetching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.