Server Quality Checklist

Tool Scores

• call_endpointA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full disclosure burden and does well: it mentions PAID pricing, the x402 payment flow (fetches challenge, signs USDC on Base, retries), and the required WALLET_PRIVATE_KEY. It does not detail error handling or return format, but the core behavioral traits—cost, payment execution, and key requirement—are transparently stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at roughly three sentences but packs essential information upfront: cost, generic purpose, usage instruction, payment flow, and key requirement. Each clause adds value, though slight redundancy exists ('generic way' + 'ANY route' could be tighter). It is well-structured and front-loaded with critical cost/purpose info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's generic nature and no output schema, the description covers key operational concerns: cost, payment flow, key requirement, and how to obtain the correct body (via get_endpoint_spec). It does not mention failure behavior or return format explicitly, but 'returns the result' implies the route's output is returned, which is adequate for a generic caller.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description reinforces the 'body' as 'a JSON body matching its input schema' and gives path examples, but adds little beyond the schema. The 'use get_endpoint_spec' hint is useful but not a substitute for schema-level detail, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'the generic way to call ANY route in the Utility Grid' with a specific verb (call) and resource (routes). It distinguishes itself from sibling tools like get_endpoint_spec (which retrieves specs) and list_capabilities (which lists capabilities), making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it is the generic route caller, and it explicitly advises 'use get_endpoint_spec first if unsure' about the body schema, naming a sibling tool as a prerequisite. It also mentions requirements (WALLET_PRIVATE_KEY) and cost, though it does not explicitly state when not to use it (e.g., for discovery or when no payment key is available).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_capabilitiesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral disclosure. It states the tool is free, always reads the live /openapi.json (never stale), and describes the two invocation modes. It does not disclose rate limits, pagination, or the exact response structure, but for a read-only catalog that is largely sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with 'FREE — no wallet needed' and the core verb. The long parenthetical enumeration of categories is verbose but provides valuable scope information. Every sentence contributes, though the category list could be trimmed without losing essential function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description covers cost, live data behavior, both invocation modes, and category examples. It does not address error cases, response format, or limits, but the core decision-making information is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the optional 'category' parameter with examples ('math', 'vision'). The description adds that passing a category lists every route with price and description, but this is more behavioral context than parameter semantics. Since schema coverage is 100%, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (browse the ForgeMesh Utility Grid catalog) and clearly differentiates from siblings by explicitly offering two modes: a category overview with no arguments, or listing routes in a category. This distinguishes it from search_capabilities and get_endpoint_spec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: no arguments yields a category overview with route counts; passing a category yields routes with price and description. It also notes the data source is live. However, it does not explicitly state when not to use this tool or point to alternatives like search_capabilities for its exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_capabilitiesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It adds useful context: it's free (no wallet), searches across specific fields (path, operation id, description). But it doesn't explicitly state it's read-only, disclose any side effects, or describe the return format. This is acceptable for a search tool but lacks some behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the most important info (free, search scope). No wasted words, every sentence earns its place. Well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, good schema coverage, and no output schema, the description is quite complete. It covers what it searches, when to use it, and cost. Missing return format details, but that's not critical for a search tool. The sibling context is available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions, so baseline is 3. The description adds value by giving examples ('chess', 'timezone', 'background removal') and clarifying what the query searches across (path, operation id, description), which enriches the meaning of the 'query' parameter beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a keyword search across every route's path, operation id, and description, with concrete examples. It also differentiates from siblings by indicating it's for when you don't know the exact route name or category.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when you don't know the exact route name or category,' giving a clear context. It also notes 'FREE — no wallet needed,' a cost-related guideline. However, it doesn't explicitly name alternatives or state when not to use it, so not a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• daily_402A
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility. It discloses critical behavior: the operation is paid ('PAID ($0.001)'), requires authentication ('Requires WALLET_PRIVATE_KEY'), and describes the return payload. It also notes deterministic rotation, giving an insight into selection logic. Missing are potential error conditions or rate limits, but the most important safety and auth aspects are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it starts with the cost and core function, followed by return details, optional parameter, and auth requirement. Four short sentences, each adding essential information without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description is complete. It explains the return value (description, price, schema, example), the deterministic rotation, the optional replay feature, and the authentication prerequisite. No critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the single parameter at 100%, including format and purpose. The description's mention of 'Optional date override to replay a past day's pick' mirrors the schema description and adds no additional semantic detail beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'one featured x402 endpoint per UTC day' and what it returns ('Returns what it does, its price, input schema, and a worked example'). This distinguishes it from sibling tools like search_capabilities or get_endpoint_spec, which focus on broader discovery rather than a daily featured pick.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on when to use the tool: for the daily featured endpoint, with an optional date override to replay past days. However, it does not explicitly mention alternatives or when-not-to-use scenarios, though the daily-specific nature is implicitly distinct from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_service_directoryA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It discloses the $0.05 cost and the WALLET_PRIVATE_KEY requirement, which are critical for an agent to decide whether to call. It also specifies the output fields and machine-readable nature, though it omits error cases or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the cost warning, and packs essential details such as content, use case, filter, and auth requirement without any redundant words. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-optional-parameter directory lookup, the description covers purpose, when to use, cost, auth requirements, and expected output content. It lacks explicit error handling details but is otherwise complete enough for an agent to decide and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the category parameter with its allowed values (100% coverage), but the description adds behavioral meaning by labeling it an 'Optional category filter' and repeating the categories in a practical context. This goes slightly beyond the schema's bare value list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a machine-readable registry of production x402 services, listing name, category, route count, and price range. This distinguishes it from sibling tools like list_capabilities and get_endpoint_spec, which focus on capability listing or endpoint details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states it is useful for an agent deciding which paid tool/service to reach for next, giving a clear when-to-use context. It does not mention alternative tools or exclusions, but the use case is well-defined and sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_specA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: free ('FREE — no wallet needed'), source ('straight from the live OpenAPI discovery doc'), and content (price, schema, examples). It doesn't cover error handling or what happens if the route doesn't exist, but for a read-only spec tool, this is sufficient. No annotations provided, so description carries the burden; it does so reasonably.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose. Every sentence provides valuable information: first describes outputs, second gives usage instructions and context. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema or annotations, the description completely covers what the tool does, how to use it, and why it's useful. It explains the 'free' aspect, the route path requirement, and that it returns examples and schema. For a single-parameter tool, this is fully sufficient guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers the parameter with description and examples (100% coverage), giving a baseline of 3. The description adds extra value by specifying the leading slash is optional ('with or without a leading slash') and reiterates examples, clarifying the expected input format beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies exactly what the tool does: 'Full call spec for one route: price, input JSON schema, a worked request example, and a worked response example'. It clearly differentiates from siblings by indicating it's a preparation step before call_endpoint and distinct from listing/searching capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use before call_endpoint: 'Use this before call_endpoint to know exactly what body to send.' Also clarifies that no wallet is needed, establishing when it's appropriate. The path format guidance ('with or without a leading slash') provides clear usage details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.