Server Quality Checklist

Tool Scores

• lei_lookupA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explicitly discloses that lapsed/retired/annulled entities are returned with a flag instead of hidden (and why), and states the exact cost per call and payment method. This goes beyond basic expectations but does not cover all potential behaviors (e.g., rate limits or error handling), so a 4 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding essential information: purpose, return fields, and edge-case/cost behavior. It is front-loaded and free of filler, though slightly longer than minimal. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description provides a strong overall picture: what it does, what it returns, how edge cases are handled, and the cost. It lacks details on response format (e.g., JSON structure) but that is not required since no output schema exists. The schema covers parameters, and the description covers broader context well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions, so the baseline is 3. The description reinforces the purpose of q versus lei ('Search by company NAME, not just by identifier') and the schema provides the rest. It adds some nuance but does not introduce new parameter-level meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'lookup' with a clear resource ('LEI from the GLEIF golden copy') and explicitly states search by company name. This distinguishes it from all sibling tools, none of which relate to LEI or entity lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states the primary use case (looking up an LEI by company name) and contrasts it with using an identifier, implying the q parameter for name searches. It does not mention alternative tools because none are relevant, but it gives practical guidance on behavior (e.g., lapsed entities included) and costs, which helps the agent decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reverse_geocodeA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the null-and-reason behavior for ocean/unmapped points, the OpenStreetMap Nominatim data source, and the cost/payment mechanism, which are important runtime behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, followed by behavioral and cost information. No filler or repetition of schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 2 params, but there is no output schema, so the description should explain the return value more fully. It mentions 'structured components' but doesn't enumerate them, and it doesn't mention potential errors aside from the null case. Still, cost and data source add context beyond the minimum.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both lat and lon already described. The description adds no parameter-specific details beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'turn latitude/longitude into a street address with structured components', using a specific verb and resource. By calling it 'Reverse geocoding', it distinguishes itself from the sibling 'geocode' tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly frames the tool as reverse geocoding, indicating its use case, but does not name alternatives or state when not to use it. The sibling tool 'geocode' implies the forward direction, but no explicit exclusion is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• secure_randomA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds significant context: the randomness source (CSPRNG), the uniform distribution via rejection sampling (avoiding modulo bias), the cost ($0.003 USDC per call), the network (Base mainnet), and the payment mechanism (EVM_PRIVATE_KEY). This goes well beyond basic schema info, though it doesn't cover error handling or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding value: the first defines outputs, the second explains a statistical property, and the third covers use case and cost. It is front-loaded with the core purpose, contains no filler, and remains under the typical length limits.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity and lack of output schema, the description covers the key contextual aspects: what it returns (bytes in hex/base64url, integers), how the randomization works, the cost, and the intended usage scenario. It falls short of a 5 because it doesn't precisely define the returned data structure or specify how to choose between hex vs base64url output, but it is sufficient for an agent to operate the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful semantics by explaining the two operational modes: random bytes (when min/max are absent) and uniform integers (when min/max are provided). It also clarifies that integer generation is rejection-sampled to be uniform, which gives context for interpreting the min/max and count parameters beyond their schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: generating cryptographically secure random bytes (hex/base64url) or uniform integers in an inclusive range. It uses a specific verb ('generate') and resource ('random values'), and the sibling tools are all unrelated (geocoding, search, etc.), making this tool's purpose distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'For randomness nobody else could have seen, generate it yourself — this is for callers that cannot.' This tells the agent when to use the tool (when local generation isn't possible) and implies an alternative (self-generation). However, it doesn't explicitly enumerate alternative tools or exclusions beyond this contrast.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• geocodeA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does an excellent job. It discloses that the tool returns ranked candidates with multiple fields (display name, coordinates, bounding box, address components, importance score), explains the significance of the importance score for handling ambiguity, names the data source (OpenStreetMap Nominatim), and transparently states the cost and payment mechanism. This is rich behavioral context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four tight sentences, each earning its place: purpose, return value/ambiguity handling, data source, and cost. It is front-loaded with the core purpose and contains no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description compensates by enumerating the return fields (candidates, display name, coordinates, bounding box, address components, importance score). It also covers cost, data source, and automatic payment behavior. For a single-parameter tool, this is comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of the parameter (q) with a clear description and example. The tool description adds no additional semantic detail about the parameter itself, so it lands at the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Forward geocoding' and explicitly states 'turn an address or place name into latitude/longitude', which is a specific verb+resource that clearly distinguishes it from the sibling tool 'reverse_geocode'. It leaves no ambiguity about the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states this is 'Forward geocoding', which implies the use case (address/place name → coordinates) and implicitly contrasts with reverse geocoding. It provides clear context but does not explicitly name alternatives or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sanctions_screenA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses key behaviors: word-order insensitivity due to name format, riskLevel values and 'clear' scoping, error behavior when lists fail to load (never returns a false clear), and the $0.03 USDC cost with payment mechanism. These go beyond simple capability claims and provide essential operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: it covers purpose, scope, output, match behavior, limitations, error semantics, and cost in a compact paragraph. The information is front-loaded and structured logically.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a paid external screening tool with no output schema, the description covers the essential context: what lists are used, what the response contains (programs, remarks, score), riskLevel interpretations, failure semantics, and pricing. Users can accurately predict tool behavior without further documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter is already well-documented. The description adds the 'SURNAME, Given' detail to clarify why word order doesn't matter, which enhances the 'name' parameter, but most parameter semantics come from the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs OFAC sanctions screening, specifically checking a person or company against the US Treasury SDN and Consolidated lists. It uses a specific verb ('Checks') and identifies the resource and scope, making it distinct from the unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool (screening persons/companies) and explicitly scopes the 'clear' result to the US lists, noting EU, UK, and UN are not screened. It does not name alternatives, but no sibling handles sanctions screening, so exclusion guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• url_to_markdownA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly describes what the tool does under the hood (Firefox reader-mode extraction, text extraction for PDFs), what it refuses (private/loopback/cloud-metadata addresses), what it returns (title, byline, excerpt, word count, truncation flag), and the cost/payment mechanism. This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and packs in critical information like limitations, error handling, security restrictions, and pricing without any fluff. Every clause earns its place, and the structure flows logically from purpose to behavior to constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description covers all essential contextual information: input type, extraction method, edge cases (client-rendered pages), security guards, return fields, and cost. For a tool with a single parameter, this is fully self-contained and leaves the agent well-equipped to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for the single 'url' parameter, stating 'Public http/https URL of an article or PDF.' The description adds some context about address restrictions and extraction behavior, but it doesn't significantly extend the parameter's meaning beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetch a public article or PDF URL and return clean Markdown.' It clearly distinguishes from siblings like geocode/weather by focusing on URL-to-Markdown conversion. It also specifies two extraction modes (reader-mode for HTML, text extraction for PDFs), making the intended functionality unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it's for public articles/PDFs, and explicitly notes that JavaScript is not executed, so client-rendered pages will return an error. It also lists restrictions (private, loopback, cloud-metadata addresses refused). While it doesn't name alternative tools, the sibling set is unrelated, so no additional when-to-use guidance is necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and excels: it discloses sponsored rows exclusion, that results are not fetched/verified, and provides cost/payment specifics. It also details the exact fields returned, offering high transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five concise sentences each earn their place: core function, exclusions, verification caveat, pairing suggestion, and cost. The description is front-loaded with the primary action and contains no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter search tool, the description is complete: it explains output fields, limitations (unverified, sponsored excluded), cost, and a workflow. With no output schema, it adequately describes return values and behavioral nuances.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add parameter-specific semantics beyond what the schema documents for 'q' and 'count'; it focuses on tool behavior and cost instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'a free-text query returns ranked organic results with title, real destination URL, display URL and snippet,' clearly defining the tool's function. It distinguishes itself from siblings by focusing on web search and even suggests a complementary tool (url_to_markdown).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the pairing directive 'Pair with url_to_markdown to read any result' and warns that results are unverified, helping set expectations. However, it does not explicitly name alternatives beyond url_to_markdown or state precise 'when not to use' conditions, so it is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• weatherA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does well: discloses data source (Open-Meteo), cost ($0.003 USDC), payment mechanism (EVM_PRIVATE_KEY), output format (WMO code + text), and a model-output caveat. This is thorough behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two dense sentences covering scope, output fields, data caveats, complementary tools, and cost. Every sentence earns its place and the most important info is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description enumerates current and forecast fields, return encoding (WMO + text), model output note, and cost. It is self-sufficient for an agent to understand what will be returned and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all 3 parameters with descriptions (100% coverage). The description adds context by confirming 'any coordinates worldwide' and 'up to a 7-day daily forecast,' reinforcing the meaning of lat/lon/days without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'Weather for any coordinates worldwide' with a detailed list of current conditions and forecast data. It is specific and distinct from sibling tools like geocode and web_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides usage context by saying 'Pair with geocode to go from a place name to coordinates,' indicating when this tool is appropriate. It doesn't explicitly exclude alternatives, but no competing weather tool exists among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.