Server Quality Checklist

Tool Scores

• assess_country_riskA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavioral traits. It discloses return format and score interpretation but omits details like idempotency, authorization needs, or data sources. Adequate for a simple risk assessment tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose and output, second provides usage guidance and threshold. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one parameter, no output schema), the description fully explains input, output, and usage context. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description adds minimal extra meaning beyond the schema (e.g., 'name or ISO-2 code'. Does not elaborate on validation or formatting beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'assess', resource 'country risk', and output '0-10 risk score and band'. Distinguishes from siblings like assess_industry_risk by specifying country risk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call: for applicant's country and beneficial owner's country if different. Provides actionable threshold for escalation (score >= 7). Lacks explicit exclusions but contextually sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_businessA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool queries SEC EDGAR, explains match/no-match outcomes, and notes escalation condition. Does not explicitly state read-only nature or discuss rate limits/authentication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus a note, front-loaded with purpose. Every sentence earns its place with no wasted text. Compact and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no output schema, so description should explain return values. It implies a match result but does not explicitly describe the output format or fields. Missing this key detail for a simple lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with descriptions for both parameters. Description adds no new semantics beyond the schema; both params are adequately documented. Baseline 3 as schema already covers meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it looks up a business in the SEC EDGAR registry to verify registration. It uses specific verb+resource (look up, SEC EDGAR registry) and distinguishes from sibling tools (risk assessment, screening, identity verification) by focusing on SEC registration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call for any business applicant' and provides important interpretation guidance (no-match expected for LLCs, red flag for claimed public companies). Lacks explicit exclusions or alternative tool references for similar lookups.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screen_pepA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that output is not a live result but a 'defined-but-not-live status' with INTEGRATION_POINT value. Warns against misinterpreting it as clean. No annotations, so description carries burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with cautionary note. Slightly verbose but each sentence adds value. Efficient for explaining a nuanced behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, return value, and caveat. No output schema, so description compensates by explaining the result semantics. Could mention authentication or rate limits but sufficient for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single 'names' parameter. Description adds context about processing these names but no new format or constraints beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Screen' and specific resource 'PEP status'. Distinguishes from sibling 'screen_sanctions' by focusing on PEP.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when not to treat result as final (integration point), implying manual due diligence. Provides context but no direct 'when not to use' compared to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screen_sanctionsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses the live screening behavior and the significance of matches. It does not cover rate limits or error handling but adequately sets expectations for a screening tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second gives usage instruction and rule. No fluff, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and lack of output schema, the description covers purpose and usage but does not describe the return value or error states, which would help an agent interpret results fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minimal extra meaning beyond the schema, just reiterating 'full names' and providing context for the disqualifying rule.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'screen' and the resource 'live OFAC SDN sanctions list'. It distinguishes from sibling tools like screen_pep by specifying the exact list and usage for applicants, beneficial owners, and company names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit instructions to call for every applicant, beneficial owner, and company name, and mandates that potential matches be treated as disqualifying pending human review. It lacks explicit when-not-to-use or alternatives but is very directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_identityA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that validation is structural and not identity confirmation, preventing misuse. It also lists possible return values, adding behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus a crucial note; no filler. The most important information (purpose and limitation) is front-loaded, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately covers purpose, behavioral limitations, and return statuses. It omits error handling details but is sufficient for an agent to understand tool scope and use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaning by explaining what validations are performed (SSN rules, DL format, address presence). This goes beyond the schema's generic property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it validates identity documents structurally, specifying checks on SSN format/range per SSA rules, driver's license format by state, and address presence. It returns specific statuses like VALID_FORMAT, distinguishing it from risk assessment and screening siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes it is structural validation only, not authoritative match, and that INVALID_FORMAT is a red flag. While it doesn't explicitly state when to use versus alternatives, it sets expectations and distinguishes from authoritative verification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_industry_riskA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description fully discloses behavioral traits: return values (PROHIBITED, STATE_CONDITIONAL etc.) and their implications ('hard gate — decision must be FAIL', 'drive ESCALATE'). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose, clear enumeration of return values, and actionable guidance. No redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and three parameters, description fully covers purpose, usage, return categories, and parameter roles. Complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds value by explaining the free-text nature of industry, optional NAICS, and why state is needed ('cannabis can be resolved correctly').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Classify a business industry for AML risk') and lists all four return categories, clearly distinguishing this tool from siblings like assess_country_risk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call for every business applicant' and instructs to pass state for correct resolution. Does not explicitly exclude scenarios or name alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.