Server Quality Checklist

Tool Scores

• screenshot_dataB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds value by stating the cost ('$0.02 USDC per call'), which is a key behavioral trait not covered by the schema. However, it lacks details on other aspects such as rate limits, error handling, or output format, leaving gaps in transparency for a tool that performs data extraction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of two sentences that efficiently convey the tool's purpose and cost. Every sentence earns its place by providing essential information without redundancy or fluff, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (data extraction from images) and the absence of both annotations and an output schema, the description is partially complete. It covers the purpose and cost but lacks details on behavioral traits, output format, and usage guidelines. This leaves significant gaps for an agent to understand how to invoke and interpret results effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description does not add any semantic details beyond what the schema provides, such as examples or constraints for 'extraction_hint'. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description neither compensates nor detracts.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Extract structured data from screenshots of dashboards, tables, forms, or UIs.' It specifies the verb ('extract') and resource ('structured data'), and identifies the source material ('screenshots'). However, it does not explicitly differentiate from sibling tools like 'document_intelligence' or 'invoice_extract', which may also extract data from documents, leaving some ambiguity about when to choose this tool over others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the cost per call, which is a usage consideration, but does not specify contexts, prerequisites, or exclusions. Without explicit when-to-use or when-not-to-use instructions, the agent lacks clear direction for selection among sibling tools like 'alt_text' or 'sentiment_analysis'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• alt_textA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds important context about the cost ($0.01 USDC per call), which isn't captured in structured fields. However, it doesn't describe other behavioral aspects like rate limits, authentication needs, response format, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with only two sentences, both of which earn their place. The first sentence states the core purpose, and the second provides critical cost information. There's zero wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate. It covers the core purpose and cost but lacks information about output format, error handling, or when to use versus alternatives. The absence of an output schema means the description should ideally explain what the tool returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so all parameters are documented in the input schema. The description adds no additional parameter semantics beyond what's already in the schema (image_url, image_base64, style). The baseline score of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Generate') and resource ('accessible alt text descriptions for images'), making it immediately understandable. It distinguishes itself from sibling tools like document_intelligence or sentiment_analysis by focusing specifically on image accessibility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While it mentions a cost per call, it doesn't specify scenarios where alt text generation is appropriate versus other image-related tools like screenshot_data, nor does it mention prerequisites or constraints beyond the cost.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• document_intelligenceA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and adds valuable behavioral context: it discloses the cost ('$0.05 USDC per call') and blockchain details ('x402, Base mainnet'), which are critical for usage decisions. It also implies mutation/processing behavior through 'analyze' and specifies input formats (URL or base64). However, it doesn't mention rate limits, error conditions, or output format details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: the first clause states the core purpose, followed by key input constraints and cost details. Every sentence earns its place with no wasted words. The structure efficiently communicates essential information in minimal space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters with full schema coverage but no annotations and no output schema, the description is moderately complete. It covers purpose, input formats, analysis types, and cost, but lacks output format details, error handling, and sibling differentiation. For a tool with significant functionality and cost implications, more behavioral context would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds marginal value by mentioning 'image URL or base64' and listing analysis types, but doesn't provide additional syntax, format, or usage details beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze document images — extract text, summarize, classify, or extract tables.' It specifies the verb ('analyze') and resource ('document images'), and lists the specific analysis types. However, it doesn't explicitly differentiate from sibling tools like 'invoice_extract' or 'screenshot_data' which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions cost but doesn't specify scenarios where 'document_intelligence' is preferred over siblings like 'invoice_extract' (for invoices) or 'alt_text' (for accessibility). No prerequisites, exclusions, or comparative context is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sentiment_analysisA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively adds context by specifying the cost ('$0.01 USDC per call'), which is a key behavioral trait not covered by the schema. However, it doesn't mention other aspects like rate limits, error handling, or response format details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently adds cost information in a single, waste-free sentence. Every part of the description earns its place by providing essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is somewhat complete by stating the purpose and cost. However, it lacks details on output structure (e.g., format of 'score, label, and key phrases'), error cases, or usage context, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('text' and 'language') adequately. The description does not add any meaning beyond what the schema provides for these parameters, maintaining the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Analyze') and resource ('sentiment of text'), and distinguishes it from siblings by specifying what it returns ('score, label, and key phrases'). It goes beyond the name to explain the output, making it highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'pii_detect' or 'document_intelligence', nor does it mention any prerequisites or exclusions. It lacks context for tool selection among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_extractA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it's a paid service ('Cost: $0.03 USDC per call'), returns structured JSON data, and processes images via URL or base64. However, it doesn't mention rate limits, error conditions, or processing time.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste: the first explains purpose and output, the second provides critical cost information. Every element earns its place and is front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description does well by explaining what data is extracted ('vendor, line items, totals, dates, and more') and the cost implication. However, it could better describe the JSON structure or error scenarios given the absence of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Extract structured fields'), the resource ('from invoice or receipt images'), and the output format ('as JSON'). It distinguishes itself from siblings by focusing on invoice/receipt data extraction rather than general document analysis or other tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'invoice or receipt images' and mentions cost, but doesn't explicitly state when to use this tool versus alternatives like 'document_intelligence' or 'screenshot_data'. No guidance on prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pii_detectA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the core functionality (detect and redact PII) and output (returns detected PII types and redacted text), and adds valuable context with the cost per call. However, it lacks details on rate limits, error handling, or specific PII types covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by output details and cost information in two concise sentences. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is fairly complete: it covers purpose, output, and cost. However, it could improve by specifying PII types or error scenarios, slightly reducing completeness for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('text' and 'redact') thoroughly. The description does not add any parameter-specific semantics beyond what the schema provides, such as examples or format details, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('detect and redact') and resource ('PII in text'), and distinguishes it from siblings by focusing on PII detection rather than alternative text analysis tasks like sentiment analysis or document intelligence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for PII detection scenarios but provides no explicit guidance on when to use this tool versus alternatives (e.g., for general text processing vs. PII-specific tasks) or any prerequisites. It mentions cost, which hints at financial considerations, but lacks clear when/when-not directives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.