Server Quality Checklist

Tool Scores

• get-artwork-by-artistC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination support, which is useful context, but doesn't describe other important behaviors like whether this is a read-only operation (implied by 'Get'), error handling, rate limits, authentication needs, or what the response format looks like. For a tool with no annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences. The first sentence states the core purpose, and the second adds important behavioral context about pagination. No wasted words, though it could be slightly more structured with clearer separation of purpose versus guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and pagination behavior, but doesn't address response format, error cases, or differentiation from sibling tools. With no output schema, the description should ideally provide more information about what gets returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema: it mentions pagination with the page parameter (already in schema) and implies the artist id comes from 'search-for-artist' (hinted in schema description). No additional syntax, format, or usage details are provided beyond what's in the structured fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get artworks by artist id in the Art Institute of Chicago collection.' It specifies the verb ('Get'), resource ('artworks'), and scope ('Art Institute of Chicago collection'), but doesn't explicitly differentiate it from sibling tools like 'get-artwork-by-id' or 'search-for-artist' beyond the artist focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance. It mentions pagination support with the page parameter, but doesn't specify when to use this tool versus alternatives like 'search-for-artist' or 'full-text-search'. No exclusions, prerequisites, or context for tool selection are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-for-artistC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the search function but doesn't mention whether it's read-only, if there are rate limits, authentication needs, or what the return format looks like (e.g., paginated results). This leaves significant gaps for a tool with three parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is insufficiently complete. It doesn't explain what the search returns (e.g., artist details, IDs), how results are structured, or any behavioral traits like pagination handling, which are critical for a search tool with three parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for 'name', 'limit', and 'page' parameters. The description doesn't add any additional semantic context beyond what the schema provides, such as search behavior or result formatting, but the schema adequately covers the basics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search for artists') and the resource domain ('Art Institute of Chicago collection'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'full-text-search' or 'search-by-title', which might also involve searching the same collection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-artwork-by-artist' or 'search-by-title'. It lacks any mention of prerequisites, exclusions, or comparative context with sibling tools, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-artwork-by-idB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions retrieving 'additional information' and an image 'if available', which hints at data retrieval but doesn't cover aspects like rate limits, authentication needs, error handling, or response format. For a read operation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and includes a practical note about the ID source. It avoids unnecessary details, though it could be slightly more structured by separating the workflow hint into a second sentence for better clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple read operation with 2 parameters), 100% schema coverage, and no output schema, the description is adequate but incomplete. It covers the basic purpose and parameter context but lacks details on behavioral traits like response format or error handling, which are important for a tool with no annotations or output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('id' and 'includeImage') fully. The description adds minimal value by referencing the ID from 'search-by-title' tool, but it doesn't provide additional syntax, format details, or usage examples beyond what the schema specifies. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get additional information, including an image if available') and resource ('about a specific artwork by its ID from the Art Institute of Chicago'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-artwork-by-artist' or 'search-by-title' beyond mentioning the ID parameter, which slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by referencing the 'search-by-title' tool for obtaining the ID, suggesting a workflow, but it doesn't explicitly state when to use this tool versus alternatives like 'get-artwork-by-artist' or 'search-by-medium'. It provides some context but lacks clear exclusions or direct comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-by-mediumB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination support, which is useful context beyond basic functionality, but doesn't cover other important aspects like rate limits, authentication needs, error handling, or what the response format looks like. The description doesn't contradict any annotations (since there are none), but it's incomplete for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that directly state the tool's purpose and a key feature (pagination). There's no wasted text, and it's front-loaded with the main functionality. It could be slightly improved by integrating the pagination note more seamlessly, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with pagination), no annotations, and no output schema, the description is minimally adequate. It covers the basic purpose and pagination but lacks details on response format, error cases, or how results are structured. Without annotations or output schema, more behavioral context would be helpful for the agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all three parameters (medium, limit, page). The description adds no additional meaning about parameters beyond what's in the schema—it mentions the 'page' parameter for pagination, but the schema already describes this. Baseline 3 is appropriate when the schema does all the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('artworks by medium in the Art Institute of Chicago'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'search-by-title' or 'full-text-search', which would require mentioning that this tool focuses specifically on medium-based filtering rather than other search criteria.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search-by-title' or 'full-text-search'. It mentions pagination support, which is a technical feature, but doesn't help the agent decide between this and other search tools based on use case or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-by-titleB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses pagination behavior ('Pagination is supported with the page parameter'), which is useful context beyond the schema. However, it doesn't cover other behavioral traits like rate limits, authentication needs, error handling, or what the search returns (e.g., partial matches, case sensitivity). The description doesn't contradict annotations since there are none.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second adds key behavioral context (pagination). Both sentences earn their place with no wasted words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with pagination), no annotations, and no output schema, the description is somewhat complete but has gaps. It covers the basic purpose and pagination, but lacks details on search behavior (e.g., fuzzy matching), result format, or error cases. For a search tool with 3 parameters, it's adequate but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (title, limit, page) with descriptions and defaults. The description adds minimal value beyond this: it mentions pagination with the page parameter, which is already in the schema. No additional semantics or usage examples are provided, so it meets the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for artworks by title in the Art Institute of Chicago.' It specifies the verb ('search'), resource ('artworks'), and scope ('by title'), distinguishing it from siblings like search-by-medium or get-artwork-by-artist. However, it doesn't explicitly differentiate from full-text-search, which might also search titles, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like full-text-search or search-by-medium. It mentions pagination support but doesn't clarify if this is the preferred tool for title searches over other search methods. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• full-text-searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses pagination behavior ('Pagination is supported with the page parameter'), which is valuable. However, it doesn't mention rate limits, authentication requirements, error conditions, or what the search results look like (format, fields returned).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste - the first states the core purpose and scope, the second adds crucial behavioral information about pagination. The description is appropriately sized and front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 3 parameters and no output schema, the description provides adequate basic information about purpose and pagination. However, without annotations or output schema, it should ideally mention more about the search behavior (case sensitivity, partial matching), result format, or limitations. The description is minimally complete but could be more comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds minimal value beyond the schema - it mentions the 'page' parameter for pagination but doesn't provide additional context about query syntax, search behavior, or result ordering that isn't already in the parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for artworks') and resource ('Art Institute of Chicago collection'), with precise scope ('metadata contains mention of the query term'). It distinguishes from siblings like 'search-by-title' or 'search-by-medium' by specifying full-text metadata search rather than field-specific searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching metadata with query terms, but doesn't explicitly state when to use this tool versus alternatives like 'search-by-title' or 'search-by-medium'. No guidance is provided about when NOT to use this tool or about prerequisites for effective searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.