Server Quality Checklist

Tool Scores

• get_artwork_imageA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that not every record has an image and that artic requests usually fail with 403. However, it does not specify the returned image format (e.g., binary, base64) or any size limits. The mention of 'short caption' is vague. More details on what exactly 'return it' means would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary use case and includes essential behavioral notes in a second paragraph. It is concise enough, but could be slightly shorter by combining sentences. Overall, every sentence serves a purpose, and the structure is logical.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers key aspects: when to use, failure modes, fallback strategy, and return of caption. It does not detail the caption format or the exact image return mechanism, but these are minor omissions. The description is largely complete for an AI agent to decide correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and descriptions exist for both parameters. The description adds extra context: it explains that the 'source' parameter is the museum and that artic has issues, and that 'id' is from search results. This provides real-world semantic value beyond the schema definitions, earning a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('download the image for a single artwork') and the result ('return it so it can actually be viewed, plus a short caption'). It distinguishes from sibling tools by specifying that it retrieves the image data, not just metadata, and mentions fallback behavior. However, it does not explicitly state the return format (e.g., base64 or URL), which could be clearer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides excellent guidance: 'Use this whenever the user wants to see a work, or when you need to judge what a work actually depicts rather than trusting its title.' It also explicitly warns against using for Art Institute ('artic') due to bot challenges, and suggests preferring other sources and falling back to the museumUrl. This clearly tells when to use the tool and when to avoid it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that sources requiring an API key will be marked unavailable until configured, adding important behavioral context about conditional availability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, no redundancy, and front-loads the primary action. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description fully covers its behavior: listing sources and indicating availability. No additional information is necessary for proper use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline score is 4. The description does not need to add parameter info, and it correctly avoids extraneous details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists museum data sources and their availability. It uses specific verbs and resources ('list', 'museum data sources'), and distinctly separates this tool from sibling tools that deal with artworks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives or any prerequisites. While the purpose is clear, it fails to set usage context or mention when this tool is needed (e.g., before querying other tools).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_artworkA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that rights come back in two fields (isPublicDomain and license) with caveats about when isPublicDomain is set, and notes that fields are only present where the museum publishes them. This goes beyond basic functionality, though it could mention potential missing fields or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two well-organized paragraphs. The first states the main purpose and fields, the second explains rights fields in detail. Every sentence adds value, with no waste. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only two simple parameters and no output schema, the description provides good context about return fields and their semantics. It mentions limitations ('where the museum publishes them'). Could be improved by noting possible null fields or error cases, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds that the id is 'as returned by search_artworks' and lists the sources in the enum context. This adds minor value but does not significantly enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches full catalog details for a single artwork by source and id, listing specific fields like medium, dimensions, department, culture, credit line, and rights. This distinguishes it from sibling tools such as search_artworks (summary) and get_artwork_image (image).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool to confirm attribution or rights before relying on a work, and notes that search results carry only a summary. This provides clear when-to-use guidance, though it does not explicitly list when not to use or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_artworksA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It explicitly states the search is not semantic, warns about weak relevance ranking, details source-specific quirks (e.g., smithsonian field filters dropping silently, japansearch returning Japanese-only titles, tepapa image licensing restrictions), and explains the time cost of an 'all' search.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized with sections (IMPORTANT, CHOOSING A SOURCE, Source-specific quirks). Every sentence adds necessary context; however, the length could slightly tax LLM context windows. It is front-loaded with the core purpose and key warnings, earning a high score but not perfect conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple sources, non-semantic search, weak ranking, field filters, language differences), the description is remarkably thorough. It covers return format, behavioral quirks, and actionable guidelines for all major scenarios. The lack of output schema is compensated by explaining what results contain ('source' and 'id').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, providing baseline meaning for each parameter. The description adds significant value by elaborating on the 'query' parameter (preferred format, translation advice), the 'source' parameter (detailed guidance on choosing sources with cultural/regional recommendations and quirks), and the 'limit' parameter (default behavior and implications).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a clear verb-resource pair ('Search museum collections for artworks') and immediately distinguishes the tool's output from siblings by noting it returns 'source and id used with get_artwork or get_artwork_image'. It provides specific enough detail to differentiate from list_sources, get_artwork, and get_artwork_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives extensive when-to-use guidance: it explains that keyword search is not semantic, instructs translating user intent into catalog vocabulary, recommends preferring known artist/title over subject descriptions, and provides a last-resort strategy. It also advises scanning the entire result list rather than assuming the first hit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.