Server Quality Checklist

Tool Scores

• get_documentA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It transparently describes the return format as a JSON string with specific fields, but does not disclose error handling, rate limits, or authentication needs. It is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with purpose, and structured with Args and Returns sections. Every sentence adds value with no redundancies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter, presence of an output schema (though not shown), and clear description of what is returned, the description is largely complete. Minor gap: no differentiation from sibling tools, but the simplicity of the tool mitigates this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description explains the zbmath_id parameter in detail, providing example and clarifying that it is a numeric identifier. This adds significant meaning beyond the schema definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full metadata for a single zbMath document by numeric ID, using the verb 'Retrieve' and specifying the resource. It does not explicitly differentiate from siblings like get_author or search_documents, but the purpose is clear and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a numeric ID is available and metadata is desired. It does not provide when-not or alternatives explicitly, which would be helpful given sibling tools exist, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_softwareA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It correctly states the tool is for reading metadata (non-destructive), but it does not disclose potential error behavior (e.g., invalid ID) or any authentication needs. This is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loads the purpose, and uses a clear Args/Returns format. Every sentence adds value, though the format is slightly more verbose than necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no nested objects, has output schema), the description covers the core functionality, argument, and return fields adequately. It is complete enough for an agent to use without confusion.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by explaining that 'software_id' is 'the numeric swMath software identifier'. The schema only specifies type integer, so the description provides necessary context. Schema coverage is 0%, making this compensation important.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Retrieve' and clearly identifies the resource as 'metadata for a zbMath / swMath software entry by its numeric ID'. This clearly distinguishes it from sibling tools like 'get_author' or 'search_documents'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a numeric software ID is available, but it does not explicitly state when to use this tool versus alternatives. No exclusions or alternative suggestions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_authorA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that the tool returns a JSON string with specific fields (name variants, institutions, publication count), implying a read-only operation. It does not cover error handling or rate limits, but for a simple retrieval, the behavioral implications are clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with Args and Returns sections, making it easy to parse. It is concise but includes necessary details. It could be slightly more streamlined, but overall it is well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, output schema exists), the description covers the key aspects: what it does, what input it needs, and what output to expect. It does not mention error responses, but for a retrieval tool, this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no description for the single parameter 'author_id' (0% coverage), but the description adds valuable examples (e.g., 'euler.leonhard' or 'gauss.carl-friedrich'), making the expected format explicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve', the resource 'profile information for a zbMath author', and the method 'by their profile ID'. It distinguishes from sibling tools like get_document and get_software, which focus on documents and software respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the user needs the author ID to use the tool, but it does not provide explicit guidance on when to use this tool versus alternatives, nor does it specify any exclusions or prerequisites beyond having the ID.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It transparently describes the return format (JSON string with fields) and pagination behavior. The tool is clearly read-only with no destructive effects mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences plus a Returns paragraph, well-structured with clear Args/Returns sections. Every sentence adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is comprehensive for a search tool with three parameters and an output schema. It covers query syntax, pagination, result fields, and includes an example. No additional context is needed for correct tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description is essential. It thoroughly explains all three parameters, including the free-text nature of query, the range and default for results_per_page, and the zero-based page index. This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches zbMath Open for mathematical documents using a free-text query. It explicitly distinguishes from sibling tools like get_document and structured_search by focusing on free-text search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context with parameter details (query, results_per_page, page) and describes pagination. However, it does not explicitly state when to use this tool versus alternatives, which are sufficiently different to imply usage but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• structured_searchA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses return format (JSON with count and documents) and parameter behavior, but does not explicitly state read-only nature or other side effects. However, search context implies safe operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: one-line purpose, usage guidance, parameter list with examples, and return description. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 optional parameters and existence of output schema, the description covers all parameters, usage, and return. It distinguishes from siblings, making it complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains all 8 parameters with examples (e.g., author 'Euler', msc_code '11', defaults for results_per_page and page), adding significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'Search zbMath documents using structured field filters', with a specific verb and resource. It distinguishes itself from the sibling 'search_documents' by noting it is for restricting results to specific fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use this instead of search_documents when you want to restrict results to specific fields such as author name, MSC subject class, or year range.' This provides clear condition and alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.