Server Quality Checklist

Tool Scores

• get_metadataB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses only the basic operation and return type ('MetadataResponse'), but lacks information about prerequisites, error behavior, rate limits, or any non-obvious side effects. The include_files/include_versions args are mentioned but only as schema parameter semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and logically structured: a one-sentence overview, an Args block, and a Returns line. There is no redundant text, though the Args block is formatted with colons instead of bullets, which is slightly less scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description relies on the vague statement 'Returns: MetadataResponse containing the record metadata.' It does not explain what fields are in the response, how include_files/versions affect the output, or any edge cases. The absence of usage context further limits completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description compensates by explaining all three parameters: record_id as 'The ID of the Zenodo record', include_files as 'Whether to include file information', and include_versions as 'Whether to include version information'. These explanations provide the meaning missing from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Retrieve detailed metadata for a Zenodo record.' The verb 'retrieve' and the resource 'metadata' distinguish it from siblings like search_records and list_files, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention that this is for fetching a record by ID rather than searching, nor does it reference sibling tools. The agent must infer usage from the tool name and args.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_recordsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only mentions return type ('Dictionary containing search results') but discloses no behavioral traits such as rate limits, pagination, authentication requirements, or whether the operation is read-only. There is no indication of side effects or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with separate Args and Returns sections. It avoids excess verbiage, though the Args section partly duplicates schema information. Overall, it is efficient and front-loaded with the purpose statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with 3 parameters and no output schema, the description provides the essential purpose, parameter meanings, and return type. However, it lacks usage context (e.g., when to use vs alternatives) and gives no detail about the returned dictionary structure. It is adequate for invocation but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all three parameters, but 'query: Search query' is tautological. 'max_results: Maximum number of results to return' and 'sort: Sort order (bestmatch, mostrecent)' add useful semantics, including sort options not present in the schema. However, descriptions are minimal and lack constraints or format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Search for records in Zenodo,' which is a specific verb (Search) and resource (records in Zenodo). This clearly distinguishes it from sibling tools like get_citation, detect_data_type, get_metadata, and list_files, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any exclusions, prerequisites, or context that would help an agent decide between search_records and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_data_typeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the return format (dictionary with data type and confidence score), which is useful. However, without annotations, it does not state whether the operation is read-only, what permissions are required, or how errors are handled. This leaves gaps for a tool that could involve network calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear purpose statement followed by a brief Args/Returns section. It is front-loaded with the main action and contains no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description provides the essential information about input and output. However, it does not explain what confidence score means, possible data type values beyond the initial three, or fallback behavior for unclassifiable records.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines record_id as a string with no explanation. The description compensates by specifying that it is 'The ID of the Zenodo record to analyze', giving semantic meaning. However, it does not specify the expected format (e.g., numeric ID or DOI), leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Determine' with a clear resource ('a Zenodo record') and identifies the classification categories (dataset, software, article). This distinguishes it from sibling tools like search_records or get_metadata, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for classification but does not explicitly state when to use this tool versus siblings, nor does it mention any exclusions. There is no guidance on what to do if the record is not one of the listed types.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses the action (listing files) and the return type (dictionary with files and metadata), but it does not mention permissions, error behavior, or pagination. For a simple read-only tool, it's adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and structured into Args and Returns sections, with each sentence adding value. It is front-loaded with the primary purpose and does not waste words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and no output schema, the description covers the essential context: what it lists, the parameters, and the return format. It lacks edge-case details (e.g., record not found), but that is acceptable given its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It provides meaningful explanations for both 'record_id' ('The ID of the Zenodo record') and 'include_metadata' ('Whether to include file metadata'), going beyond the schema's bare names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List all files available in a Zenodo record.' It uses a specific verb ('list') with a specific resource ('files in a Zenodo record'), and it distinguishes from siblings like search_records and get_metadata by focusing on file listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (when you need to see files in a Zenodo record) but does not explicitly contrast with sibling tools or provide exclusions. It does not say when to prefer this over search_records or get_metadata, but the purpose is distinct enough to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_citationA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses the return type (CitationResponse) but does not mention potential errors, accepted format values beyond examples, or any read-only guarantees. The behavioral scope is minimally covered but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a concise docstring with a clear one-sentence purpose, followed by structured Args and Returns sections. It contains no fluff and efficiently communicates essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description covers purpose, parameters, and return type. It lacks details on error handling or a definite list of supported formats ('etc.' is vague), but overall it is reasonably complete given the tool's simplicity and lack of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly explains both parameters: record_id as 'The ID of the Zenodo record' and format as 'The citation format (bibtex, apa, etc.)'. Since the schema only provides field names and defaults, the description adds meaningful examples and clarifies the expected semantics, compensating for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get a citation for a Zenodo record in the specified format.' It uses a specific verb ('Get') with a clear resource ('citation for a Zenodo record') and distinguishes itself from sibling tools like search_records and get_metadata by focusing on citation generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a citation is needed for a Zenodo record, providing clear context. However, it does not explicitly state when not to use this tool or mention alternatives, such as using get_metadata for record details instead of a formatted citation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.