Server Quality Checklist

Tool Scores

• read_paper_toolB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only states it fetches as markdown. Does not disclose rate limits, authentication, size limits, or what happens if paper not found. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, no wasted words. However, oversimplification leaves gaps in parameter and behavior details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one required param and output schema, description is adequate but missing input details and edge cases. Could be more complete for a fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage 0%: description does not mention the arxiv_id parameter or its format. Fails to add meaning beyond the schema for the single required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'fetch', resource 'full text of a paper', and output format 'as markdown'. Distinguishes from siblings like get_paper_details_tool (metadata) and list_daily_papers_tool (listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives. Implies usage through purpose but lacks guidance on when not to use or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paper_details_toolB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It implies a read-only operation but does not explicitly state side effects, authentication needs, or error behavior. The list of returned fields gives some context but is minimal for disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that efficiently lists key return fields without extraneous information. It is front-loaded with the action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and one parameter, the description lacks context about error handling, invalid inputs, and differentiation from sibling tools. It is minimally sufficient but not complete for an agent to fully understand behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter (arxiv_id) with 0% coverage (no description in schema). The tool description does not elaborate on the parameter's format, constraints, or examples, failing to compensate for the missing schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'detailed paper metadata' and lists specific fields like title, authors, abstract, GitHub repo, etc., which distinguishes it from siblings like list_daily_papers_tool (list) and read_paper_tool (likely full text).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or scenarios where other tools would be preferred, leaving the agent to infer usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_daily_papers_toolA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose all behavioral traits. It only states the basic function (list papers for a date) and default behavior (today), but does not mention pagination, rate limits, or what happens when no papers exist for the given date.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that conveys the essential information without any unnecessary words or structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional parameter) and the presence of an output schema, the description covers the core purpose and parameter. It is complete enough for an agent to decide when to invoke it, though it could optionally hint at the output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the input schema has 0% description coverage, the description adds meaning by specifying the date format (YYYY-MM-DD) and default value (today). For a single optional parameter, this provides adequate guidance, though additional details like allowed range could help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'list' and the resource 'papers featured on Hugging Face Papers' with a specified date format. It distinguishes from siblings like get_paper_details_tool (details) and read_paper_tool (reading), which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing papers on a specific date but does not explicitly state when to use this tool versus siblings, nor does it provide when-not conditions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_code_link_toolA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states the purpose but fails to disclose behavioral traits such as whether the operation is read-only, potential failure modes, or any side effects. For a simple lookup, more transparency is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the main action with no extraneous words. It is efficient and immediately informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no annotations) and the presence of an output schema, the description sufficiently covers the core functionality. It could be improved by noting potential errors, but for a basic resolver it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the parameter, so the description must compensate. It adds that the single parameter is an 'arXiv ID', which is essential, but does not specify the expected format (e.g., including version or prefix). This provides minimal added meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'resolve', the resource 'arXiv ID to GitHub repository URL', and the context 'from the corresponding HF Papers page'. This distinguishes it from sibling tools that deal with paper details, daily lists, or reading papers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have an arXiv ID and need the GitHub repo, but does not explicitly state when not to use it or mention alternatives. However, the context is clear enough for an agent to infer appropriate usage given the siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.