Server Quality Checklist

Tool Scores

• load_whitepaperA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions downloading and loading a PDF, implying a write operation, but fails to detail critical aspects like required permissions, rate limits, error handling, or what 'knowledge base' entails. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by parameter and return details. It uses bullet points for clarity without unnecessary verbosity. However, the return statement could be more concise, and some behavioral details are omitted, slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 0% schema coverage and no output schema or annotations, the description is moderately complete. It covers purpose and parameter meanings adequately but lacks output details (only a vague success/failure message) and behavioral context like side effects or error conditions, which are crucial for this type of tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context for both parameters: 'project_name' is explained with cryptocurrency examples (e.g., 'bitcoin', 'ethereum'), and 'url' specifies it's for a PDF download. This clarifies semantics beyond the bare schema, though it doesn't cover all potential edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Load a whitepaper PDF from a URL into the knowledge base') with both verb and resource, and distinguishes it from sibling tools like 'ask_whitepapers', 'list_available_projects', and 'search_whitepaper' by focusing on ingestion rather than querying or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to add a whitepaper to the knowledge base, but lacks explicit guidance on when to use this tool versus alternatives like 'search_whitepaper' for existing content or prerequisites such as URL accessibility. It provides basic context without exclusions or detailed comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_whitepaperA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it performs a web search via DuckDuckGo and returns JSON-formatted results, but does not mention behavioral traits such as rate limits, authentication needs, error handling, or whether it accesses external APIs. For a tool with no annotations, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose, followed by clear sections for parameters and returns. Each sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (web search with one parameter) and no annotations or output schema, the description covers the basic purpose, parameters, and return format. However, it lacks details on behavioral aspects like error cases or search constraints, which could be important for an agent. It is adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% coverage. It explains that 'project_name' refers to 'the name of the cryptocurrency project' and provides examples ('bitcoin', 'ethereum'), clarifying the parameter's purpose and expected format. This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for a cryptocurrency project's whitepaper PDF') using a specific resource ('using DuckDuckGo'), which distinguishes it from sibling tools like 'load_whitepaper' (likely for loading content) and 'list_available_projects' (likely for listing projects). It provides a concrete verb+resource combination that is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the resource (cryptocurrency whitepapers) and search engine (DuckDuckGo), but does not explicitly state when to use this tool versus alternatives like 'ask_whitepapers' or 'load_whitepaper'. It provides context but lacks clear exclusions or named alternatives, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_whitepapersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it's a search operation (implied read-only), returns up to 5 results, and filters by project. However, it lacks details on permissions, rate limits, error handling, or how results are formatted beyond being a string, leaving gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The parameter and return sections are structured clearly, though the return explanation could be slightly more concise. Every sentence adds value, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and low schema coverage (0%), the description is moderately complete. It covers purpose, parameters, and basic return info, but lacks details on output structure (e.g., format of the string), error cases, or behavioral constraints like pagination. For a search tool with 2 parameters, this leaves some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the schema: it explains that 'query' finds 'relevant whitepaper content', 'project_name' filters by 'cryptocurrency project' with an example ('bitcoin'), and clarifies that 'None' searches all whitepapers. This covers both parameters thoroughly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search the knowledge base for information') and resource ('whitepaper content'), distinguishing it from siblings like 'list_available_projects' (listing), 'load_whitepaper' (loading specific content), and 'search_whitepaper' (likely more focused search). It explicitly mentions filtering by project as an optional feature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use optional filtering ('optionally filtered by project') and implies usage for general knowledge base searches. However, it doesn't explicitly state when NOT to use this tool or name specific alternatives among siblings, such as when to prefer 'search_whitepaper' over this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_projectsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists projects and returns JSON, but lacks details on rate limits, authentication needs, data freshness, or error handling. It adequately describes the core behavior but misses operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by structured sections for parameters and returns. Every sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is mostly complete. It explains the action, source, and return format. However, it could enhance completeness by mentioning data derivation from PDF filenames more prominently or noting any limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly notes 'Parameters: None,' which adds clarity beyond the empty input schema. With 0 parameters and 100% schema coverage, the baseline is 4, as the description confirms no inputs are required, compensating for any potential ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all cryptocurrency projects') and resource ('available in the knowledge base'), distinguishing it from sibling tools that focus on whitepaper operations rather than project listing. It precisely communicates what the tool does without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'available in the knowledge base,' which helps differentiate from other data sources. However, it lacks explicit guidance on when to use this tool versus alternatives like 'search_whitepaper' for filtering, or prerequisites for accessing the knowledge base.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.