Server Quality Checklist

Tool Scores

• browse_collectionsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states 'Browse,' implying a read-only operation, but does not explain output format, pagination, data freshness, or authentication requirements. Critical traits are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no waste. It is concise and to the point, but could include more information (e.g., output hint) without becoming verbose. The brevity does not harm comprehension but limits completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple schema (1 optional param), no output schema, and presence of related sibling tools, the description fails to explain what the output looks like (list of collections? providers? both?) or how it relates to search and metadata tools. An agent lacks sufficient context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the only parameter 'provider' with a clear description ('Filter by specific data provider'). The tool description does not add additional meaning beyond the schema, but since schema description coverage is 100%, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Browse'), the resource ('available collections and providers'), and the scope ('OpenArchives.gr'). It distinguishes itself from sibling tools like search_archives (search) and get_document_metadata (metadata), as browsing collections/providers is a distinct operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., when to browse vs search). The description does not mention any prerequisites, use cases, or exclusions. An agent would have no context to choose this tool over search_archives or get_document_types.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_metadataB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It only states the function without indicating read-only nature, side effects, permissions, or performance implications. The lack of transparency leaves the agent uninformed about safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, efficiently conveying the core purpose. However, it could include more useful information without being verbose, so it's slightly above average but not perfect.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and the need for the agent to understand what 'full metadata' entails, the description is incomplete. It does not explain the return structure, potential limitations, or what data fields to expect, leaving significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains both parameters fully. The description adds no semantic value beyond what the schema provides, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'get full metadata' and specifies the resource 'a specific document using its URI'. This distinguishes it from sibling tools like 'browse_collections' which are for browsing, and 'search_archives' which are for searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or scenarios. The description is too minimal to help an agent choose this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_archivesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose behavioral traits such as pagination behavior, default sorting, or whether the search is free-text or filtered. The agent gets minimal insight into what happens beyond the parameter descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single-sentence description is concise and front-loaded with the core action. However, it could be slightly more structured to include usage context, but overall it earns its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters, no output schema, and no annotations, the description is minimally adequate. It names the aggregator but does not explain expected response format, language handling, or how pagination interacts with queries. Some gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 6 parameters have schema descriptions (100% coverage), so the description adds no additional semantic value beyond the schema. Per rules, baseline is 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for documents in a specific Greek scientific content aggregator (OpenArchives.gr). It directly differentiates from sibling tools which focus on collections, metadata, or types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like browse_collections or get_document_metadata. The description does not specify use cases or exclusions, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_typesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description should fully disclose behavior. It only states the basic function, omitting details like authentication, rate limits, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no wasted words; front-loaded with the core action and context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Minimally adequate for a simple tool with one optional parameter, but lacks details about return structure or behavior when language is unsupported.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear parameter description in the schema. The tool description adds no additional meaning beyond the schema's explanation of the language parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves document types and specifies their use for filtering searches, distinguishing it from sibling tools like browse_collections or search_archives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly indicates use for filtering searches, but no explicit guidance on when to use vs alternatives or exclusions for specific contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.