Server Quality Checklist

Tool Scores

• pl_callA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already include readOnlyHint, idempotentHint, and destructiveHint=false. The description adds meaningful context: parameters are passed verbatim, validation happens at the aggregator before forwarding, and specific error codes are enumerated. This enriches the behavioral model without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise opening, practical examples, parameter details, and a list of errors. Slightly long for an escape hatch but every sentence contributes necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generic escape-hatch tool, the description is highly complete. It covers use cases, parameter semantics, how to discover native names, validation behavior, and error handling. The output schema is present, so return-value details are not needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates well. It explains the 'arguments'/'args' alias and clarifies that either can be used (not both), and that keys must be native parameter names, directing to pl_list_sources for discovering them. Source and tool are self-explanatory given the context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'call any native tool of any registered source.' It distinguishes itself from sibling unified tools by positioning as an 'escape hatch' for operations not covered, with concrete examples (citator, DPA statistics, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use the tool: 'For operations the unified tools do not cover.' It also provides alternatives for finding native parameter names (pl_list_sources) and lists possible error cases, fully guiding the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pl_get_documentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds substantial behavioral detail: pagination at ~5000 characters, the exact response fields (content, page, total_pages, has_more), the deprecated page_number alias, native pagination for isap, and a concrete extra example for eu-compliance. It also enumerates error types, enriching the transparency beyond the annotation baseline. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph, but every sentence conveys necessary information: pagination, native pagination, extra per source, and error codes. No filler or redundancy. While not broken into bullet points, its length is justified by the tool's complexity. It is front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's high complexity (source-specific IDs, optional extra, pagination modes, error taxonomy), the description covers all functional aspects necessary for correct invocation. It includes edge cases like page_number conflicts, missing required extra, and upstream_error with guidance to search first. An output schema exists, so return-value details are not needed, and the description's pagination overview complements it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description bears full responsibility for parameter meaning. It does this thoroughly: document_id is explained with source-specific formats, page is clarified as 1-based, extra is given a concrete example, and page_number is identified as a deprecated alias. Source is implicitly understood from pl_list_sources. This fully compensates for the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening phrase 'Fetch the full document' uses a specific verb and resource, clearly distinguishing it from sibling tools like pl_search (search) and pl_list_sources (list sources). It also delineates scope by enumerating document types and referencing per-source ID formats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is clear: it explains how to fetch documents and how pagination works, and it references pl_list_sources for document_id format and pl_search for pagination comparability. However, it does not explicitly state when to use this tool versus alternatives or provide 'when not to use' exclusions, though it hints at searching first before fetching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pl_list_sourcesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnly, idempotent, and non-destructive hints, and the description adds valuable context: it explicitly states that the no-arg mode 'reads a local registry, spawns nothing' and that the source_id mode 'connects to that connector (lazy spawn on first use)'. It also discloses possible errors, adding transparency beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and well-structured, using a first-sentence summary followed by mode-specific details and error notes. It is longer than necessary but every sentence provides useful information, making it efficient rather than verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers both execution modes, optional filters, return values (catalog fields, live schemas), side effects (lazy spawn), and error cases. Given the tool's two-mode complexity and the presence of an output schema, this is a complete and self-sufficient description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description thoroughly explains both parameters: source_id triggers connector inspection, group filters by 'pl' or 'eu', and no arguments yields the full catalog. It even illustrates the native_params mapping with an example, fully compensating for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List available Polish/EU legal data sources, or inspect one source's live tool schemas.' It specifies both a verb and resource, and distinguishes it from sibling tools like pl_search and pl_get_document by focusing on discovery and schema inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each mode: 'Without arguments' for the full catalog, 'With source_id' for live schemas, and 'Optional group filter' for narrowing by 'pl' or 'eu'. It also notes the use case for native tools not covered by unified tools, but does not explicitly name alternative tools or when-not-to-use scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pl_searchA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations already declaring readOnly/openWorld/idempotent behavior, the description adds substantial context: parameter normalization (0- vs 1-based pages), clamping of limit, verbatim merging of extra filters, and a comprehensive list of error codes. It discloses deprecated page_number alias and source-specific behaviors beyond the structured hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose sentence, then systematically covers parameter behavior in a compact block, followed by an error list. Every sentence carries operational value, and the length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers parameter semantics, error handling, source adapters, and cross-references to sibling tools (pl_list_sources and pl_get_document) for additional context. An output schema exists, so return-value details are not needed in the description; the provided information is complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% parameter documentation in the schema, the description compensates fully by explaining date format, page numbering, limit clamping, extra native filters, and the deprecated page_number alias. Even though query and source are not individually defined, their meanings are self-evident from the schema and the first sentence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Unified search across any registered source (SAOS, NSA, ISAP, EUREKA, KIO, UODO...),' using a specific verb and resource scope. It clearly distinguishes itself from siblings like pl_get_document and pl_list_sources by focusing on search across sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes when to use the tool ('Unified search across any registered source') and cross-references pl_list_sources for native parameter names, but it does not explicitly state when to avoid this tool or name direct alternatives beyond the sibling tool names. This is clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.