Server Quality Checklist

Tool Scores

• searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and not destructive, so the description adds behavior like sorting by date descending and error types ('invalid_filter', 'upstream_error'), which is helpful context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat long but every sentence adds value. It front-loads the database name and document counts, then details filters, sorting, and errors. Could be slightly more concise, but no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters with 100% schema coverage, no output schema, and no nested objects, the description adequately explains usage context including search behavior, sorting, and errors. It covers the complexity well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions, but the tool description adds critical context like prefix matching for signature, the meaning of full_phrase, and that category_ids come from list_categories. This provides value beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the EUREKA database of Polish tax documents, specifies the document types and counts, and distinguishes from siblings like search_by_signature which is signature-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied for general text search with filters, but there is no explicit guidance on when to use this vs siblings like get_interpretation or list_categories. No when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_categoriesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond the annotations by specifying the return format (dictionary with 28 items) and potential error type ('upstream_error'). The annotations already cover read-only, idempotent, and non-destructive behavior, so the description complements them well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence stating the purpose, usage hint, and error note. No superfluous information. It front-loads the key action and is well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description covers all essential information: what is returned (dictionary, size 28, content examples), how to use the result (in search), and error handling (upstream_error). It is fully complete for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so the description's explanation of the output (dictionary mapping IDs to names) and its purpose (feeding into search) adds meaning. With zero parameters, the baseline is 4, and the description fulfills this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a dictionary of EUREKA document categories (id to name) with 28 items, listing examples. It specifies the verb 'returns' and the resource 'categories'. While it does not explicitly distinguish from sibling tools, it mentions usage in search, providing context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: to obtain category IDs for the search tool's category_ids parameter. This gives clear context. However, it does not mention when not to use it or provide alternatives, so it misses explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_interpretationA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds valuable behavioral details: it returns only the first 3000 characters of the full content (truncation), HTML is stripped to text, and error conditions are listed. This goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and input, then details output and errors. Every sentence adds necessary information with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description covers the essential aspects: what it does, what input it expects, what it returns (including truncation), and possible errors. The absence of output schema is compensated by describing return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (one parameter with description). The description adds a format hint (numeric ID can be string or number) and ties it to the concept of ID_INFORMACJI from search results, but the schema already covers meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a full EUREKA document by ID, listing specific returned fields (metadata, thesis, first 3000 chars of content). It distinguishes itself from sibling tools (search returns summaries, not full documents) by specifying the input source (from search results).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the tool is used after obtaining an ID from 'search' results, and lists three error types (missing_arg, not_found, upstream_error). It does not explicitly mention when not to use it, but context implies it's for full document retrieval after search, which is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_signatureA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint. Description adds that it is a shortcut and lists possible errors, which provides some additional context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise two-sentence description that is front-loaded with purpose and includes equivalent tool reference and error types.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, description covers purpose, usage, and errors adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. Description adds meaning by explaining 'full or prefix' and providing an example parameter value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches for documents by signature (full or prefix). Distinguishes from sibling 'search' by being a shortcut with signature parameter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says it is equivalent to search with signature parameter, providing clear use context. Mentions error types but does not explicitly state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.