Server Quality Checklist

Tool Scores

• get_legal_precedentB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says 'Fetch' (implying read-only) but does not confirm if authentication is needed, what happens on invalid input, or any rate limits. No mention of return type beyond 'complete text and metadata'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the action ('Fetch') and specifies the resource and identifiers. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description mentions 'complete text and metadata', which gives a hint but is not detailed. It lacks information on pagination, error handling, or whether both parameters are simultaneously usable. Sufficient for a simple fetch but could be improved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter described minimally. The tool description summarizes the two parameters as 'by its signature or slug' but adds no new semantic information. Baseline 3 is appropriate since the schema already covers meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the complete text and metadata of a legal ruling using a signature or slug. It distinguishes itself from the sibling 'search_legal_precedents' (which is for searching) by specifying the exact identifiers needed for retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_legal_precedents'. It does not mention prerequisites, such as needing a known signature or slug, nor does it indicate when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagnose_orchestrationA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It does not state whether the tool modifies data, requires special permissions, or how it handles invalid inputs. The term 'diagnose' suggests a read-only operation, but this is not explicitly confirmed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that immediately conveys the action and purpose. No redundant words or information, front-loaded with the verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter, no output schema, and no annotations, the description adequately covers the core purpose. However, it lacks details on return format, error handling, or edge cases, which would make it more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage for the single parameter 'id' with a clear description. The tool description adds no further meaning beyond 'Report UUID to diagnose', which is already in the schema. Baseline 3 is appropriate as the schema already does the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Analyzes', the resource 'generated sections of a report', and the dual purposes 'diagnose Chyłka AI progress or find AI fallback errors'. It effectively distinguishes from sibling tools like get_report or list_reports, all of which serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The context implies it is for diagnostic purposes, but there are no 'use when' or 'do not use when' statements, nor mention of prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reportA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It implies a read operation ('Fetch full details') but lacks details on authentication, rate limits, side effects, or what 'full details' includes. It is adequate for a simple fetch but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, no wasted words. Front-loaded with verb and resource. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter fetch tool, the description is mostly complete. It states the purpose and access method. Lacks output schema, but given low complexity, this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (one parameter 'id' with format/pattern described). The description adds no extra meaning beyond the schema. Baseline 3 is appropriate as schema does the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetch full details'), the resource ('ZomPP report'), and the method ('by its ID'). It distinguishes from siblings like 'list_reports' (likely a list without full details) and 'get_legal_precedent' (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings like 'list_reports' or 'search_legal_precedents'. It does not mention prerequisites or conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_legal_precedentsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool supports full-text search across specified courts, but does not mention read-only nature, authentication needs, rate limits, or what happens on no results. Basic transparency but lacking depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose. No unnecessary words; every sentence earns its place. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks information about what the tool returns (e.g., list of matching precedents with metadata). Given no output schema, the description should at least hint at the return format. Also, no mention of pagination beyond limit parameter. Incomplete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds no parameter-specific meaning beyond the schema; 'supports full-text search' aligns with the query parameter but does not elaborate on format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches a database of legal precedents, specifically highlighting Supreme Court (SN) and CJEU (TSUE) rulings, and confirms full-text search capability. This effectively distinguishes it from siblings like get_legal_precedent and list_reports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching legal precedents but provides no explicit guidance on when to use this tool versus alternatives like get_legal_precedent for retrieving a single precedent. No when-not-to-use or prerequisite conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_reportsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description bears full burden. It correctly implies a read-only list operation but does not explicitly confirm non-destructiveness. It mentions pagination via the limit parameter but lacks details on ordering, empty results, or response format, which are acceptable for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with only two sentences, front-loading the key purpose and pagination instruction. Every word earns its place; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is adequate for a simple list tool. It covers the main action and pagination but omits details like default ordering and result structure. Could be more complete with response hints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description adds a hint to use limit for pagination, which is valuable, but does not mention the status parameter. This provides marginal extra value beyond the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'ZomPP reports', with the Polish term in parentheses for clarity. It also mentions pagination using the limit parameter, which differentiates it from the sibling tool get_report that retrieves a single report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when listing reports is needed and mentions pagination, but it does not explicitly contrast with sibling tools like get_report or search_legal_precedents. There is no guidance on when not to use this tool, such as for single report retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.