Server Quality Checklist

Tool Scores

• saos_get_judgmentA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a notable side effect: in local mode, it saves an .md archival file. It also explicitly details the return content (metadata, verbatim sententia, full justification). Given no annotations exist, this is valuable behavioral disclosure, though it leaves 'local mode' and permissions unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main action and followed by concrete details about returns and side effects. There is no filler or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter retrieval tool with no output schema and no annotations, the description adequately covers what is returned and the side effect. It leaves a gap in explicitly linking to the search workflow, but the schema parameter description fills that gap, overall making the description complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single 'id' parameter, which already explains the ID comes from search results. The description only adds the redundant 'po ID' phrase, so it contributes no additional semantic meaning beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Pobiera' (retrieves) and resource 'pełne orzeczenie z SAOS' (full judgment from SAOS), scoped by ID. This clearly distinguishes it from siblings such as saos_search_judgments (search) and saos_list_courts (list courts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention that it should be used after obtaining an ID from saos_search_judgments, nor does it address exclusions or alternatives. Usage context is only implied by the schema parameter description, not the description text.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• saos_search_judgmentsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behavioral traits: exact phrases require quotes, unquoted queries become AND across words causing false hits, keyword is an editorial taxonomy independent of content, and judgmentType=SENTENCE targets substantive judgments. This goes beyond the schema and provides valuable operational nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (golden rule, strategy, examples) and front-loaded with the main purpose. Although somewhat long, each section earns its place by offering actionable guidance for a search tool with 10 parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex search tool (10 params, no output schema), the description covers the most critical parameters: query semantics, keyword behavior, judgmentType, ccCourtType, caseNumber, and date narrowing. It also provides troubleshooting. It omits minor details like pagination or judgeName, but those are self-explanatory from the schema, so the description is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds substantial semantic value by explaining how to format queries (quoting), the meaning of keyword as a curated taxonomy, and how to combine parameters for better precision. Examples illustrate query+keyword and query+judgmentType combinations, exceeding baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Wyszukuje orzeczenia sądowe w bazie SAOS' which is a specific verb+resource (searches court judgments in SAOS) and adds sort order. This clearly distinguishes it from sibling tools like saos_get_judgment (retrieval) and saos_list_courts (court list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context and a detailed layered strategy for when to use quotes, combine query with keyword, and add judgmentType or ccCourtType. It also includes concrete examples and troubleshooting advice. However, it does not explicitly contrast with alternative tools, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• saos_list_courtsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses the output structure (ID, name, type) and the fact that it returns a list. Since it's a simple read-only list operation with no side effects, this is sufficient. It doesn't mention auth or rate limits, but they are not critical for this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, focused sentences. Every word earns its place, providing the return type and usage context without any fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters, no annotations, and no output schema. The description fully covers what is returned and when to use it, making it complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema coverage is 100% and the baseline is 4. The description adds no parameter details, but none are needed since no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a list of common courts with ID, name, and type, using a specific verb and resource. It also differentiates from siblings by explicitly mentioning its role in providing court IDs for filtering in saos_search_judgments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit usage guidance: 'Use to find the court ID for filtering in saos_search_judgments.' This tells exactly when to use the tool and references a sibling tool, making the context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.