Server Quality Checklist

Tool Scores

• list_recentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full behavioral disclosure burden. It does reveal the ordering behavior (newest publication date first), but it says nothing about limit semantics, pagination, response shape, or whether all dispositions are included. This is insufficient for a tool with no annotation safety net.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single Japanese sentence with no filler, front-loading the core behavior. Every word earns its place and the structure is appropriately sized for such a simple list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description is the only documentation, and it is too sparse to fully support invocation. It leaves unanswered questions about the limit parameter, the nature of the returned dispositions, and how this tool relates to its siblings. It is minimally viable but clearly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter, 'limit', with 0% description coverage. The description never mentions this parameter, so an agent must rely entirely on the parameter name to infer its meaning. The description adds no semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('返す' / returns) and resource ('処分' / dispositions) with an explicit ordering key (公表日, newest first). This is clear and not a tautology, though it does not explicitly distinguish itself from sibling tools like search_cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a usage context: retrieving dispositions ordered by newest publication date. However, it gives no explicit guidance on when to prefer this tool over search_cases, get_case, or stats, and no exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_caseA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full behavioral disclosure burden. It only states the basic read operation and gives no details about what happens when the id is not found, what fields are returned, or any permissions/rate limits. For a simple getter this is a minimal but not transparent description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no filler. It front-loads the essential information: the operation, the resource, and the selection method. Every word contributes to the meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter get-by-id tool, the essential invocation information is present: the id is required and used to fetch a single case. The lack of output schema and error behavior is a minor gap, but given the tool's simplicity, the description is largely adequate for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter, id (string), with 0% description coverage, and the description says 'id で' (by id). This adds little beyond the parameter's name, which already implies it is the identifier. It does not describe id format, constraints, or how to obtain a valid id, so it fails to compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '1件の処分を id で取得する' states a specific action ('取得する' = get), a resource ('処分' = case/disposition), and the singular scope ('1件'). It clearly differentiates from sibling tools like search_cases (search), list_recent (list), and stats (aggregate), since it is the only one that fetches one item by id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use this tool when you have a specific id to fetch one case. However, it does not explicitly state when not to use it or name alternatives, despite the sibling tools making the intended contrast obvious. This is implied usage guidance, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_casesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a key constraint: only the Consumer Affairs Agency's published content and source URL are returned, not full orders or metadata. It also enumerates filterable fields. It does not mention sorting, pagination behavior, or what happens with no filters, but the search semantics are clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences with front-loaded verb and resource. The first sentence defines the capability; the second defines the output contract. No redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter search tool with no output schema, the description covers the tool's purpose, filters, and return format. Combined with the schema's parameter descriptions, an agent can call it. Remaining gaps (result ordering, filter combination semantics, behavior with no query) are not addressed, but they are secondary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 83%, so the schema already documents most parameters (dates, limit, query, provision). The description restates query/provision filters but adds no new parameter-level detail beyond the schema, and it omits the order_type filter entirely. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action (search) and a precise resource (orders under the Premiums and Representations Act), and clarifies the searchable fields. It implicitly differentiates from siblings get_case/list_recent/stats by framing itself as a filtered search that returns only published content and source URLs. However, it never names the sibling alternatives, so differentiation is indirect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to find cases by business name, product name, provision, or text and only the published content plus source URL is sufficient. It does not explicitly state when not to use it or mention alternatives like get_case for full details or list_recent for recent items, leaving routing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• statsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden but only states that it returns a breakdown. The verb '返す' implies a read-only aggregation, but there is no mention of side effects, authentication, data scope, or edge cases such as empty results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Every element (breakdown, grouping dimensions) earns its place and nothing is redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter stats tool, the description tells the agent what the tool returns and how data is grouped, which is the core information needed. It lacks an explicit output shape, but with no output schema the description still communicates the essential return semantics clearly enough for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the input schema already fully covers parameters (vacuously). The description adds useful context about the output dimensions, which is more than required for parameter semantics; baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('返す' returns) and resource (件数の内訳, a breakdown of counts) with explicit grouping dimensions (year, disposition type, article). This clearly distinguishes it from siblings that retrieve individual cases or case lists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for aggregate statistics rather than case retrieval, so an agent can infer when to use it. However, it does not explicitly name conditions, exclusions, or alternatives compared to siblings like search_cases or list_recent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.