Server Quality Checklist

Tool Scores

• search_keywordC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only references the underlying API endpoint (e-Gov法令API v2 GET /keyword) but does not disclose behavioral traits such as pagination behavior, result limits, rate limits, or whether the operation is read-only. The keyword search nature implies read-only, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and directly to the point. It avoids unnecessary words and is easy to parse. However, its brevity means it lacks explanatory richness that would make it more helpful, so it does not earn a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and the tool is a search with pagination parameters (limit, offset), the description should explain expected return format or pagination behavior. It does not, and also lacks notes on result ordering or response structure. The tool is minimally complete but leaves gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all four parameters (keyword, limit, offset, law_type) have descriptive text. The description adds no extra meaning beyond what the schema already provides, but it does align with the 'keyword' parameter. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the function: searching for laws containing a specified keyword in the law text ('法令本文内に指定したキーワードを含む法令を検索する'). It specifies the verb (search), resource (laws), and scope (keyword match). However, it does not explicitly distinguish itself from the sibling 'search_laws', so it only partially differentiates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like search_laws. The description only states what it does, with no mention of use cases, exclusions, or comparative advantages. This leaves the agent to infer usage purely from the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_law_revisionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral transparency. It only mentions the API endpoint and does not disclose whether the operation is read-only, what the response format is, whether pagination or rate limits exist, or any error conditions. This is especially lacking for a retrieval tool with potentially complex output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is specific and front-loaded, with the API endpoint added in parentheses for extra context. There is no redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema, no annotations), the description is minimal and does not explain the return value or provide any additional context about the amendment history response. However, the schema covers parameters, and the purpose is clear, so it is minimally viable for a simple retrieval operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all four parameters with 100% coverage, so the description adds no additional parameter semantics. The baseline of 3 is appropriate since the schema does the heavy lifting, and the description does not compensate with extra context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('取得する' - retrieve) and the specific resource ('改正履歴一覧' - amendment history list) and even references the exact API endpoint. This unambiguously distinguishes it from sibling tools like search_laws, get_law_data, or download_law_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention any exclusions, prerequisites, or comparisons with sibling tools, leaving the agent to infer usage solely from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_law_dataA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does mention the API endpoint and JSON output, but it omits critical details such as error handling, rate limits, permission requirements, or the structure of the returned JSON. For a read operation, this is a notable gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the essential action and inputs without unnecessary detail. It is efficiently written and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, leaving the description to provide context. It discloses the return format (JSON) but not the structure or content of the law text. It also lacks any guidance on the optional parameters elm and law_full_text_format, though the schema covers them. As a result, the description is minimally sufficient but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters. The tool description does not add additional meaning beyond what is in the schema; it primarily restates the main parameter. Thus, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's function: retrieving law text (本文) in JSON format using a law ID, law number, or revision ID. It also names the underlying API endpoint, which adds specificity. This distinguishes it from sibling tools like search_laws or get_law_revisions, which handle different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a law ID/number/revision ID and need the law text, but it does not explicitly state when to use this tool versus alternatives like get_law_revisions or get_attachment. No exclusions or alternative conditions are mentioned, leaving the decision to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_law_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does disclose the key behavior: saving to a temporary file and returning its path. However, it omits details about temp file lifecycle (e.g., cleanup), error handling, or authentication requirements, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently packs the action, parameters, file formats, and return behavior without excessive verbosity. It is slightly dense with the API endpoint reference, but every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description explains the return value (temp file path) adequately. However, it lacks mention of error cases, temp file handling after reading, or when to prefer sibling tools, making it adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters clearly. The description reiterates that law_id_or_num_or_revision_id accepts ID/number/revision ID and file_type is the download format, but adds no new meaning beyond the API endpoint path, which is already evident.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads law text in a specified file format (xml/json/html/rtf/docx) and returns the path to a temporary file. This specific verb+resource combination distinguishes it from siblings like get_law_data or search_laws, which are more about structured data or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a file download is needed, and it provides context on required parameters (law ID/number/revision ID, file type). However, it does not explicitly name alternative tools or state when not to use this tool, so the guidance remains implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_lawsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It reveals it wraps the e-Gov Law API v2 GET /laws but does not mention pagination, return structure, rate limits, or any side effects. This is minimal disclosure for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the tool's purpose and key parameters. No redundant information; it is appropriately brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for starting a search but lacks details about the response format or any constraints on the API. Since there is no output schema, the description could have specified the returned fields or pagination behavior, but it doesn't. The tool has 5 parameters, so a bit more context would help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for all five parameters (100% coverage), so the description adds little. It highlights law_title, law_type, and asof, which are already documented, and omits limit/offset entirely. It does not add meaningful usage context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a list of laws (法令一覧) using law name, law type, and as-of date. This verb+resource+parameters distinction separates it from siblings like get_law_data (which retrieves specific law details) or search_keyword (likely full-text search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear context for usage: to search law listings by metadata fields. It does not explicitly mention alternatives or exclusions, but the distinct purpose makes the intended use clear relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_attachmentA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key side effect: saving the file to a temporary directory and returning its path. It also mentions the HTTP GET endpoint, implying a read-only operation. However, it omits details like authentication requirements, file size limits, or cleanup behavior, preventing a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core function, mentions the side effect (temp file + path), and includes the API endpoint in parentheses. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description explains the return value (path) and file types, which is essential. It also indicates the API. However, it does not explicitly clarify how src selects among multiple attachments, and given the tool's simplicity, this is a minor gap. Overall, the context is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters documented in the schema. The description merely restates that law_revision_id is required and does not mention the src parameter at all, adding no significant meaning beyond the schema. Baseline of 3 applies because the schema is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool retrieves attachment files (images, PDF, Zip) for a specified law_revision_id, saves them to a temporary file, and returns the path. It also names the exact e-Gov API endpoint, distinguishing it from siblings like download_law_file by focusing on attachments for a specific revision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an attachment for a given law revision is needed, but it does not explicitly explain when to use this tool instead of alternatives such as download_law_file or get_law_data, nor does it provide any exclusions. This is a clear implied usage but lacks explicit differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.