Server Quality Checklist

Tool Scores

• search_jaish_tsutatsuC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the source (JAISH) and searchable content types, but lacks critical details: it doesn't specify the output format (e.g., list of results with titles/dates/links), pagination behavior, error handling, rate limits, or authentication requirements. For a search tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: the first states the core functionality, and the second adds context on searchable laws. There's no fluff or redundancy, and it's front-loaded with the main purpose. However, it could be slightly more efficient by integrating the law examples into the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (search tool with 3 parameters, no output schema, and no annotations), the description is incomplete. It lacks output details (what the search returns), error cases, and usage guidelines relative to siblings. Without an output schema, the description should ideally hint at the result structure (e.g., 'returns a list of notifications with titles and links'), but it doesn't, leaving the agent uncertain about the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters (keyword, limit, max_pages) with descriptions and examples. The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain keyword matching (e.g., partial/full), default values beyond the schema, or interactions between parameters. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'search for safety and health-related notifications from JAISH using keywords.' It specifies the resource (JAISH notifications) and the action (keyword search), and mentions relevant laws (Labor Safety and Health Act, Pneumoconiosis Act, etc.). However, it doesn't explicitly differentiate from sibling tools like 'search_mhlw_tsutatsu' or 'search_law,' which likely search different sources or content types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'search_mhlw_tsutatsu' (likely for MHLW notifications) and 'search_law' (likely for laws), it's unclear when to prefer this JAISH-specific search over others. There's no mention of prerequisites, exclusions, or comparative contexts, leaving usage decisions ambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_mhlw_tsutatsuC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the search capability and domain areas, it doesn't describe important behavioral aspects like authentication requirements, rate limits, pagination behavior beyond the page parameter, error conditions, or what the search results look like. For a search tool with no annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently convey the tool's purpose and scope. The first sentence states the core functionality, and the second provides domain context. There's no wasted text, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the search returns (formats, structure, fields), doesn't mention authentication or rate limiting, and provides minimal behavioral context. The description should do more to compensate for the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already fully documents both parameters (keyword and page). The description doesn't add any parameter-specific information beyond what's in the schema, such as search syntax, keyword matching behavior, or result ordering. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for administrative notices from the Ministry of Health, Labour and Welfare database using keyword search. It specifies the resource (通達/administrative notices) and verb (検索/search), but doesn't explicitly differentiate from sibling tools like search_jaish_tsutatsu or search_law beyond mentioning the specific ministry scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the types of administrative notices that can be searched (labor standards, employment insurance, safety and hygiene, social insurance), but provides no guidance on when to use this tool versus the sibling tools like search_jaish_tsutatsu or search_law. There's no explicit when/when-not usage context or alternative tool recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_lawB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the API source ('e-Gov法令API v2') which adds implementation context, but doesn't disclose important behavioral traits like rate limits, authentication requirements, error handling, or what the response format looks like (especially critical since there's no output schema). For a search tool with no annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the core purpose. The second sentence adds useful context about when to use it and the API source. Every sentence earns its place, though it could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (format, structure, fields), error conditions, or important behavioral constraints. For a search tool that likely returns complex legal data, this leaves significant gaps for an AI agent to understand how to properly interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema (keyword usage context is implied but not detailed). With high schema coverage, the baseline score of 3 is appropriate - the description doesn't compensate but doesn't need to.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '労働・社会保険関連の法令をキーワードで検索する' (search labor and social insurance laws by keyword). It specifies the resource (laws) and action (search), but doesn't explicitly differentiate from sibling tools like 'get_law' or other search tools. The mention of 'e-Gov法令API v2' adds technical context but not sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: '法令名が分からない場合に使用' (use when you don't know the law name). This implies when to use it (keyword-based search rather than known-name lookup), but doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools. The guidance is helpful but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jaish_tsutatsuB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states what the tool does (retrieve text) and the input requirement. It doesn't describe what happens on failure, rate limits, authentication needs, response format, or whether it's a read-only operation. The description is minimal and lacks important behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - just two sentences that directly state the purpose and usage requirement. Every word earns its place with zero waste or redundancy. It's front-loaded with the core purpose followed by implementation guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what format the text returns in (HTML, plain text, structured data), error conditions, or any behavioral characteristics. While the single parameter is well-documented in the schema, the overall context for using this tool effectively is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single 'url' parameter. The description adds marginal value by reinforcing that URLs should come from 'search_jaish_tsutatsu' and specifying it's for '通達本文' (notification text), but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '安全衛生情報センター（JAISH）の通達本文を取得する' (retrieve the full text of JAISH notifications). It specifies the resource (JAISH notifications) and the action (retrieve text), but doesn't explicitly differentiate from sibling tools like get_law or get_mhlw_tsutatsu beyond mentioning JAISH specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'search_jaish_tsutatsu で取得した url を指定' (specify the URL obtained from search_jaish_tsutatsu). This establishes a workflow dependency, but doesn't explicitly state when NOT to use it or mention alternatives like get_mhlw_tsutatsu for different notification sources.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lawA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the API source (e-Gov法令API v2) and abbreviation support, but doesn't describe error conditions, rate limits, authentication requirements, or what happens when parameters are omitted. For a tool with 5 parameters and no annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that each serve clear purposes: stating the core functionality and providing important implementation context. There's zero wasted text, and the information is front-loaded with the primary purpose stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description is somewhat incomplete. While it states the purpose clearly and mentions the API source, it doesn't address behavioral aspects like error handling, response format, or practical usage constraints that would help an agent invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema - it mentions abbreviation support which relates to the law_name parameter, but doesn't provide additional semantic context about parameter interactions or usage patterns beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('取得する' - retrieve/obtain) and resource ('日本の法令から特定の条文' - specific articles from Japanese laws/regulations). It distinguishes from sibling tools by specifying it retrieves specific articles rather than searching (like search_law) or getting different document types (like get_jaish_tsutatsu).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning the e-Gov API and abbreviation support, but doesn't explicitly state when to use this tool versus alternatives like search_law. It doesn't provide exclusion criteria or clear differentiation from sibling tools beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mhlw_tsutatsuA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions that long notifications may be paginated (implying multi-page retrieval), it doesn't cover other important behavioral aspects like authentication requirements, rate limits, error conditions, or what happens when invalid parameters are provided. The description provides some context but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that each serve a clear purpose: the first states the tool's function, the second explains the parameter requirement. There's zero wasted language and it's effectively front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only retrieval tool with 2 parameters and 100% schema coverage, the description provides the minimum viable information. However, without annotations or an output schema, it should ideally explain more about the return format, error conditions, or pagination behavior beyond what's implied in the parameter descriptions. The description is adequate but has clear gaps in behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description adds minimal value beyond the schema - it mentions that data_id comes from 'search_mhlw_tsutatsu' (implied in schema) and that long notifications may be paginated (implied in schema's page_no description). The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '厚生労働省の通達本文を取得する' (retrieve the text of MHLW notifications). It specifies the resource (MHLW notifications) and the action (retrieve text), but doesn't explicitly differentiate from sibling tools like 'get_jaish_tsutatsu' or 'get_law' beyond mentioning the source agency.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: it requires a data_id obtained from 'search_mhlw_tsutatsu'. This establishes a workflow dependency. However, it doesn't explicitly state when NOT to use it or mention alternatives like the sibling 'get_jaish_tsutatsu' for different agency notifications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.