Server Quality Checklist

Tool Scores

• get_train_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core read behavior and the ability to see normal operation, but it does not mention that onlyTroubled defaults to true (so by default only troubled lines are returned) or describe the output format. The parenthetical about normal operation could be ambiguous without schema details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. It front-loads the main purpose (area-based operation status retrieval) and then adds the line-filter detail, making every sentence earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, but with no output schema and no annotations, the description should clarify default behavior (e.g., onlyTroubled defaulting to true) and indicate what the response contains. It also does not position itself among siblings. These gaps make it adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions the 'line' filter and normal status, but it largely restates what the schema already says (e.g., area-based retrieval, line filter). It adds little beyond the schema parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb '取得する' (retrieve) with a clear resource '鉄道の運行情報（遅延・運転見合わせ等）' and a clear scope 'エリア別'. It also mentions line filtering and normal-operation detection, which clearly distinguishes it from sibling tools like search_station, search_route, and get_timetable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool to get railway operation status by area, optionally filtered by line. However, it does not explicitly state when not to use it or mention alternatives (e.g., use get_timetable for schedules), so it misses full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_routeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does state the output data (time, duration, fare, transfers, routes), and explains the mode parameter's effect. However, it does not explicitly confirm the operation is read-only (though '検索' implies it), nor does it describe error handling, whether multiple route options are returned, or pagination. These omissions leave some behavioral aspects unclear, but the core behavior is adequately conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the core action and output summary. Every sentence contributes essential information: the first describes purpose and return data, the second clarifies mode usage. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 12 parameters and no output schema, the description provides a strong foundation by listing the key return fields and special mode options. It does not mention the boolean filters (useExpress, useAirplane, etc.) or sorting variants, but these are well-described in the schema. For an agent to correctly invoke the tool, the combination of schema and description is sufficient, though more context on default behavior (e.g., current time defaults) would round it out.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, so the baseline is 3. The description adds little beyond this, except a natural-language restatement of the mode parameter's special values (first/last train). No additional parameter meanings or format details are introduced, so it neither improves nor degrades the schema information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states '出発地から目的地までの電車経路を検索する' (search train routes from departure to destination), a specific verb and resource. It further distinguishes itself from siblings by detailing the outputs: '時刻・所要時間・運賃・乗換回数・各区間の路線と番線を返す' (returns time, duration, fare, transfers, route and platform), clearly separating it from station search or timetable tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly defines the tool's context (route search between two points) and adds specific guidance for special modes: '終電は mode=last_train、始発は mode=first_train' (last train via mode=last_train, first train via mode=first_train). However, it does not explicitly name alternative tools or exclusion criteria, so it falls short of a 5. The context is clear enough for an agent to know when to select it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_stationA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that station codes are returned, and additional route/direction info with groupId is only returned when the candidate narrows to one. However, it does not explain behavior for multiple or zero matches.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy, front-loaded with the primary purpose and followed by the conditional behavior and usage context. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers the purpose, conditional behavior, and usage context. It could be improved by describing the response format for multiple candidates, but it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the query parameter with an example ('駅名（例: 新宿）'), and the description does not add significant new information about parameter syntax or format. Per the rubric, high schema coverage (100%) yields a baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching for station codes from station names. It also specifies the conditional return of route/direction lists with groupId, distinguishing it from sibling tools like search_route and get_timetable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states that this tool is used as a pre-stage for get_timetable, providing clear usage context. It does not explicitly exclude alternatives like search_route, but the role is obvious enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_timetableA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals a non-obvious behavior: ambiguous line/direction leads to an error containing a candidate list, prompting the user to refine. It does not mention rate limits or return format, but for a read-only timetable lookup, the disclosed ambiguity handling adds meaningful insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences. The first sentence states the core purpose, and the second explains the error-handling behavior. There is no redundant or extraneous information, and the structure is front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must provide sufficient context. It covers the core function and the key ambiguity error behavior. The schema itself references search_station for station codes, which mitigates the need for the description to do so. Overall, it is fairly complete for a straightforward retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all six parameters, including examples and enums. The description adds no additional parameter-specific semantics beyond restating station/line/direction as the main selectors. Since the schema already documents parameters thoroughly, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves departure timetables for a specified station, line, and direction. This is distinct from the sibling tools (search_station, search_route, get_train_status). The verb '取得する' and resource '発車時刻表' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: obtaining timetables with optional line/direction filtering. It also instructs the user to refine and retry when ambiguous line/direction causes an error. However, it does not explicitly mention when not to use this tool or compare it to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.