Server Quality Checklist

Tool Scores

• search_routeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the description need not restate safety. It adds valuable behavioral context by explaining that station names require language conversion, providing concrete examples, and summarizing option meanings (e.g., timeType, ticket). This goes beyond annotations and helps the agent understand input requirements, though it does not cover all edge cases like error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured with clear sections (IMPORTANT, conversion tables, examples, options summary). Each section serves a distinct purpose: no fluff, and the conversion information is essential for correct use. It is front-loaded with the core function, though it could be marginally tighter by merging redundant option explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 19 parameters and no output schema, the description compensates by covering critical operational details: language conversion, option semantics, and examples. It does not explain the result structure, but that is not required without an output schema. For a complex tool, this description is reasonably complete, though it could mention nonexistent station handling or API limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with every parameter described, so baseline is 3. The description adds significant value beyond schema by offering conversion tables (English/Chinese → Japanese kanji), examples for via stations, and human-readable translations for enum values (e.g., departure=出発). This enhances parameter understanding and helps agents use the tool correctly, especially for station names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Search train routes between stations in Japan using Yahoo! Transit,' which clearly states the verb (search), resource (train routes), and geographic scope (Japan). It distinguishes itself by specifying the Japanese rail context, making its purpose unambiguous even without sibling comparisons.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Station names MUST be in Japanese kanji/kana. Convert before calling' followed by detailed conversion tables and examples. It clearly specifies when to use the tool (for Japan train routes) but does not explicitly discuss when not to use it or mention alternatives, as there are no sibling tools. This is strong contextual guidance but stops short of the full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.