Server Quality Checklist

Tool Scores

• search_locationsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds context about the volunteer-run service and usage policy, enhancing transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with core purpose, then provides policy guidance. Sentences are efficient and informative, though the warning could be slightly condensed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description mentions output types (IDs, coordinates). It covers service nature and policy. Parameters are fully defined in schema, making the description fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add significant detail about parameters beyond the schema; it only mentions the query purpose indirectly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds stop IDs and coordinates for a station or stop name, using specific verbs and resources. It distinguishes from sibling tools like search_connections and execute_api_action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage warnings and policy links, but does not directly compare to siblings or state when not to use. However, the context is clear and responsible.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_api_actionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds context beyond annotations: mentions support for GET and computational POST, rejection of arbitrary URLs, truncation of large responses, and usage policy. However, the mention of POST operations may cause confusion with the read-only annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with purpose first, then details and warnings. While slightly long, every sentence adds value. Could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers allowed operations, truncation, and usage policy, but lacks information on error handling or response format. Adequate but not exhaustive given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. The description adds value by explaining the purpose of maxResponseBytes (truncation) and acknowledgeHeavyRequest (required for heavy actions), going beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Execute', the resource 'read-only operation from the allowlisted Transitous/MOTIS action catalog', and distinguishes it from sibling tools like search_api_actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly states to use search_api_actions to obtain the action ID and parameter fields. Provides responsible usage warnings and prerequisites, but does not explicitly state when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_api_actionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint; the description adds that it is read-only, what fields are returned, and includes usage warnings about the service. No contradiction with annotations, but could mention error behavior or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, starting with the core purpose and then adding behavioral context and warnings. It is efficient but includes necessary usage guidance without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description details return fields (action IDs, methods, etc.). It includes documentation and OpenAPI links. For a two-parameter tool, this provides adequate context for the agent to understand usage and integration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema (100% coverage). The description adds useful guidance: 'use an empty string to list the catalog' for query, which enhances schema descriptions. No additional hints for limit beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search or list'), the resource ('complete allowlisted Transitous/MOTIS API action catalog'), and the return fields. It distinguishes this tool from siblings like execute_api_action and search_connections by focusing on API action discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the tool's role in discovering actions for use with execute_api_action. It includes responsible usage warnings about Transitous being volunteer-run and asks for contact before resource-intensive use, but does not explicitly state when not to use this tool vs. alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_connectionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and no destructiveness. The description adds value by listing return details (legs, transfers, times, realtime status, tracks, accessibility) and stating limitations (no ticket sales, accuracy not guaranteed). However, it does not explicitly mention that the tool queries an external service or that rate limits may apply, though the usage warning implies responsible usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise, front-loaded with the primary purpose, and every sentence adds value. It efficiently covers purpose, return values, limitations, and usage policy without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (10 parameters, no output schema), the description covers purpose, return values, limitations, and usage policy. While it does not detail the exact output structure, mentioning 'legs, transfers, times, available realtime status, tracks, and accessibility fields' provides sufficient context. The lack of an explicit output schema is compensated by the description, but some agents might benefit from more structured return info.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter already has a description. The tool description does not add additional parameter semantics beyond what is in the schema. Per guidelines, baseline is 3, and no extra information is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Plan public transport journeys between two station names using Transitous/MOTIS.' The verb 'plan' and resource 'journeys' are specific, and the tool clearly distinguishes from siblings like execute_api_action or search_locations which handle generic actions or location lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description provides clear guidance: 'It does not sell tickets or guarantee accuracy.' It also includes a detailed usage warning about Transitous being volunteer-run, best-effort, and intended for FOSS/non-profit use, with specific instructions on User-Agent, caching, traffic limits, and contacting before intensive use. This effectively tells the agent when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.