Server Quality Checklist

Tool Scores

• bmlt_get_server_infoA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds context about network call behavior and return data, which is consistent and non-contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a front-loaded purpose, followed by usage, args, and returns. Efficient, though could use clearer section separation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool with one optional param and no output schema, the description adequately covers purpose, usage, parameters, and returns, making it complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter, so the description adds minimal new info beyond restating the default URL. It does provide usage context, but baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves metadata about a BMLT root server, specifically version and geographic center. This distinguishes it from sibling tools which handle formats, meeting details, service bodies, and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists use cases: verifying reachability, checking version, or getting coordinates. It does not explicitly exclude other uses, but the context and sibling tools provide reasonable differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bmlt_get_formatsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior; the description adds context on default URL and response format options (markdown or json), slightly exceeding what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and well-organized with a clear statement, bullet points for use cases, and an Args list. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and annotations cover safety. The description explains the return value as a list of codes, names, and descriptions, which is sufficient for understanding the output. No output schema exists, but the description fills the gap adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description repeats parameter details but adds the default URL value and format options, providing minimal additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves all meeting format codes and descriptions, with specific examples like 'O' for Open. It distinguishes from sibling tools by focusing on formats rather than meetings, servers, or service bodies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage scenarios are provided: discovering format codes before filtering, explaining codes to users, and listing all meeting types. No exclusions or comparisons to alternatives are given, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bmlt_get_meeting_detailsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds that it returns full meeting record with specific fields (name, day, time, etc.), which is useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: 4 sentences total. Front-loaded with purpose, then usage hint, then args/returns. Every sentence adds value with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity, rich annotations, 100% schema coverage, and no output schema needed, the description fully covers purpose, usage, parameters, and return content. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so baseline is 3. Description adds meaning by explaining meeting_id comes from search results, and provides default values and purpose for root_server_url and service_body_ids.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Retrieve full details for a specific NA meeting by its BMLT meeting ID'. It distinguishes from sibling tools like bmlt_search_meetings by specifying this is for getting a complete record after search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this after bmlt_search_meetings to get the complete record for a specific meeting', providing clear context. However, no explicit when-not-to-use or alternatives, but usage is well implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bmlt_get_service_bodiesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds value by specifying return fields (IDs, names, types, URLs, helplines) and providing an example. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections, bullet points, and an example. Every sentence adds value without redundancy. It is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the return list and includes an example. It covers all necessary context for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already describes parameters. The description adds practical guidance by showing defaults and an example (root_server_url). This enhances understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Retrieve all service bodies' and explains their role as organizational units in BMLT. It distinguishes from sibling tools by explicitly mentioning their use in filtering meeting searches via bmlt_search_meetings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists use cases: discovering service body IDs, viewing hierarchy, and finding helpline/website. It provides good context for when to use this tool, though it lacks explicit exclusions or when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bmlt_search_meetingsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds value by explaining the return format ('Formatted list of meetings with name, day, time, location, formats, and links') and query behavior without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening, list of parameters, and examples. It is slightly long due to detailed parameter descriptions, but every sentence adds value. It could be more concise, but it remains clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 11 optional parameters and no output schema, the description covers all parameters with defaults, formats, and examples. It explains the return format and provides multiple usage examples, making it complete for an AI agent to decide when and how to use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage, but the description adds significant value by providing default values (root_server_url, service_body_ids, radius_miles) and clarifying accepted formats (weekdays as names or numbers, start_time as HH:MM, format codes like 'O', 'VM'). Examples tie parameters to real queries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search for Narcotics Anonymous meetings in the BMLT database' and 'Returns a list of meetings matching the given filters.' It distinguishes from sibling tools which focus on getting formats, details, server info, or service bodies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides examples like showing Portland meetings, finding open meetings near downtown, virtual meetings, and morning meetings. It notes that all filters are optional and omitting them returns all meetings for the default service body. It does not explicitly state when not to use or compare to alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.