Server Quality Checklist

Tool Scores

• get_facility_detailA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and idempotentHint=true, indicating a safe, idempotent read operation. The description adds transparency by disclosing partial name matching and best match behavior, which is beyond what annotations provide. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the tool's purpose and what it returns, followed by a brief note on matching behavior and usage context. No fluff; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one simple parameter, rich annotations, and a clear description of return values (even without an output schema), the description is complete. It covers purpose, behavior, usage context, and data details sufficiently for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only one parameter 'name' with full schema description coverage (100%), so the baseline is 3. The description in the schema already explains the parameter well. The main description adds little beyond the schema for parameter meaning, but it does mention partial matching behavior which is marginally useful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves the full profile of one specific treatment facility, listing specific attributes (address, phone, programs, insurance, SAMHSA status, browse URL). It distinguishes from sibling tools by explicitly saying to use after search_facilities for drilling into a named facility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions when to use: 'Use after search_facilities when the user wants to drill into a named facility.' It also notes partial name matching and best match behavior. While it doesn't explicitly list when not to use, the guidance is clear and contextual.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_facilitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and idempotentHint=true. The description adds behavioral context by specifying the search scope (state required, optional other filters) and what the response includes (matched facilities with fields listed). It does not contradict annotations and adds useful information about the tool's non-destructive, idempotent nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences covering purpose and usage plus a one-sentence cross-reference to sibling tools. Front-loading the key action and output, with zero filler. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters (1 required) and no output schema, the description is sufficiently complete. It explains the tool's purpose, output fields, required state parameter, and links to sibling tools. However, it does not mention default limit behavior (schema covers this) nor pagination, but those are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for all 5 parameters. The description summarizes the parameter usage but does not add significant new meaning beyond the schema. Baseline 3 is appropriate as schema does the heavy lifting; the description provides high-level context without enriching parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a curated SAMHSA-sourced directory of 12,338 US addiction treatment facilities by state, city, treatment type, and insurance, and lists what it returns (name, city/state, programs, insurance, phone, browse URL). It also explicitly distinguishes from siblings by directing to get_facility_detail for full profiles and list_states for location coverage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Search the directory... Use this to find candidates — then call get_facility_detail for one facility's full profile.' It also tells when to use alternatives: 'If the user is uncertain about location coverage, call list_states first' and implicitly references get_treatment_types for treatment type uncertainty.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_treatment_typesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint. Description adds that it returns markdown-formatted explanations, providing additional behavioral detail beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states core purpose and output, second provides usage guidance. No extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully explains what the tool returns (definitions, types, format) and when to use it. No missing details given zero parameters and no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100% (empty). Baseline for 0 parameters is 4. Description properly omits parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the specific verb 'Get' and resource 'definitions of all treatment program types', listing example types and output format. Distinguished from siblings which deal with facilities, states, and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use: when uncertain about treatment_type for search_facilities or when user asks about differences between types. Provides clear context for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_statesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent; description adds value by specifying output structure (array of {state, stateAbbr, count}) and coverage context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded purpose and immediate use guidance, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a zero-parameter tool with good annotations; covers purpose, usage, and output format without gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters; baseline score of 4 applies as description does not need to add param info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists US states with treatment facility data and per-state counts, distinguishing it from sibling tools like search_facilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using it as a first step when location is unclear or before calling search_facilities, providing clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.