Server Quality Checklist

Tool Scores

• get_state_summaryA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing the tool's behavior. It states what data is returned (totals for systems, branch outlets, collections, usage) but gives no details about output format, error handling, data sources, or any constraints on state_code. This is a minimal disclosure for a read operation with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with a clear one-sentence purpose followed by a straightforward parameter table. Every sentence earns its place, and the format makes the key information easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is mostly adequate, but it leaves gaps. It explains what is returned (totals of specific metrics) but does not describe the return value's structure, which would be helpful given the absence of an output schema. It also doesn't mention any limitations or edge cases, making it complete only at a high level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the lack of parameter documentation. It does this by defining state_code as a 'Two-letter state abbreviation' with an example ('IL'), which adds crucial meaning beyond the bare type:string in the schema. The single parameter is well-explained, though it doesn't cover case sensitivity or valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get statewide public library totals' and specifies the exact metrics (systems, branch outlets, collections, and usage) and scope (one state). This specific verb+resource combination distinguishes it from siblings like get_library or get_outlet, which target individual entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates this is for statewide aggregate data, which implicitly tells the agent to use this when needing totals rather than individual library or outlet details. However, it lacks explicit when-not-to-use guidance or named alternatives, so it doesn't fully distinguish from related search/find tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_libraryA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a read-only 'Get' operation and lists returned data categories, but does not mention error behavior, prerequisites, or limitations. It adds some context but is not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two-sentence description plus a one-line parameter doc. Front-loaded with purpose, no redundant text. The example is concise and useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  One-parameter tool with no annotations and no output schema. The description lists the five data categories returned (collections, staffing, visits, programs, finances), which is good context, but doesn't specify response format or error cases. Adequate for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only type 'string' for fscs_id with 0% description coverage. The description's Args section explains it is an IMLS FSCS ID and provides an example ('IL0021'), adding essential semantic value that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool retrieves a full profile for a library system by FSCS ID, with specific verb 'Get' and resource 'library system'. Lists included data categories (collections, staffing, visits, programs, finances), distinguishing it from siblings like get_state_summary and get_outlet.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when you have an FSCS ID, and lists what the profile includes, but does not explicitly discuss alternatives or exclusions. This is clear context for a direct get-by-ID tool, though explicit comparisons to search_libraries or find_* tools would strengthen it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_librariesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It adds useful semantics (full-text vs. substring matching, limit/offset defaults, two-letter state format) and clarifies that it searches systems, not outlets. However, it does not describe the return format or how multiple filters are combined.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose, immediately defines the key term 'system', and uses an organized Args list. Every sentence provides necessary information with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 params, no annotations, no output schema), the description covers the core semantics and constraints well, including the 'at least one' requirement. It omits the response structure and filter combination behavior, but overall is sufficient for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description fully compensates by explaining every parameter: name (full-text search), state (two-letter code), city (substring match), limit (1-100, default 20), offset (default 0). This is rich, exact information that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool 'searches public library systems' with specific filters (name, state, city). The definition of a system vs. branch distinguishes it from sibling tools like 'get_outlet', and the scope is specific and action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a clear usage constraint ('Provide at least one of name, state, or city'), but does not explicitly mention when to use this tool over siblings like 'find_libraries_near_address' or 'find_libraries_near_coordinates'. Alternative selection is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_libraries_near_addressA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses key behavioral details: results are sorted by distance, temporarily-closed branches are excluded by default, and the include_closed flag allows including them. It stops short of describing error handling or exact return shape, but this is reasonable for a read-only lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single-sentence purpose followed by a clean Args list. Every sentence adds value, including ranges and defaults, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Parameter coverage is excellent, and the purpose is clear. However, with no output schema, the description does not explain what fields each returned branch contains (e.g., name, address, distance, hours). It also does not provide guidance on when to use this vs. find_libraries_near_coordinates, which is a notable gap given the sibling tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema itself provides no parameter descriptions (0% coverage), but the tool description fully compensates: each parameter is documented with type, range or default, and address includes a concrete example. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and resource ('public library branches'), and clearly states the geographic scope ('nearest to a US street address') and sorting behavior. This distinguishes it from siblings like find_libraries_near_coordinates and search_libraries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates this tool is for US street address-based lookups, which establishes the appropriate context. It does not explicitly name alternatives or exclusions, but the use case is clear and unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_outletA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It says 'Get' which implies a read-only operation and lists included fields, giving response expectations. However, it does not mention error behavior (e.g., missing ID) or auth requirements, which are minor for a get tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose in one sentence, followed by an Args block that is one line. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-ID tool with one parameter, the description covers what it does, what the argument is, and what the response includes. The sibling tools provide context for when to use this one. No output schema, but the description's mention of fields is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only has 'outlet_id' as a string with no description (0% coverage). The description adds meaning by defining it as 'IMLS outlet ID', which tells the agent what kind of value to supply. This is a meaningful addition beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Get the full record for one library branch (outlet) by its ID' and lists the record content. This distinguishes it from sibling search/get tools by focusing on a specific outlet ID retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you have an outlet ID and need the full record. It does not explicitly mention alternatives or when not to use, but the context of siblings (search, find near) suggests this is the direct lookup. There is no clear exclusion, but it doesn't name alternatives, hence 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_libraries_near_coordinatesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses sorting by distance, radius/limit bounds, and include_closed default. It also notes performance (skips geocoding). However, it omits edge-case behavior (e.g., no results) and any error conditions, preventing a top score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is stated in one sentence, usage guidance in one sentence, and parameters are listed in a clean, scannable format. Every word contributes; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with no output schema, the description provides the core behavior and all parameter constraints. It does not specify the exact structure of returned branches, but this is a minor gap given the tool's simple purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All five parameters are explained with types, units (WGS84, miles), ranges (0.1–50, 1–100), and defaults. Given schema description coverage is 0%, this fully compensates and adds meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds public library branches nearest to coordinates, sorted by distance. It distinguishes itself from sibling find_libraries_near_address by noting it skips geocoding and is faster, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use this tool when coordinates already exist and directly compares to the alternative, providing clear when-to-use and why. This is a model example of usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.