Server Quality Checklist

Tool Scores

• refresh_taxonomyA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It merely says 're-fetch' without disclosing side effects (e.g., whether it mutates local state, makes external network calls, or blocks). This is minimal behavioral disclosure for a tool that likely has an update-like side effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence communicates purpose and usage guidance with no filler. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is sufficient: it states what the tool does and when to use it. It does not need to explain return values due to the output schema, though it could mention any blocking or performance implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so per the rubric the baseline is 4. The description correctly implies there are no configurable inputs and focuses on the action itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('re-fetch'), resource ('filter taxonomy'), and source ('from the portal'). It distinguishes itself from sibling tools (list_tag_categories, list_tags) which are read/list operations, whereas this is a refresh operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear usage trigger: 'use if tags seem stale.' This provides context on when to invoke this tool, though it does not explicitly contrast with alternatives or mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tag_categoriesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source of behavioral information. It states the tool 'lists' categories and mentions the output includes 'how many options each has,' which implies a read-only operation. However, it does not explicitly mention whether output is sorted, paginated, or otherwise constrained. For a simple listing tool, this is acceptable but not deeply transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two complete sentences with clear front-loading: the first sentence states the core function, the second provides a pointer to a related tool. No wasted words; every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (zero parameters) and the presence of an output schema (per context signals), the description is sufficiently complete. It explains what the tool returns (categories with option counts) and how to drill down further using list_tags. Sibling context further clarifies where this tool fits.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema has no properties with 100% coverage. The description therefore does not need to explain parameter meaning. The baseline for zero-parameter tools is 4, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: 'List the filter categories available on the CDTM portal and how many options each has.' This uses a specific verb and resource, and it distinguishes itself from the sibling tool list_tags by explaining that list_tags is used for seeing options within a category.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a direct cross-reference: 'Use list_tags(category) to see the options.' This implies the intended workflow and alternatives, though it does not explicitly state when not to use this tool. Still, the usage context is clear for a simple list-all tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tagsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the only source of behavioral information. It discloses the case-insensitive filtering behavior and the optional default, adding some value beyond the schema. It does not cover potential error conditions or response characteristics, but for a simple list operation this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a category list, with the main purpose front-loaded in the first sentence. Every sentence adds value; there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, two parameters, and an existing output schema, the description covers purpose, parameters, and behavioral nuances. It could be improved by explicitly linking to list_tag_categories for authoritative category values, but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description fully compensates: it explains the 'category' parameter with enumerated examples and the 'contains' parameter as a case-insensitive substring, including its optionality. This gives an agent complete parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific action ('List available tag names') on a defined resource ('filter category'), with an optional filter. This clearly distinguishes it from sibling tools like list_tag_categories, which handle categories rather than tag names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool by enumerating valid categories and noting the optional substring filter. However, it does not explicitly mention alternatives or when not to use it, such as cross-referencing list_tag_categories for available categories.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_membersA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses that tag arguments are resolved against the portal's taxonomy, returns member cards with specific fields plus totalCount/totalPages, and explains pagination via page parameters. It doesn't cover auth, rate limits, or sorting, but provides strong behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (about 90 words) and front-loaded with the primary purpose. Each sentence adds value: scope, taxonomy resolution guidance, and return/pagination details. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 10 parameters, the description covers the essential aspects: what it searches, how tag filters work, how to get valid tag names, what the response contains, and pagination. It omits sorting and result limits, but overall it's sufficiently complete for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains that all tag arguments take human-readable names resolved against taxonomy, gives an example using skills, states combining filters narrows results, and mentions page pagination. This covers the key parameter semantics effectively, though it doesn't enumerate each filter individually.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches CDTM community members (students + alumni), using a specific verb and resource. It distinguishes itself from sibling tools like list_tags and get_member_profile by focusing on member search and result cards.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance to use list_tags() first when unsure of exact tag names, and notes that combining filters narrows the search. It doesn't explicitly contrast with get_member_profile for single-member lookups, but the context makes the search-versus-profile distinction clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_member_profileA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of explaining behavior. It discloses that the tool returns the page's text content (bio, tags, links, etc.), which is useful behavioral information. It does not explicitly state that the operation is read-only or discuss error handling, but 'Fetch' implies a non-destructive read.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the core purpose, and includes a useful example without excess. Every sentence contributes value, and the structure is clean.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has a single required parameter and an output schema, the description sufficiently covers the input format, the source of valid inputs, and the nature of the returned content. It is complete for a simple fetch operation, with no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only specifies the parameter name and type, so the description adds critical meaning by explaining that 'profile' can be a profile URL or a UUID from search results, with a concrete example. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and resource ('member's full profile page'), and distinguishes itself by explaining the input can be a profile URL or UUID from search results. This clearly separates it from sibling tools like search_members, which find members rather than fetch a full profile.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool—after obtaining a profile URL or UUID from search results—and includes an example. However, it does not explicitly mention when not to use it or name alternative tools, so it stops short of fully explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.