Server Quality Checklist

Tool Scores

• explain_regexA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description discloses important behavioral traits: the tool never executes the pattern, returns a parsed boolean, returns an error rather than throwing an exception on unparseable input, and provides the parser's best guess at the problem. This adds real context beyond the readOnly/idempotent hints and contains no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args, Returns, Examples, and Error Handling sections, and it front-loads the core purpose. It is slightly redundant with the schema in the Args section, but the overall organization makes it easy for an agent to parse and apply.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one parameter and no output schema, the description fully compensates by specifying the exact JSON return shape, the meaning of each field, error behavior, and a concrete usage example. An agent has enough information to call the tool correctly and interpret its result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description's Args section essentially repeats the schema: pattern string, 1-1000 chars, no delimiters. It adds no new semantic information beyond what the input schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Parse a regular expression into a real syntax tree and describe in plain English what it matches.' It also clearly differentiates the tool from running a regex by explicitly saying it 'never executes the pattern, only analyzes its source,' which distinguishes it from execution-oriented tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use guidance: 'Use this to sanity-check a regex... before shipping it' and includes both a 'Use when' example and a 'Don't use when' exclusion. It does not explicitly name check_redos_risk as the alternative for security risk assessment, so it misses the full when/when-not/alternatives pattern, but the context is otherwise clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_redos_riskA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive. The description adds critical behavioral context: the tool never executes the pattern, is safe on untrusted or malicious input, returns errors rather than exceptions on parse failure, and is heuristic with possible missed risks. This is significant added transparency with no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headings for purpose, usage, arguments, return format, examples, and error handling. Every section contributes useful operational information and the purpose is front-loaded within the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since no output schema exists, the description provides the full JSON return structure, error behavior, and usage boundaries, making the tool actionable. Minor ambiguity about regex flavor is acceptable given the structural check nature, and the description is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: the pattern parameter is fully described with type, min/max length, and an example in the schema. The description repeats the 'without delimiters' point but adds no new semantic meaning beyond what the schema already provides. Baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: statically analyzing a regex's structure for the two classic causes of catastrophic backtracking (nested quantifiers and ambiguous alternation). This clearly distinguishes it from a generic regex explainer like the sibling explain_regex, even without naming it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when' scenarios (generated regex before use, reviewing untrusted regex) and a clear 'Don't use when' caution that it is a heuristic structural check, not a formal verifier. This gives the agent clear selection criteria and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.