Server Quality Checklist

Tool Scores

• preview_mobB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It correctly implies a read-only preview operation, but it does not mention side effects, authentication needs, rate limits, or output format. More detail is needed for a complete behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without extraneous information. It is front-loaded and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate given the presence of an output schema (not shown), which likely details return values. However, it omits information about default behaviors (e.g., what happens when seed is null) and does not clarify the relationship between count and seed. Overall, it is minimally sufficient but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It mentions 'count/seed', linking them to the parameters, but does not explain what seed controls (e.g., randomness source) or the meaning of count beyond quantity. It adds basic semantic value but lacks detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Preview') and the resource ('random mob members'), and mentions the key parameters (count/seed). It distinguishes the tool from siblings like 'list_personas' and 'read_report' by focusing on a specific preview of mob generation. However, it could be more explicit about what 'mob members' refers to.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no scenarios where it should or should not be used. It is a minimal statement of function without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_personasA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description implies a read-only operation (listing) and specifies that it covers built-in personas, which is useful behavioral context. However, it does not explicitly state non-destructiveness or any authentication needs. Still, it is clear and not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It directly states the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and the presence of an output schema, the description provides sufficient context for a simple listing tool. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, schema coverage is 100%. The description adds meaning by specifying the scope ('built-in ringleader personas') and the content ('what each one catches'), which is beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list', the resource 'built-in ringleader personas', and the additional information they provide ('what each one catches'). It effectively distinguishes from sibling tools like preview_mob and read_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines are provided. The description does not indicate when to use this tool vs alternatives, nor does it mention any prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_reportA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It declares the tool returns the newest markdown report, which implies read-only behavior. However, it does not specify behavior when no reports exist or potential errors, but the simplicity of the tool limits gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences and 18 words. The main action is front-loaded in the first sentence, and the second sentence adds essential default context with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one optional parameter and an output schema (which defines return structure), the description is largely complete. It mentions the report type (markdown) and connects to run_crowd_test. A minor gap is lack of error handling details, but for a simple read tool this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'report_dir' is optional with default null. The description adds meaning by explaining it specifies 'under a directory' and defaults to '~/crowd-test-reports', which clarifies the parameter's role beyond the schema (which only shows type and default).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool returns the newest crowd-test markdown report under a directory, with a clear verb ('return') and resource ('newest crowd-test markdown report'). It distinguishes itself from sibling tools (list_personas, preview_mob) by focusing on reading a report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating the default directory and noting that run_crowd_test writes there, implying this tool is used to read those reports. However, it does not explicitly state when not to use it or provide alternatives, but the context is sufficient given simple siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.