Server Quality Checklist

Tool Scores

• lint_promptA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions no LLM call and lists check categories, but does not disclose return format, side effects (none expected but not stated), or any required privileges. The behavior is partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and examples, no redundant information. Every word earned its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (3 params, no nested objects), the description covers the what and broad categories but lacks information about output format (e.g., list of issues). With no output schema, return value should be described for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds marginal value by explaining how draft enables comparison rules and context provides background, but the prompt parameter is simply restated. No additional formats or constraints beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks prompts for common issues, listing specific categories (unresolved placeholders, suspect acronym expansions, leaked meta-commentary). It distinguishes from sibling tools as a linting tool, with a clear verb and resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for pre-checking prompt quality before finalizing, but does not explicitly state when to use vs alternatives or when not to use. The 'No LLM call' hint is useful but lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimize_promptA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It states the tool uses a local LLM (free, no external cost) and modifies the prompt, but it does not disclose potential resource usage, rate limits, or side effects. The description is adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and front-loaded. It earns its place, but could be slightly more informative without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 11 parameters and no output schema or annotations. The description does not explain return values, the optimization process, or when to use specific parameters. It provides a high-level overview but lacks the detail needed for complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 11 parameters are described in the schema (100% coverage), so the description does not need to add meaning. It adds no further explanation beyond the schema. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it optimizes a rough prompt draft using a local LLM. It also provides context (before sending to a paid API) that distinguishes it from sibling tools like check_health, generate_system_prompt, lint_prompt, and score_prompt.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when preparing prompts for paid APIs to reduce costs, but it does not explicitly state when to use this tool versus alternatives like lint_prompt or generate_system_prompt. No exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_system_promptA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the key behavior: drafting then auto-linting and auto-scoring. However, it does not detail other aspects like return format or authentication needs. The 'rigor' behavior is explained, but overall transparency is adequate yet not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first sentence covers the core purpose and auto-validation, second explains the key 'rigor' option. No filler, every sentence adds value. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter tool with no output schema, the description is somewhat minimal. It does not describe what the returned output looks like (e.g., combined prompt with scores) or mention optional parameters like transcript or failure_modes. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal new meaning beyond the schema; it repeats the 'both' option explanation already present in the schema's description. The role parameter is mentioned generally.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool drafts a system prompt for a role and then auto-lints and auto-scores it. It uses a specific verb 'Drafts' and distinguishes itself from sibling tools (lint_prompt, score_prompt) by combining these steps in one call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is an all-in-one for generating and evaluating prompts but does not explicitly state when to use it versus sibling tools. No alternatives or exclusions are mentioned, leaving guidance implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_promptA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions scoring dimensions, comparison mode, and output (deltas/verdict) but lacks details on side effects, auth, or rate limits. For a non-destructive scoring tool, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, efficient and front-loaded with the core purpose. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema is provided, so the description should clarify return format. It mentions grades 1-5 and deltas/verdict for comparison, but lacks specifics on output structure (e.g., JSON with per-dimension scores). For a tool with 4 parameters and no output schema, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds context about the baseline parameter switching to comparison mode and mentions per-dimension deltas and verdict, which enriches understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool grades a prompt on five specific dimensions with a 1-5 scale and has a comparison mode with baseline. It distinguishes itself from siblings like lint_prompt or optimize_prompt by focusing on scoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for scoring prompts but does not explicitly advise when to use this tool over siblings. No when-not-to-use or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_healthA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It describes the action but does not disclose whether it is read-only, idempotent, or has side effects. For a health check, no side effects are expected, but this is implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the core purpose, and contains no unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should clarify what the return value looks like (e.g., boolean, health status). It names the engines but fails to explain the expected output format, which is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds no new parameter meaning beyond what the schema already provides (e.g., 'model' override and 'engine' selection are fully described in the schema). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it checks if the configured LLM engine (Ollama or Anthropic) is reachable and ready. It specifies the verb 'checks' and the resource 'LLM engine', and distinguishes from the sibling tool 'optimize_prompt' by focusing on health status rather than prompt optimization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when verifying engine availability, which is clear context. However, it does not explicitly state when not to use or mention alternatives, though the sibling tool is conceptually distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.