Server Quality Checklist

Tool Scores

• log_outcomeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the logging action and data collected but does not disclose side effects, authentication needs, or rate limits. For a logging tool, the description is adequate but not rich; it assumes the agent understands it is write-only and non-destructive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first enumerates key fields concisely, the second explains the strategic value. Every word contributes. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and no annotations, the description adequately explains the tool's role in the workflow (post-grade_prompt), what data it collects, and why it matters. It could be improved by mentioning the return value (e.g., taskId or confirmation), but it is largely complete for a logging tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all 7 parameters have descriptions). The tool description adds contextual value by explaining the significance of 'namedRootCauseAndLocation' as the strongest predictor, but it does not add new parameter details beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('records') and resource ('real-world outcome for a graded prompt'), listing concrete fields (succeeded, turns, scope creep, etc.). It clearly distinguishes from sibling 'grade_prompt' which scores before use, and explains the broader context of Promptest MCP analytics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used after a prompt has been used and previously scored via grade_prompt. It also explains the value ('lets Promptest MCP tell you what actually predicts good outcomes'). However, it does not explicitly state when NOT to use it or name alternative tools for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_prompting_historyA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It describes returned data including aggregate stats and a specific finding. Missing details like data freshness or pagination, but sufficient for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence front-loads purpose. Slightly long but contains essential information. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with one optional param and no output schema. Description covers history and stats adequately. Lacks explanation of limit behavior if omitted, but schema covers it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (limit) with 100% schema coverage. Schema already provides description. Description does not add new semantic detail beyond output context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Returns' with specific resource 'personal history of graded prompts and logged outcomes, plus aggregate stats'. Distinct from sibling tools explain_rubric, grade_prompt, log_outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies personal review use case. Does not explicitly state when to use vs siblings, but sibling names make differentiation clear. Lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grade_promptA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes output (grade, breakdown, guidance, taskId) but no annotations exist. Lacks disclosure on side effects, safety, or rate limits. CRG-RIS term is unexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first defines purpose, second describes return values and follow-up. No fluff, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description fully covers return values (grade, breakdown, guidance, taskId) and next steps. Adequate for simple tool with one parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter promptText with schema description 'The exact prompt text to grade.' Schema covers 100%, description adds 'exact' nuance but no major additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly defines verb 'scores', resource 'prompt', and the rubric categories (Specificity, Clarity, Scope, Verification, Constraint Calibration). Distinguishes from sibling tools: explain_rubric, get_prompting_history, log_outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Indicates usage for prompts about to be sent or just sent. Provides actionable follow-up to call log_outcome. Does not explicitly state when not to use or exclude alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_rubricA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It correctly describes the tool as non-grading and explanatory, but lacks additional details like idempotency or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words, delivering essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple explanatory tool with no parameters and no output schema, the description covers the purpose and usage context adequately, though it could hint at the output format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description need not explain parameter details. It adequately covers the tool's purpose, meeting the baseline for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool explains the rubric and study findings, and explicitly notes it does not grade anything, distinguishing it from the sibling 'grade_prompt' tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides context for when to use ('before you start') but does not explicitly mention when not to use or offer alternatives beyond the implicit sibling distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.