Server Quality Checklist

Tool Scores

• explain_lockA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the tool 'explains' lock acquisition, without disclosing whether the DDL is executed, if a database connection is needed, or if there are any side effects. This is a significant gap for a tool that could potentially run DDL.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 13 words, front-loading the action and the resource. It contains no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema or annotations, the description provides only the core function. It does not explain what the explanation looks like, how pg_version changes behavior, or what 'impact' refers to. Adequate but with clear room for additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters (sql and pg_version) with 100% coverage. The tool description adds no additional parameter detail beyond its core purpose, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Explain') and a clear resource ('PostgreSQL lock a DDL statement acquires'), distinguishing it from sibling tools that focus on migration analysis or fixes. It unambiguously states the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for analyzing lock behavior of DDL statements, but provides no explicit context or alternatives. It does not mention when to use this tool versus sibling tools like analyze_migration or check_before_apply.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_migrationA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses that the tool performs analysis and returns violations, risk level, and lock analysis, which is useful. However, it does not explicitly state whether it is read-only, requires a database connection, or executes the SQL, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose and mentions key return values. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters, full schema coverage, and no output schema, the description adequately covers purpose and high-level outputs. It lacks detailed guidance on the structure of violations or risk level, but this is not critical for selection and invocation given the schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both parameters (sql and pg_version) have meaningful descriptions in the schema. The tool description adds no additional parameter semantics beyond what the schema provides, which aligns with the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes a PostgreSQL migration SQL for safety issues and specifies its outputs (violations, risk level, lock analysis). It is specific enough to distinguish from siblings like analyze_migration_dir, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a single migration SQL to analyze, but provides no explicit guidance on when to use this tool versus sibling tools like check_before_apply or suggest_fix. No when-not or alternative mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggest_fixA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does add useful context by stating that it returns fixed SQL and a list of changes, but 'safe violations' is vaguely defined, and there is no disclosure of side effects, permissions, or whether changes are applied directly. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action and scope, and it includes the return value. Every word earns its place, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must explain the return value; it does say it returns fixed SQL and a list of changes, but without detailing the structure of the changes list. It also doesn't clarify the definition of 'safe violations' or how this tool relates to check_before_apply, which could be important given the sibling tools. Overall, it is functional but leaves several gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters (sql and pg_version), so the baseline is 3. The description adds no additional parameter-specific semantics beyond what the schema already provides, and no extra context about how the parameters affect the fix behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Auto-fix' with a clear resource ('PostgreSQL migration SQL') and scope ('safe violations'). It also states what it returns (fixed SQL and list of changes), making it clearly distinguishable from siblings like analyze_migration and check_before_apply.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when there are safe violations to fix) but does not explicitly mention alternatives or exclusions. No context about when to choose suggest_fix over check_before_apply or analyze_migration is provided, so the usage guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_migration_dirA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It mentions the output structure (per-file results plus aggregate summary) and implies a read-only analysis, but it does not explicitly state that files are not modified, nor does it mention any requirements or side effects. This is useful context but leaves gaps about safety and exact behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and includes return format and usage guidance without any fluff. Every word adds value, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and no annotations, the description covers the main purpose, output shape, and suggested use cases. It stops short of explaining what 'analyze' means in terms of rules or checks, but the sibling tools like 'list_rules' and 'get_rule' imply a rule system. Overall, it is fairly complete for making an informed selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema already documents all three parameters (path, pattern, pgVersion) with clear descriptions. The tool description adds no additional parameter-level meaning, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Analyze every migration file in a directory' with a specific verb and resource scope, distinguishing it from the sibling 'analyze_migration' by emphasizing directory-wide analysis. It also notes the return format (per-file results plus aggregate summary), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage context: 'Use this to audit a whole migrations folder before a release, or to find which existing migration introduced a risky pattern.' This clearly indicates when to choose this tool over alternatives, though it does not explicitly name the single-file sibling or give exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'List' implies a read-only operation, and the phrase 'with descriptions' indicates return content. However, it does not disclose potential auth requirements, pagination, or ordering behavior. For a simple listing tool this is minimally adequate but lacks richer behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action verb 'List', and every word earns its place. It is succinct and clearly structured, with no redundant or vague language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only listing tool, the description covers the core purpose and return essence. It could optionally mention how the output relates to get_rule or describe the expected response structure, but given the low complexity, it is nearly complete. The absence of an output schema is partially mitigated by the note about descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, so the baseline is 4 per the rubric. The description adds no parameter-specific info, but none is needed since the tool takes no inputs. Schema coverage is effectively 100% with no properties, so there is no gap to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with a clear resource: 'all available MigrationPilot safety rules with descriptions.' This distinguishes it from sibling tools like get_rule, which presumably fetches a single rule, and from analysis or fix tools. It unambiguously states the scope and output content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for enumerating all safety rules, but it does not explicitly state when to prefer this over alternatives. The sibling get_rule suggests a complementary use case (retrieving a specific rule), but no explicit when-to-use or alternatives are mentioned, leaving the guidance implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_before_applyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool is non-mutating (it is a 'check' that returns a verdict), explains config resolution ('exactly like the CLI'), and instructs the agent on failure behavior. It doesn't explicitly state it makes no writes to the database, but the safety-gate framing and 'returns a verdict' imply it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three compact sentences, front-loaded with 'Safety gate.' Each sentence provides necessary context: when to call, what it does, and how to handle the verdict. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 params and no output schema, the description covers the critical points: use case, behavior, and outcome ('pass/fail verdict'). It could describe the verdict structure or error cases, but for a safety gate the essential info is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds only minor reinforcement (e.g., 'exactly like the CLI' for configPath, 'exact SQL' for sql) but no substantive meaning beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a 'Safety gate' with the specific verb 'call this BEFORE writing or executing any PostgreSQL DDL or migration.' It states the function (resolves config, returns verdict) and differentiates from siblings like analyze_migration or suggest_fix because it is explicitly a gate that blocks application.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use ('BEFORE writing or executing any PostgreSQL DDL or migration') and provides the rule for fail ('do not apply the migration'). However, it doesn't explicitly state when NOT to use it (e.g., for read-only queries), leaving the exclusion to interpretation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ruleA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals that the tool returns documentation details (what it reports, why it matters, auto-fix capability, configuration) and implies it is a read-only lookup operation. It does not state potential side effects or permissions, but for a documentation retrieval tool, the behavior is clear and appropriately disclosed. The addition of specific return content adds value beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each earning its place. The first sentence defines the tool's output comprehensively, and the second sentence gives the exact usage trigger. There is no redundancy or fluff, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with only two parameters and no output schema, the description is complete. It explains what the tool does, what content it returns, when to use it, and how it relates to the migration workflow. The lack of an output schema is mitigated by the description's clear enumeration of return contents. Sibling tools are implicitly distinguished, and no critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('ruleId' and 'sql') well-documented in the schema. The description does not add parameter-specific information, but it also does not need to because the schema already provides clear examples and semantics. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('full documentation for one MigrationPilot rule'), and enumerates exactly what the documentation includes. It also distinguishes the tool from siblings by emphasizing 'one rule' and the use case of needing reasoning before rewriting, which is unique among the listed sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: 'Call this when a violation ID appears and you need the reasoning behind it before rewriting a migration.' This provides a clear trigger, but it does not explicitly name alternative tools or contrast with them. Since it gives a direct use case without exclusions, it earns a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.