Server Quality Checklist

Tool Scores

• handle_diagnose_locksA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by explaining the tool's multi-step behavior: fetching all lock information, applying optional filters, formatting results into list+summary structure, and potential error conditions (DataApiError). It doesn't mention permissions, rate limits, or side effects, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured with clear sections (purpose, args, returns, raises). While efficient, the parameter explanations could be slightly more concise, and the purpose statement could be more front-loaded before diving into implementation details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a diagnostic tool with 3 parameters, no annotations, and no output schema, the description provides good coverage: clear purpose, parameter semantics, return format (list of dictionaries), and error conditions. It could improve by explaining the summary structure or providing example output, but overall it's reasonably complete given the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic explanations for all three parameters: target_pid (filter by process ID), target_table_name (filter by table with schema qualification note), and min_wait_seconds (minimum waiting time with default). The descriptions add meaningful context beyond basic schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('identifies', 'fetches', 'filters', 'formats') and resource ('active lock contention in the cluster'). It distinguishes itself from siblings by focusing specifically on lock diagnostics rather than general health, performance, or table operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through parameter explanations (filtering by PID, table name, wait time) but doesn't explicitly state when to use this tool versus alternatives like handle_check_cluster_health or handle_diagnose_query_performance. No explicit when-not-to-use guidance or named alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_monitor_workloadA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes that the tool executes SQL scripts concurrently, returns a dictionary with raw results or exceptions, and handles individual script failures gracefully by including exceptions in the result dict. It also mentions that critical configuration errors raise DataApiError. However, it doesn't specify performance characteristics, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, execution details, args, returns, raises) and front-loaded with the core functionality. While comprehensive, some sentences could be more concise, such as the detailed explanation of the return dictionary structure which is somewhat verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a workload analysis tool with 2 parameters, no annotations, and no output schema, the description provides substantial context about behavior, parameters, return format, and error handling. It explains the concurrent execution of SQL scripts, the dictionary return structure with success/failure results, and different error scenarios. The main gap is lack of information about what specific workload metrics are analyzed beyond the general categories mentioned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides excellent parameter semantics beyond the basic schema. While schema description coverage is 0%, the description clearly explains that time_window_days is the 'lookback period in days for workload analysis' with a default of 2, and top_n_queries determines 'number of top queries to consider' with a default of 10. This adds meaningful context about what these parameters control in the analysis.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'analyzes cluster workload patterns over a specified time window' with specific verbs (analyzes, executes, gathers) and resources (cluster workload, SQL scripts). It distinguishes from siblings like handle_check_cluster_health or handle_diagnose_query_performance by focusing on comprehensive workload analysis rather than specific health checks or query diagnostics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for analyzing workload patterns over time, but doesn't explicitly state when to use this tool versus alternatives like handle_diagnose_query_performance or handle_execute_ad_hoc_query. There's no guidance on prerequisites, exclusions, or specific scenarios where this tool is preferred over sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_get_table_definitionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by detailing success/error return structures, specific exception types (TableNotFound, DataApiError), and the SQL script execution behavior. However, it doesn't mention performance characteristics, rate limits, or authentication requirements that would be helpful for a database tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, execution details, Args, Returns, Raises) and every sentence adds value. It's appropriately sized for a tool with 2 parameters and complex return behavior, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and no output schema, the description provides excellent coverage of parameters, return values, and exceptions. The main gap is lack of guidance on when to use versus sibling tools, but otherwise it's quite complete for the tool's complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides explicit parameter documentation in the Args section, clearly explaining what schema_name and table_name represent. With 0% schema description coverage, this comprehensive parameter documentation fully compensates and adds significant value beyond the bare input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieves the DDL statement') and resource ('for a specific table'), distinguishing it from sibling tools like handle_execute_ad_hoc_query or handle_inspect_table. It explicitly mentions the SQL script execution aspect, providing precise functional context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing table DDL, but doesn't explicitly state when to use this tool versus alternatives like handle_inspect_table or handle_execute_ad_hoc_query. No guidance is provided on prerequisites, error handling expectations, or specific scenarios where this tool is preferred over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_inspect_tableA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does so effectively. It discloses the concurrent execution of multiple scripts, the mixed return types (raw results, DDL strings, or Exception objects), and specific error conditions (TableNotFound, DataApiError). However, it omits details like rate limits, authentication needs, or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, Args, Returns, Raises) and front-loaded key information. It avoids redundancy, but the Returns section is slightly verbose in detailing success/error cases; some details could be condensed without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides substantial context: purpose, parameters, return structure, and error handling. It adequately covers the tool's complexity (2 params, mixed outputs). However, it lacks examples of return values or script names, which would enhance completeness for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explicitly lists and explains the two parameters (schema_name and table_name) in the Args section, clarifying their roles in identifying the Redshift table. This adds meaningful context beyond the bare schema, though it could elaborate on format constraints (e.g., case sensitivity).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieves detailed information') and resource ('about a specific Redshift table'), distinguishing it from siblings like handle_get_table_definition (which likely fetches only DDL) and handle_diagnose_query_performance (which focuses on queries rather than table metadata). The mention of 'various inspection scripts covering design, storage, health, usage, and encoding' provides concrete scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly suggests usage when detailed table metadata is needed, but lacks explicit guidance on when to choose this over alternatives like handle_get_table_definition or handle_monitor_workload. It does not specify prerequisites or exclusions, though the error conditions hint at when-not scenarios (e.g., table not found).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_check_cluster_healthA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: concurrent execution of scripts, aggregation of results into a dictionary, error handling approach (individual script errors captured in dictionary vs. critical errors raised as DataApiError), and the distinction between basic and full diagnostic levels.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, execution behavior, args, returns, raises) and front-loaded with the core purpose. While comprehensive, some sentences could be more concise, such as the detailed explanation of the return dictionary which is slightly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and 0% schema description coverage, the description provides substantial context including purpose, parameters, return format, and error handling. However, it doesn't mention authentication requirements, rate limits, or potential side effects on the cluster, which would be helpful given the diagnostic nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description fully compensates by providing detailed semantic explanations for both parameters: 'level' options ('basic' for operational status, 'full' for comprehensive checks) and 'time_window_days' purpose (lookback period for time-sensitive checks like queue waits). It also mentions default values and provides concrete examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'performs a health assessment of the Redshift cluster' with specific verbs ('executes diagnostic SQL scripts', 'aggregates results') and distinguishes it from siblings by focusing on comprehensive cluster health rather than specific issues like locks, query performance, or table inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use different levels ('basic' for operational status, 'full' for comprehensive checks) and mentions time-sensitive checks, but doesn't explicitly state when to choose this tool over sibling tools like handle_diagnose_query_performance or handle_monitor_workload for similar health-related tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_diagnose_query_performanceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so well. It describes what data gets fetched, how it's synthesized into a structured report, and documents specific error conditions (QueryNotFound, DataApiError). It also mentions the formatting utility and the tool's ability to optionally fetch historical data. While it doesn't mention rate limits or authentication needs, it provides substantial behavioral context for a diagnostic tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured with clear sections: purpose statement, what it fetches, how it processes data, args documentation, returns documentation, and raises documentation. Every sentence earns its place, though the returns section could be slightly more concise. The information is front-loaded with the core purpose stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a diagnostic tool with 2 parameters, no annotations, and no output schema, the description provides substantial context. It explains what data gets collected, how it's processed, parameter meanings, and error conditions. The main gap is the lack of detail about the exact structure of the returned dictionary or what specific metrics/alerts are examined, but given the tool's complexity, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter semantics. It explains that query_id is 'the numeric ID of the Redshift query to analyze' and that compare_historical controls whether to 'fetch performance data for previous runs of the same query text' with its default value. This adds crucial meaning beyond the bare schema types (integer, boolean).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('analyzes', 'fetches', 'synthesizes') and resources ('query's execution performance', 'query text, execution plan, metrics, alerts, compilation info, skew details, historical run data'). It distinguishes from sibling tools like handle_check_cluster_health or handle_diagnose_locks by focusing specifically on query performance analysis rather than cluster health or lock diagnosis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you need to analyze a specific query's performance with detailed metrics and recommendations. It doesn't explicitly state when NOT to use it or name specific alternatives among siblings, but the context is sufficiently clear for an agent to understand this is for query performance diagnosis rather than general cluster monitoring or table inspection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• handle_execute_ad_hoc_queryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing behavioral traits. It describes the return structure in detail (success vs error cases), mentions potential exceptions raised (DataApiConfigError, SqlExecutionError, etc.), and notes that 'Actual return might be handled by FastMCP error handling for raised exceptions.' This provides comprehensive behavioral context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It begins with the core purpose, then provides usage context, followed by parameter documentation, return value details, and exception information. Every section adds value, though the detailed exception list could be slightly condensed. Overall, it's efficiently organized with clear sections.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (executing arbitrary SQL queries via Redshift Data API) and the absence of both annotations and output schema, the description provides substantial context. It covers purpose, usage guidelines, parameter semantics, return structure, and potential exceptions. The main gap is lack of information about query limitations, performance implications, or security considerations for arbitrary SQL execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema. With 0% schema description coverage (schema only shows sql_query is a required string), the description explains that 'sql_query: The exact SQL query string to execute.' This clarifies the parameter's purpose and format. While it doesn't provide SQL syntax guidance, it adequately compensates for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Executes an arbitrary SQL query provided by the user via Redshift Data API.' It specifies the exact action (execute SQL query), the mechanism (Redshift Data API), and distinguishes it from specialized tools by calling it an 'escape hatch for advanced users or queries not covered by specialized tools.' This differentiates it from sibling tools like handle_get_table_definition or handle_inspect_table.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Designed as an escape hatch for advanced users or queries not covered by specialized tools.' This provides clear guidance that this tool should be used when other specialized tools (the siblings listed) don't cover the needed functionality, establishing clear alternatives and usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.