Server Quality Checklist

Tool Scores

• convertB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It explains what each mode does at a high level, but it does not disclose side effects such as creating files, overwriting, directory scanning behavior, error conditions, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with a clear opening purpose and structured mode sections. It is longer than average but the length is justified by the four distinct modes and their parameter lists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (14 parameters, four modes), the description does a solid job of laying out mode-specific required parameters and basic semantics. It is not fully complete because several parameter meanings and behavioral outcomes are left implicit, but it is sufficient for basic invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists mode-specific parameters and some allowed values (style: production|notebook, target: spark|glue), but it does not explain the meaning of parameters like table_info, optimization_level, include_glue_template, output_dir, or recursive, leaving many parameters underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool converts SQL to PySpark code and supports batch file/directory processing and PDF extraction. It distinguishes itself by specific action and resources, though it does not explicitly name sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description defines four distinct modes with their required parameters, which gives an agent enough context to select a mode. However, it does not explicitly state when to use this tool over siblings like analyze, optimize, or refactor, nor does it offer exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyticsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral disclosure burden. It discloses deprecation, defines two behavioral modes, and lists associated parameters. However, it doesn't mention side effects, error behavior, or whether this is purely read-only, though the analytics nature implies a query operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured, with the deprecation warning front-loaded and modes presented clearly. There is minor redundancy between the opening line and the mode descriptions, but no wasted words overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values don't need explanation. The description covers modes, deprecation, and parameter mapping. It remains incomplete around parameter semantics and when exactly to fall back to this tool versus 'review', leaving some ambiguity for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It maps 'optimization_type' and 'limit' to the optimization mode and 'limit' to the usage mode, which adds useful structure beyond the schema. However, it doesn't explain acceptable values for 'optimization_type' or the precise meaning of 'limit' beyond the schema's type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: providing analytics on optimization effectiveness and usage patterns. It further clarifies by defining two distinct modes, 'optimization' and 'usage', with explicit outputs for each. While the deprecation note adds context, it doesn't obscure what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Prefer review', giving the agent a clear alternative and strongly signaling not to use this tool unless necessary. The phrase 'Still registered this minor version' implies a fallback/backward-compatibility role. It doesn't detail conditional cases, but the deprecation guidance is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• glue_dataA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries full behavioral burden. It implies job generation but does not disclose side effects, resource provisioning, permissions, idempotency, or failure behavior. The deprecation warning adds some context, but the safety and mutation profile of the operation remains unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but well-structured with mode headers and parameter lists. The deprecation warning is front-loaded, and each section conveys necessary mode-specific information, though some repetition across modes could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 14 parameters, no annotations, and no schema descriptions, so the description carries a heavy burden. It covers mode-specific required parameters well and an output schema exists, but it leaves strategy value semantics and operational behavior unexplained. This is adequate but with notable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does meaningfully map parameters to each mode and marks required ones, which is valuable. However, it does not explain the meaning or accepted values of parameters like incremental_strategy, cdc_strategy, bookmark_strategy, or transformation_sql beyond their names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Generate AWS Glue data processing jobs — incremental, CDC, bookmarks,' naming the specific action and resource. The opening 'Deprecated. Prefer glue_job' explicitly distinguishes it from its sibling, so an agent can tell it apart immediately.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Deprecated. Prefer glue_job,' giving a direct when-not-to-use instruction and naming the alternative. The mode breakdown further tells the agent which mode to select for incremental, CDC, or bookmark needs, though it does not deeply elaborate on conditions beyond deprecation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyzeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden: it clearly marks the tool as deprecated and describes what each mode analyzes. It does not disclose operational constraints or response behavior, though the output schema exists and the analysis-focused verbs imply read-only behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bold headings and compact mode blocks; the deprecation warning is front-loaded and every listed parameter relates to a specific mode. There is no filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mode-based tool with 11 parameters and no annotation coverage, the description provides enough per-mode detail to invoke it correctly, and the output schema covers the return shape. It could be more complete by explaining optional parameter meanings and the relationship to preferred replacement tools, but the core calling contract is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by mapping every mode to its relevant parameters and marking required ones, such as pyspark_code for data_flow and directory_path for codebase. It stops short of defining the semantics of scan_depth, table_info, or workspace_name, but the mode-to-parameter mapping is the most critical missing guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes SQL or PySpark code and enumerates four distinct modes: sql_context, data_flow, codebase, and workspace. However, it does not meaningfully differentiate analyze from its preferred siblings beyond a bare deprecation pointer, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The deprecation notice advises preferring convert/glue_job/review, and each mode lists its parameters, giving a rough sense of when a mode applies. There is no explicit when-to-use versus when-not-to-use guidance or conditions for choosing analyze over the preferred replacement tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• glue_jobA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly describes what each mode generates, but it does not mention side effects, credentials, filesystem writes, or whether 'manage' implies modifying existing AWS resources. The behavior is not misleading, but it is incomplete for a tool with no annotation safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a one-sentence summary followed by clearly formatted mode sections and parameter lists. It is longer than average, but every section earns its place given the tool's four distinct behaviors, and the front-loaded summary helps an agent orient immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's high complexity, 23 parameters, and no annotations, the description does a solid job covering all modes and associating parameters appropriately. The presence of an output schema reduces the need to explain return values. Missing details like allowed values for template_type or source_format and mode-specific examples prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by grouping all 23 parameters into their relevant modes and marking required parameters such as pyspark_code and job_name. It adds meaning beyond the bare schema, though it still relies on parameter names to carry most semantic weight rather than describing values or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the resource ('AWS Glue job configurations and templates') and the action ('Generate and manage'), and it enumerates four distinct modes that make the tool's scope concrete. It does not explicitly contrast with sibling tools like glue_schema or glue_s3, so it misses the top tier, but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The mode breakdown gives clear context for which mode to choose: template, dynamic_frame, properties, or sql_conversion, each with its own parameter list. However, there is no explicit guidance on when glue_job should be preferred over sibling tools or when a mode should not be used, so usage guidance remains implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• contextA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It does disclose deprecation status, the real-time behavior of the 'assist' mode, and the purpose of each mode. However, it does not mention side effects, permissions, data mutation, or error behavior, which is a notable gap for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with the deprecation warning front-loaded and modes broken into clear labeled sections. The mode/parameter layout is easy to scan. The only minor issue is the vague 'work with' phrase in the overview, which adds little information, but overall the structure is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema and the description covers modes, required parameters, and deprecation guidance, an agent can select and invoke each mode correctly. It lacks examples of context_data structure and a fuller explanation of what 'assist' returns, but for a deprecated compatibility tool with an output schema, the coverage is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by tying each parameter to a specific mode and marking required ones (e.g., conversion_id and context_data for store, sql_query or selected_text for assist). This adds meaning beyond the raw schema, although it does not describe the shape or expected format of context_data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is deprecated and that it stores, retrieves, and works with SQL/PySpark conversion context, then enumerates three distinct modes (store, get, assist). Each mode has a concrete verb and resource, so an agent can generally tell what the tool does. The phrase 'work with' in the overview is vague, but the mode list resolves most ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description opens with 'Deprecated. Prefer convert. Still registered this minor version,' which is explicit routing guidance away from this tool to a sibling. It clearly tells the agent when not to use it, though it does not describe under what conditions, if any, this tool should still be chosen over convert.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimizeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the code mode 'does not rewrite the input' and that outputs are recommendations/estimates, which is useful. However, it does not state whether the other modes have side effects, what happens with invalid inputs, or any environment prerequisites such as a running Spark session.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The deprecation warning is front-loaded, followed by a one-sentence purpose and compact mode bullets. The repeated parameter lists are a minor redundancy but every section earns its place and the format is easy for an agent to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema covers return values, so the description does not need to describe them. It covers deprecation, modes, and mode-specific parameters, but misses allowed values for optimization_level and the structure of table_info; for a 5-parameter tool with no annotations, this is an incomplete picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds a useful mode-to-parameter mapping and distinguishes code from pyspark_code by mode. It does not define allowed values for optimization_level or the expected shape of table_info, leaving important invocation details unresolved.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a deprecation notice and then states 'Optimize PySpark code and recommend performance improvements,' a specific verb and resource. The mode list further clarifies what each invocation produces. It identifies review as preferred but does not explain how review differs functionally, so it stops short of perfect sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Deprecated. Prefer review,' which is a direct when-not-to-use directive with a named alternative. Each mode section also states the exact parameters required for that mode, giving clear invocation guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reviewA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must carry the full burden. It describes the modes but does not explicitly state side effects, permissions, or that it is read-only. Given the output schema exists, some behavioral context is implied but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with clear headers and backticks for modes. Every sentence adds value, and the main purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need not be explained. The description covers the three modes and parameter dependencies. It could mention limitations or prerequisites, but overall it is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining the role of each parameter per mode (e.g., 'code' required in code mode, 'code_samples' in patterns/duplicates). This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reviews PySpark code for issues, patterns, and refactoring opportunities. It differentiates three modes, but does not explicitly distinguish it from the sibling tool 'analyze', which could be seen as similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit modes and the parameters required for each, guiding when to use each mode. However, it does not specify when not to use this tool or mention alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• glue_schemaA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing side effects, permissions, or mutation risk. It says 'Manage' and lists actions like evolve/define, but never states whether catalog schemas are modified, whether operations are reversible, or what side effects occur. The deprecation note adds context but not behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a deprecation banner, one-line summary, and bold mode subsections that each list only relevant parameters. There is no filler; even the 'Still registered this minor version' line adds availability context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists and all 14 parameters are covered by mode-based breakdowns, an agent has enough to invoke the tool correctly. Missing is any statement about side effects or exact mode string values, but the mode headings make those inferable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by mapping every parameter to its mode, indicating which are required, and adding type guidance ('dict/list') not present in the schema. This is far more informative than the bare property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear deprecation warning and a preferred alternative, then identifies the resource (Glue Data Catalog schemas) and three specific actions with concrete outcomes: detect schema and generate table definition, generate evolution strategy, and generate catalog table definition. This distinguishes it from siblings like glue_job and glue_s3 by naming the exact domain and outputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Deprecated. Prefer glue_job', which tells an agent not to use this tool when a current alternative exists. Mode-specific parameter lists give contextual usage for each mode, though it does not state conditions under which glue_schema should still be used despite the deprecation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• glue_s3A
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full behavioral burden, and it does well: it explains that path heuristics are used, that no AWS API call is made, and that figures are not measured. It also says the consolidate mode 'Generate[s] Glue job' rather than executes one, but it does not fully clarify whether any side effects or external changes occur, leaving some ambiguity around resource mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the deprecation warning, followed by a one-line summary and clearly separated mode sections. The mode parameter lists are somewhat repetitive but serve a functional purpose because the schema alone does not indicate which parameters apply to which mode.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 15-parameter, multi-mode tool, the description covers the primary context: deprecation, mode purposes, required parameters, and behavioral constraints like 'no AWS API call.' The output schema exists, so return-value documentation is not required here, but per-parameter semantics and side-effect clarity remain partially underserved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It does add value by grouping parameters by mode and marking which are required per mode, which is not evident from the schema's sole 'mode' requirement. However, it does not explain the meaning of many parameters, such as query_patterns, data_size_gb, consolidation_strategy, or enable_small_file_optimization, relying mostly on self-explanatory names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear deprecation notice and explicit preferred alternative ('Prefer glue_job'), then states exactly what the tool does: 'Analyze S3 data layouts using path heuristics (no AWS API call).' The mode sections further define specific behaviors — analyze, optimize, consolidate — so an agent can distinguish this tool from siblings like glue_job, glue_schema, and glue_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent to prefer glue_job instead and notes that glue_s3 is 'Still registered this minor version,' which is clear when-not-to-use guidance. The mode breakdown also provides situational guidance for when each mode is appropriate, such as path-heuristic suggestions versus generating a Glue job for small-file consolidation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It adds useful behavior beyond the schema by explaining empty-query defaults for conversions and patterns and by using verbs like 'retrieve' and 'search' to imply read-only behavior. It does not discuss side effects, auth, or rate limits, but those are less critical for a search-style tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a deprecation warning up front, a one-line summary, and clearly labeled mode sections. Each sentence provides useful operational guidance with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The mode breakdown and parameter semantics give enough detail to call the tool correctly, and the output schema covers return-value shape. The main gap is not clarifying how the context mode relates to the sibling context tool or exactly how key differs from conversion_id.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so well by mapping each mode to its applicable parameters and explaining empty-query behavior for conversions and patterns, plus the context mode's key/conversion_id inputs. This adds meaning far beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches stored conversions, code patterns, and context data, and it breaks behavior into three named modes. It names convert as the preferred alternative, but it does not fully differentiate from the sibling context tool, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The deprecation notice explicitly tells the agent to prefer convert, which is strong when-not-to-use guidance. It also explains mode-specific parameter usage, but it does not spell out when search should still be used over convert or other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_statusA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavior. It discloses deprecation status and the supported operations, and the cancel mode implies mutation, but it does not describe side effects, reversibility, permissions, or behavior beyond the mode names. Some added context exists, but not deep behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The deprecation warning is front-loaded, followed by a terse mode-by-mode breakdown. The markdown structure makes the conditional parameter requirements easy to scan, and there is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all modes and their parameters, deprecation, and the preferred alternative. The output schema exists, so return values need not be spelled out, but the status filter values and mode-specific edge behavior are not specified, leaving minor gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema descriptions are absent (0% coverage), but the description compensates by mapping each mode to its relevant parameters and marking job_id as required for status and cancel. It does not define allowed values for status or clarify limit semantics, so it is helpful but not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as managing batch processing jobs, then enumerates four distinct modes (status, cancel, active, recent), each with a specific action and resource. It also differentiates itself from its sibling convert by stating that convert(mode="batch_dir") is preferred.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly marks the tool as Deprecated, instructs the agent to prefer convert(mode="batch_dir"), and notes it is still registered this minor version. This gives a clear when-not-to-use signal and names the alternative, while the mode list clarifies internal usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refactorA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure, and it does so well by explaining what each mode does: replacing duplicate patterns, extracting utilities, and generating pipeline or project code. It also discloses the deprecation status and that the tool remains registered. It does not mention side effects or return behavior, but the described behaviors are sufficiently concrete for an agent to anticipate what will happen.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the most important decision-related information: deprecation and the preferred alternative. It then uses a clear mode-based structure with bullet-style parameter lists. Despite its length, the size is justified by 16 parameters and three modes, and every section contributes actionable guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's high complexity—16 parameters, three modes, no annotations, and zero schema description coverage—the description is quite complete: it identifies all modes, their purposes, required parameters, and the preferred replacement. It does not explicitly restate the 'mode' parameter as a field in the schema, but the mode names are clearly used as the expected values. Since an output schema exists, the lack of return-format details is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does by mapping parameters to specific modes and marking which are required. It adds meaning beyond the bare JSON schema by grouping parameters under 'patterns,' 'utilities,' 'pipeline,' and 'project' contexts, which the schema alone does not convey. Some parameter semantics remain implicit from names, but the mode-based organization materially improves parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear deprecation notice and names the preferred alternative, review, which immediately distinguishes this tool from its siblings. It then states the core action ('Refactor PySpark code and generate pipeline structures') and enumerates three specific modes with distinct behaviors, so an agent can tell what the tool does and how it differs from related tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Deprecated. Prefer review. Still registered this minor version,' giving an unambiguous when-not-to-use instruction and naming the alternative. Within the modes, it lists mode-specific required parameters, which further guides invocation by telling the agent which parameters to supply for each mode.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s3_sourceA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. The verb 'Analyze' and phrases like 'Analyze S3 data source structure, format, and optimization opportunities' signal a read-only analysis operation. The deprecation status is also a useful behavioral disclosure. It does not explicitly state that no data is modified or mention permission requirements, but the analysis framing makes the safety profile reasonably clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a deprecation banner followed by two short mode bullets. Every sentence adds useful information, and there is no redundant repetition of schema defaults or obvious filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to describe return values. It covers the two modes, their parameters, and the preferred alternative. The only notable gap is the lack of an explicit statement about side effects or access requirements, but for a read-oriented analysis tool this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by mapping parameters to modes: 's3_path (required), include_schema_inference' and 'table_path (required), analyze_history.' It also clarifies conditional requiredness that the flat schema cannot express. The boolean parameters are named but not elaborated, though their purposes are fairly self-evident from their names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly opens with a deprecation notice and then states the tool's purpose: 'Analyze S3 data sources and Delta tables.' It enumerates two specific modes, 'analyze' and 'delta', each with its own resource target, making the tool's scope explicit and distinguishable from siblings like glue_job or glue_s3.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs agents to prefer glue_job: 'Deprecated. Prefer glue_job. Still registered this minor version.' It also gives mode-specific guidance by listing which parameters apply to which mode, so an agent can decide when to call this tool and with what arguments.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.