Server Quality Checklist

Tool Scores

• stop_pipelineA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only repeats the action 'stop a running pipeline' and describes parameters. It does not disclose whether stopping is reversible, whether it interrupts in-flight work, what happens to pipeline state, or whether permission is required.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely compact, front-loads the core action in the first sentence, and then lists parameters in a structured Args block with no filler or redundant elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with an output schema, the description is close to sufficient, but the lack of behavioral transparency and any guidance about verifying pipeline state creates a gap. An agent could invoke the tool correctly, but it would not know what to expect beyond the fact that the pipeline is stopped.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does add meaning for the database parameter by explaining that it 'defaults to the connection's configured database,' which the schema's null default does not convey. The pipeline_name explanation is largely a restatement of the property name, but for two parameters the overall semantics are adequately covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Stop a running pipeline.' This clearly distinguishes the tool from siblings like create_pipeline, start_pipeline, drop_pipeline, and alter_pipeline, so an agent can identify the correct operation without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'running' implies the tool should be used only on pipelines that are currently active, but the description does not explicitly say when to use this tool versus alternatives. It does not mention checking pipeline_status first, nor does it contrast stopping with dropping or restarting a pipeline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral context. 'Show' indicates a read-only inspection, and the default-database note adds useful behavior when the parameter is omitted. However, it does not explicitly state that no data is modified, what happens on a missing table, or any permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose, followed by a minimal Args section. There is no filler; every line adds information about the operation or its parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter introspection tool with an output schema available, the description covers the action and all argument semantics, including the database fallback. It lacks explicit usage guidance and behavioral caveats, but those gaps are minor given the tool's simplicity and the output schema's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by documenting both parameters. It adds real meaning beyond the schema by clarifying that omitting database falls back to the connection's configured database, which is not inferable from the schema's null default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Show column definitions for a table.' This clearly distinguishes the tool from siblings like list_tables (which lists tables) and run_sql (which executes arbitrary SQL), even though no sibling is named.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to use this tool versus alternatives such as run_sql or list_tables. The intended use is only implied by the verb 'Show,' and there is no mention of when not to use it or which sibling to prefer for broader schema exploration.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pipeline_ddlA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Get' implies a read-only retrieval and the database default is disclosed, but the description does not state behavior for missing pipelines, permissions, or whether any side effects occur. It does not contradict the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well organized: one clear purpose sentence followed by a brief, relevant Args list. Every sentence adds useful information without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter retrieval tool, the description covers the return value, the required parameter, and the optional parameter's default behavior. It does not cover error cases or explicitly route between siblings, but the output schema supplies additional structure and the use case is straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description documents both parameters. The database parameter explanation ('defaults to the connection's configured database') adds real meaning beyond the schema's nullable default of null, which is valuable because the schema itself has no property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') with a precise resource ('full CREATE PIPELINE statement') and outcome ('reproduces an existing pipeline'). This clearly distinguishes it from siblings like create_pipeline, drop_pipeline, and list_pipelines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use case is implied: you call this when you need the DDL to reproduce an existing pipeline. However, the description does not explicitly mention alternatives or when not to use it, leaving the agent to infer the choice from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pipelinesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description itself must convey behavioral expectations. It does disclose the key output trait—returning pipeline states (Running/Stopped/Error)—which is useful. However, it does not mention that this is a read-only operation, whether permissions are required, or what happens with an invalid or missing database. It is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. The primary purpose is front-loaded, and the parameter explanation is minimal and directly useful. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one optional parameter and an output schema present, the description is nearly complete. It gives the resource, scope, state categories, and default parameter behavior. It could be improved by noting the absence of side effects, but this is a minor gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the description fully explains the one parameter: database is the target database and defaults to the connection's configured database. This adds real semantic value beyond the schema's type/default fields and compensates for the lack of schema-level documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List all') and a clear resource ('pipelines in a database'), and it differentiates itself from the more specific pipeline_status sibling by explicitly covering all pipelines and their current state. This is unambiguous and lets an agent understand the exact scope of the tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: whenever you need the full set of pipelines and their states in a database. It does not explicitly contrast this with pipeline_status or other read-only siblings, nor does it give exclusions. The default-database note is helpful context but not a full usage guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It usefully notes that views are included and that the database parameter defaults to the connection's configured database. It does not mention permissions, output sorting, or whether system tables are included, but the 'list' wording strongly implies a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short, front-loaded with the core purpose, and adds an Args section only where it adds value. Every sentence earns its place and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and an output schema available, the description covers the essential details: what is listed, the target database, and the default behavior. It is slightly incomplete only in not routing the agent to sibling tools for related tasks like listing databases or describing a specific table.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines 'database' as a nullable string with default null and 0% schema description coverage. The description compensates by explaining the parameter means 'Database to list tables from' and clarifying the default behavior. This goes beyond the schema, though it could still specify expected format or accepted values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('List') and resource ('tables (and views) in a database'), making the tool's purpose immediately clear. It is naturally distinguished from sibling tools like list_databases and describe_table because it explicitly targets tables/views within a database.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you need to list tables or views from a database. However, it does not explicitly mention alternatives or exclusions, such as using list_databases to discover available databases or describe_table for individual table details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• alter_pipelineA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral transparency burden. It discloses the requirement for a full statement and that alterable clauses are source-specific, which is valuable. However, it does not mention permissions, reversibility, effects on a running pipeline, or the destructive potential of resetting offsets.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized: a one-sentence purpose, a short use-case expansion, and a clear Args block. Every sentence earns its place; there is no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with an output schema, the description covers the essential usage: what SQL to supply, what the database parameter does, and why a full statement is needed. It lacks some safety/permission context, but the core calling contract is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates by defining alter_pipeline_sql as 'The full ALTER PIPELINE ... statement' and explaining that database defaults to the connection's configured database. This adds meaning beyond the bare property names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Alter' and the resource 'existing pipeline', and specifies that it operates from a full ALTER PIPELINE statement. It distinguishes itself from siblings like create_pipeline and drop_pipeline by mentioning it changes connection strings/credentials or resets offsets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete use cases ('change the connection string/credentials or reset offsets') and explains why a full statement is required, referencing create_pipeline for the same rationale. It does not explicitly state when not to use this tool or list alternatives, but the common-use framing provides clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• drop_pipelineA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It usefully reveals the automatic stopping of running pipelines and that if_exists turns failure into a no-op. It does not explicitly warn that deletion is permanent or mention permission requirements, but the core side effects are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loads the primary purpose in the first sentence. The Args block is organized and every sentence contributes useful guidance with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential operation, side effects, and parameter semantics, and an output schema exists so return-value documentation is not required. It is nearly complete, though a brief note about irreversibility would have fully closed the gap for a destructive action with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates by explaining all three parameters: pipeline_name's role, database's default to the connection database, and if_exists's no-op behavior. This adds real meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Delete a pipeline,' a specific verb and resource that clearly identifies the operation. It also notes that running pipelines are stopped automatically, which differentiates dropping from simply stopping a pipeline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for permanent removal and explains that running pipelines are stopped as part of the drop. However, it does not explicitly contrast drop_pipeline with stop_pipeline or state when dropping is inappropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_pipelineA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It does so well by revealing that the tool only checks for the 'CREATE [OR REPLACE] PIPELINE' prefix before running, and that the pipeline is not started automatically. These are meaningful non-obvious behaviors beyond the basic 'create' action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: it leads with the action, explains why the design is statement-based, states critical behavioral caveats, and includes a representative example. Every section serves a purpose and there is minimal redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, validation behavior, start semantics, and parameter meanings, and an output schema exists to describe return values. It does not mention error cases or permission requirements, but for this tool the provided context is sufficient for correct use and routing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains create_pipeline_sql as the full statement text, provides a concrete example, and clarifies that database defaults to the connection's configured database. This adds real meaning beyond the parameter names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Create a new pipeline from a full CREATE PIPELINE statement.' It clearly distinguishes this tool from siblings like start_pipeline, drop_pipeline, and alter_pipeline by focusing on creation from a complete statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells the agent that creating a pipeline does not start it, and directs it to call start_pipeline afterwards. It also explains why the tool takes a full statement rather than structured parameters, which gives context for when it is appropriate. It does not state when not to use alternatives like run_sql, but the guidance is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_databasesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that results depend on the connected user's visibility, implying permission-based filtering and no modifications. Although no annotations exist, the read-only nature of a list action is sufficiently conveyed by the verb 'List'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence front-loads the action and resource, with the scope qualifier placed at the end. Every word earns its place; there is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, an existing output schema, and a simple read-only enumeration, the description covers everything an agent needs to invoke the tool correctly. The user-visibility qualifier is a useful contextual detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters and the input schema is empty. There are no parameter semantics for the description to clarify, so it correctly remains brief.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a clear resource ('databases'), and an explicit scope ('visible to the connected user'). It is easily distinguishable from sibling tools like list_tables and list_pipelines because it names the object type directly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternative guidance is provided, but the operation's purpose is self-evident: it enumerates databases, which is a distinct step from listing tables or pipelines. The context is clear and there are no exclusions or preconditions to state.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pipeline_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the transparency burden. It signals a read-only lookup through 'Get the current state' and discloses the important edge behavior of returning an empty row list when the pipeline doesn't exist. It doesn't discuss permissions or rate limits, but for a status-read tool the key behavior is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence adds value: action, equivalence, missing-pipeline behavior, and the two parameters. The key semantics are front-loaded before the args detail, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple two-parameter lookup with an output schema available, so return details need not be repeated. The description fully equips an agent to select and call the tool correctly, including the optional database context and empty-result behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no property descriptions (0% coverage), so the Args section must compensate. It explains pipeline_name as the lookup key and database as the owning database with a useful default tied to the connection's configured database, adding meaning beyond the bare names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence names a specific action and resource: 'Get the current state of one pipeline by name.' The SHOW PIPELINES equivalence and the 'single pipeline' phrasing clearly separate it from list_pipelines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys this is for a targeted single-pipeline lookup and that a missing name yields an empty list rather than an error. It does not explicitly name list_pipelines as the alternative for retrieving all pipelines, so the when-not-to-use guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_pipelineA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It explains the synchronous vs. immediate-return distinction, the constraint that limit_batches only works in foreground, and that if_not_running turns a start into a no-op instead of an error. This is strong behavioral disclosure, though it does not cover potential failure modes or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a clear one-sentence purpose followed by a concise parameter list. Every line adds information, and the most important behavior is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and the sibling set, the description covers the essential behavior and parameter semantics. It could be slightly more complete by explicitly contrasting with test_pipeline or pipeline_status, but for this tool's complexity it is quite sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It explains all five parameters, including the database default, the meaning of foreground, the conditional validity of limit_batches, and the effect of if_not_running. This goes well beyond the bare schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Start a pipeline so it begins (or resumes) loading data.' This clearly distinguishes start_pipeline from siblings like stop_pipeline, create_pipeline, or pipeline_status, and even adds the nuance that starting can resume an existing pipeline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical usage context, noting that foreground mode is 'Useful for a one-off load or for testing a pipeline end to end.' It does not explicitly name alternatives or say when not to use this tool, so it misses the top tier, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sqlA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral transparency burden. It clearly indicates the tool executes arbitrary SQL, includes DML/DDL, returns results, and clarifies that max_rows only truncates returned rows without limiting statement processing. It stops short of explicitly warning that arbitrary DDL/DML may be destructive or require elevated permissions, though this is strongly implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: purpose first, routing guidance second, then a clear Args list. Every sentence earns its place, and there is no redundant or vague wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for an arbitrary-SQL tool of this complexity. It covers scope, exclusions, parameter behavior, and output-truncation semantics. An output schema exists to define return values, and sibling-tool differentiation is handled explicitly, so an agent has enough to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must fully explain the parameters. It does: sql is 'the statement to execute,' database defaults to the connection's configured database, and max_rows 'truncate[s] returned rows' without affecting how many rows the statement processes. This adds meaningful semantics beyond the sparse schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Run one arbitrary SQL statement against SingleStore and return the results.' It also distinguishes itself from pipeline-specific tools by explicitly listing what types of SQL it is for versus what should go to sibling pipeline tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use this for SELECT/DML/DDL that isn't pipeline-specific.' It also names the excluded category and gives a reason to prefer alternatives: pipeline tools 'validate the statement type and are easier to call correctly.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_pipelineA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and handles it well. It explicitly discloses that nothing is written to the destination table, that testing a running pipeline causes a SingleStore error, and that the pipeline must already exist and be stopped. These are important behavioral traits beyond what the schema shows.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, then prerequisites, then side effects, then parameter details. Every sentence adds value, and the length is appropriate for a tool with nontrivial preconditions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no schema-level parameter descriptions, the description covers all essential context: prerequisites, error condition, side-effect-free behavior, and parameter semantics. The output schema exists to describe return values, so the description does not need to explain them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description documents all three parameters with meaningful semantics: pipeline_name identifies the pipeline, database defaults to the connection's configured database, and limit restricts how many rows/messages are pulled. This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Test an existing pipeline') plus the key behavioral distinction ('without loading it into the table'), which clearly separates it from start_pipeline, stop_pipeline, and create_pipeline. It also states the validation goal, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit preconditions: the pipeline must already exist and must be stopped first, and it directly tells the agent to call stop_pipeline before this if needed. It also clarifies the tool's use case ('purely for validating source/format/transform config works'), which helps an agent decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.