Server Quality Checklist

Tool Scores

• run_saved_questionB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It only says 'execute' and does not mention return format, error behavior, authentication needs, side effects, or whether card parameters are supported or ignored. The effect of running a card is underspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences, and front-loaded with the primary action. Every word adds value, and it avoids unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description is incomplete. It does not explain what the response contains (e.g., query results, row counts, errors), whether parameters can be passed to the card, or what happens for cards that require parameters. These are meaningful gaps even for a simple one-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes card_id as 'The saved question/card ID' with 100% coverage. The description adds minimal semantic value ('by ID') and mentions card type compatibility, but does not clarify ID format, whether it's a number or string (schema says number), or any restrictions on which card IDs are valid. Baseline 3 applies due to high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a saved question (card) by ID, using a specific verb and resource. It also explicitly mentions support for both MBQL and native SQL cards, distinguishing it from sibling tools like run_native_query and list_saved_questions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: the tool is for running an existing saved card by ID. However, it does not explicitly state when to prefer this over alternatives, nor does it mention any exclusions (e.g., cards with parameters, unsupported card types). The mention of MBQL and native SQL gives some context but no direct comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_questionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states that the tool lists saved questions/cards, which implies a read-only operation, but it fails to explicitly mention that it has no side effects, or that it respects any permissions or rate limits. There is no additional context about return format, pagination, or ordering.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence: 'List all saved questions/cards, optionally filtered by database.' Every part is informative, and no wasted words. It is front-loaded with the core action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a low-complexity tool with one optional parameter and no output schema. The description explains the tool's core function and the only parameter. However, it does not mention what the returned data structure looks like or any possible pagination, which would help an agent use the output. Given the simplicity, it is minimally adequate but leaves room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for database_id is 100% covered, providing complete parameter documentation. The description text adds no further meaning beyond restating the optional filter, so the baseline of 3 is appropriate as the schema carries the burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and target resource: 'List all saved questions/cards' with an optional filter by database. This directly distinguishes it from siblings like run_saved_question and get_card_metadata, which operate on individual saved questions rather than listing them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool lists all saved questions/cards, optionally filtered by database. However, it does not explicitly mention when to use this tool versus alternatives, nor does it provide any exclusions or prerequisites. The implied usage is clear for a simple listing tool, but no explicit guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only restates the basic action and target scope, adding no insight into read-only nature, result limits, search matching semantics, permissions, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that communicates the tool's essence without any wasted words. It is appropriately concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple search tool, covering the core purpose and relying on a well-documented schema for parameters. However, with no output schema and no annotation, it does not explain return values, result format, or any search limitations, leaving some gaps for an agent fully understanding the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters thoroughly (query as 'Search keyword' and type with enum explanations). The description adds no new information about parameters beyond what the schema provides, so it meets the baseline for 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'search' with a clear resource scope ('questions, dashboards, and tables') and mechanism ('by keyword'). This distinguishes it from sibling list/get tools, which enumerate or retrieve specific items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies a use case: finding items across multiple types by keyword. It provides clear context but does not explicitly mention exclusions or alternatives (e.g., when to use list_dashboards instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_dashboardsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It implies a read-only action via 'list' but doesn't explicitly disclose additional behavioral traits such as pagination, ordering, or performance implications. It is not misleading but is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence with no unnecessary words. It is front-loaded and perfectly sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless list tool with no output schema and no annotations, the description is reasonably complete. It clearly states the action and scope. It could optionally mention return shape, but that is not essential for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. The description adds no param semantics because there are none to describe. Baseline for 0 params is 4, which is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all dashboards.' uses a specific verb (list) and resource (dashboards), clearly distinguishing it from siblings like list_databases and list_tables. It leaves no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance is provided. It doesn't mention how this differs from get_dashboard or when one might prefer list_saved_questions, leaving the agent to infer usage context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_native_queryA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the full burden of behavioral disclosure. It reveals important traits: permission requirements, mandatory LIMIT, rejection of write operations and multi-statement queries, and comment stripping. This goes well beyond the schema. It does not mention return format or pagination, which would push it to a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, and every sentence provides essential information: what it does, permission requirement, LIMIT warning, and rejection rules. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers permissions, query constraints, and validation behavior, making it largely complete for a safe read-only SQL tool. It omits explicit mention of return format, but for a SQL query tool this is often implied. The lack of output schema slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The schema already documents both parameters (query with LIMIT advice, database_id). The description's mention of LIMIT and write rejection adds context but not parameter-level detail beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Run a native SQL query against a Metabase database' – a specific verb and resource. It is clear in intent, but it does not explicitly distinguish itself from sibling tools like query_dataset or run_saved_question, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning required permissions, LIMIT clause, and rejected operations, but it does not explicitly state when to prefer this tool over alternatives such as query_dataset or run_saved_question. There is no when-not-to-use or alternative recommendation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dashboardA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. The verb 'Get' implies a read-only operation, and specifying the return content is helpful. However, it doesn't mention potential errors, permissions, or any edge cases. For a simple read tool this is adequate though not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence, immediately stating the action and the object. There is no filler, repetition, or unnecessary detail, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description gives a high-level summary of the return value (cards and layout) that is likely sufficient for an agent to understand the tool's purpose. It could be more detailed about the exact response structure, but for a getter this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully covers the single parameter (dashboard_id) with a description. The tool description adds no additional semantic value to the parameter itself, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and clearly identifies the resource (a dashboard) plus the exact content returned (cards and layout). This distinguishes it from sibling tools like list_dashboards (which lists dashboards) and get_card_metadata (which fetches a single card).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear context: use this when you need a specific dashboard's cards and layout. It doesn't explicitly name alternatives or exclusions, but the intent is unambiguous and fits well among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states the return fields (table ID, name, schema) and implies a read-only listing operation. However, it does not mention any potential limitations (e.g., whether system tables are included, ordering, or permission requirements) that could affect behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short, information-dense sentences that state the action and the output. There is no unnecessary detail or repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description covers the core functionality (what it does and what it returns). It does not address potential edge cases like pagination or access control, but these are less critical for a basic table listing operation. Overall it is sufficiently complete for selecting and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema documents the single parameter database_id with a description that already includes the pointer to list_databases, so schema coverage is 100%. The tool description itself does not add any parameter-level information beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all tables in a database') and specifies the resource (tables within a database). It distinguishes itself from siblings like list_databases (which lists databases) and get_table_fields (which accesses fields) by focusing on table listing. The return fields are also specified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool is used to list tables within a specific database, and the parameter description points users to list_databases for obtaining the database_id, providing a clear prerequisite. However, it does not explicitly mention when to use this versus siblings like get_table_fields or preview_table, so no exclusions are given. This gives clear context but lacks explicit alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_tableA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It mentions 'small sample' but does not explicitly state that the operation is read-only, that the sample may be non-representative, or any performance implications. While the act of 'returning rows' implies non-mutation, important caveats about data shape representativeness are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and purpose. Every sentence earns its place, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple preview tool with no output schema, the description adequately covers purpose, usage context, and parameter implications. It could be more explicit about the return format, but the mention of 'data shape' sufficiently implies columns and sample rows. Given the tool's simplicity, this is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already well-documented in the schema. The description adds only a general notion of 'small sample' which aligns with the 'limit' parameter but does not provide additional detail beyond what the schema already explains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return a small sample of rows from a table') and the resource ('table'). It also indicates the purpose ('understand its data shape') and distinguishes it from sibling tools by emphasizing inspection of JSON structures and field formats, setting it apart from query or metadata tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool: 'before writing queries' to inspect JSON columns or field formats. However, it does not explicitly state when not to use it or mention alternatives, though the sibling list implies it is a lightweight preview compared to full query tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_table_fieldsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. It implies a read-only metadata fetch ('Get') and adds useful context that field IDs are required for later queries. However, it does not explicitly state whether this is a safe read operation, mention authorization requirements, or note return format/limitations, leaving important behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with a clear purpose statement followed by a critical usage directive. Every word earns its place, with no filler or redundancy. This is exemplary conciseness for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one well-documented parameter and no output schema, the description is complete: it states what it returns (field IDs, names, types), when to use it (before query_dataset), and why (filters/aggregations need field IDs). There are no obvious gaps given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the single parameter (table_id) with its own description ('The table ID (use list_tables to find it)'). The tool description adds minimal param-specific meaning beyond referencing the table context and the need for field IDs, so it does not elevate above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Get field IDs, names, and types for a table' — a specific verb ('Get') with a clear resource ('field IDs, names, and types for a table'). This clearly distinguishes it from sibling tools like list_tables (which lists tables) and query_dataset (which queries data), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'You MUST call this before building query_dataset calls — field IDs are required for filters and aggregations.' This names a dependent tool and reason, providing clear context. However, it does not explicitly mention when not to use it or name alternatives, so it falls short of the full 'when/when-not/alternatives' bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cross_db_overlapA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and discloses key behavior: fetching both sets, in-memory intersection, a 500K row cap per side, and cancellation support. This is strong transparency, though it doesn't discuss return format or error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose/example, handling mechanism, and limits/cancellation. Each sentence earns its place, with no filler or redundancy. The structure is front-loaded and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with nested parameters and no output schema, the description covers the core mechanics, limitations, and a real-world example. It doesn't explain the return value format or sample_size specifics, but these are inferable or covered by the schema, making it adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds minimal parameter-specific guidance (only mentions adding filters for large tables), but doesn't elaborate on source structure, sample_size behavior, or filter syntax beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Compare a field across two databases to find overlapping values') with a concrete example (email addresses across CRM and marketing databases). It clearly distinguishes this tool from siblings by focusing on cross-database intersection, which no other sibling handles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on when to use the tool (when cross-DB joins are limited) and practical guidance (add filters for tables larger than 500K rows). While no explicit alternatives are named, the context makes usage conditions clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_card_metadataA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the non-execution behavior and specifies the returned definition components (query type, database, SQL/MBQL), which is valuable transparency. It does not mention permissions or error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose and key contrast without extra words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter schema and no output schema, the description's mention of expected return content (query type, database, SQL/MBQL) is sufficient for an agent to select and invoke the tool correctly. It lacks only a note on error handling, but that is minor for this low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage of the single parameter (card_id) with its description. The tool description offers no additional parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and names the resource 'saved question's definition', explicitly contrasting with execution by adding 'without executing it'. This clearly distinguishes it from sibling tools like run_saved_question.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without executing it' implies use when only metadata is needed, but it does not explicitly name alternative tools or provide when-not scenarios. This is clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_datasetA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on full responsibility for behavioral disclosure. It reveals auto-pagination through the 2000-row cap, the 500K row limit, the file path returned in the response, and configurable output directory via environment variable with default. While it doesn't discuss error handling or authorization, it covers the key operational traits an agent needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a distinct purpose: stating the action and key capability, giving usage context, and noting limits and configurability. It is front-loaded and without redundancy, every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately complex export tool with no output schema, the description addresses all essential aspects: purpose, usage case, alternative, row cap, output location configuration, and the return value (file path). It leaves no significant gaps that would cause an agent to misuse the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all parameters with 100% coverage (fields, filter, format, order_by, database_id, table_id). The description adds a small hint about using filters for larger tables but doesn't deeply expand parameter semantics beyond what the schema provides. Baseline 3 applies given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool exports full row-level data to a local CSV or JSON file, with a specific verb and resource. It distinguishes itself from siblings by noting auto-pagination through the 2000-row cap and explicitly contrasting with query_dataset for analytical queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance ('when you need complete data for external analysis') and names the alternative tool ('For analytical queries, use query_dataset with aggregations instead'). It also gives a practical constraint for larger tables ('add filters for larger tables').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_databasesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that it returns all databases and specifies the output fields, signaling a read-only listing operation. It doesn't mention pagination or access restrictions, but for a simple list tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action and resource, with no unnecessary words. It is maximally concise while still conveying the essential scope and return fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there are no parameters and no output schema, the description completely covers what the tool does and returns. It states the action, the resource scope, and the returned attributes, leaving no ambiguity for an agent selecting or invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the baseline score is 4. The description needs to explain no parameter details because there are none; the schema already fully covers this with an empty properties object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'List', the resource 'databases', and the scope 'all databases connected to Metabase' with specific fields (ID, name, engine type). This distinguishes it from sibling tools like list_tables, which focus on tables.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this tool to enumerate available databases and their basic attributes. It doesn't explicitly mention alternatives or exclusions, but its purpose is self-evident relative to the sibling tools, so no further guidance is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_datasetA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the hard 2000-row server cap, pagination behavior, the is_truncated flag, and limits on pagination (<10K rows). It stops short of detailing error handling or auth, but adds important context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence adds value: core action, limits, pagination strategy, alternatives, and examples. It is not as terse as an ideal high-score, but it is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description includes key behavioral details (row cap, is_truncated, pagination) and directs users to alternative tools. It covers the essential operational knowledge needed to use this tool effectively, making it complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so a baseline of 3 is warranted. The description adds MBQL examples for aggregation, filter, and breakout, clarifying parameter syntax that would otherwise be cryptic. This exceeds the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific verb and resource: 'Run an MBQL query against a Metabase database.' It distinguishes itself from sibling tools (run_native_query, run_saved_question, export_dataset) by focusing on MBQL querying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use alternatives: use export_dataset for large exports, use a date breakout for time comparisons, and use pagination only for small result sets (<10K rows). This gives clear decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.