Server Quality Checklist

Tool Scores

• redash_queryC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose behavioral traits such as side effects of delete/archive, synchronous vs asynchronous run, authentication needs, or rate limits. The description is silent on these critical aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) and front-loaded with the purpose. However, it is terse and could benefit from grouping or examples. Still, it is efficient with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (12 parameters, many actions, no output schema, no annotations), the description is incomplete. It does not explain return values, error handling, or action-specific behaviors like scheduling or exporting. Considerable gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (83%), so the baseline is 3. The description adds some context for the query parameter (MongoDB JSON format) and max_rows (default, prevents huge responses) but overall adds minimal value beyond the schema. The main description only lists actions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Manage Redash queries' which is generic, and then lists actions. It conveys the general domain but lacks specificity about what the tool does as a whole. It does not differentiate from sibling tools like redash_alert or redash_dashboard.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. No when-not-to-use, prerequisites, or exclusions are mentioned. It merely lists possible actions without context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash_dashboardC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, and the description does not disclose behavioral traits such as destructive effects (e.g., delete), authorization needs, or rate limits. The actions imply different behaviors but are not explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and concise, but it lacks structure. It is a single sentence listing actions, which is efficient yet insufficient for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters, no output schema, and no behavioral details, the description is incomplete. It does not explain return values, action-specific requirements, or interactions between parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds little beyond the schema: it lists actions but does not clarify which parameters apply to each action (e.g., id required for get, publish, delete). With 60% schema description coverage, the description does not compensate for gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it manages Redash dashboards and lists five specific actions (list, get, create, publish, delete), clearly indicating the verb and resource. It distinguishes from sibling tools (redash_alert, etc.) by focusing on dashboards.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives or when not to use it. The description only enumerates actions without clarifying contexts or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash_vizC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only says 'Create visualizations' without detailing side effects, permissions, idempotency, or limitations. Does not reveal that some parameters (datetime, stacked, suffix) are type-specific.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise single sentence. However, it front-loads the main purpose and lists types efficiently. Could be slightly improved by grouping type-specific parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters and no output schema, the description is too brief. Doesn't explain that datetime/stacked/suffix are only relevant for certain types, nor what the tool returns (e.g., visualization ID). Missing key context for proper invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is ~63% (5 of 8 params described). The description merely repeats the type enum already in schema. Does not clarify query_id (e.g., must exist) or name (e.g., unique?). No additional value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Create visualizations' and lists supported types (pie, line, bar, counter). Sibling tools (alerts, dashboards, etc.) are distinct, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. Doesn't mention prerequisites (e.g., existing query_id) or situations where other tools (like redash_query) might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash_alertB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the brief description does not disclose behavioral traits such as authentication requirements, rate limits, side effects of delete/update, or error handling. For a multi-action tool, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, front-loading the core purpose and action list. It is efficient with words, but could benefit from brief structuring around actions for improved readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 8 parameters and multiple actions, the description lacks context on return values for each action, prerequisites, or default behavior. Schema coverage helps but the description alone is incomplete for safe autonomous use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (88%), so the parameter semantics are mostly documented by the schema. The description adds no extra parameter information beyond repeating the action names, which are already enumerated in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Manage Redash alerts' and lists all five actions (list, get, create, update, delete), clearly indicating the tool's purpose and distinguishing it from sibling tools that manage other Redash entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description only lists possible actions without providing guidance on when to use each action or when to prefer this tool over alternatives. No context is given for selection, leaving the agent to infer from schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash_data_sourcesB
  3.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits such as authentication requirements, pagination, or whether it returns metadata. The tool is a simple list but lacks necessary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the action and resource. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, output schema, or annotations, the description is adequate for understanding the basic purpose. However, it lacks any additional context like authentication or output format, leaving gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the schema is complete. The description adds no extra meaning but is sufficient for an empty parameter list. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all available data sources' clearly states the verb (List) and resource (data sources). It distinguishes from sibling tools like redash_query or redash_dashboard which have different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The description implies a straightforward listing but does not address exclusions or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• redash_widgetA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool performs add, move, and delete operations, implying mutation, but lacks details on reversibility, permissions, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—one sentence—and front-loads the core purpose. It is efficient, though slightly terse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no output schema), the description is minimal. It omits return values, error cases, and any usage context, leaving the agent with insufficient information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (88%), so the schema already documents parameter purposes. The description adds some context by grouping parameters by action (e.g., 'for add/move'), but does not provide significant additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Manage dashboard widgets' and lists specific actions (add, move, delete), using a verb+resource pattern that distinguishes it from sibling tools like redash_dashboard or redash_query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by listing actions but provides no explicit guidance on when to use this tool versus alternatives. For example, it does not mention that to create a dashboard one should use redash_dashboard instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.