Server Quality Checklist

Tool Scores

• count_karte_eventsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'gets the count' which implies a read-only operation, but doesn't specify whether it requires authentication, has rate limits, returns paginated results, or what happens with null parameters. The default date values are mentioned, but other behavioral aspects are undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by parameter documentation. Each parameter explanation is concise and informative. While efficient, the Japanese-only text might limit accessibility for non-Japanese speaking agents, and the structure could be slightly improved with clearer section headers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description doesn't need to explain return values. However, for a 4-parameter counting tool with no annotations, the description should ideally provide more behavioral context about what 'count' means (e.g., is it aggregated counts when using group_by?). The parameter documentation is excellent, but overall context about the tool's behavior remains incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic information for all 4 parameters. It explains what each parameter does, provides example syntax ('event_name = "purchase"'), specifies date format requirements (YYYYMMDD), and documents default values (2 days ago for date_from, yesterday for date_to). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'KARTEイベントの件数を取得する' (Get the count of KARTE events). It specifies the verb ('取得する' - get/count) and resource ('KARTEイベント' - KARTE events), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'query_karte_events' which might also retrieve event data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'query_karte_events' or 'execute_karte_sql', nor does it specify use cases, prerequisites, or exclusions. The parameter documentation implies filtering and grouping capabilities, but no contextual usage instructions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_karte_events_schemaA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states that the tool retrieves metadata including column names, types, and descriptions, which implies a read-only operation. However, it doesn't cover aspects like error handling, authentication needs, rate limits, or response format details beyond metadata. The description adds some context but lacks comprehensive behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, consisting of two sentences that directly state the purpose and what is returned. There is no wasted language or redundancy, and it is front-loaded with the main action. Every sentence earns its place by adding clear information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity is low (0 parameters, read-only operation) and an output schema exists, the description is reasonably complete. It explains the purpose and the type of metadata returned, which aligns with the output schema's role in detailing return values. However, it could be more complete by addressing usage context or behavioral aspects like error cases, but for a simple schema retrieval tool, it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description doesn't mention any parameters, which is appropriate since none exist. It adds value by explaining what metadata is returned (column names, types, descriptions), compensating for the lack of parameter documentation. Baseline is 4 for 0 parameters, as the description provides useful output semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'KARTEイベントテーブルのスキーマを取得する' (Get the schema of the KARTE events table). It specifies the verb '取得する' (get/retrieve) and the resource 'スキーマ' (schema), making it understandable. However, it doesn't explicitly differentiate from sibling tools like 'query_karte_events' or 'execute_karte_sql', which might also involve schema-related operations indirectly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions what the tool does but doesn't specify scenarios for usage, prerequisites, or exclusions. Given sibling tools like 'count_karte_events' and 'query_karte_events', there's no indication of when schema retrieval is preferred over data querying or counting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_karte_eventsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: automatic date constraints with defaults, parameter defaults, and limit range (1-10000). However, it doesn't mention important aspects like whether this is a read-only operation, potential rate limits, authentication requirements, or what happens with large result sets beyond the limit parameter. The description adds useful context but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by detailed parameter documentation. It's appropriately sized for a 6-parameter tool. The only minor improvement would be front-loading more critical behavioral information before the parameter details, but overall it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, but with output schema), the description is quite complete. It thoroughly documents all parameters and their semantics. The presence of an output schema means the description doesn't need to explain return values. The main gap is lack of behavioral context around permissions, rate limits, or error conditions, but the parameter documentation is comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It does this excellently by providing detailed parameter documentation in the Args section: each parameter's purpose, format examples (YYYYMMDD, 'event_name = "click"'), default values, constraints (limit 1-10000), and special cases (None for no limit). This adds substantial meaning beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'KARTEイベントデータを取得する' (retrieve KARTE event data). It specifies the verb (retrieve) and resource (KARTE event data). However, it doesn't explicitly differentiate from sibling tools like count_karte_events or execute_karte_sql, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by mentioning automatic date constraints (default: 2 days ago to yesterday), which helps understand when to use it for date-filtered queries. However, it doesn't explicitly state when to use this tool versus alternatives like count_karte_events (for counting) or execute_karte_sql (for custom SQL), leaving the guidelines somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_karte_sqlA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and excels. It reveals critical behavioral traits: guardrails on SQL execution, mandatory constraints for specific tables, automatic LIMIT application (1000) by default, rejection of destructive SQL operations, and the dry_run option for scanning only. This provides comprehensive insight into how the tool behaves beyond basic execution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: it starts with the core purpose, followed by key behavioral rules, and ends with a structured Args section. Every sentence earns its place by providing critical information without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a SQL execution tool with 3 parameters, 0% schema coverage, no annotations, but an output schema present, the description is complete enough. It covers purpose, usage rules, behavioral constraints, and parameter semantics thoroughly. The output schema handles return values, so the description doesn't need to explain them, making this a well-rounded specification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully, which it does. The Args section explicitly explains each parameter's semantics: 'sql' is the SQL statement to execute, 'dry_run' confirms scan volume only when True, and 'no_limit' skips automatic LIMIT addition when True. This adds essential meaning beyond the bare schema, clarifying how parameters affect tool behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'カスタムSQLを実行する（ガードレール付き）' (Execute custom SQL with guardrails). It specifies the verb (execute) and resource (custom SQL) and distinguishes it from sibling tools like count_karte_events, describe_karte_events_schema, and query_karte_events by focusing on raw SQL execution rather than predefined queries or metadata operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: it states when to use specific constraints ('krt_pockyevent_v1_* テーブルへのクエリには _TABLE_SUFFIX 制約が必須' - queries to krt_pockyevent_v1_* tables require _TABLE_SUFFIX constraint), when alternatives apply (automatic LIMIT 1000 unless no_limit is True), and exclusions ('破壊的SQL（DROP, DELETE等）は拒否される' - destructive SQL like DROP, DELETE is rejected). This clearly guides when and how to use this tool versus other approaches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.