Server Quality Checklist

Tool Scores

• query_funnel_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the tool's analytical purpose, it doesn't describe important behavioral traits: whether this is a read-only operation, what permissions might be required, potential rate limits, response format, or data freshness. For a query tool with 6 parameters, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently state the tool's purpose and utility. The first sentence establishes the core function, and the second explains its analytical value. There's no wasted text, though it could be slightly more structured with explicit usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with 6 parameters, no annotations, and no output schema, the description is insufficiently complete. It lacks critical context about behavioral traits (read-only status, permissions, rate limits), output format, and when to use versus sibling tools. While the schema covers parameters well, the description doesn't compensate for the missing annotations and output schema, leaving the agent with incomplete operational understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides no parameter-specific information beyond the tool's general purpose. However, with 100% schema description coverage, all 6 parameters are well-documented in the input schema with clear descriptions, enums for 'interval', and required fields. The description doesn't add value beyond the schema, but the schema provides adequate parameter documentation, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get funnel conversion data' with the specific goal of 'analyzing conversion rates through multi-step user flows and identifying drop-off points.' This provides a specific verb ('Get') and resource ('funnel conversion data') with clear analytical intent. However, it doesn't explicitly distinguish this from sibling tools like 'query_insights_report' or 'query_segmentation_report' which might also analyze user behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance, stating only that it's 'Useful for analyzing conversion rates through multi-step user flows and identifying drop-off points.' This implies context but doesn't specify when to use this tool versus alternatives like 'query_insights_report' or 'query_segmentation_report' from the sibling list, nor does it mention prerequisites or exclusions. No explicit when/when-not guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_insights_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'Get' operation but doesn't clarify if it's read-only, requires authentication, has rate limits, returns paginated results, or what format the output takes. The description adds minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that efficiently convey the core purpose. Every word earns its place, with no redundant information or unnecessary elaboration. It's appropriately sized for a tool with good schema documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what insights reports contain, what format they return in, whether date filtering is required, or how this differs from other query_* report tools. The minimal description leaves significant gaps in understanding the tool's behavior and output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions 'saved insights reports' which aligns with the report_id parameter but provides no additional syntax, format, or usage details for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'saved insights reports', with additional context about accessing 'pre-configured analytics reports and dashboards'. It distinguishes from siblings like query_funnel_report or query_segmentation_report by focusing on insights reports specifically, though it doesn't explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like query_funnel_report or query_segmentation_report. It mentions 'useful for accessing pre-configured analytics reports' but gives no context about prerequisites, exclusions, or comparative use cases with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_retention_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. The description mentions analyzing patterns but doesn't disclose important behavioral traits like whether this is a read-only operation, what permissions are required, whether it's resource-intensive, what the output format looks like, or any rate limits. For a complex analytics tool with 9 parameters, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that efficiently convey the tool's purpose. The first sentence states what it does, and the second explains its utility. There's no wasted verbiage, though it could be slightly more structured by front-loading more specific information about the analysis type.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, analytics function) and lack of both annotations and output schema, the description is insufficiently complete. It doesn't explain what the tool returns, what format the retention analysis takes, whether it's a heavy operation, or how results should be interpreted. For a sophisticated reporting tool, users need more context about output and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 9 parameters thoroughly. The description adds no parameter-specific information beyond what's in the schema. It mentions 'retention patterns' and 'cohort behavior' which aligns with parameters like born_event and interval, but doesn't provide additional semantic context beyond what the schema descriptions already cover.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze user retention patterns' with specific goals of 'understanding how well you retain users over time and identifying cohort behavior.' It uses a specific verb ('analyze') and identifies the resource ('user retention patterns'), but doesn't explicitly differentiate from sibling tools like query_funnel_report or query_segmentation_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the tool is 'useful for understanding how well you retain users over time' but doesn't specify when to choose it over other reporting tools like query_funnel_report or query_segmentation_report, nor does it mention any prerequisites or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_segmentation_reportC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the analytical purpose but doesn't mention any behavioral traits: no information about whether this is a read-only operation, potential rate limits, authentication requirements, data freshness, or what the output format looks like. For a query tool with 7 parameters, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that directly address the tool's purpose and utility. The first sentence states the core functionality, and the second explains its analytical value. There's no wasted verbiage, though it could be slightly more structured by front-loading more specific information about what makes this segmentation unique.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 7-parameter query tool with no annotations and no output schema, the description is insufficiently complete. It doesn't address what the tool returns (no output schema exists), doesn't mention any constraints or limitations, and provides minimal behavioral context. Given the complexity of segmentation analysis and the rich parameter set, users need more guidance about what to expect from this operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no specific parameter information beyond what's in the schema - it doesn't explain how parameters interact, provide examples of valid 'where' JSON strings, or clarify the relationship between 'event' and 'on' parameters. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Segment events by properties' with the goal of 'analyzing how different user segments behave and comparing event patterns across groups.' This specifies both the action (segment) and resource (events by properties), though it doesn't explicitly differentiate from sibling tools like query_funnel_report or query_retention_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions the tool is 'useful for analyzing how different user segments behave,' but doesn't specify when to choose it over sibling tools like query_funnel_report, query_insights_report, or aggregated_event_property_values. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_eventsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the time range ('last 31 days') and high-level use cases, but it doesn't cover critical behavioral aspects such as whether this is a read-only operation, if there are rate limits, authentication requirements, or what the output format looks like (e.g., list structure, pagination). For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with two sentences: the first states the core functionality, and the second provides usage context. There's no unnecessary repetition or fluff, and it's front-loaded with the main purpose. It could be slightly more detailed without losing conciseness, but it efficiently conveys key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a data querying tool with 3 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., safety, performance), output format, and explicit differentiation from siblings. While it covers the basic purpose and usage context, it doesn't provide enough information for an agent to fully understand how to invoke and interpret results, especially without structured output guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters (project_id, type, limit) with descriptions and enums for 'type'. The description doesn't add any parameter-specific details beyond what's in the schema, such as explaining the differences between 'general', 'average', and 'unique' event types. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but doesn't detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a list of the most common events over the last 31 days.' It specifies the verb ('Get'), resource ('most common events'), and time scope ('last 31 days'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'get_today_top_events' or 'aggregate_event_counts', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by stating it's 'useful for identifying key user actions, prioritizing feature development, and understanding overall platform usage patterns.' This gives some guidance on when to use it, but it doesn't explicitly mention when not to use it or name alternatives like 'get_today_top_events' for different time ranges, leaving room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_profilesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'query' and 'filtering,' which imply a read-only operation, but it doesn't specify whether this requires authentication, rate limits, pagination behavior, or what the output format looks like. For a tool with 4 parameters and no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose in the first sentence. The second sentence adds useful context without redundancy. It's appropriately sized for the tool's complexity, though it could be slightly more structured by explicitly linking to parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain the return values, error conditions, or behavioral details like pagination or authentication needs. For a query tool with filtering capabilities, this lack of context makes it harder for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'filtering' and 'profile properties,' which loosely relate to the 'where' and 'select' parameters, but it doesn't provide additional syntax, examples, or constraints. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Query user profiles with filtering.' It specifies the resource (user profiles) and the action (query with filtering). However, it doesn't explicitly differentiate from sibling tools like 'profile_event_activity' or 'list_saved_cohorts', which might also involve user data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some implied usage context: 'Useful for finding users based on profile properties and analyzing user segments.' This suggests when to use it, but it doesn't offer explicit guidance on when not to use it or name alternatives among the sibling tools, such as 'profile_event_activity' for event-related queries or 'list_saved_cohorts' for pre-defined groups.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aggregated_event_property_valuesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool is for analysis, implying it's likely a read-only operation, but doesn't confirm this or describe other traits like rate limits, authentication needs, error handling, or what the output looks like (e.g., format, pagination). For a tool with 6 parameters and no annotations, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences: the first states the purpose, and the second provides usage context. It's front-loaded with the core function and wastes no words. However, it could be slightly more structured by explicitly naming the tool's output or limitations to enhance clarity without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema), the description is minimally adequate. It covers the purpose and high-level usage but lacks details on behavioral traits, output format, and differentiation from siblings. For a data analysis tool with multiple parameters, more context is needed to fully guide an agent, but it meets a basic threshold without being misleading.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, providing clear details for all 6 parameters (e.g., 'event' as 'The event name to analyze properties for'). The description adds no additional parameter semantics beyond what's in the schema, such as examples or constraints. With high schema coverage, the baseline score is 3, as the schema adequately documents parameters without extra help from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Analyze specific event properties and their values.' It specifies the verb 'analyze' and the resource 'event properties and their values.' However, it doesn't explicitly differentiate from siblings like 'query_segmentation_report' or 'aggregate_event_counts,' which might also analyze event data but with different scopes or outputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance: 'Useful for understanding property distributions and identifying common values.' This suggests the tool is for statistical analysis of property values. However, it doesn't explicitly state when to use this tool versus alternatives (e.g., 'query_segmentation_report' for segmentation or 'aggregate_event_counts' for event counts), nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aggregate_event_countsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral disclosure. It mentions the tool is 'useful for analyzing trends' but doesn't specify whether it's read-only, requires authentication, has rate limits, returns paginated results, or what format the counts are in. For a data query tool with zero annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise: two sentences that directly state the purpose and utility without redundancy. Every word earns its place, and it's front-loaded with the core functionality. No structural issues or wasted verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema, no annotations), the description is insufficient. It doesn't explain what the output looks like (counts per time period), how results are structured, or any limitations (e.g., date range constraints). For an aggregation tool with multiple parameters and no output schema, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no parameter-specific information beyond what's in the schema—it doesn't explain how parameters interact (e.g., how 'unit' affects aggregation) or provide examples. Baseline 3 is appropriate when the schema does all the work, though the description could have added value with context like default behaviors.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get event counts over time periods' specifies the verb (get) and resource (event counts) with temporal aggregation. It distinguishes from siblings by focusing on count aggregation rather than property values, funnels, or profiles. However, it doesn't explicitly name alternatives like 'query_segmentation_report' for similar analytics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context: 'Useful for analyzing event volume trends and identifying patterns in user activity over time' suggests when to use it. However, it lacks explicit guidance on when to choose this tool over alternatives like 'query_insights_report' or 'get_top_events', and doesn't mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• custom_jqlB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool runs queries for analytics, but doesn't disclose critical behavioral traits such as whether it's read-only or has side effects, authentication requirements, rate limits, error handling, or the format of results. For a query tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: it starts with the core purpose ('Run custom JQL queries') and follows with a brief context sentence. Both sentences earn their place by clarifying the tool's role and use case, with zero wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a custom query tool with no annotations, no output schema, and 2 parameters, the description is incomplete. It lacks details on behavioral aspects (e.g., safety, performance), result format, error conditions, and how it differs from sibling query tools. While the schema covers parameters, the overall context for an agent to use this tool effectively is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't add any parameter-specific information beyond what the input schema provides. Since schema description coverage is 100% (both parameters have descriptions in the schema), the baseline score is 3. The description doesn't compensate with additional details about parameter usage, syntax examples, or constraints, so it doesn't exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run custom JQL (JSON Query Language) queries.' It specifies the action ('run') and resource ('JQL queries'), and distinguishes it from 'standard reports' by mentioning 'advanced analytics and custom data analysis beyond standard reports.' However, it doesn't explicitly differentiate from all sibling tools like 'query_funnel_report' or 'query_insights_report' that also involve querying, so it doesn't reach a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by stating it's 'useful for advanced analytics and custom data analysis beyond standard reports,' which implies when to use this tool (for complex queries not covered by standard reports). However, it doesn't explicitly mention when not to use it or name specific alternatives among the sibling tools (e.g., use 'query_insights_report' for predefined insights vs. this for custom JQL), leaving the guidance somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_today_top_eventsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the tool is for 'quickly identifying' and 'real-time activity,' hinting at efficiency, but fails to disclose critical behavioral traits: whether it's read-only, potential rate limits, authentication needs, data freshness, or what 'top' means (e.g., by count, revenue). For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a second sentence explaining use cases. Both sentences earn their place by adding value, with no redundant or vague language. However, it could be slightly more structured by separating purpose from guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters with full schema coverage, the description is moderately complete. It covers the purpose and usage context adequately but lacks behavioral details (e.g., safety, performance) and output information, which are important for a tool fetching real-time data. This leaves gaps for an AI agent to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters (project_id, type, limit). The description adds no parameter-specific information beyond what's in the schema, such as explaining 'top' in relation to 'type' or 'limit.' With high schema coverage, the baseline is 3, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get today's top events from Mixpanel.' It specifies the resource (events) and temporal scope (today), and distinguishes it from siblings like 'get_top_events' by emphasizing 'today's' and real-time monitoring. However, it doesn't explicitly differentiate from all siblings like 'aggregate_event_counts' or 'query_insights_report'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context: 'Useful for quickly identifying the most active events happening today, spotting trends, and monitoring real-time user activity.' This suggests when to use it (for real-time, top events analysis), but lacks explicit guidance on when not to use it or alternatives among siblings (e.g., vs. 'get_top_events' for non-today data or 'query_segmentation_report' for detailed analysis).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_cohortsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool lists cohorts and is useful for discovery and ID retrieval, but doesn't describe key behaviors like whether it returns all cohorts or is paginated, what the output format looks like, or any permissions or rate limits. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by a useful context sentence. It's appropriately sized with two sentences that add value, though it could be slightly more structured by explicitly separating purpose from usage guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and usage context but is incomplete. It doesn't cover output details, behavioral traits, or advanced usage scenarios, which are important for a tool with one parameter and no structured safety or output information. This is minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents the single parameter 'project_id' with its type and optionality. The description adds no additional parameter information beyond what's in the schema, such as format examples or default values. This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('user cohorts in the project'), making the purpose understandable. It distinguishes from siblings like 'list_saved_funnels' by specifying cohorts, but doesn't explicitly contrast with other list/query tools like 'query_profiles' or 'query_segmentation_report' that might also involve user segments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by mentioning it's 'useful for discovering existing user segments and getting cohort IDs for analysis,' which suggests when to use it. However, it lacks explicit guidance on when to choose this tool over alternatives like 'query_profiles' or 'query_segmentation_report' for similar purposes, and doesn't mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• profile_event_activityB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool retrieves data ('Get data') but doesn't disclose behavioral traits like whether this is a read-only operation, potential rate limits, authentication requirements, or what format the returned data takes. The description is functional but lacks operational transparency needed for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences. The first sentence states the core purpose, and the second provides use case context. Both sentences add value without redundancy. It could be slightly more front-loaded by mentioning key constraints earlier, but overall it's well-structured and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with full schema coverage but no annotations and no output schema, the description provides adequate purpose and context but lacks behavioral and output information. For a data retrieval tool with multiple parameters, the description should ideally mention what kind of data structure is returned or any limitations. It's minimally viable but has clear gaps in operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. This meets the baseline expectation when schema coverage is complete, but doesn't provide additional semantic context about how parameters interact or affect results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get data for a profile's event activity' (verb+resource). It distinguishes from siblings by focusing on individual user journeys rather than aggregated data, but doesn't explicitly name alternatives. The description adds context about use cases (understanding journeys, troubleshooting, analyzing behavior), which enhances clarity beyond a basic statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('useful for understanding individual user journeys...') which suggests when to use it, but doesn't explicitly state when NOT to use it or name specific alternatives. It doesn't provide clear guidance on choosing between this tool and sibling tools like query_profiles or aggregated_event_property_values for similar purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_saved_funnelsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool lists funnels and is useful for discovery and ID retrieval, but it doesn't disclose key behavioral traits such as whether it returns all funnels or is paginated, what the output format is, or any permissions or rate limits. For a listing tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: it starts with the core purpose ('List available saved funnels in the project.'), followed by a concise usage note. Both sentences earn their place by providing essential information without waste, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is somewhat complete but has gaps. It explains the purpose and usage but lacks details on output behavior, which is critical for a listing tool. Without annotations or an output schema, the description should ideally cover more about what is returned, but it only partially meets the needs for contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with one parameter ('project_id') documented as 'The Mixpanel project ID. Optional since it has a default.' The description adds no additional parameter semantics beyond what the schema provides. According to the rules, with high schema coverage (>80%), the baseline score is 3, as the schema does the heavy lifting and the description doesn't compensate with extra details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List available saved funnels in the project.' It specifies the verb ('List') and resource ('saved funnels'), and distinguishes it from siblings by focusing on funnels rather than events, cohorts, or other reports. However, it doesn't explicitly differentiate from similar listing tools like 'list_saved_cohorts' beyond the resource name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidelines: it's 'useful for discovering existing funnel analyses and getting funnel IDs for further analysis.' This suggests when to use it (for discovery and ID retrieval), but it doesn't explicitly state when not to use it or name alternatives (e.g., 'query_funnel_report' for detailed analysis). The guidance is helpful but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.