Server Quality Checklist

Tool Scores

• query_trendsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It states the output is a time series and mentions optional breakdown, but it omits critical details like authentication requirements, rate limits, error handling, what happens when events are missing, or whether results are capped. The agent lacks information about side effects or invocation consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, perfectly front-loaded with the core action. Every sentence adds value: the first states the primary function, the second specifies the return format and optional feature. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and moderate complexity (5 parameters, 1 required), the description covers the return type (time series) and breakdown capability but lacks details on date range handling, default behavior without breakdown, pagination, or errors. It is minimally sufficient for a basic understanding but incomplete for confident invocation, especially without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter having a description. The tool description adds overall context ('returns a time series...') but does not enhance understanding of individual parameters beyond what the schema provides. For example, it doesn't explain how 'breakdown' interacts with the interval or how defaults for date_from and date_to work. Baseline 3 is appropriate as the description reiterates schema concepts without significant additional value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool runs a trends query to get event counts over time, returning a time series optionally broken down by a property. The verb 'run' and resource 'trends query' are specific, and the tool distinguishes itself from siblings like list_events (which likely lists raw events) by focusing on aggregated time series data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as list_events or get_insights. It does not specify prerequisites, typical use cases, or scenarios where this tool is preferred. The agent is left to infer usage from the tool name and schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_feature_flagsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits like authentication, rate limits, pagination, or side effects. It only states what is listed, missing any behavioral context. For a read-only listing, even minimal transparency would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that directly states the action and output fields. It is front-loaded, efficient, and contains no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description is clear for a simple list tool, it lacks mention of potential pagination, exhaustive nature of the list, or any limitations. Given no output schema, some behavioral context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the single parameter (active_only) with a description in the schema itself (100% coverage). The tool description does not add any extra meaning beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists feature flags and specifies the fields included (keys, enabled status, rollout percentages, targeting conditions). This is specific and leaves no ambiguity about what the tool does, especially given sibling tools are in different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines are provided. The description does not indicate when to use this tool over siblings, when not to use it, or any prerequisites or context. The agent is left to infer usage from the tool's purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_personsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It mentions returning a paginated list but omits details like auth requirements, rate limits, or side effects. The read-only nature is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The purpose and two modes are front-loaded, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality but lacks details on return format, pagination behavior, and potential errors. Given good schema coverage and no output schema, some gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with well-described parameters. The description confirms the two modes but adds no additional meaning beyond the schema's parameter descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up person profiles in PostHog, with two modes: by distinct ID or paginated list. This distinguishes it from sibling tools (get_feature_flags, get_insights, etc.) which address different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. While siblings are different, the description does not explicitly state when to use get_persons, nor does it mention when-not or provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_insightsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must convey behavioral traits. It indicates the tool is read-only (fetching saved insights) and mentions caching via the 'refresh' parameter, but it does not disclose error handling (e.g., non-existent IDs), performance implications, or authentication requirements. Sufficient for a simple retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the action and resource, and avoids filler. Every word contributes to understanding, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should provide context on return values. It says 'return its result data' but does not describe the structure or content of that data. Given the simplicity of the tool (2 params, no nested objects), this is acceptable but leaves some ambiguity about the response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for both parameters (100% coverage). The tool description adds no new parameter information beyond what the schema offers, so a baseline score of 3 is appropriate. The 'refresh' parameter's purpose is clearly stated in the schema, and the description does not expand on it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('Fetch') and resource ('PostHog insight by ID') and mentions common use cases (funnels, retention, trend charts). It implicitly distinguishes from sibling tools like 'query_trends' which likely handle ad-hoc queries rather than saved insights, though no explicit differentiation is made.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states when to use the tool ('retrieve saved insights') and provides examples, but it does not specify when not to use it or mention alternative tools like 'query_trends' for ad-hoc analysis. The guidance is adequate but lacks explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_eventsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description mentions the default limit of 100 events and filtering support, but lacks details on ordering, pagination, or how date range interacts with 'recent'. With no annotations, more behavioral context would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the main purpose. No wasted words, but could be structured to highlight key behaviors first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and moderate complexity, the description covers filtering and default limit but does not mention return format, authentication, or error conditions. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 5 parameters. The description adds no extra meaning beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches events from PostHog and supports filtering by name and date range. It is specific and distinguishes from siblings like get_feature_flags or get_insights.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the description (fetch recent events with filters) but there is no explicit guidance on when to use this tool over alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.