Server Quality Checklist

Tool Scores

• get-aggregationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'gets' data, implying a read-only operation, but doesn't mention any behavioral traits such as rate limits, authentication requirements, data freshness, or potential side effects. For a tool with 11 parameters and no annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and appropriately sized, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, no output schema, and no annotations), the description is insufficient. It doesn't explain what the aggregated data looks like, how results are formatted, or any constraints on usage. For a data retrieval tool with many parameters, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are documented in the input schema. The description adds no additional meaning or context beyond what's already in the schema, such as explaining relationships between parameters or providing usage examples. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get aggregated analytics data') and the source ('from Fathom'), which is specific and unambiguous. However, it doesn't differentiate this tool from its siblings (like 'list-events' or 'list-sites'), which might also retrieve analytics data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list-events' or 'list-sites'. It lacks explicit instructions on context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-current-visitorsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves current visitors but doesn't explain what 'current' means (e.g., real-time, last hour), whether it's a read-only operation, if there are rate limits, or what the output format looks like. This leaves significant gaps for a tool that likely involves data retrieval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that efficiently conveys the core purpose without any wasted words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of retrieving analytics data, no annotations, and no output schema, the description is insufficient. It doesn't cover behavioral aspects like data freshness, permissions, or response structure, leaving the agent with incomplete information to use the tool effectively in context with its siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters. The description adds no additional semantic context beyond what's in the schema (e.g., it doesn't clarify what 'detailed' information includes or provide examples). This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('current visitors for a Fathom Analytics site'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'list-events' or 'get-aggregation' which might also retrieve visitor-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or compare it to siblings like 'list-events' for historical data or 'get-aggregation' for summarized metrics, leaving the agent with no contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-sitesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action ('List all...') but doesn't disclose behavioral traits like pagination, rate limits, authentication requirements, or what happens if no sites exist. For a list operation with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose with zero wasted words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a list operation with no annotations and no output schema, the description is incomplete. It lacks details on return values, error handling, or behavioral constraints, which are crucial for an agent to use the tool effectively in this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'limit' fully documented in the schema. The description adds no additional meaning about parameters, such as default behavior or format details, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all Fathom Analytics sites on the account'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-account' or 'get-aggregation', which might also retrieve account-related data, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-account' or 'list-events'. It lacks context on prerequisites, such as authentication needs, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-accountB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states it 'gets' information, implying a read-only operation, but doesn't specify authentication requirements, rate limits, error conditions, or what 'account information' includes. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, with every word contributing to understanding. No waste or redundancy is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate but incomplete. It specifies what is retrieved ('account information') but not the format or scope of the return data. For a read operation with no structured output documentation, more detail on the response would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description adds no parameter information, which is appropriate here. Baseline is 4 for zero parameters, as no compensation is needed for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('Fathom Analytics account information'), making the purpose understandable. It doesn't explicitly differentiate from sibling tools like 'get-aggregation' or 'list-sites', but the specificity of 'account information' provides some implicit distinction. This is clear but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-aggregation' or 'list-sites'. It doesn't mention prerequisites, context, or exclusions. Without any usage instructions, the agent must infer based on tool names alone, which is insufficient for optimal selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-eventsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context by mentioning automatic pagination handling, which is a key behavioral trait beyond the basic 'list' function. However, it doesn't cover other aspects like rate limits, authentication needs, or error handling, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all events') and adds valuable behavioral context ('automatically handles pagination') without any wasted words. Every part earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the main action and pagination behavior, but lacks details on return values, error cases, or integration with sibling tools, leaving room for improvement in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already documents both parameters ('site_id' and 'limit') fully. The description implies filtering by site but doesn't add syntax or format details beyond what the schema provides, aligning with the baseline score when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('events for a Fathom Analytics site'), making the purpose evident. However, it doesn't explicitly differentiate from sibling tools like 'get-aggregation' or 'get-current-visitors', which might also retrieve event-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as 'get-aggregation' for summarized data or 'get-current-visitors' for real-time events. It lacks explicit when/when-not statements or named alternatives, offering only basic context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.