Server Quality Checklist

Tool Scores

• list_activitiesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool lists activity types and notes 'Free (0 credits)', which implies no cost, but lacks details on permissions, rate limits, response format, or whether it's read-only (implied by 'List' but not explicit). For a tool with zero annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List available activity types') and adds essential details (categories and cost). There is zero waste, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has gaps. It explains the return value (activity types and cost), which is good, but lacks usage guidelines and full behavioral transparency. For a low-complexity tool, this is minimally viable but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameters need documentation. The description adds value by explaining what the tool returns (activity types with categories and cost info), which compensates for the lack of output schema. This exceeds the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'available activity types', specifying the categories (FREE, PUBLIC, PRIVATE). It distinguishes from siblings like 'create_activity' or 'list_sessions' by focusing on activity types rather than sessions or creation. However, it doesn't explicitly differentiate from all siblings (e.g., 'get_stats' might also involve activity data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_sessions' or 'get_stats'. It mentions 'Free (0 credits)' which hints at cost implications but doesn't specify context or prerequisites for usage, leaving the agent to infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_sessionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context: it's free (0 credits), includes elapsed_sec for active sessions, and specifies a single session lookup. However, it doesn't cover other behavioral aspects like error handling, response format, or whether it's read-only (implied but not stated).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose and key details (elapsed_sec, free), and the second sentence documents the parameter. Every sentence earns its place with no wasted words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no annotations, no output schema), the description is somewhat complete but has gaps. It covers the purpose, cost, and parameter semantics, but lacks details on output format, error cases, or how it integrates with siblings. For a simple lookup tool, this is adequate but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema, which has 0% coverage. It explains that ak_id is 'The session ID to look up,' providing clear semantics. Since there's only one parameter and the schema lacks descriptions, the description effectively compensates, though it could add more detail like format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get details for a single session, including elapsed_sec if active.' It specifies the verb ('Get details') and resource ('a single session'), distinguishing it from siblings like list_sessions (which lists multiple sessions) and start/stop_session (which modify sessions). However, it doesn't explicitly differentiate from get_stats, which might overlap in functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'a single session' and providing the required parameter (ak_id), suggesting it's for retrieving details of a specific session. However, it lacks explicit guidance on when to use this tool versus alternatives like list_sessions (for multiple sessions) or get_stats (for broader statistics), and does not mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_activityA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key traits: the activity is private (visibility constraint), free (cost implication), and has a character limit (max 64 characters). However, it lacks details on permissions, error handling, or response format, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a clear Args section. Every sentence adds value: the first states what it does and key constraints, the second lists parameters with useful details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete. It covers the creation action, privacy, cost, and parameters, but lacks information on return values, error cases, or integration with siblings. For a 2-param tool with minimal structured data, it's adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning: activity_name has a max length constraint (64 characters) and description is optional, which aren't in the schema. This covers both parameters well, though it doesn't detail format or validation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('custom PRIVATE activity'), specifying it's visible only to the user's account and free. It distinguishes from siblings like list_activities (which lists) and start_session (which starts sessions). However, it doesn't explicitly differentiate from other potential creation tools, keeping it at 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for creating private activities, but doesn't explicitly state when to use this vs alternatives like list_activities or start_session. No exclusions or prerequisites are mentioned, leaving some ambiguity about context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool is 'Free (0 credits)' which is useful behavioral context, and mentions it returns both 'completed and active sessions'. However, it doesn't describe response format, pagination behavior beyond parameters, error conditions, or authentication requirements. The description adds some value but leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with purpose statement first, then organized parameter documentation. Every sentence serves a purpose - the first states the tool's function and cost, the parameter section provides essential usage details. It could be slightly more concise by combining some parameter explanations, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with 0% schema coverage and no output schema, the description does well on parameters but leaves gaps. It doesn't describe the return format (what a 'session' object contains), doesn't mention error handling, and while it documents parameters thoroughly, the overall context for a listing tool with filtering and pagination could be more complete about result structure and limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It provides detailed parameter documentation including: date format specifications (YYYY-MM-DD), activity_id meaning (0 = no filter), default values (limit default 20, offset default 0), and range constraints (max 50). This adds substantial meaning beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'completed and active sessions', making the purpose immediately understandable. It distinguishes from siblings like 'check_session' (specific session) and 'start_session' (creation). However, it doesn't explicitly contrast with 'list_activities' which might be a related listing tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Free (0 credits)' which provides some usage context about cost, but doesn't explicitly state when to use this tool versus alternatives like 'check_session' for individual sessions or 'list_activities' for activities. It implies usage through parameter descriptions but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing key behavioral traits: it mentions a cost ('Costs 1 credit'), describes what it returns (specific metrics), and explains a performance benefit ('Offloads all calendar arithmetic to the server'). It doesn't cover error handling or rate limits, but given the lack of annotations, this is a solid effort that adds meaningful context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured: three sentences that efficiently convey cost, return values, and a key benefit. Every sentence earns its place by providing essential information without waste, and it's front-loaded with the core purpose. No extraneous details are included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple read operation with no parameters) and the absence of both annotations and an output schema, the description is quite complete: it explains what the tool does, its cost, what it returns, and a server-side advantage. However, it doesn't detail the exact format of the return values (e.g., data types), which could be helpful since there's no output schema, leaving a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description appropriately doesn't discuss parameters, as none exist, and instead focuses on the tool's output and behavior, which adds value without redundancy. No compensation is needed since there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get pre-computed session aggregates' with specific metrics returned (total sessions, total seconds, current streak, credits remaining). It distinguishes itself from siblings like list_sessions by focusing on aggregated statistics rather than raw session listings. However, it doesn't explicitly contrast with check_session which might also provide some statistical information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the mention of 'Costs 1 credit' and 'Offloads all calendar arithmetic to the server,' suggesting this tool should be used when you need aggregated metrics without client-side computation. However, it doesn't explicitly state when to use this versus alternatives like list_sessions for raw data or check_session for session status, nor does it mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_sessionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it costs 1 credit, automatically records time, returns an ak_id needed for future operations, and has default values for all parameters. It doesn't mention error conditions or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Perfectly structured with purpose statement first, cost disclosure second, behavioral details third, and parameter explanations in a clear Args section. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations and no output schema, the description is quite complete - covering purpose, cost, behavior, and all parameters. It could mention what happens if the session fails to start or provide more detail about the return format beyond 'including its ak_id'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all three parameters: activity_id (with default and reference to list_activities), timezone (with format example and default), and intended_sec (with meaning of default value). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Start' and resource 'new behavioral session', distinguishing it from sibling tools like 'stop_session' or 'check_session'. It specifies this creates a new session with a unique identifier (ak_id) needed for later operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (to begin a session) and references 'list_activities' for activity options, but doesn't explicitly state when NOT to use it or compare it directly to alternatives like 'create_activity'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_sessionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and adds valuable behavioral context: it discloses that the operation is 'Free (0 credits)', explains server-side computation of 'actual_sec' automatically, and clarifies that the agent doesn't need to track elapsed time. This goes beyond the basic 'stop' action to include cost and automation details, though it lacks information on permissions or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by key behavioral details (cost, automation), and ends with parameter explanation. Every sentence earns its place with no redundant or vague language, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (a mutation operation with no annotations or output schema), the description is mostly complete: it covers purpose, usage context, behavioral traits, and parameter semantics. However, it lacks details on potential side effects (e.g., what happens to session data after stopping) or error cases, leaving minor gaps for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate fully. It explicitly defines the single parameter 'ak_id' as 'The session ID returned by start_session', adding crucial meaning not present in the schema (which only lists it as 'Ak Id' with type integer). This provides clear semantics and usage guidance for the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Stop') and resource ('an active session'), distinguishing it from siblings like 'start_session' (which initiates) and 'check_session' (which queries). It goes beyond tautology by specifying the action on a particular resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by mentioning it stops 'an active session' and references 'start_session' as the source for the required session ID. However, it does not explicitly state when NOT to use this tool (e.g., for inactive sessions) or name alternatives like 'check_session' for verification, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.