Server Quality Checklist

Tool Scores

• delete_scheduleC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the action without behavioral details. It doesn't disclose if deletion is permanent, requires specific permissions, has side effects, or provides confirmation. This is inadequate for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste. It's appropriately sized and front-loaded, efficiently conveying the core purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It lacks critical context like behavioral risks, confirmation details, or error handling, leaving significant gaps for safe agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'task_name' parameter. The description adds no additional meaning beyond implying the parameter identifies the schedule, matching the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('a scheduled task by name'), making the purpose immediately understandable. It doesn't distinguish from sibling tools like 'list_schedules' or 'schedule_recurring_task' beyond the obvious destructive nature, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing schedule), exclusions, or relationships with siblings like 'list_schedules' for verification or 'schedule_recurring_task' for creation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_session_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'Get[s] the current status and state', implying a read-only operation, but doesn't specify whether it requires authentication, has rate limits, returns real-time or cached data, or handles errors. This leaves significant behavioral gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste—it directly states the tool's purpose without redundancy or unnecessary elaboration. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and a simple parameter, the description is incomplete. It lacks details on what 'status and state' entail, potential return values, error conditions, or how it differs from sibling tools. For a tool with no structured data beyond the input schema, this minimal description is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'session_id' documented in the schema. The description adds no additional meaning about the parameter beyond what the schema provides, such as format examples or contextual usage. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'current status and state of a Jules session', which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'manage_session' which might also involve session operations, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'manage_session' or other session-related operations. It lacks context about prerequisites, typical use cases, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral context. It mentions two actions but doesn't disclose what 'approve_plan' does (e.g., triggers execution, changes session state), what 'send_message' is for (e.g., feedback to Jules, user notification), or any side effects, permissions, or rate limits. This leaves critical gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just one sentence that front-loads the core purpose. Every word earns its place, and there's no redundancy or fluff, making it efficient for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (managing sessions with mutations), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what happens after actions (e.g., response format, session state changes), error conditions, or how it fits into broader workflows with sibling tools, leaving the agent under-informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters well. The description adds no additional meaning beyond implying that 'message' is tied to 'send_message', which is already clear from the schema's enum and description. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('approve plans', 'send feedback') and resource ('active Jules session'), making it immediately understandable. However, it doesn't differentiate this tool from its siblings like 'get_session_status' or explain how session management relates to other session/coding/scheduling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an active session), exclusions, or relationships with sibling tools like 'get_session_status' for checking session state or 'create_coding_task' for task creation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_recurring_taskC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions server-managed execution and offline capability, which adds some context. However, it lacks critical details such as permissions required, whether this is a mutation (implied by 'schedule'), error handling, rate limits, or what happens on schedule conflicts. For a tool with 8 parameters and no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences that are front-loaded: the first states the core purpose, and the second adds behavioral context. There's no wasted text, but it could be slightly more structured by explicitly mentioning key parameters or usage scenarios.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no annotations, no output schema), the description is incomplete. It lacks details on return values, error conditions, interaction with sibling tools (e.g., how this relates to 'list_schedules'), and operational constraints. For a scheduling tool with multiple parameters, this leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 63% (5 out of 8 parameters have descriptions), so the description must compensate for gaps. The description adds no specific parameter information beyond what's implied by 'cron schedule' (relating to 'cron_expression'). It doesn't explain the purpose of parameters like 'task_name', 'prompt', or 'source' in context, nor does it clarify the role of defaults like 'auto_create_pr'. Baseline 3 is appropriate as the schema does moderate lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Schedule a Jules task to run automatically on a cron schedule.' It specifies the verb ('schedule'), resource ('Jules task'), and key behavior ('run automatically on a cron schedule'). However, it doesn't explicitly differentiate from sibling tools like 'create_coding_task' or 'list_schedules' beyond the scheduling aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal usage guidance. It mentions 'The server manages execution even when offline,' which implies reliability but doesn't specify when to use this tool versus alternatives like 'create_coding_task' for one-off tasks or 'delete_schedule' for removal. No explicit when/when-not instructions or prerequisites are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_schedulesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action ('List') but doesn't disclose behavioral traits such as pagination, sorting, error handling, or what 'locally-managed' entails. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded and appropriately sized for a simple listing tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It lacks details on return values, error conditions, or what 'locally-managed' means in context. For a tool with zero structured data, more behavioral context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter information is needed. The description appropriately doesn't add param details, earning a baseline score of 4 for not overloading with unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all locally-managed scheduled tasks'), providing a specific purpose. It doesn't explicitly differentiate from sibling tools like 'delete_schedule' or 'schedule_recurring_task', but the scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_session_status' or 'manage_session'. The description implies usage for listing tasks but lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_coding_taskA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool 'returns immediately with a session ID' and that progress should be monitored separately, which are useful behavioral traits. However, it doesn't cover aspects like error handling, rate limits, authentication needs, or what happens if the session fails, leaving gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and immediate return, followed by monitoring instructions. Every sentence earns its place by providing essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a mutation tool. It covers the basic purpose and monitoring aspect but lacks details on return values (beyond session ID), error cases, or side effects. This is adequate but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds no additional meaning beyond what the schema provides, such as explaining how parameters interact or providing examples. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('creates') and resource ('new Jules coding session'), distinguishing it from sibling tools like get_session_status or manage_session. It specifies the immediate return of a session ID, which helps differentiate it from tools that might wait for completion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning to 'monitor progress via jules://sessions/{id}/full resource', suggesting this tool initiates a task that runs asynchronously. However, it lacks explicit guidance on when to use this versus alternatives like schedule_recurring_task or manage_session, and no prerequisites or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.