Server Quality Checklist

Tool Scores

• jules_send_messageB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It conveys that the tool sends a message and requires an ongoing session, but does not disclose side effects, authentication needs, or whether the message is stored/visible to the session. The verb 'send' provides basic transparency, but no deeper behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundancy. Every word contributes to the core purpose, and it avoids unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential action and resource, and the schema covers all parameters. However, it lacks information about return values, error conditions, or constraints beyond 'ongoing' (e.g., session must be active, message length limits). Given the tool's simplicity and good schema coverage, this is minimally viable but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter ('message', 'account_id', 'session_id') already meaningfully described. The description adds no extra parameter semantics, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Send') and resource ('follow-up guidance, feedback, or adjustments to an ongoing Jules session'), distinguishing it from list/get/approve/delegate siblings. However, it could more explicitly contrast with 'delegate_task' or 'approve_plan' to fully disambiguate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for ongoing sessions but provides no explicit guidance on when to use this tool versus alternatives, no prerequisites, and no exclusion criteria. It does not mention that 'approve_plan' or 'delegate_task' should be used for other actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_list_sessionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It fails to disclose optionality of parameters, the meaning of 'recent', response format, or pagination behavior, leaving gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with a clear verb and essential details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool, the description is adequate but lacks details such as output format, pagination behavior, and whether account_id is required or scopes the list. With no output schema and no annotations, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The description adds the status filter (active and completed) not present in the schema, but does not clarify the account_id optionality or default behavior when omitted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists recent active and completed Jules sessions, specifying the resource and scope. It is easily distinguishable from sibling tools like jules_list_accounts and jules_get_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for an account' implies use with account_id, and the active/completed wording suggests filtering, but no explicit guidance is given on when to prefer this over jules_get_session or other list tools. Usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_get_diffA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool works on both completed and active sessions and indicates the output type, suggesting a read-only operation. However, it does not explicitly state safety, authentication requirements, or potential edge cases (e.g., empty diff).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and object. Every word contributes, with no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple extraction tool, the description adequately specifies the return value ('unified git patch diff') and a key constraint (session state). While no output schema exists, the description names what is returned. It could mention details like error handling or account_id behavior, but overall it is complete enough for this scale.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for both session_id and account_id. The tool description adds no additional meaning about parameters, so the baseline of 3 applies per the rubric.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Extract') and resource ('unified git patch diff') from a Jules session, making the tool's purpose unambiguous. It also distinguishes itself from sibling tools like jules_get_session or jules_list_activities by focusing on diffs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, or any exclusions. The description implies its use for retrieving diffs, but an agent is left without context on when to select it over other session-related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_get_sessionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It reveals the tool is a read operation ('Retrieve') and lists output categories, but does not disclose potential side effects, authentication requirements, rate limits, or error behavior. This is basic transparency but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that front-loads the action and enumerates the primary returned data. Every word is informative and no redundant repetition occurs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with two well-described parameters and no output schema, the description adequately explains what the tool returns. It lacks examples or notes on edge cases, but the scope is straightforward and the sibling context does not require deeper disambiguation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage with descriptions for both session_id and account_id. The tool description adds no additional parameter explanation beyond implying session_id identifies the specific session, so the schema carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve' and the resource 'specific Jules session', enumerating what is fetched (status, outputs, PR links, metadata). It distinguishes itself from sibling list tools by focusing on a single session rather than listing multiple.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for a specific Jules session' implies when to use this tool versus listing sessions, but it does not explicitly name alternatives or state when not to use it. No guidance is provided relative to siblings like jules_get_diff or jules_list_activities.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_list_accountsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. The verb 'List' implies a read-only, non-destructive operation, but the description does not explicitly state behavioral traits such as authentication requirements, side effects, or return format. This is minimally sufficient for a simple list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, coherent sentence that is entirely on point. It is front-loaded with the action and resource, with no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless listing tool, the description provides the essential context: what is being listed. It does not detail the return structure, but this is somewhat obvious for a list operation and there is no output schema to complement. The absence of a mention of differentiation from jules_list_sources is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, which per rubric is a baseline score of 4. The description adds no parameter-specific meaning since there are none to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with a clear resource: 'configured Google Jules / GitHub accounts and routing aliases'. This distinguishes it from sibling tools like jules_list_sources and jules_list_sessions, which target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as jules_list_sources. It merely states what the tool does, leaving the agent to infer usage context without explicit exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_delegate_taskA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the entire execution pipeline: spin up a VM, clone repo, plan changes, run tests, apply modifications, open PR. It also mentions 'prompt boosting & verification contracts.' However, it omits async behavior, return format, and potential failure modes, which would be valuable for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and followed by a compact workflow summary. Every clause adds information, with no fluff. The inclusion of 'prompt boosting & verification contracts' is brief but adds context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main process but lacks critical operational context: it does not say what the tool returns, whether it is asynchronous, or how to track progress via sibling tools like jules_list_sessions. With no output schema and 8 parameters, this is a notable gap for a complex delegation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The tool description provides a high-level workflow but does not add parameter-level details beyond the schema. For example, it mentions 'run test commands' but the schema already describes test_command in detail. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Delegate an autonomous coding, refactoring, bug-fixing, or documentation task to Google Jules.' It also details the full workflow (VM sandbox, clone, plan, test, modify, PR), distinguishing it from sibling tools like list_sources or approve_plan, which are clearly auxiliary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by describing what the tool does, but it does not explicitly state when to use it over siblings or mention prerequisites like using jules_list_sources to validate source_name. It provides no guidance on handling plan approval or monitoring the delegated task.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_list_activitiesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It clearly states the output is a chronological timeline and lists content categories, but it does not explicitly note that the operation is read-only, mention permissions, or describe any side effects or edge-case behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action and then provides a parenthetical list of content types. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description provides a reasonable overview but omits details about the return format, pagination, or error behavior. It is adequate for a simple listing tool but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already describes both parameters. The description does not add extra meaning beyond implying that the timeline is for a session, which matches session_id. Thus, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Get the chronological event timeline of a session'. It also enumerates the content types (agent thoughts, execution plans, bash outputs, diffs, PRs), which distinguishes it from siblings like jules_get_session or jules_get_diff.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when a chronological timeline of session activities is needed. However, it does not explicitly mention alternatives or situations when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_approve_planA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the required session state, which is useful context, but it does not explain what happens after approval (e.g., the plan is executed), any permissions needed, or whether the action is reversible. This is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the action and condition without any wasted words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple approval action with two well-described parameters and no output schema, the description covers the essential purpose and condition. It could mention post-approval effects, but given the low complexity, it is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters (session_id and account_id) have descriptions. The tool description does not add any additional meaning beyond what the schema already provides, aligning with the baseline score for full coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Approve') and clearly identifies the resource ('generated execution plan') and the context ('session paused in AWAITING_PLAN_APPROVAL state'). It is distinct from sibling tools like jules_delegate_task or jules_send_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage condition: the session must be in AWAITING_PLAN_APPROVAL state. While it doesn't explicitly mention alternatives or exclusions, the state condition effectively guides when this tool should be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jules_list_sourcesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'List' implies a read-only operation, but it does not disclose any additional behavioral details such as permissions, return format, or whether 'across defaults' means all accounts. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no extraneous words. It efficiently communicates the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two optional parameters and no output schema, the description is sufficient. It clearly states the action and scope. The only minor gap is the ambiguous 'across defaults' phrasing, but overall it is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds a slight hint about account_id ('for a specific account or across defaults') but does not provide new meaning beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('GitHub repositories connected to Google Jules'), and clarifies the scope ('for a specific account or across defaults'). It clearly distinguishes from sibling tools like jules_list_accounts and jules_list_sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when you need to list connected repositories) and mentions the optional account context. It does not explicitly name alternatives or exclusions, but the context is clear enough for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.