Server Quality Checklist

Tool Scores

• create_sessionA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the key steps (clone, execute, PR), but omits important traits such as auto_pr defaulting to true, the long-running/asynchronous nature of the session, and potential side effects like modifying the repo or needing authentication. This is a moderate disclosure, not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—two short, direct sentences. It front-loads the purpose ('Create a new Jules coding session') and immediately explains the behavior, with no filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is only partially complete. It explains what the tool does at a high level but does not describe return values, session lifecycle, how to track progress, or how it differs from create_session_from_issue. For a 5-parameter mutation tool, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, so the baseline is 3. The tool description adds no additional parameter meaning beyond restating the auto_pr behavior in prose; it does not clarify formats, relationships, or constraints beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Create a new Jules coding session') and the resource, with a specific verb and context. It explains the session's workflow (clone repo, execute task, optionally create a PR), which differentiates it from sibling tools like list_sessions and get_session, and from create_session_from_issue, which presumably starts from an issue rather than a direct prompt.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the primary use case (starting a session to run a coding task on a repo) but does not explicitly state when to use this tool over create_session_from_issue or when to use list_sessions/get_session. No exclusions or alternative recommendations are provided, so guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sessionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. 'Get details' clearly implies a read-only operation with no side effects, but the description does not disclose behavior around errors, authorization, or return format beyond the mention of status and PR info.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the key action and resource. Every word adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple get-by-ID tool with one well-documented parameter. Without an output schema, the description provides some return details (status, PR info), which is adequate for such a simple operation. Minor gaps exist around error handling, but these are acceptable given the tool's low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (session_id) with 100% description coverage. The description does not add additional meaning beyond confirming that the session is 'specific', so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves details of a specific Jules session, including status and PR info. The verb 'Get' is specific, and 'specific Jules session' distinguishes it from siblings like list_sessions or create_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The word 'specific' implies this tool is for fetching a single session by ID, hinting that list_sessions is for multiple sessions. However, there is no explicit guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It reveals the scope ('your') and that status is included, but it does not disclose pagination, sorting, error behavior, or explicitly confirm read-only behavior. 'List' implies non-mutating, but more context would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with a clear subject, verb, and object. There is no redundant or unnecessary wording; it efficiently conveys the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with one well-documented parameter, the description is sufficient. It communicates scope and return content ('with their status'), and the schema covers the limit. Minor missing details like return format or pagination are not critical given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not mention the limit parameter. However, the input schema already provides 100% coverage with a clear description of the limit, so no additional semantic value is needed from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'your Jules sessions' and adds a distinguishing detail 'with their status'. This clearly separates it from siblings like create_session and get_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is for listing all of the caller's sessions, but it does not explicitly state when to use this over get_session or mention any exclusions. No alternatives are named, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_session_from_issueA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that the tool fetches issue details and creates a session with auto-PR, but it does not explain side effects (e.g., whether it creates a branch, modifies the issue, or what 'auto-PR' entails). This is moderate transparency but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the verb and key action, and contains no filler. Every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description must explain behavior and return value. It explains the action but omits what the session creation returns, any side effects, or prerequisites. This is adequate for a simple 3-parameter tool but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and each parameter has a clear description. The tool description does not add additional meaning beyond the schema, which is acceptable since the schema already documents the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb+resource: 'Create a Jules session from a GitHub issue.' It distinguishes itself from the sibling create_session by specifying the source (GitHub issue) and the auto-PR feature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you want to turn a GitHub issue into a Jules session with auto-PR. It provides clear context but does not explicitly state when to avoid this tool or mention alternatives, such as using create_session for non-issue-based sessions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.