Server Quality Checklist

Tool Scores

• iris_list_draftsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It offers the source folder ('staging mail folder') and an ordering hint ('recent'), but it does not mention authentication needs, whether results are scoped to the current user, or how the 'limit' parameter affects behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or repetition. Every word contributes to identifying what the tool lists and where the data comes from.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter list tool with an output schema, the core listing behavior is adequately conveyed. Yet the description omits authentication context and usage conditions, leaving an agent to guess when it is safe or appropriate to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate by explaining the single 'limit' parameter. The property name and default of 10 make the meaning partially obvious, but the description adds no detail about whether it caps results, enables pagination, or controls page size.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb and resource ('List recent messages') and identifies the source ('staging mail folder'), which distinguishes it from sibling create/update/delete tools. However, it does not explicitly name 'drafts' or an alternative read tool, so it is clear but not fully differentiating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to call this tool versus alternatives, nor whether authentication (via iris_login) is required first. The description implies a read operation but leaves the selection context entirely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_create_draftA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral burden. It discloses that the message is NOT sent, that it goes to a user-defined folder (with default), and that the folder is created on first use. However, it does not mention any required authentication (e.g., that the user must have logged in via iris_login), potential side effects (e.g., overwriting existing drafts), or the exact return format. These gaps are meaningful for a tool that creates a draft.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and efficiently front-loads the critical fact that the message is staged and not sent. It avoids redundancy and directly states the key behavioral constraint. However, it could be slightly more structured (e.g., explicitly listing parameter hints) but overall it is concise and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters (3 required) and no annotations, the description is not complete for an agent to call it correctly. It lacks explanations for most parameters, does not mention the output schema (though one exists), does not describe error conditions or authentication requirements, and does not clarify whether the folder creation is automatic. While the core behavior is clear, the operational details are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description is the primary source for parameter semantics. Only reply_to_message_id is explained (to draft a threaded reply). The other six parameters (to, cc, bcc, subject, body, html) are only listed with types and titles, with no explanation of format, defaults, or interdependencies. This provides minimal assistance to an agent in correctly constructing the arguments.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Compose a message into the staging mail folder') and resource (draft), and explicitly clarifies that the message is NOT sent, which distinguishes it from a send operation. It also mentions the default folder and the reply_to_message_id for threaded replies. This clearly differentiates it from siblings like iris_update_draft and iris_delete_draft.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this to create a draft that a human will later send, not to send directly. It also hints at the reply_to_message_id for threaded replies. While it doesn't explicitly name alternative tools (e.g., 'use iris_update_draft for modifying existing drafts'), the create vs. update/delete distinction is obvious from the tool name and siblings. It provides enough guidance for an agent to know when to select it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_update_draftA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explicitly states the partial-update behavior, which is a valuable disclosure beyond the schema. However, it omits error handling, idempotency, or what happens if the draft does not exist, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, focused sentences with no wasted words. The core action is front-loaded, and the second sentence clarifies the update semantics effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and presence of an output schema, the description is mostly adequate. However, missing context on error behavior or authentication (given no annotations) leaves room for improvement, making it minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The phrase 'Only the fields you pass are changed' adds key partial-update semantics beyond the raw schema. Yet no per-parameter details are given, though parameter names are self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Revise' with 'existing draft' and 'in place,' clearly identifying the target resource and distinguishing it from creation or other operations. This directly differentiates the tool from siblings like iris_create_draft.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context (update an existing draft) is implied but no explicit when/when-not guidance or alternatives are given. It doesn't mention that creating a new draft should use the sibling tool, leaving usage to be inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_auth_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden, and it does reasonably by communicating that this is a read-only reporting operation returning identity and scopes. It does not mention side effects or whether an existing session is required, but the reporting verb strongly implies non-destructive introspection.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence containing no filler. It front-loads the action and delimits exactly what the report will contain.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only status checker with an output schema present, the description is fully sufficient: it tells the agent what the tool reports and implies no required setup. No important missing context is identifiable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100%, so no parameter documentation is needed. The description contributes to semantics by listing the output dimensions (signed-in state, identity, scopes), which compensates for the lack of any exposed data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Report'), a clear resource ('iris signed-in state'), and the exact information returned (whether signed in, which user, what scopes). It is clearly distinct from siblings like iris_login and iris_create_draft, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool — to inspect the current auth status — but does not state any exclusions nor explicitly contrast it with the login/draft siblings. An agent can infer the right use case, but the guidance is not made explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_loginA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does explain the interactive flow (returns URL/code for human entry, requires a follow-up call) and the condition for reuse, which is valuable. However, it does not disclose potential side effects (e.g., invalidating existing tokens), whether the call is blocking, or error behavior if invoked multiple times. A 3 reflects this moderate level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the primary action, then immediately gives the return values and next step, and ends with the usage condition. Every sentence earns its place and there is no redundancy with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description explains the meaningful output (URL and code) in plain terms, which the schema may not convey. It also covers the workflow (call finish after) and the lifecycle condition. For a login-initiation tool with no parameters, nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description has no parameter semantics to add. The baseline for 0 parameters is 4; the description appropriately mentions 'the mailbox' but that refers to the overall context, not a specific parameter. There is no gap to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start a device-code sign-in'), the resource ('the mailbox'), and the specific protocol (device-code), distinguishing it from siblings like iris_login_finish (which completes the sign-in) and iris_auth_status (which presumably checks status). It also mentions the return values (URL and code), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Only needed once, or after the refresh token lapses.' It also names the next step, iris_login_finish(), and implies that if you already have a valid token you don't need to call this. This is clear guidance on when to use vs. alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_login_finishA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It transparently reveals that the tool waits up to ~60 seconds and how it behaves when the code hasn't been entered (returns a notice and can be retried). It does not explicitly describe success behavior or side effects, but the existence of an output schema may cover the return format. The disclosure of timing and retry is valuable and beyond what the empty schema provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundant words. The first sentence states the action and prerequisite; the second explains the wait and retry behavior. Information is front-loaded, with the primary purpose stated before the behavioral details. Every sentence earns its place, and there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, an output schema, and a simple role in the login flow, the description covers the essential usage context. It explains the prerequisite (iris_login was called), the triggering condition (code entered), the wait behavior, and the retry logic. Nothing critical is missing that would prevent an agent from calling it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The schema has no properties to describe, and the description does not need to add parameter-level meaning. Any context about the sign-in flow is already covered by the purpose and usage guidelines, so the parameter semantics are trivially satisfied.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool completes a device-code sign-in initiated by iris_login(), using a specific verb (complete) and resource (sign-in). It immediately distinguishes itself from iris_login, which starts the flow, and from other sibling tools like iris_auth_status and draft management functions, leaving no ambiguity about its role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to call it ('after entering the code in the browser') and provides retry guidance ('can simply be called again' if the code isn't entered). While it implies the prerequisite of having called iris_login(), it does not explicitly name alternative tools for excluding use cases, such as iris_auth_status for checking status. However, the flow is so clearly delineated that an agent would not confuse it with other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• iris_delete_draftA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly labels the operation as 'Destructive' and explains the confirmation requirement and human-approval condition. While it implies irreversibility through the word 'destructive', it does not explicitly state that the deletion is permanent or mention any side effects. However, it covers the critical safety behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the purpose ('Delete a draft') and immediately provides a critical warning about the confirmation flag. Every word earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete operation, the description adequately covers the essential behavioral aspects: destructive nature, confirmation requirement, and human-approval condition. The presence of an output schema (not shown) likely covers return values and errors. While it could explicitly state that deletion is irreversible, 'destructive' effectively communicates this. Overall, it is complete enough for an agent to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions (0% coverage), so the description must compensate. It adds crucial meaning to the confirm parameter by stating it is required and only to be set after human approval, which the schema does not convey. The other parameter, draft_id, is self-explanatory given the tool's purpose, so the description's contribution is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the operation as 'Delete a draft' with a specific verb and resource. It is distinct from sibling tools like iris_create_draft, iris_update_draft, and iris_list_drafts, which involve different actions. No ambiguity exists about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance: confirm=true is required and should only be set after human approval. This tells the agent when it is appropriate to invoke the tool and under what conditions, effectively differentiating it from non-destructive alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.