Server Quality Checklist

Tool Scores

• intercom_pendingC
  2.6/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description contradicts the annotations: it says 'Read', but annotations declare readOnlyHint=false and destructiveHint=true, suggesting side effects or mutation. The mark_read parameter's impact is not disclosed, creating a serious behavioral inconsistency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no wasted words. It is concise and structurally efficient, though it omits important behavioral and parameter details that are scored elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and mixed annotations, the description leaves the return value, side effects, and mark_read behavior unexplained. The annotation contradiction further undermines the agent's ability to use the tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention mark_read, its default value, or the consequences of setting it to true. The agent cannot understand the only parameter from either the schema or the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific operation: reading unread inbound messages and unresolved asks scoped to the current Codex session. This distinguishes it from siblings like intercom_send or intercom_reply, which are clearly different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a polling/checking use case but gives no explicit guidance on when to use this tool over alternatives like intercom_list or intercom_status. No exclusions or sibling comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_statusA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds the specific status elements returned, which is useful but does not add behavioral traits beyond the annotations. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence, front-loaded with the main purpose and followed by a clear list of included items. Every word contributes information; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only status tool with safety annotations already provided, the description is nearly complete. It names all reported categories. However, it leaves minor ambiguity about how 'pending asks' relates to the sibling intercom_pending tool, which could be clarified without much extra text.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and schema description coverage is 100% (vacuously). With no parameters to describe, the description's silence is appropriate. Baseline for 0-param tools is 4, and nothing is lacking.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Show intercom connection status') and enumerates concrete contents: connection status, active sessions, unread messages, and pending asks. It clearly distinguishes itself from action-oriented siblings like send/reply/ask, though it does not explicitly differentiate from intercom_pending, which may overlap on 'pending asks'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The sibling list includes intercom_pending and intercom_list, which could serve similar informational purposes, but the description never mentions them or gives selection criteria. The usage context is entirely implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_listA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful scopng context—'intercom-connected Pi or Codex sessions' and 'on this machine'—but does not disclose return format, ordering, filtering, or other runtime behavior. No contradiction with the annotations is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler. The verb 'List' and the key resource and location qualifiers are front-loaded, making the core purpose immediately clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list tool, the description is usable and the annotations provide the safety context. However, the two optional parameters are not semantically explained, no output schema exists to clarify return values, and there is no guidance distinguishing this from status/pending siblings, so the description is not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter meaning. The phrase 'on this machine' weakly aligns with the default scope=machine, but the description does not explain the directory/repo scope values or the meaning of include_self at all.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource: 'intercom-connected Pi or Codex sessions' on 'this machine'. This differentiates it from siblings like intercom_reply, intercom_send, and intercom_ask, which are communication actions, and from intercom_status/intercom_whoami, which query status or identity rather than enumerate sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when an agent needs to enumerate local intercom sessions, but it does not explicitly state when to prefer this tool over siblings such as intercom_status or intercom_pending. There are no clear exclusions or alternative routing clues beyond the verb and resource.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_set_summaryA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false, destructiveHint=true, and idempotentHint=true, covering the mutation and repeat-call behavior. The description adds the transparency that the summary is published for other sessions to discover, which is useful beyond the schema. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence with all necessary information: the action, object, and purpose. No filler or redundancy. Front-loaded with the key verb 'Publish' and the resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple: one required parameter, no output schema, no nested objects. The description plus annotations provide enough to call it correctly. It states what to put in the summary, why, and the mutating behavior is captured by annotations. Minor omission: it doesn't explicitly say that calling it again overwrites the previous summary, though that's implied by idempotentHint and the nature of a status setter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for the 'summary' parameter. It specifies that the summary should be 'short' and should describe what this Codex session is doing, giving meaningful content guidance. However, it does not clarify format, tone, or whether the summary replaces a previous one. For a single param, this is adequate but not thorough.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Publish' with the resource 'short status summary' and specifies the audience and purpose: helping other sessions discover what this Codex session is doing. This clearly differentiates from sibling tools like intercom_reply or intercom_send, which focus on interactive communication, and from intercom_status, which likely retrieves status rather than setting it. It lacks an explicit contrast with siblings but the purpose is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage: use it when you want other sessions to discover what this session is currently working on. That is a contextual signal, but there is no explicit when-to-use versus alternatives or exclusions. It does not mention that intercom_status could be an alternative for status retrieval, so the guidance remains implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_replyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already flag this as a destructive, non-idempotent mutation, and the description does not contradict those hints. The description adds that it targets pending inbound asks and that 'to' is a sender/session selector, but it does not disclose concrete side effects such as whether the ask is resolved or removed after replying. With annotations carrying the safety profile, a middle score is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the main action and no redundant phrasing. The only minor flaw is the awkward 'Use to plus which=...' construction, which would benefit from quotes around parameter names, but it remains compact and useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool description covers the main purpose and the multi-ask selection case, but leaves gaps around the optionality of 'to' (what recipient is used when omitted) and the post-reply effect on the pending ask. No output schema is present, so the return value is also unspecified, which is acceptable for a send action but still leaves room for additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions already cover 'to' and 'which'; the description adds the dependency between them (use together when one sender has multiple unresolved asks), which is helpful. However, the required 'message' parameter has no description beyond its name and type, and the description does not clarify what happens when 'to' is omitted. With 67% schema description coverage, this is adequate but not outstanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('Reply') and a specific resource ('pending inbound ask'), which clearly separates it from generic intercom_send and other siblings. The second sentence adds selector nuance, making it even easier to distinguish from intercom_pending and intercom_ask.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives clear context for when to use: there must be a pending inbound ask to reply to. It also provides a concrete conditional for using 'to' plus 'which' when one sender has multiple unresolved asks. It does not explicitly name alternative tools or when not to use it, but the intended situation is clearly implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_sendA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds the behavioral trait 'non-blocking', which is not fully captured by the annotations, and clarifies target resolution by name, full ID, or unique ID prefix. It does not contradict the annotations and provides useful context beyond the structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence containing only essential information: the action, the target, the non-blocking behavior, and accepted target identifiers. No filler or repetition; the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple send operation, but it omits explicit guidance on alternatives and does not describe what the caller should expect in return (there is no output schema). It covers the core send behavior but leaves some operational context to inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It meaningfully explains the 'to' parameter by listing accepted formats, but it does not clarify the semantics of 'message' or 'attachments' beyond their names. The schema provides structure, but the description leaves most parameter meaning implicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Send') with a clear resource ('direct message to another intercom session') and explains the acceptable target formats ('by name, full ID, or unique ID prefix'). This clearly differentiates it from siblings like intercom_reply, which likely targets the current session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'non-blocking' and 'another intercom session', suggesting it is for outbound direct messages rather than replies. However, it does not explicitly name alternatives or state when not to use this tool, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_teamA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the word 'live' which implies real-time current data, but no additional behavioral context like freshness guarantees or potential inconsistencies. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that states purpose and argument requirement with no wasted words. Every element earns its place and the structure is immediately parseable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only tool with robust annotations, the description gives what the tool does and that no arguments are needed. The phrase 'live coworkers' leaves slight ambiguity (currently online vs. current team roster), but overall it is sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema already reflects that with an empty properties object. The description explicitly states 'No arguments are required,' reinforcing the schema and preventing an agent from inventing arguments. This matches the baseline for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Show' and the specific resource: 'your current manager and the live coworkers owned by that manager.' This distinguishes it from siblings like intercom_whoami (shows self) and intercom_list (generic list), making the tool's function unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied by the purpose: use when the agent needs to show team structure under the current manager. However, there is no explicit guidance on when to choose this over alternatives or when it is inappropriate. The note 'No arguments are required' addresses invocation but not selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_askA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals an important behavioral trait: this is a blocking question that should only be used when the reply is required for the next step. Annotations provide readOnly=false and destructive=false but do not explain the blocking/waitting nature, so the description adds meaningful context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The primary condition is front-loaded, and the alternative sibling is mentioned immediately after, making the guidance easy to scan and apply.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives strong when-to-use and routing guidance, and the timeout_ms schema adds useful blocking/error behavior. However, there is no output schema and no explicit statement about what the tool returns, and the recipient/attachment semantics remain underspecified. It is adequate for simple use but incomplete for full confidence in all invocation paths.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25%; only timeout_ms is documented. The description indirectly hints that 'to' refers to another intercom session and 'message' is the question, but it does not explain message format, how to identify the target session, or what attachments mean. With such low schema coverage, the description needed to compensate more and does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: ask another intercom session a question, and attaches a clear condition (only when the next step depends on its reply). It also differentiates the tool from intercom_send by naming what that sibling is for, so an agent can select the right tool without relying solely on the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: only when the next step depends on the reply. It also gives a direct alternative: use intercom_send for assignments, progress/status checkpoints, and notifications. The timeout_ms schema description reinforces this by recommending intercom_send plus intercom_pending for longer work.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• intercom_whoamiA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds that the returned identity is meant for targeting, but it does not disclose the return format or any further behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with no wasted words. The key information is front-loaded: what the tool returns and why it is useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter, read-only identity lookup, the description is sufficient. It clearly tells the agent what the tool produces and why that output matters, and no additional invocation guidance is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and 100% schema coverage, so the parameter semantics burden is minimal. The baseline for a zero-parameter tool is 4, and the description does not need to explain parameter behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Return'), a specific resource ('this Codex session's intercom identity'), and a clear purpose ('for reliable targeting'). This distinguishes it from the sibling intercom tools, which handle other actions like sending, replying, or listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for reliable targeting' gives clear context for when to call this tool: before actions that need to address the current session. However, it does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.