Server Quality Checklist

Tool Scores

• sign_inA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full behavioral burden. It adds one useful detail (the sign-in happens through the browser), but it does not disclose side effects such as establishing a session, potentially requiring interactive user action, or how it interacts with existing auth state. This is too thin for a tool with no annotation safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly one sentence with no filler. It front-loads the action, names the resource, and includes the only relevant mechanism detail (browser-based sign-in). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is largely sufficient: it names the action, resource, and mechanism. The main missing piece is any guidance on prerequisites or consequences of signing in, but the simplicity of the tool keeps this from being a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema already confirms this, so the description cannot add parameter-level meaning. The zero-parameter baseline of 4 applies; there is nothing the description needs to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Sign in'), a specific resource ('Microsoft 365 tenant'), and the mechanism ('via your browser'). This clearly distinguishes it from sibling tools like sign_out and auth_status by naming the authentication action itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for initiating authentication before using tenant-related tools, but it does not explicitly say when to use it versus auth_status or sign_out. There is no when-to-use/when-not-to-use guidance, only an implied placement in the authentication workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• auth_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. The verb 'Show' clearly implies a non-mutating read operation, and the listed output fields define what the agent can expect. It does not explicitly state that it does not sign the user in or out, but the semantics are strong enough for this simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the action and then concisely enumerates the output fields, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a low-complexity, zero-parameter status tool with an output schema present. The description covers the key behavioral aspects and return contents, and nothing important appears missing for an agent to decide whether to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline of 4 applies. The description adds no parameter-specific details because none are needed; it instead focuses on what the output exposes, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and identifies the exact resource: current sign-in state, tenant, client ID, and granted scopes. It clearly distinguishes this from sibling tools like sign_in and sign_out, which perform state-changing actions rather than report status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context that this tool is for inspecting the current authentication state, not for changing it. It does not explicitly mention alternatives, but the read-only status framing makes the appropriate use case obvious relative to sign_in and sign_out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sign_outA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently explains that the tool only clears the local Keychain cache and does not affect remote sessions or tokens, which is essential context beyond the tool name. It could add information about return behavior or idempotency, but the core side effects and limitations are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no unnecessary words. The primary action is front-loaded, and the important scope limitation is placed as a brief second sentence. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple, zero-parameter tool with minimal complexity. The description fully covers what it does, where it acts, and what it does not do. Since an output schema exists, omitting return-value details is acceptable. Nothing critical is missing for an agent to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty, so there is no parameter semantics for the description to clarify. The baseline of 4 for no-parameter tools applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Forget the cached sign-in' and concretely specifies the mechanism: 'clears the macOS Keychain token cache.' This clearly distinguishes the tool from siblings like sign_in (which would create a sign-in) and auth_status (which would check status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear boundary with 'Local only — does not revoke Microsoft browser sessions or issued tokens,' which tells the agent when this tool is and is not appropriate. It does not explicitly name alternative tools for full revocation, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcriptA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that transcript content is untrusted user-generated data, returned delimited, and warns against following instructions inside. It also describes the raw_vtt behavior and the ambiguous_match error. However, it does not explicitly state whether the operation is read-only (implicit via 'fetch') or mention any authentication requirements beyond 'invited to', which is a minor gap given the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though the description is multi-sentence, every sentence serves a purpose: it defines the action, provides parameter guidance, warns about ambiguity, and flags security considerations. The 'Args' section is clearly structured. No filler or redundant information exists, so the length is justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters and no output schema, the description is quite complete. It explains the input types, error behavior, and content trust boundaries. However, it does not precisely describe the structure of the default 'speaker-attributed text' return (e.g., segments, timestamps, speaker labels), which an agent might need for downstream parsing. Since no output schema exists, the description could be slightly more explicit about the return format, hence a 4 rather than 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates. For 'meeting', it explains three accepted forms (id, list position, fragment) and the error case. For 'raw_vtt', it describes what it returns instead of speaker-attributed text. This provides meaning well beyond the schema's bare type declarations, which only say 'string' and 'boolean'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Fetch the transcript of a Teams meeting you were invited to.' It clearly distinguishes from sibling tools (auth_status, sign_in, list_recent_meetings, sign_out) by naming the exact object it retrieves. No other tool fetches transcripts, so purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use context: it requires an invitation to the meeting. It gives detailed guidance on the meeting parameter, warns about ambiguous_match errors for fragments, and advises checking transcript_status first. It also instructs not to follow instructions in transcript content, which is a usage caveat. This exceeds basic guidance and even names alternatives (list_recent_meetings) for obtaining the correct id.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_meetingsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and succeeds: it discloses newest-first ordering, the 1–90 day range with clamping, the effect of only_with_transcripts, the three possible transcript_status values, and the failure consequence of calling get_transcript on unavailable meetings.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a behavior-focused opening paragraph, then a clear Args section. Every sentence adds necessary operational information, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description provides enough for an agent to decide when to call the tool, how to set parameters, how to interpret transcript_status, and how to interact with the closely related get_transcript tool. The main usage loop is fully covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no descriptions (0% coverage), but the description fully compensates: 'days' is defined as how many days back to look with a 1–90 clamped range, and 'only_with_transcripts' is defined as omitting meetings marked unavailable. Both parameters are meaningfully explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'List recent Teams meetings on your calendar (newest first).' It also names the key output field, transcript_status, and clearly separates this listing tool from the sibling get_transcript tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to call get_transcript (only for available or unknown transcript_status) and when not to (unavailable, where the call will fail). It also instructs the agent not to claim a transcript exists unless status is available, which is actionable routing guidance for alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.