Server Quality Checklist

Tool Scores

• calendarA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructiveHint=true and readOnlyHint=false; the description adds specific behavioral details: delete_event 'permanently deletes and notifies attendees', create_meeting 'sends invitations; confirm with user first', etc. It explains the 'output' parameter behavior for get_event. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy due to the number of operations (15), but it is well-structured: a brief intro followed by a bulleted list with each verb, its purpose, and required params. Information is front-loaded with the purpose. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (33 parameters, 15 operations) and no output schema, the description covers each operation's behavior and required inputs. It does not explain return values but that is acceptable without an output schema. The 'help' operation provides a fallback for full documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds operational context by listing required parameters per verb (e.g., delete_event requires event_id). It also directs users to operation='help' for full parameter reference. This adds meaning beyond what the schema alone provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool handles 'Calendar operations for Microsoft Outlook via Microsoft Graph' and lists 15 distinct operations with specific verbs and resources. This distinguishes it from sibling tools (account, mail, system) which cover different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use similar operations (e.g., 'create_event' for personal events vs 'create_meeting' for meetings with attendees). It does not explicitly contrast against sibling tools, but the domain difference is obvious. The 'help' operation is recommended for full documentation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• accountA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include destructiveHint=true, and the description adds behavioral context by noting that 'remove' is irreversible and 'logout' preserves configuration. This goes beyond the annotations. However, it does not disclose details like rate limits, authorization requirements, or failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a brief purpose statement, an explanation of the verb list format, and then concise bullet-like lines for each operation. No extraneous information, front-loaded with the tool's scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, 7 operations) and no output schema, the description covers all operations and their required parameters. It directs to help for full details but lacks information on return values or error handling, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers all parameters with descriptions (100% coverage). The description adds meaning by grouping parameters per verb and indicating which are required for each operation, enhancing understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for 'Account management for Microsoft accounts connected to the Outlook MCP server.' It lists specific operations (add, remove, list, login, logout, refresh, help) with brief explanations, distinguishing it from sibling tools like mail, calendar, and system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each operation, including required parameters and a reference to operation='help' for full details. While it does not explicitly state when not to use this tool, the sibling tool names (mail, calendar, system) make the context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mailA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses several behavioral traits beyond the readOnlyHint/idempotentHint annotations: it explains that only read verbs are registered by default and that write verbs are gated by MailManageEnabled, notes that search_messages results are ranked by relevance rather than chronologically, and mentions output modes for get_message (bodyPreview vs. raw). It also points to an in-tool help command for further reference. This is rich, non-obvious context that annotations do not provide, and it aligns with the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized for a multi-operation dispatcher. It is front-loaded with the main purpose and configuration context, then uses a clear bulleted list for each verb. Every sentence earns its place, and the structure makes it easy for an agent to scan and find the relevant operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (18 parameters, multiple operations, no output schema), the description is remarkably complete. It explains configuration-dependent verb registration, lists all active verbs with their required parameters, highlights key behavior (relevance ranking, output modes), and directs the agent to operation='help' for full details. There are no nested objects or output schema to worry about, so this is sufficient for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful semantic value by connecting each operation to its required parameters (e.g., get_message requires message_id) and by explaining the output parameter semantics for get_message. It also clarifies that operation='help' provides the full parameter reference. This goes beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Mail operations for Microsoft Outlook via Microsoft Graph.' It then lists each active operation (help, list_folders, list_messages, get_message, search_messages) with a clear action and scope, which fully distinguishes it from sibling tools like account, calendar, and system. The wording is unambiguous and operational.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong internal usage guidance by mapping verbs to their purposes and required parameters, and by explaining configuration-dependent availability of read versus write verbs. However, it does not explicitly compare this tool to alternatives (account, calendar, system) or state when not to use it, leaving that to the tool name and domain clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• systemA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond the annotations: it notes that about works 'without requiring authentication', status performs 'no Graph call', and it clarifies dependencies like 'Requires: query' and 'Requires: slug'. These details disclose operational traits that are not present in the readOnlyHint or idempotentHint annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a brief overview sentence, a general instruction about the operation parameter, and a bullet-style list of verbs with their actions and required parameters. Every sentence serves a purpose, and the format is scannable and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has six sub-operations and no output schema, the description provides sufficient detail for selection and invocation. It describes the return value of each verb (e.g., about returns 'build identity and host environment metadata', status returns 'version, accounts, uptime, config', search_docs returns 'ranked snippets with line numbers') and notes that help gives the full parameter reference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all parameters with detailed descriptions (100% coverage), so the baseline is 3. The description adds semantic value by linking specific parameters to their verbs (e.g., 'search_docs requires query', 'get_docs requires slug') and instructing users to consult help for optional parameters. This mapping is not fully explicit in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool as system diagnostics and authentication utilities, and enumerates six specific verbs with distinct actions (help, about, status, list_docs, search_docs, get_docs). Each verb has a one-line explanation, so the purpose is specific and unambiguous, and it is naturally distinguished from sibling tools like account, calendar, and mail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance, such as 'Start with system.about when troubleshooting' and 'use search_docs first to identify the slug' for get_docs. It also tells users to call operation='help' for the full parameter reference. It does not explicitly contrast with sibling tools, but the system-level scope is implied, making the usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.