Server Quality Checklist

Tool Scores

• create-accountC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Create' implies a write operation, but lacks details on permissions, side effects (e.g., if it overwrites existing accounts), error handling, or response format. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no wasted words, making it highly concise and front-loaded. It efficiently communicates the core action without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a creation tool with no annotations, 0% schema coverage, no output schema, and a nested object parameter, the description is insufficient. It lacks details on parameter semantics, behavioral traits, and expected outcomes, leaving significant gaps for the agent to operate effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter ('accountData') with 0% description coverage, and the tool description provides no information about what 'accountData' should contain (e.g., required fields like name, email, or structure). This leaves the parameter completely undocumented, failing to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and the resource ('a new account in Dynamics 365'), making the purpose understandable. It distinguishes from siblings like 'fetch-accounts' (read) and 'update-account' (modify), but doesn't explicitly differentiate from other potential creation tools, keeping it at a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'update-account' or prerequisites for account creation. The description merely states what it does without context, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update-accountC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Update' implies a mutation, but fails to mention critical details like required permissions, whether changes are reversible, error handling, or rate limits. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and resource, making it easy to parse quickly, which is ideal for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a mutation tool with no annotations, 0% schema coverage, and no output schema, the description is incomplete. It lacks details on parameters, behavioral traits, return values, and usage context, making it inadequate for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the schema provides no parameter details. The description adds no information about the parameters 'accountId' (e.g., format, source) or 'accountData' (e.g., allowed fields, structure), failing to compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Update') and resource ('an existing account in Dynamics 365'), making the tool's purpose understandable. However, it doesn't differentiate from sibling tools like 'create-account' or 'fetch-accounts' beyond the basic verb distinction, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create-account' for new accounts or 'fetch-accounts' for reading. It lacks context on prerequisites, such as needing an existing account ID, or exclusions, leaving the agent without clear usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-user-infoC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states a read operation ('Get'), implying it's likely safe, but doesn't disclose behavioral traits such as authentication requirements, rate limits, error conditions, or what happens if no user is found. This leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and appropriately sized for a simple tool, though it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple but vague purpose, the description is incomplete. It doesn't explain what 'user info' includes, how results are returned, or any constraints, making it inadequate for an agent to use effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add param details, which is appropriate here, but it could hint at implicit parameters (e.g., user ID context), though not required. Baseline is 4 for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('Get') and resource ('user info from Dynamics 365'), which provides a basic understanding of purpose. However, it's vague about what specific user information is retrieved and doesn't differentiate from sibling tools like 'fetch-accounts' or 'create-account', leaving ambiguity about scope and distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, context (e.g., when user info is needed over account data), or exclusions, leaving the agent without usage direction beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-associated-opportunitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'fetch' (implying a read operation) but doesn't cover critical aspects like authentication requirements, rate limits, error handling, or what the return format looks like (e.g., list of opportunities with fields). This leaves significant gaps for an agent to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of fetching data from Dynamics 365, no annotations, no output schema, and low schema coverage, the description is incomplete. It lacks details on behavioral traits, parameter usage, and expected outputs, which are essential for an agent to invoke this tool correctly in a real-world context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'for a given account,' which hints at the 'accountId' parameter's purpose, but doesn't explain the parameter's format (e.g., GUID), source, or constraints. This adds minimal value beyond the schema's structural definition, aligning with the baseline for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetch') and target resource ('opportunities for a given account from Dynamics 365'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'fetch-accounts' or 'get-user-info' in terms of scope or relationship, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description implies it's for fetching opportunities linked to an account, but it doesn't specify prerequisites, exclusions, or compare it to other tools like 'fetch-accounts' for broader account data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch-accountsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'fetch' which implies a read operation, but doesn't specify whether this requires authentication, has rate limits, returns paginated results, or what format the data comes in. This leaves significant gaps for a tool interacting with an external system like Dynamics 365.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states exactly what the tool does with zero wasted words. It's appropriately sized for a simple tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that fetches data from an external system like Dynamics 365 with no annotations and no output schema, the description is incomplete. It doesn't explain what kind of accounts are fetched, whether there are filters or sorting options, what the return format looks like, or any error conditions. The agent would need to guess these important details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the schema already fully documents the absence of parameters. The description appropriately doesn't add parameter information beyond what's in the schema, maintaining a baseline score of 4 for tools with no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Fetch accounts from Dynamics 365' clearly states the action (fetch) and resource (accounts from Dynamics 365), making the purpose immediately understandable. However, it doesn't differentiate this tool from its sibling 'get-associated-opportunities' which might also fetch account-related data, so it misses the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create-account' or 'update-account'. It doesn't mention any context, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.