Server Quality Checklist

Tool Scores

• get_vrops_alertsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavior. It mentions return format but fails to state that the operation is read-only, nor does it discuss authentication, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and structured with Args and Returns sections, making it easy to scan. However, it could be more front-loaded with the key purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple retrieval tool with an output schema, but it lacks usage context, behavioral details, and differentiation from siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant value by specifying that resource_ids are comma-separated vROps resource IDs, which is not in the schema. This compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets alerts for specific resources using resource IDs. While it differentiates from siblings by focusing on alerts, it does not explicitly contrast with detect_incidents or query_events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No prerequisites, conditions, or exclusions are mentioned, leaving the agent without context for proper selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_eventsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions the return format and default values but does not explicitly state that this is a read-only, non-destructive operation. The agent must infer that from the verb 'search' and the return of log events.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with a clear title, an Args block with parameter descriptions, and a Returns section. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, but with an output schema), the description covers the main aspects: input parameters with defaults and return format. It could mention that search_term is optional and the relationship between field_name, field_operator, and field_value, but it's largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the input schema by explaining each parameter's purpose and constraints (e.g., 'limit: default 100, max 10000'). However, it does not list the accepted values for field_operator (e.g., CONTAINS, NOT_CONTAINS), which would enhance clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and resource 'log events' with a specific scope 'in VMware Aria Operations for Logs'. This distinguishes it from sibling tools like detect_incidents or get_vrops_alerts, which deal with different resource types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description does not mention when not to use it or provide any context about preferred usage scenarios relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_dashboardsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool uses a deprecated legacy API and will return empty results on newer systems, which is important behavioral context for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no fluff. The first sentence states the purpose, and the subsequent sentences provide critical caveats. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and an output schema, the description is fairly complete. It identifies the tool's purpose and its limitation (empty on new appliances). Could mention the return type, but output schema covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100%. The description adds no parameter information, which is acceptable because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List saved dashboards from Aria Operations for Logs', specifying the verb and resource. It does not explicitly differentiate from sibling tools, but the action is distinct from detect_incidents, query_events, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the legacy API and deprecation, warning that it returns empty on newer appliances. This provides some usage context but lacks explicit alternatives or when-not-to-use guidance compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_versionA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full burden. It mentions probing key API endpoints but does not detail which endpoints or specify side effects (e.g., read-only). The description is adequate but lacks some behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff. The description is front-loaded with the main purpose and efficiently adds detail about probing endpoints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters, and an output schema exists. The description covers the key aspects (version and feature probing), so it is complete for this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so schema coverage is 100% by default. The description adds value by explaining that the tool returns version info and probes endpoints for feature detection, beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'VMware Aria Operations for Logs appliance version and API surface'. It distinguishes from sibling tools like detect_incidents or query_events, which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to obtain version and feature availability). No explicit exclusions or alternatives are stated, but with no similar sibling tools, the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_vrops_resourcesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries burden. Discloses return format (JSON array with IDs, names, types) and prerequisite. Could mention case sensitivity or partial match behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with 5 sentences including Args and Returns. Well-structured and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with one parameter and output schema, description covers purpose, prerequisite, parameter, and return. Complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds meaning: 'Resource name to search for (VM name, host name, etc.)'. Compensates effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'find' and resource 'resources in VMware Aria Operations by name.' Distinguishes from siblings like get_vrops_alerts and query_events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a specific use case (correlating Log Insight events) and a prerequisite (VROPS_BASE_URL). Does not explicitly mention when to avoid use or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_incidentsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It transparently describes that the tool queries events, groups by signature, and returns clusters above a threshold. It does not mention any side effects or mutability, but the verb 'detect' implies a read-only operation. However, it lacks explicit non-destructive declaration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a one-line purpose, a process overview, a parameter list with explanations, and a return value summary. Every sentence is informative and efficiently written without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, and an output schema exists, the description covers purpose, process, parameter semantics, and return format adequately. It provides sufficient context for an AI agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description compensates fully with clear explanations for all five parameters: lookback_minutes, search_term, event_limit, mass_threshold, max_incidents. Each has a concise purpose and default value stated, adding significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool detects mass log incidents using signature clustering with the Stormbreaker engine, distinguishing it from siblings like query_events (raw events) and list_dashboards (dashboards). It specifies the verb 'detect' and the resource 'incidents' with a clear methodology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for detecting mass incidents via parameters like lookback_minutes, search_term, and mass_threshold. It does not explicitly state when to avoid using it or mention alternatives among siblings, but the specialized intent is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.