Server Quality Checklist

Tool Scores

• aws_callA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals that parameters are passed exactly as AWS documents them, indicating a passthrough to the AWS API. However, with no annotations provided, it fails to disclose potential side effects (e.g., mutating resources), authentication requirements, or what happens on errors or returns, leaving the agent without a full picture of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loading the core purpose and then explaining each parameter in a clear, linear structure. While it could be slightly more compact, it is efficient and well-organized, with no unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's generic nature (any AWS API call) and the absence of an output schema, the description does not explain what the tool returns after execution, how to interpret errors, or any prerequisites like authentication. These gaps leave the agent under-informed for a potentially complex operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Each parameter is explained in detail: service includes boto3 service name examples (ec2, s3, lambda, rds); action provides both PascalCase and snake_case examples (RunInstances vs run_instances); params is defined as a JSON object exactly as AWS documents, with a bucket example. This substantially adds meaning beyond the schema's bare types and required flags, fully compensating for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool calls any AWS API operation against a local Floci instance, specifying the verb 'Call' and the resource 'any AWS API operation' on the target. It distinguishes itself from sibling tools by being generic, though it doesn't explicitly compare, the purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the sibling tools (floci_status, floci_inventory). The description does not mention alternatives, exclusions, or prior conditions, leaving the agent to infer that it's for arbitrary AWS calls without knowing when the siblings would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• floci_inventoryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. 'Summarize' inherently signals a safe, non-destructive read operation, which is adequate basic transparency. However, it does not disclose details an agent might need — whether results are scoped to a specific account/region, whether there are pagination or rate limits, or what a 'summary' contains (counts vs. names vs. configurations).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly-wound sentence that leads with the verb, names the resource scope, and enumerates the services. Every clause earns its place and there is zero filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter inventory tool with no output schema and no annotations, the description covers the essential ground: what it does, its scope, and the eight services covered. A minor gap is that it does not describe the return shape of the summary, but since the agent is not required to construct any arguments, nothing is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, there is no schema meaning for the description to supplement, so the baseline of 4 applies. The description adds genuine value by specifying exactly which services are included in the inventory and by clarifying that the output is a summary of currently deployed resources rather than a full configuration dump.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Summarize'), a precise resource ('resources currently deployed on Floci'), and enumerates the covered services (S3, DynamoDB, Lambda, EC2, SQS, SNS, ECS, RDS). This clearly distinguishes it from the sibling aws_call (an execution tool) and floci_status (a status check) — an agent can tell it apart without opening either schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The verb 'Summarize' and the term 'inventory' imply this is a read-only overview used when a user wants an at-a-glance resource listing rather than executing operations (aws_call) or checking status (floci_status). However, the description never explicitly states when to prefer this tool over the alternatives, nor does it state exclusions or prerequisites. Usage context is implied, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• floci_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It states the tool checks reachability but does not disclose what 'reachable' entails (e.g., HTTP status, latency, auth requirements) or the exact return format. For a health-check tool, this is acceptable but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is efficient and front-loaded, with no wasted words. It states the action and target clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description is mostly complete. However, it does not specify the nature of the result (e.g., boolean, status message) or any error handling. A slight gap, but acceptable for a simple health check.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and schema coverage is 100% (trivially). Per guidelines, zero parameters warrant a baseline of 4. The description adds no param-specific details because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Check') and resource ('configured Floci instance'), clearly distinguishing this status-check tool from siblings like aws_call and floci_inventory. The purpose is unambiguous and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates its use case (verifying reachability) which naturally separates it from inventory or calls. It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to select it when connectivity is in question.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.