Server Quality Checklist

Tool Scores

• create_virtual_cardC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'Returns secure card details,' which adds some context about output behavior, but fails to address critical aspects like authentication requirements, rate limits, error conditions, or whether the creation is reversible. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just two sentences that directly state the tool's purpose and what it returns. There is no wasted language, and the information is front-loaded appropriately, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of creating a financial instrument (a virtual card), the lack of annotations, no output schema, and 0% schema description coverage, the description is incomplete. It doesn't cover parameter meanings, error handling, security implications, or detailed return values, which are crucial for such an operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the input schema provides no descriptions for the two parameters (merchant, amount_cents). The tool description does not add any semantic meaning for these parameters—it doesn't explain what 'merchant' represents, what format 'amount_cents' should be in, or any constraints. This leaves the parameters largely undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a single-use virtual card') and the resource ('via Privacy.com'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate this tool from its siblings (get_funding_sources, secure_auto_fill), which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings or any alternatives. It mentions the tool's purpose but offers no context about prerequisites, appropriate scenarios, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• secure_auto_fillC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'robust heuristics' but doesn't explain what these entail (e.g., error handling, security measures, or performance). It lacks details on permissions needed, side effects (e.g., page changes), or response behavior, which is critical for a tool handling sensitive payment data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that is front-loaded with the core action ('finds and fills payment fields') and includes key details ('on the current page', 'using robust heuristics'). There is no wasted text, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (handling payment data with 5 required parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like security, error handling, or return values, which are essential for safe and effective use. The description alone is insufficient for an agent to invoke this tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning parameters are undocumented in the schema. The description doesn't add any meaning to the parameters (url, pan, cvv, exp_month, exp_year), such as explaining their roles (e.g., 'pan' as primary account number) or formats. With 5 required parameters and no schema descriptions, the description fails to compensate, leaving semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('finds and fills') and resource ('payment fields on the current page'), and mentions the method ('using robust heuristics'). It distinguishes itself from sibling tools like 'create_virtual_card' and 'get_funding_sources' by focusing on form filling rather than card creation or data retrieval. However, it doesn't explicitly differentiate from potential similar tools not listed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., being on a payment page), exclusions (e.g., when manual input is preferred), or compare it to sibling tools. The context is implied ('current page'), but explicit usage instructions are missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_funding_sourcesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation, implying read-only behavior, but doesn't mention potential authentication requirements, rate limits, or what the output format looks like (e.g., list structure, fields). This leaves significant gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is front-loaded and appropriately sized for a simple list tool, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema), the description is minimally complete by stating what it does. However, it lacks details on behavioral aspects like authentication or output format, which would be helpful even for a simple tool, especially with no annotations to fill those gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, and it appropriately doesn't mention any. A baseline of 4 is applied for zero parameters, as it avoids unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'available funding accounts connected to Privacy.com', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'create_virtual_card' or 'secure_auto_fill', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_virtual_card' or 'secure_auto_fill'. It lacks context about prerequisites, such as whether authentication is needed or if it should be used before creating cards, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.