Server Quality Checklist

Tool Scores

• additionD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Tool has no description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tool has no description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has no description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tool has no description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Tool has no description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• factorialC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but reveals nothing about computational limits, error handling (e.g., for negative numbers or non-integers), performance characteristics, or output format. This leaves significant behavioral gaps for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function with zero wasted words. It's appropriately sized for a simple mathematical operation and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's mathematical simplicity and lack of annotations/output schema, the description is incomplete. It doesn't address potential constraints (e.g., integer inputs, recursion limits), error conditions, or return value format, leaving the agent with insufficient context for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'a number' which aligns with the single parameter 'a' of type number, but adds no semantic context about valid ranges (e.g., non-negative integers), units, or examples. This provides minimal value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('calculates') and resource ('factorial of a number'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools, which is unnecessary since factorial is mathematically distinct from operations like addition or stock trading tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. While the mathematical nature makes usage obvious in some contexts, there's no explicit mention of use cases, prerequisites, or comparisons with sibling tools like addition for different mathematical operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Generate-SessionC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool generates a session token, implying a write operation that creates new data, but doesn't disclose behavioral traits like authentication requirements, rate limits, token expiration, or side effects. This is a significant gap for a tool that likely involves security-sensitive operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'Generate a new session token using request token'. It is front-loaded with the core action and resource, with zero wasted words. Every part of the sentence contributes to understanding the tool's basic function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (authentication/token generation tool), lack of annotations, no output schema, and low schema coverage, the description is incomplete. It doesn't explain what a session token is used for, the response format, error conditions, or security implications. For such a tool, more context is needed to guide safe and effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'request token' as the input, which aligns with the single parameter 'requestToken', but adds no meaning beyond the schema's type and requirement. It doesn't explain what a request token is, how to obtain it, or its format, leaving the parameter semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate a new session token using request token'. It specifies the verb ('Generate') and resource ('session token'), and mentions the required input ('request token'). However, it doesn't differentiate from sibling tools, which are unrelated (e.g., Buy-Stock, factorial), so it doesn't need sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a request token first), context for usage, or exclusions. Without annotations or sibling tools for authentication, the agent has no hints about usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Show-PortfolioC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'shows' data, implying a read-only operation, but doesn't clarify aspects like whether it requires specific permissions, how it handles errors, or what the output format is. This leaves significant gaps for a tool that likely interacts with user data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, straightforward sentence that efficiently conveys the core action and target. It's front-loaded with the main purpose and avoids unnecessary elaboration, making it appropriately concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral traits like authentication needs, error handling, or output structure, which are crucial for a tool that accesses user-specific data. This leaves the agent with insufficient context to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't add parameter details, as there are none to explain, which aligns with the baseline expectation for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose ('shows the current portfolio') and specifies the target resource ('of the given user'), which is clear. However, it doesn't differentiate this tool from potential siblings like 'Buy-Stock' or 'Sell-Stock' beyond the basic verb 'shows', making it somewhat vague in distinguishing its specific role within the server's toolset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., user authentication), exclusions, or comparisons to other tools like 'Buy-Stock' or 'Sell-Stock', leaving the agent with no usage context beyond the basic purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Buy-StockC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a buy operation at market price, implying a financial transaction with real-world consequences, but lacks critical details: it doesn't mention authentication requirements, potential costs/fees, execution risks (e.g., market volatility), confirmation mechanisms, or error handling. For a high-stakes tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('buys a stock') and includes essential context (platform, parameters). There's no wasted verbiage, but it could be slightly more structured (e.g., separating purpose from parameter roles) for optimal clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (financial transaction with real-world impact), lack of annotations, no output schema, and low schema coverage, the description is incomplete. It omits critical context: expected output (e.g., order confirmation, trade ID), error cases, dependencies (e.g., user session), and behavioral nuances like rate limits or idempotency. The agent lacks sufficient information to use this tool safely and effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'stock' refers to the stock name and 'qty' is the quantity to buy, and implies these are required for the transaction. However, it doesn't specify format details (e.g., stock symbol conventions, integer vs. decimal quantities) or constraints (e.g., minimum order size). The description provides basic semantics but leaves key details ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'buys a stock in the zerodha trading platform for the given quantity and stock name at the current market price for the given user'. It specifies the verb ('buys'), resource ('stock'), platform ('zerodha trading platform'), and key parameters. However, it doesn't explicitly differentiate from its sibling 'Sell-Stock' beyond the obvious verb difference, which keeps it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions no prerequisites (e.g., user authentication, market hours), no exclusions (e.g., when not to buy), and doesn't reference sibling tools like 'Sell-Stock' or 'Show-Portfolio' for context. The agent must infer usage solely from the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Sell-StockB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the action is a sell transaction at market price, but doesn't mention critical behavioral aspects like authentication requirements, transaction costs, settlement time, risk warnings, or what happens on execution failure. For a financial transaction tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, reasonably efficient sentence that packs key information: action, platform, parameters, pricing mechanism, and user context. While dense, it avoids unnecessary words and front-loads the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a financial transaction tool with no annotations, no output schema, and 0% schema description coverage, the description is incomplete. It lacks information about return values, error conditions, side effects, authentication needs, and platform-specific constraints that would be essential for safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'stock name' and 'quantity' but doesn't provide format details (e.g., stock symbol format, quantity constraints like minimum lots), validation rules, or examples. The description adds basic meaning but insufficient detail for reliable parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('sells a stock'), target resource ('in the zerodha trading platform'), and scope ('for the given user'). It distinguishes from sibling tools like 'Buy-Stock' by specifying the opposite trading action, and from 'Show-Portfolio' by being an execution rather than query tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('sells a stock... at the current market price') but doesn't explicitly state when to use this versus alternatives like 'Buy-Stock' or prerequisites. It mentions the platform (Zerodha) and user context, but lacks explicit guidance on timing, conditions, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.