Server Quality Checklist

Tool Scores

• estimate_costB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. While it states the tool returns 'estimated USD cost', it doesn't disclose important behavioral traits: whether this is a read-only operation (likely, but not stated), whether it requires authentication, rate limits, accuracy of estimates, or what happens with invalid inputs. For a cost estimation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that efficiently convey the core functionality. The first sentence states the purpose, the second specifies what it returns. No wasted words, though it could be slightly more structured by separating purpose from return value more clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a cost estimation tool. It doesn't explain the return format (beyond 'estimated USD cost'), error conditions, whether estimates are guaranteed, or how they're calculated. For a tool that could influence financial decisions, more context is needed about the estimation's reliability and limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema, mentioning 'prompt length and model' which aligns with the 'prompt' and 'model' parameters but doesn't provide additional semantic context. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Estimate the cost of an AI request before making it' with specific verb ('estimate') and resource ('cost of an AI request'). It distinguishes from sibling tools like 'ask_ai' (executes request), 'check_balance' (checks account status), and 'list_models' (lists available models). However, it doesn't explicitly contrast with these siblings in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('before making it') suggesting this should be used prior to actual AI requests. However, it doesn't provide explicit guidance on when to use this vs. alternatives like 'ask_ai' (which presumably incurs actual cost) or 'check_balance' (which shows current balance). No exclusions or specific scenarios are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_aiA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the payment mechanism ('Costs are automatically deducted from your Solana wallet balance in USDC') and supported models ('Supports Claude and GPT-4 models'). However, it lacks information about rate limits, error handling, response format, or authentication requirements, which are important for a paid service tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three sentences that each serve a distinct purpose: stating the core function, explaining the payment mechanism, listing supported models, and providing usage guidance. There's no wasted language, and the information is front-loaded with the most critical details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a paid service tool with 4 parameters, no annotations, and no output schema, the description provides adequate but incomplete context. It covers the payment mechanism and supported models well, but lacks information about response format, error conditions, rate limits, or authentication requirements that would be important for proper tool invocation in a production context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, providing complete parameter documentation in the structured fields. The description doesn't add any parameter-specific information beyond what's already in the schema, so it meets the baseline expectation. No additional semantic context is provided for the parameters beyond what the schema already covers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Send a prompt to an AI model'), identifies the resource ('via SolanaProx'), and distinguishes this tool from its siblings (check_balance, estimate_cost, list_models) by focusing on execution rather than querying or listing. It provides a complete purpose statement with both the primary function and the payment mechanism.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Use this for any AI inference task'), which directly addresses its primary use case. However, it doesn't explicitly mention when NOT to use it or provide specific alternatives among the sibling tools (e.g., using estimate_cost first for cost estimation or list_models for model discovery), leaving some room for improvement in comparative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_balanceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return values (USDC and SOL balances) and their purpose (for AI requests), which is useful behavioral context. However, it lacks details on permissions, rate limits, or error handling, leaving gaps in transparency for a financial tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by return details. Every sentence adds value without waste, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and return context, which is adequate for a simple read operation. However, as a financial tool, it lacks details on authentication, data freshness, or error cases, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the single parameter 'wallet'. The description does not add specific parameter semantics beyond implying balance checking, but with 0 required parameters and high schema coverage, the baseline is strong. The description's focus on output compensates slightly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check'), resource ('SolanaProx balance'), and scope ('current'), distinguishing it from siblings like 'estimate_cost' or 'list_models'. It specifies what is returned ('available USDC and SOL balance'), making the purpose explicit and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to check balances for AI requests, but does not explicitly state when to use this tool versus alternatives like 'estimate_cost' for cost estimation or 'ask_ai' for making requests. No exclusions or prerequisites are mentioned, leaving usage context somewhat implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the behavioral trait of listing models with pricing, which is useful context. However, it lacks details on potential rate limits, authentication needs, response format, or pagination behavior, leaving gaps in behavioral understanding for a tool that likely involves network calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all available AI models') and adds essential context ('on SolanaProx with their pricing'). There is zero waste, and every word contributes to understanding the tool's function without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema) and lack of annotations, the description is minimally complete by stating what it does. However, it could be more complete by addressing behavioral aspects like response format or usage constraints, which are important for a listing tool in an AI service context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, focusing instead on the tool's purpose. A baseline of 4 is applied as it effectively handles the lack of parameters without unnecessary elaboration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all available AI models') and the resource ('on SolanaProx'), with explicit mention of what information is included ('with their pricing'). It distinguishes itself from siblings like 'ask_ai', 'check_balance', and 'estimate_cost' by focusing on model enumeration rather than interaction, balance checking, or cost estimation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'available AI models on SolanaProx', suggesting this tool is for discovering models before using them (e.g., with 'ask_ai'). However, it does not explicitly state when to use it versus alternatives like 'estimate_cost' for pricing details or provide any exclusions or prerequisites for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.