Server Quality Checklist

Tool Scores

• mpesa_transaction_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, idempotent, non-destructive. Description adds valuable prerequisite info (env vars) beyond annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both essential. Front-loaded with purpose, then prerequisite. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool with output schema and rich annotations, description is mostly complete. Missing rate limits or side-effect details, but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameter with example. Description adds no new semantics, so baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear specific verb (Query) and resource (transaction status) with receipt number. Distinguishes from sibling tools like airtime_send and sms_send.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions required environment variables but does not explicitly state when to use this tool vs alternatives like mpesa_stk_query. Usage is implied by name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• airtime_sendA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructive/destructiveHint=true. Description adds valuable sandbox behavior disclosure. Does not discuss other aspects like auth or rate limits, but the added sandbox note is useful beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each adding distinct value: action, use cases, sandbox note. No fluff, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (return values not needed), the description covers purpose, common usage, and sandbox behavior. Lacks prerequisites or error scenarios, but sufficient for a simple tool with good annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and already describes each parameter in detail (including minimum amount in production). Description does not add new parameter information beyond what's in the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states action (send airtime top-up) and target (specific network subscribers). Common use cases provided. Distinguishes from siblings like mpesa_stk_push which are for money transfers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Lists common use cases (NGO incentives, survey rewards, agent payouts) and mentions sandbox mode behavior. Does not explicitly compare to alternatives, but given sibling tools, context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sms_sendA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true and idempotentHint=false. The description adds valuable behavioral details: the maximum recipient limit, geographic coverage, and that it returns per-recipient status and cost. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, each serving a distinct purpose: what the tool does, its capacity and scope, and its return value. No wasted words, and the most critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three well-documented parameters and an existing output schema, the description covers the essential aspects: operation, capacity, geographic scope, and return format. It is sufficiently complete for an agent to understand and use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema itself clearly documents all three parameters. The description adds no additional parameter-level meaning beyond what is in the schema (e.g., it mentions Unicode support which is already in the message description). Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends SMS via Africa's Talking, specifies the maximum recipients (1,000), mentions geographic coverage, and indicates return of per-recipient status and cost, distinguishing it from sibling tools like airtime_send or mpesa_stk_push which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (sending SMS to one or many recipients) and includes practical limits (1,000 recipients). It does not explicitly mention when not to use it or compare to alternatives, but the sibling tools are sufficiently different that no confusion arises.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mpesa_stk_pushA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations: it explains that the tool is async, that the customer enters their PIN to complete payment, and that a CheckoutRequestID is returned for follow-up. This aligns with destructiveHint=true and provides useful actionable information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with four short sentences, all of which provide necessary information. It front-loads the primary action and sequentially covers process, return value, and follow-up. There is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential aspects of a payment initiation tool: what it does (trigger STK push), the user interaction (customer enters PIN), the result (CheckoutRequestID), and the asynchronous follow-up (use mpesa_stk_query after 10-30 seconds). With an output schema present, the description does not need to detail return fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for all parameters. The tool description does not add additional meaning beyond the schema (e.g., no examples or format clarification). Baseline 3 applies as schema already documents parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool triggers an M-Pesa STK Push, sends a payment prompt to the customer's phone, and returns a CheckoutRequestID for tracking. It distinguishes itself from sibling tools like mpesa_stk_query by noting that the query tool is for checking completion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the sibling tool mpesa_stk_query (after 10-30 seconds) to check completion. It implies that this tool is for initiating payments, but does not explicitly state when not to use it or mention alternatives like airtime_send.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mpesa_stk_queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, idempotentHint), the description adds polling timing and result code meanings, disclosing behavioral traits like non-immediate result and cancellation handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, each essential: purpose, timing advice, and result interpretation. No extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one parameter and existing output schema, the description fully covers polling strategy, result codes, and usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description merely restates the parameter's purpose ('CheckoutRequestID from mpesa_stk_push response') without adding additional meaning or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Check the status of an STK Push request' with a specific verb and resource, distinguishing it from siblings like mpesa_stk_push (initiates) and mpesa_transaction_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to poll 10-30 seconds after calling mpesa_stk_push, and interprets result codes (0=success, 1032=cancelled, 1037=timed out), providing clear when-to-use and expected outcomes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.