Server Quality Checklist

Tool Scores

• cooperative_rights_queryD
  1.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits but only gives a vague topic. It does not state whether the tool is read-only, destructive, or requires authentication, leaving agents uninformed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short, but at the expense of clarity. It is under-specified for an agent to understand its purpose or usage effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one parameter and an output schema exists, the description still fails to provide sufficient context about what the tool does, how to use it, or what to expect as output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'topic' is of type string with no enums and schema description coverage is 0%. The description adds no meaning beyond the schema, failing to clarify what topics are acceptable or expected.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Rights and protections for Kenya cooperative members. DEMO.' is vague and does not clearly specify what the tool does beyond its name. It lacks a specific verb and resource, and does not distinguish it from sibling tools like cooperative_benefits or chama_formation_guide.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The sibling tools suggest related but different functionalities, but no explicit conditions or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chama_formation_guideC
  2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only says 'Guide' and 'DEMO', with no information on whether it is read-only, destructive, requires authentication, or returns dynamic data. The behavioral transparency is very low.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief (7 words), but this brevity results in under-specification rather than conciseness. It lacks structure and fails to convey essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low schema coverage and no annotations, the description is very incomplete. It does not explain the output, the meaning of parameters, or any constraints. The presence of an output schema is not leveraged.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention any parameters. It adds no meaning beyond the input schema itself. The parameters 'members' and 'purpose' are left unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it is a 'Guide to forming a chama/investment group in Kenya', which indicates the purpose. However, it is vague and does not specify what the tool actually does (e.g., returns text, steps, etc.). The 'DEMO' tag suggests it is a demonstration, not a production-ready tool, but this is not clarified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention any prerequisites, exclusions, or context. The word 'DEMO' hints it may be intended for testing, but this is insufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cooperative_benefitsC
  2.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It does not mention side effects, required permissions, or data sensitivity. The 'DEMO' label hints at limited functionality but is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short, but the 'DEMO' tag adds noise without value. It is structured as a single sentence, but does not front-load critical information like the fact that it is a demo.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (1 optional parameter, siblings in similar domain), the description is incomplete. It neither explains the tool's scope nor distinguishes it from alternatives. The output schema is present but unaddressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not explain the 'coop_type' parameter. It does not specify allowed values, despite a default of 'sacco'. The output schema exists but is not described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Benefits and obligations of Kenya cooperative membership. DEMO.' uses a noun phrase instead of an action verb, making the tool's purpose unclear. The 'DEMO' tag may confuse agents about its functionality. It fails to distinguish from sibling tools with similar scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings (e.g., cooperative_rights_query, sacco_loan_guide). The description lacks context for appropriate usage scenarios or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sacco_loan_guideC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. States it covers loan types and process but does not disclose if it is read-only, if results are limited (DEMO), or any side effects. Insufficient for agent to understand behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very short at one sentence, but includes ambiguous 'DEMO'. Not excessively verbose, but could be more informative without adding length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists so return values need not be explained, but description is too vague about input parameters and usage. Does not compensate for missing parameter semantics or usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no descriptions in schema). Description does not mention loan_type or sacco_name parameters, leaving their meaning and defaults unexplained. Fails to add value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it covers SACCO loan types, eligibility, and process in Kenya. Differentiates from siblings like sacco_finder (finding SACCOS) and cooperative_benefits (benefits). However, 'DEMO' suggests incompleteness, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage for loan information, but no explicit guidance on when to use versus alternatives. No exclusions or context provided. Sibling tools exist but no differentiation is offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sacco_finderC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only 'DEMO' hints at limited functionality, but no details on behavior, error handling, or what happens when both parameters are used. Insufficient for an agent to understand side effects or states.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely short, but lacks essential information. The 'DEMO' tag adds ambiguity. Not concise in a helpful way; it is under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With two optional parameters and an output schema, the description should explain what is returned and how parameters interact. It omits return format, pagination, and the meaning of 'DEMO'. Incomplete for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It mentions 'by county or sector' but does not clarify format, allowed values, or whether parameters behave as AND or OR. For 0% coverage, more detail is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds accredited SACCOs in Kenya, with filtering by county or sector. The verb 'Find' and resource 'accredited SACCOs' are specific. It distinguishes from sibling tools like chama_formation_guide or cooperative_benefits, which are guides not a finder.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs siblings. Does not mention scenarios where one parameter is preferred over the other, or indicate limitations of the 'DEMO' version. Lacks any usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.