Server Quality Checklist

Tool Scores

• kenya_budget_summaryA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, and idempotent behavior. The description adds critical context: the data is from FY 2022/23, sourced from the Kenya Controller of Budget, and is DEMO data (not real). This goes beyond annotations and is valuable for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short, front-loaded sentences. The first states the purpose, the second gives the source, and the third notes it is demo data. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple query tool with an output schema, the description explains what data is retrieved and its demo nature. Minor gap: could mention that the language parameter affects output, but the schema description covers it. Overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description does not add meaning beyond the schema. The description mentions 'county budget allocation' which ties to the county parameter, but does not elaborate on formats or allowed values. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and clearly identifies the resource: county budget allocation, development vs recurrent split, and absorption rate. It distinguishes this tool from siblings like kenya_counties_list (county names) and kenya_county_drought, as it focuses on budget data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to get budget data) but provides no explicit guidance on when not to use it or alternatives. It does not mention siblings or suggest using kenya_counties_list for county names or kenya_county_drought for drought information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_parliament_billsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint=true, idempotentHint=true) align with the query nature. Description adds value by noting data is DEMO and real data is at parliament.go.ke, which is crucial for agent expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, returned fields, and demo data disclaimer. No wasted words, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has two simple parameters, output schema exists (context signal), and description mentions return fields. Demo data disclaimer provides completeness. Slight gap: no mention of pagination or limits, but acceptable for simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage, describing status and keyword. Description does not add new parameter-level details beyond listing return fields. Baseline 3 appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool queries Kenya Parliament bills and their legislative status, listing returned fields. This distinguishes it from siblings like kenya_budget_summary and kenya_counties_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. Context from sibling tools implies scope, but description lacks direct guidance on alternatives or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_rights_queryA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the agent knows it's a safe, repeatable read. The description adds that it returns specific article and provisions, and supports two languages, which provides useful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a purpose: purpose, output, language support. No redundant words. Front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to detail return values. It covers the source (Chapter 4), output type, and language support. Lacks mention of specific rights list, but the parameter description covers that. Adequate for a simple query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already explains parameters. The description adds that the tool works in English and Swahili, aligning with the language parameter, but does not add significant new meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Query', the specific resource 'Kenya constitutional rights from Chapter 4', and the output 'returns the specific constitutional article and its provisions'. It distinguishes itself from sibling tools like kenya_budget_summary by focusing on constitutional rights, and mentions bilingual support.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for querying constitutional rights, but does not explicitly state when to use vs alternatives, nor provides exclusions or conditions. The sibling tools are unrelated, but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_sacco_lookupA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, and idempotentHint. The description adds a critical behavioral trait: the data is DEMO and to get real data one should visit the actual registry. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, returned fields, and data source note. Front-loaded and concise, though the purpose and fields could be combined slightly more efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with annotations fully covering safety, an output schema exists (so return values are documented), and parameters are well-described. The description is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for both parameters. The description does not add additional meaning beyond what the schema provides (e.g., that 'name' can be a partial match or 'ALL'). Baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Look up Kenya SACCOs'), the resource ('SACCOs from the SASRA registry'), and lists the returned fields. It is distinct from sibling tools which cover different topics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the source (SASRA registry) and notes that the data is DEMO, implying when to use the tool. It does not explicitly state when not to use it, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_counties_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds value beyond annotations by specifying the exact number of counties (47) and the output nature (names suitable for other tools), enhancing transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences that front-load the action and purpose without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description fully covers purpose, output, and usage context, especially with annotations and output schema available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With no parameters, description adds context about return values (names), which is helpful for invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'list' and resource 'all 47 Kenya counties', distinguishing it from sibling tools that focus on specific data like budget or drought.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description indicates use with other tools, implying it's a foundational list. While no explicit when-not-to-use, the simplicity and sibling context make it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_county_droughtA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds value by stating the data source (NDMA) and that data is DEMO, which is helpful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise: four short sentences that front-load the purpose, then add phase meaning, data source, and demo warning. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema, return values are covered. The description explains data source, demo nature, and suggests an alternative for real-time data. Missing minor details like case sensitivity of counties, but overall adequate for a simple read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds example county names (Turkana, Garissa, Nairobi) and mentions language options (en/sw), which helps parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the current NDMA drought phase for any of Kenya's 47 counties, specifies the meaning of phases (1=Minimal, etc.), and distinguishes itself from sibling tools like kenya_budget_summary or kenya_counties_list by focusing on drought data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes that the data is DEMO and suggests using ndma.go.ke for real-time data, providing clear guidance on when to use this tool versus an alternative source.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.