Server Quality Checklist

Tool Scores

• gazette_searchD
  1.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose what happens when called. 'DEMO' suggests it might not perform real searches, but this is ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just one sentence, but it includes the unhelpful 'DEMO' and lacks essential details. Conciseness without substance is not valuable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description fails to explain the tool's behavior or results. It is incomplete for a search tool, leaving major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explain any parameter. 'search_type' and 'date_range' are not described at all, leaving the agent clueless about valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'Guide to searching Kenya Gazette... DEMO', but it's unclear whether this tool actually performs a search or merely provides guidance. The term 'DEMO' further obscures its functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus sibling tools like 'citizen_feedback_channels' or 'tender_search_guide'. The description lacks any context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tender_search_guideC
  2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses no behavioral traits such as side effects, authentication needs, or data returned. The short description is insufficient for transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise (two clauses), but conciseness comes at the cost of clarity. It lacks essential details, making it inadequate rather than efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (context indicates 'Has output schema: true'), the description does not mention what the tool returns or how it functions. The DEMO label and missing parameter info make it incomplete for a 2-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain parameters but fails to do so. The two optional parameters (county, sector) are not mentioned or defined.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it is a guide to Kenya government tenders, which gives a general purpose. However, it does not specify the exact action (e.g., search, list, provide instructions) and includes 'DEMO', adding ambiguity about its real functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus sibling tools (e.g., gazette_search, open_data_guide). The description only indicates domain but lacks context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_data_guideC
  2.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only says 'DEMO', implying it is non-functional or a prototype, but no disclosure of side effects, data access, or reliability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely short but not well-structured. 'DEMO' is uninformative. Could be more concise while adding key details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has one parameter and an output schema, but description is too brief to understand what the tool actually returns. Incomplete for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and description does not explain the 'data_type' parameter. No hint about expected values or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it deals with 'Kenya open government data sources and APIs', which gives a general purpose but lacks specificity (e.g., list, search, or retrieve). 'DEMO' adds ambiguity. It distinguishes from siblings as a guide, but the verb is missing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like citizen_feedback_channels or gazette_search. No context on use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parliament_trackerC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It only says 'Track... DEMO' without disclosing whether operations are read-only, any authentication needs, rate limits, or what happens on use. The 'DEMO' label is ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short (one sentence) but lacks essential details. It does not earn its place by providing key information beyond a minimal statement of purpose. Front-loading is not relevant when content is so sparse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, a single undocumented parameter, and only a minimal purpose statement, the description is wholly inadequate. An output schema exists but is not described. The tool as defined does not provide enough information for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required parameter (query_type) with no description or enum. Schema description coverage is 0%. The description does not explain what values query_type accepts or how it relates to the listed items (bills, etc.).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: tracking Kenya parliament with specific entities (bills, Hansard, committees, petitions). It distinguishes from sibling tools which cover citizen feedback, gazette, open data, and tenders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any context for when it is appropriate. The description only lists what it tracks, with no mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• citizen_feedback_channelsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavior. It only mentions 'DEMO,' implying it may be non-functional or sample data, but does not clarify whether the tool reads data, returns a list, or requires authentication. Behavioral traits are not disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise. However, it lacks structure (e.g., bullet points or sections) and omits essential details, making it under-specified rather than efficiently brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and the single parameter with no schema description, the tool is incomplete. The output schema exists but is not described in the description. The sibling tools indicate a contextual domain, but the description fails to provide enough information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, issue_type, is a required string with no description, enum, or format. The tool description mentions 'feedback, complaints' but does not map issue_type to these categories or suggest valid values. Schema description coverage is 0%, so the description should compensate, but it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it provides 'official channels for Kenyan citizen feedback, complaints, and government access,' which clearly identifies the domain. It distinguishes from sibling tools (e.g., gazette_search, parliament_tracker) that cover different topics. However, 'channels' is somewhat vague and does not specify the tool's output (e.g., list, details).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, such as open_data_guide or parliament_tracker. There are no instructions on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.