Server Quality Checklist

Tool Scores

• cultural_heritage_sitesD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only includes 'DEMO', which is ambiguous and fails to indicate side effects, permissions, or safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short but not concise because it lacks essential information. 'DEMO' adds no value, making it under-informative rather than efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having a single parameter and an output schema, the description provides almost no context. The 'DEMO' label suggests incompleteness, leaving agents unable to understand the tool's function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the 'region' parameter. The schema shows it is optional with anyOf, but no meaning is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Kenya UNESCO and national heritage sites. DEMO.' is a noun phrase that restates the tool name without specifying an action (e.g., list, retrieve). It vaguely indicates the domain but fails to convey what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool or how it relates to siblings. The description lacks any context for appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• historical_documentsC
  2.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but provides no behavioral details. It does not state if it is read-only, what operations it performs, side effects, or prerequisites. Simply stating 'Where to access' is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While extremely concise, the description is under-specified. It does not earn its place due to lack of informative content. The structure is a single ambiguous sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description fails to cover essential context: what the tool actually returns (list? links?), how the parameter affects behavior, and any usage nuances. Incomplete for a single-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description adds no meaning to the single 'document_type' parameter. The parameter remains a mystery with no hints about its purpose or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides access to historical Kenya and East Africa documents, distinguishing it from sibling tools like cultural sites or timelines. However, 'Where to access' is slightly vague and the 'DEMO' suffix adds ambiguity about its real functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus the siblings or any context-specific recommendations. The description lacks any when-to-use or when-not-to-use information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ethnic_groups_guideC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states it is a 'DEMO', implying limited or non-production data, but does not explain read-only behavior, return format, or any constraints beyond the parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, but lacks necessary detail. While it is front-loaded, it omits key information about the parameter and usage, making it underspecified rather than efficiently concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description does not need to detail return values, but it fails to explain what the tool does with the parameter or how it ties into the sibling context. A demo tool still requires clarity on its functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides zero description for the 'group' parameter, and the tool description offers no explanation of its purpose, expected values, or how it affects results. This leaves the agent completely uninformed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a guide to Kenya's major ethnic groups, cultures, and communities. It distinguishes from sibling tools like cultural_heritage_sites, which focus on locations rather than groups. However, the inclusion of 'DEMO' introduces ambiguity about its reliability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not indicate suitable contexts or prerequisites, leaving the agent without decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kenya_history_timelineC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only mentions 'timeline' and 'DEMO', but fails to explain what the tool returns (e.g., events, years) or any behavioral traits like pagination or data source.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (one sentence), which is concise, but it includes the unnecessary word 'DEMO' that adds confusion. It could be more informative without increasing length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters and an output schema, the description is incomplete. It lacks details on usage, parameter ranges, and general behavior, making it hard for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has two parameters (start_year, end_year) with 0% schema description coverage. The description does not even hint at these parameters, leaving the agent to infer functionality solely from parameter names, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides a historical timeline of Kenya from ancient times to present. It is specific and distinguishes it from sibling tools that focus on cultural sites, ethnic groups, documents, leaders, and oral history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It simply states what it is, without context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• independence_leadersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. The description only states the tool's content but does not reveal format, side effects, or any constraints. It is insufficient for understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, very concise. However, the inclusion of 'DEMO' is slightly distracting and may mislead about the tool's production status. Otherwise, it is front-loaded with the key purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and an output schema, the description is minimal but provides the core purpose. It does not describe the structure of the output, but the output schema likely covers that. The 'DEMO' flag reduces confidence in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema fully covers semantics. Per the rubric, baseline 4 applies. The description adds no parameter information, which is acceptable given no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides information about Kenya independence movement leaders and their roles. This distinguishes it from siblings like cultural_heritage_sites or kenya_history_timeline. However, it lacks an action verb, making it a noun phrase rather than an actionable instruction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as kenya_history_timeline or oral_history_resources. The description does not specify context, prerequisites, or exclusions, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• oral_history_resourcesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only states it provides resources and is a 'DEMO', which hints at limited functionality but doesn't disclose specifics like data format, access methods, or rate limits. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no fluff. Clearly states purpose and scope. The 'DEMO' tag adds a note but could be integrated or clarified. Otherwise efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with no parameters and an output schema. Description provides domain scope but is vague on what 'archives and preservation resources' entails. The output schema likely fills in details, but the description alone is minimal for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters (input schema empty, 100% coverage). With zero params, schema already fully documents the inputs. Description adds domain and scope (Kenya/East Africa), which is helpful but not essential for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides 'oral history archives and preservation resources' for Kenya and East Africa, distinguishing it from sibling tools like cultural_heritage_sites or historical_documents. However, the 'DEMO' suffix may confuse whether the tool is fully functional.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage: to fetch oral history resources for Kenya/East Africa. No explicit when-to-use or when-not-to-use compared to sibling tools. Lacks guidance on prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.