Server Quality Checklist

Tool Scores

• open_weights_directoryC
  2.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description does not disclose behavioral traits such as side effects, authentication needs, or whether it performs a read-only operation. The description is too brief to inform the agent about tool behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at one sentence, but it sacrifices necessary detail for brevity. Every word earns its place, but the content is insufficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 optional param, no annotations, output schema exists but not described), the description fails to provide a complete picture. It lacks return format overview, usage constraints, or how the output schema complements the directory listing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and description does not explain the single parameter 'use_case' (type, acceptable values, or effect on results). The description adds no meaning beyond the schema, which itself is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description is a noun phrase 'Directory...' rather than a verb phrase indicating an action. It states the domain (open-weight AI models for East Africa civic use) but doesn't specify what the tool does (e.g., list, search, or retrieve). Compared to sibling 'list_recommended_models', the purpose is ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like 'list_recommended_models' or 'check_ollama_status'. Lacks any context about preferred use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_deployment_guideC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits. It only states it is a guide, which suggests a read-only operation, but does not confirm whether it accesses external resources, requires internet, or has side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is concise and easy to parse. No redundancy exists, but it could be expanded slightly to include parameter details without losing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one optional parameter, output schema exists), the description is incomplete. It lacks context on how the guide is delivered (e.g., text, steps), what the output schema contains, and any usage hints. The presence of an output schema partially mitigates the need to describe return values, but the description remains insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single optional parameter (`device_type`), but the description provides no explanation of its meaning, allowed values, or how it affects the output. With 0% schema coverage, the description fails to compensate, leaving the agent without guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the subject matter (deploying local AI inference on modest hardware) and geographic scope (Kenya/East Africa), distinguishing it from sibling tools like `run_local_inference` or `check_ollama_status`. However, it lacks a verb (e.g., 'provides a guide' or 'returns deployment steps') so the action is implied rather than explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings such as `list_recommended_models` or `degraded_mode_guide`. The description does not mention prerequisites, alternative tools, or scenarios where the guide is applicable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_local_inferenceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose traits. It only says 'run a prompt', implying a blocking operation, but does not mention execution time, error handling, or any side effects. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the action and resource. No extraneous words, efficiently communicates the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description omits important context such as the need for Ollama to be running, model availability, or potential delays. Insufficient for a user to understand full requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to add meaning. The description does not explain the purpose of 'prompt' or 'model' beyond their names, failing to compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'run' and the resource 'a prompt through a local Ollama model'. It is distinct from sibling tools like check_ollama_status and list_recommended_models, though it does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, no prerequisites or context provided. The description lacks any usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recommended_modelsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only says 'list,' implying a read-only operation. It does not disclose aspects like authentication needs, rate limits, or whether the listing is paginated or cached.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no superfluous words, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description does not hint at return structure or pagination. It is minimal for a list tool with two parameters and multiple siblings, missing usage cues.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the two parameters ('use_case' and 'max_ram_gb'). The phrase 'for East Africa AI use cases' hints at 'use_case' but adds no practical semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'recommended open-weight models for East Africa AI use cases,' which is specific and distinguishes it from siblings like 'open_weights_directory' that likely lists all models.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as 'open_weights_directory' or 'local_deployment_guide.' The description lacks context for appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• degraded_mode_guideA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It correctly implies read-only behavior as a guide, but does not disclose any further behavioral traits such as return format or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, clear sentence that is front-loaded and contains no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and presence of an output schema (not shown but indicated), the description is adequate. It could be slightly improved by hinting at the output nature, but completeness is high given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0 parameters and schema description coverage is 100%, so baseline is 4. Description adds no parameter info, but none needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it is a guide for operating AI systems when cloud connectivity fails, specifying both the resource (guide) and the context (degraded mode). It distinguishes itself from sibling tools which are about checking status, running inference, models, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. There is no mention of when not to use it or contextual cues for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_ollama_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description describes the tool's behavior as a read-only check and listing operation. No annotations are provided, so the description carries the burden; it is adequate but does not disclose edge cases (e.g., behavior if Ollama is not installed) or output details beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that captures the essential purpose without superfluous information. Every word adds value; no restructuring needed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters) and the presence of an output schema, the description is largely complete. It covers the tool's main function. A minor gap: it could mention whether errors are thrown if Ollama is not installed, but this is not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (vacuously). The description adds no parameter details because none exist, which is acceptable. Baseline score for zero-parameter tools is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check') and the resource ('if Ollama is running locally and list available models'). It distinguishes this tool from siblings like run_local_inference or list_recommended_models by focusing on status checking rather than computation or recommendations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for verifying Ollama availability and listing models, but does not explicitly state when to use this over alternatives (e.g., before running inference) or when not to use it. Sibling tools provide context but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.