Server Quality Checklist

Tool Scores

• delete_rule_baseC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must fully disclose behavior. It indicates a destructive action but lacks details on reversibility, permissions, or error handling. The bare statement is insufficient for an agent to anticipate consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence that is front-loaded with the key action. While extremely concise, it contains no unnecessary words. It is appropriately sized for a simple tool but could benefit from minor structural improvements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and presence of an output schema, the description is adequate for basic differentiation from siblings. However, it omits behavioral context such as confirmation or side effects, leaving gaps for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must add meaning to the 'name' parameter beyond the type. It merely repeats 'by name' without explaining format, constraints, or examples, providing no additional semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and the resource ('saved rule base') with the identifying mechanism ('by name'). It unambiguously distinguishes from siblings like 'get_rule_base' or 'list_rule_bases'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or caveats. The description only states what it does, not when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rule_baseB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description only mentions retrieval. Does not disclose read-only nature, error handling (e.g., behavior when rule base not found), or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single concise sentence with no unnecessary words. Could be improved with structured formatting but appropriate for the information provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists so description needn't detail returns, but lacks mention of error conditions, prerequisites, or behavior when rule base doesn't exist.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'name' has no description in schema and the tool description adds no explanation of what 'name' refers to (e.g., name of the rule base). Minimal value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action 'Retrieve' and the resource 'Prolog source of a saved rule base', distinguishing it from siblings like 'delete_rule_base' and 'execute_prolog'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings. Does not specify prerequisites (e.g., rule base must exist) or context where listing or executing would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_prologB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must carry full burden. It mentions supported features (CLP(FD), etc.) but omits behavioral traits such as side effects, safety, error handling, or permissions. As an execution tool, it should disclose whether it modifies state or is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, each serving a distinct purpose: purpose, usage, and capabilities. No wasted words; front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 params, output schema exists), description is largely complete for the core function. It does not mention the rule_bases parameter or trace behavior, but these are well-documented in the schema. With output schema present, return values are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds some value by summarizing supported Prolog features, which is relevant to the prolog_code and query parameters. However, it does not elaborate on parameter semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (execute) and resource (Prolog code) and result (reasoning results). It distinguishes from sibling tools which manage rule bases, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs. siblings (e.g., save_rule_base, delete_rule_base). The description does not mention alternatives or provide pre/post conditions, leaving the agent to infer context from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_rule_baseA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It states 'Save' but does not disclose whether this overwrites existing rule bases, if any permissions are needed, or what the side effects are. The behavioral context is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the action and scope, then provide a usage example and contrast. Every sentence adds value; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose and usage guidelines adequately, but lacks behavioral transparency and parameter semantics. Given the existence of an output schema (which presumably documents return values), completeness is moderate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds minimal parameter detail. It mentions 'name' and 'content' but does not explain expected formats, constraints, or what constitutes valid Prolog rules. The parameters are left almost entirely to schema interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Save a named rule base containing Prolog rules that can be reused across execute_prolog calls.' It uses a specific verb and resource, and distinguishes usage from including rules directly in prolog_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidelines are given: use for stable, reusable knowledge (e.g., chess piece movement rules), and for one-time facts, include them directly in prolog_code instead. This provides clear when-to-use and when-not-to-use advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rule_basesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that results are sorted by name and describes the metadata extraction process from file comments. This adds behavioral context beyond the empty schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus a return format, front-loaded with the core purpose. Every sentence adds value without extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema, the description fully explains the return structure, sorting, and metadata source. It provides sufficient context for use alongside sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and schema coverage is 100%. Per guidelines, the baseline is 4. The description adds no parameter info, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'List all saved rule bases', which is a specific verb and resource. This clearly distinguishes it from sibling tools that delete, execute, get, or save rule bases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates that this tool lists all rule bases, implying it is for reading/listing purposes. While no explicit when-not or alternatives are given, the context of siblings makes the usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.