Server Quality Checklist

Tool Scores

• open_runA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behaviors: the Engine creates the evaluator contract, measures baseline, refuses an unbounded budget, and locks the run. It also describes the return format and warns about budget parameters. It does not cover permissions or reversibility, but provides solid context beyond trivial description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose line, usage condition, behavioral explanation, return format, and a mistake warning. It is slightly longer than minimal but every sentence contributes value, avoiding fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, when to use, key behaviors, return values, and a common mistake, making it moderately complete. However, with 8 parameters and zero schema descriptions, the lack of per-parameter explanations and absence of annotations leaves gaps in how to effectively invoke the tool, especially for non-budget parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% for 8 parameters. The description only mentions max_evals, wall_clock_s, and max_cost_usd in a warning, implying they are budget constraints and at least one must be set. Other parameters like goal_text, seed, target, workers, and evaluator_ref remain unexplained, so the description does not sufficiently compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('open and lock') on a specific resource ('evolution run'), and adds the crucial condition 'when no existing run matches the work,' which distinguishes it from sibling tools like join_run that likely handle existing runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit trigger condition ('Call this once when no existing run matches the work'), but does not explicitly name alternative tools or state when not to use it beyond this condition. The 'Mistake to avoid' note adds usage guidance for budget parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lineageA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses the return type ('list[dict]') and the error behavior ('or the standard error dict'). It also cautions against reconstructing ancestry from memory or filenames, highlighting the tool's authoritative role.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact with four short sentences, each serving a distinct purpose. The 'Mistake to avoid' adds practical guidance without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and the description covers purpose, usage, return value, and a warning. No output schema is shown, but the return description suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Program_id is only typed as a string in the schema with no description. The description references it in the return statement, implying it identifies the program, but does not elaborate on format or meaning beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Read the recorded ancestry for one evaluated program,' clearly stating the action and target. It distinguishes from siblings like next_parent by focusing on recorded ancestry rather than traversal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this when explaining how a strong program was reached or when choosing which prior operators and changes deserve closer study,' giving concrete use cases. It does not, however, name alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• next_parentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It details the return structure and emphasizes the order of operations (read before mutating). It implies a read-only nature through 'Get' and 'before editing', and even mentions 'the standard error dict'. However, it does not explicitly state side-effect-free behavior or discuss auth/rate limits, so it's not a full 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with a clear purpose statement. It includes usage instructions, a detailed return format, and a cautionary note, all of which are useful. It is somewhat dense but each sentence contributes value; the return format might be redundant given the output schema, but it remains helpful for quick reference.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple getter tool, the description provides the essential context: when to call it, what it returns, and a common mistake to avoid. Since an output schema exists, the description need not detail all return fields. The main gap is the lack of parameter semantics, but overall the tool is adequately described for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage for its two parameters (run_id, island), and the tool description does not mention them at all. It adds no meaning about what these parameters represent, their formats, or how they affect the returned data. The names are somewhat self-explanatory from context, but the description fails to compensate for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the parent and evidence needed for one mutation cycle.' This specifies a concrete verb and resource, and the instruction 'Call this at the start of every worker cycle' distinguishes it from sibling tools like 'discoveries' or 'best' by framing it as the preparatory step for mutation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('at the start of every worker cycle') and what to do with the returned data ('Read the parent files, every inspiration, every discovery, the operator hint, and any crossover parent before editing'). It also warns against mutating before reading inspirations and discoveries, providing clear usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format (including both a successful payload and an error dict), and adds behavioral rules ('never omit the artifact paths or continue after a closed status'). It does not mention permissions or side effects, but for a read operation, the context is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the purpose, then usage, return format, and a warning. It is somewhat lengthy due to the full return dict, but the format is structured and each section adds value. It could be tightened by omitting the return dict if the output schema already covers it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description adds crucial usage guidance and error handling context. For a simple 1-parameter tool, it covers the key aspects: what it reads, when to call it, what it returns, and common pitfalls. No significant information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single parameter, run_id, with no description. The tool description does not explicitly explain the parameter, though it references 'a run' throughout. Since schema coverage is 0%, the description should compensate by detailing run_id, but it relies on the name and context. This is a minor gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb and resource: 'Read closure state, progress, budget, and artifact paths for a run.' This clearly distinguishes it from siblings like submit_child or next_parent, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Call this after every submit_child and at the end of every cycle summary. Stop requesting parents when status says the run is closed.' This gives clear when-to-use context and a stopping condition. It also includes a 'Mistake to avoid' section that reinforces proper usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_childA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It details the side effects ('evaluation and archive insertion'), disallows out-of-band modifications ('The Engine rejects it'), and specifies the exact return structure including the success dict and 'standard error dict.' This provides a solid behavioral contract beyond the raw schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet information-dense, with clear sections: purpose, usage instruction, return contract, and a mistake warning. Every sentence adds value, and the structure makes it easy to parse. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 params, nested files object, no annotations) and the presence of an output schema in the description, the text is reasonably complete. It covers purpose, usage, parameters, and output. It does not detail all nested output fields, but the 'standard error dict' and named return fields provide sufficient context. Slight gap: it does not mention prerequisites like opening a run, but this is implied by sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It does so by explaining key parameters: 'Echo its parent_sample_seq' clarifies parent_sample_seq, 'use the selected or suggested operator' clarifies operator, and 'explain reasoning in notes' clarifies notes. The 'full child file contents' maps to the files parameter. While run_id and parent_id are not explicitly described, they are self-evident from context. This is strong compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's primary action: 'Submit full child file contents for evaluation and archive insertion.' This uses a specific verb ('submit') and resource ('child file contents'), and explicitly differentiates its purpose from siblings by specifying it should be called 'once after producing one mutation from the current parent bundle.' This makes it distinct from tools like next_parent or best.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual usage: 'Call this once after producing one mutation from the current parent bundle.' It also gives specific instructions on what to include (parent_sample_seq, operator, notes) and a warning to avoid modifying content outside EVOLVE-BLOCK markers. This offers strong situational guidance, though it does not explicitly name alternative tools for reference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discoveriesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return type ('list[dict]' or error dict) and adds a behavioral caution about not treating discoveries as proof. It implies read-only but doesn't explicitly confirm lack of side effects, though for a search tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sections: purpose, usage, return, and a mistake to avoid. Each sentence contributes useful information without redundancy. Well-structured for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no nested objects) and the presence of an output schema, the description is complete. It covers when to use, what it returns, and a domain-specific caution. No significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It provides some meaning for query ('focus the search on a failure, operator, or implementation technique') but does not explicitly explain domain. While the schema shows types and defaults, the description only partially adds value beyond the schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search reusable measured discoveries for a domain' with a specific verb and resource, and it distinguishes itself from sibling tools like open_run or submit_child by focusing on discovery search. The scope is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call ('before a mutation when the parent bundle needs broader evidence') and gives a contrasting mistake to avoid with submit_child, effectively providing when/why guidance. This goes beyond generic context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• join_runA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the return format ('Returns: {"island": int}') and error structure, and the warning about not inventing assignments implies that the tool assigns islands automatically. It lacks mention of side effects like marking the worker as joined, but covers key behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with each sentence serving a purpose: action, timing, parameter hint, return format, and a caution. No redundant or vague statements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only 2 required parameters and no enums, the description covers all essential aspects: what it does, when to call it, how to use parameters, and what it returns/errors. The output schema also exists, so return value details are already structured. The description is complete for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description elaborates on the 'runtime' parameter ('Pass a useful runtime label such as claude-code, codex, or a local worker identifier'), but 'run_id' is only implied by 'Join a run' and the reference to get_contract. Given 0% schema description coverage, it partially compensates but leaves run_id underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Join a run') and the result ('receive this worker's island assignment'). It distinguishes itself from siblings by placing it in a workflow ('after get_contract and before the first next_parent call').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides sequencing context ('Call this after get_contract and before the first next_parent call') and a mistake to avoid ('never invent an island number or reuse another worker's assignment'), which effectively tells the agent when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_contractA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states it is a read operation, the contract is immutable, and lists the exact return fields plus the error dict. This gives the agent a clear behavioral model.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a lead sentence, usage instruction, return list, and a mistake to avoid. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the explicit return fields, the description is complete. It also provides error handling and lifecycle guidance, making it sufficient for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the description does not mention run_id explicitly. While the context of 'run' implies run_id is the run identifier, this is not stated, so it fails to fully compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Read' and identifies the resource as 'the immutable scoring contract'. It clearly distinguishes from sibling tools like join_run by specifying this must be called before join_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this before join_run' and to reread whenever the metric, gate, target, or budget is unclear. This provides clear when-to-use guidance and even a caution against inferring or rewriting fields.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bestA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses the Engine's ownership of ranking, the best-first order, the return shape (list of dicts with up to k records), and the error case. It also warns against an anti-pattern. This is thorough behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. It uses short paragraphs for purpose, usage, return, and a caution. Every sentence contributes meaning without redundancy. This is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two parameters and an output schema (though not shown). The description covers purpose, usage timing, return format, and a mistake to avoid. This is complete for the tool's complexity and utility.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'k' as the cap on returned records and implies 'run_id' as the run identifier. It does not explicitly restate that run_id is required or that k defaults to 5, but the context is sufficient for correct usage. Slight gap in not naming run_id directly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb and resource: 'Read the best measured programs in a run.' It distinguishes from siblings by specifying 'measured ranking' and 'best-first order,' which no other sibling tool claims. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit timing ('at checkpoints or after closure') and context (when you need current leaders). It also includes a warning not to bypass this measured ranking, but it does not name specific alternative tools or exclusion scenarios, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.