Server Quality Checklist

Tool Scores

• evo2_scoreA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It discloses output structure, the causal shift causing position 0 to be unscored, optional per-position output, and explicitly warns against clinical diagnosis and pathogenicity interpretation. This is thorough and transparent for a model-scoring tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a few sentences but each adds necessary context: purpose, output details, usage, and disclaimers. It is not overly verbose and front-loads the core function. Slightly dense with parentheticals, but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description fully explains the returned fields (total_log_likelihood, mean_log_likelihood, scored_positions, optional per_position_log_likelihood) and the positional convention. It also addresses interpretation caveats. For a single-sequence inference tool, nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are well-documented in the schema. The description adds little beyond the schema; it does explain the per-position output semantics within the overall description, but that is not parameter-specific. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes model-based log-likelihood of a DNA sequence under Evo2-7B, with specific output fields. It does not explicitly differentiate from siblings like evo2_batch_score or evo2_variant_score, but the single-sequence focus is implied by the parameter list and usage text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear context: 'Use for sequence-level probability estimates' and notes this is a DNA foundation model inference tool. It does not explicitly state when not to use it or mention alternatives, but the use case is specified clearly enough without being misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evo2_variant_scoreA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it explains the output meaning (negative delta indicates lower likelihood), the reasoning behind rejecting position 1 (causal LM limitation), coordinate system default, and important limitations (not clinical, not pathogenicity). This goes well beyond a basic verb+object statement and gives the agent a clear model of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences with no redundancy. It front-loads the core purpose and output interpretation, then adds constraints and domain caveats. Every sentence carries meaningful information, and the structure makes it easy for an agent to quickly grasp the tool's role.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema or annotations, the description covers the essential aspects: what it computes, how to interpret the result, coordinate system, positional restriction, and domain limitations. It does not explicitly describe the effect of include_per_position (though schema notes default false), but that is a minor gap. Overall, an agent has sufficient context to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (83%), so the baseline is 3. The description adds semantic value by explaining the output delta and the coordinate system, but it does not elaborate on parameters that the schema already documents. It reinforces the 'sequence' as wildtype context, which is consistent with schema descriptions, but adds no new parameter-level nuance beyond the position-1 rejection rule.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Compare a single-nucleotide variant') and defines the exact output ('delta_log_likelihood (mutant − wildtype)'). It clearly distinguishes the tool from siblings by focusing on variant comparison rather than generic scoring or batch operations. The inclusion of the model name (Evo2-7B) and the caveat about clinical diagnosis further clarify its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (for single-nucleotide variant scoring) but does not explicitly compare with siblings like evo2_score or evo2_batch_score. It does provide usage constraints (1-based positions, rejection of position 1, not for clinical diagnosis) that help an agent decide applicability, yet it lacks an explicit 'use when' versus 'use instead' statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evo2_batch_scoreA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure, and it excels. It reveals the caching behavior (WT pass computed once, identical mutants forwarded once), concurrency limits (EVO2_MCP_MAX_CONCURRENCY, default 2) to respect NVIDIA rate limits, and per-variant error reporting. It also adds crucial disclaimers (not clinical diagnosis, not pathogenicity labels). This is exemplary transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place. It leads with the core purpose, then efficiently packs performance details, error handling, and disclaimers. It is slightly longer than minimal but avoids fluff. The structure is logical: purpose → efficiency → safety/limitations. This is good, though a bit dense; a 4 feels right.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a batch scoring tool with no output schema, the description covers the execution semantics, concurrency, error reporting, and clinical disclaimer. However, it omits any description of the return format or how results are structured (even per-variant errors are mentioned but not the shape). Given the complexity and no annotations, the absence of output details leaves a small but notable gap, so it stops short of a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (sequence and variants have descriptions; coordinate has an enum with a default). The description does not add parameter-level meaning beyond what the schema already provides—it merely restates the variants structure and coordinate default implicitly. It does clarify the 'many' scope, but that is more behavioral than parameter-specific. Baseline 3 is appropriate since the schema covers most parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a strong, specific verb-resource pair: 'Score many single-nucleotide variants against one wildtype sequence.' It clearly states the batch scope and distinguishes itself from single-variant tools by emphasizing 'many' and 'batch.' It also specifies the analysis type (saturation-mutagenesis), which orients the agent on the intended use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool provides a clear when-to-use directive: 'Use for saturation-mutagenesis-style analyses.' It also implies batching by explaining the reuse of the WT forward pass and bounded concurrency. However, it does not explicitly name alternative sibling tools for single-variant scoring, so the agent must infer those from the 'many' qualifier. This is solid but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evo2_score_fastaA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full disclosure burden. It states the output type (per-record total/mean log-likelihood), error reporting behavior, and importantly warns that scores are not clinical diagnostics. It does not explicitly state read-only behavior, but for an inference tool this is implied and not a significant gap given the safety disclaimers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet covers all essential aspects: purpose, input options, constraints, output, and limitations. The main action is front-loaded, and every sentence serves a purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description explains the return values clearly. It covers input constraints and error reporting, but does not specify the exact response format (e.g., whether it's a JSON object or text). For a tool with only two parameters, this is a minor gap, so a 4 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptive parameter comments, so the baseline is 3. The description adds value by clarifying the meaning of the output (per-record scores) which is not in the schema, and reinforces the sandboxing constraint for fasta_path, going beyond the schema's own description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Score every record'), the specific resource (FASTA source), and the model (Evo2-7B). It also distinguishes itself from siblings by focusing on FASTA input, making it easily separable from evo2_score or evo2_variant_score without needing to infer.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly explains when to use the tool: 'Provide EITHER fasta_text OR fasta_path', with the path restriction clearly stated. It also provides cautionary guidance on interpreting results, which helps agents avoid misuse. The description mentions no alternatives but the either/or and sandboxing rules effectively guide usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evo2_forwardA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full responsibility. It discloses mode behavior (summary is context-safe, raw only for small tensors, save writes to server output dir), ambiguity-code handling with an environment flag, endpoint-dependent layer naming, and the clinical limitation. This is exceptionally transparent for a complex inference tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose, then organizes modes and constraints in a logical flow. Every sentence provides distinct information—no filler. Dense but efficient, appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four parameters, no output schema, and no annotations, this description is remarkably complete. It covers return types, mode constraints, output directory rules, ambiguity handling, endpoint differences, and disclaimers—everything an agent needs to invoke it correctly without external documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions already cover all parameters (100% coverage), but the description adds significant value: it explains the endpoint-specific layer names (output_layer vs unembed), the save_mode implication of save_path, and the ambiguity environment flag. This goes beyond the schema to make parameter usage clearer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Run a forward pass of Evo2-7B') and explicitly contrasts with scoring tools: 'Use this when you need layer outputs for analysis, not just a scalar score.' This clearly distinguishes from sibling scoring tools and conveys the tool's core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use this tool versus alternatives ('not just a scalar score'), and details the three modes with their intended contexts. It also adds a clinical disclaimer ('does not provide clinical diagnosis'), guiding appropriate usage boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.