Server Quality Checklist

Tool Scores

• reinvent_transfer_learningA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears the transparency burden. It discloses default validation behavior, dry-run vs. launch outcomes, and that the output model serves as an agent_file for RL. This is solid but does not detail failure modes or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient, front-loaded with the core purpose, and uses brief conditional statements for key flags. No fluff, though the bullet-like format is a bit packed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 16 parameters, no schema descriptions, and no output schema, this description is incomplete. It omits crucial details about many training parameters and presumed defaults, leaving the agent to guess or inspect further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It covers dry_run, skip_validation, launch, and output_model_file, but leaves 12+ parameters (e.g., batch_size, num_epochs, device, mol2mol_pairs) unexplained, making confident use difficult.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action: 'Fine-tune a REINVENT4 prior on a focused SMILES dataset (transfer learning).' This clearly names the verb, resource, and scope, distinguishing it from sibling tools like reinforcement learning or sampling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on modes: dry_run, skip_validation, and launch, explaining what each does and when to set them. However, it does not explicitly discuss when to prefer this over RL or sampling, so a slight gap exists.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_samplingA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the behavioral disclosure burden. It discloses dry-run behavior, synchronous execution, immediate return of results, and smiles_file requirements for specific generators. However, it omits details about return format, side effects (e.g., output files), or error handling, which would be valuable for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured, front-loading the core purpose, then using a compact bullet list for generator modes. Every sentence provides distinct value without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (12 parameters, no annotations, no output schema), the description covers key behavioral and generator-specific requirements but leaves gaps around return value, output handling, and the meaning of several parameters. It is adequate but not complete for an agent to invoke correctly without additional inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning to dry_run, generator, and smiles_file (explaining when each is needed), but leaves many parameters (num_smiles, temperature, sample_strategy, output_file, randomize_smiles, unique_molecules, workdir, device) without any explanation, which is insufficient for a 12-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Generate molecules from a REINVENT4 prior model,' which clearly identifies a specific verb and resource. The generator guide further distinguishes the tool's scope by enumerating four sampling modes, making it distinct from sibling tools like reinforcement learning or transfer learning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool, including dry-run vs. synchronous execution, the smiles_file requirement for libinvent/linkinvent/mol2mol, and a generator guide explaining each use case. However, it does not explicitly compare against sibling tools or state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_reinforcement_learningA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: dry_run previews config without writing/launching, skip_validation=False validates SMILES/TOML, launch controls whether the agent runs the job, and it details required stage/component structure. It doesn't cover all side effects (e.g., file creation, error handling) but provides substantial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose, then lists behaviors and config requirements. Every sentence adds value, though the nested bullet lists make it dense. Appropriate for the complexity, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with 21 parameters, no output schema, and no annotations. The description thoroughly covers the stages structure and gives usage context, but it omits explanations for many parameters and doesn't describe return values or failure modes. It's adequate but has clear gaps for fully independent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It thoroughly explains the central 'stages' parameter and clarifies dry_run, skip_validation, and launch, but it leaves many other parameters (e.g., distance_threshold, diversity_filter, inception, sample_strategy) unexplained. This is only partial compensation for a 21-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Run staged reinforcement learning (curriculum learning supported),' which clearly states the action and resource. It distinguishes from siblings by emphasizing 'staged' and 'RL,' separating it from sampling and transfer learning tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Call reinvent_rl_scoring_design_guide first to collect scoring config interactively,' and clearly explains launch=True vs launch=False behavior, along with dry_run and validation defaults. This tells the agent when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_validate_inputA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well: it lists specific checks (line counts, duplicates, RDKit validation, generator constraints) and discloses the side effect of writing a cleaned file when clean=True. It also describes the return object, including error details, without leaving major behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections, the purpose is front-loaded in the first sentence, and every sentence provides useful information without waste. It is appropriately concise for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the lack of annotations and output schema, the description provides a thorough overview of validation checks, side effects, and a detailed return object. It is sufficient for an agent to select and invoke the tool. Minor gaps in generator-specific constraints prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It defines smiles_file as a path, explains clean's behavior, but generator's description only restates the enum values without explaining what each generator does or what constraints apply. This partial compensation earns a 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Validate') and resource ('SMILES/CSV input file') with a contextual purpose ('before passing to REINVENT4'). This distinguishes it from sibling tools like reinvent_validate_toml, which validates config files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides context for when to use the tool ('before passing to REINVENT4') but does not explicitly list alternatives or exclusion criteria. The focus on SMILES/CSV implicitly differentiates it from the TOML validator, but a direct comparison would strengthen the guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_job_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses what the tool returns (status, log tail, current epoch, result summaries, top SMILES, checkpoint paths) and its scope (both agent-launched and registered jobs). This goes beyond a simple 'get status' and helps the agent predict behavior, though it doesn't mention error handling or invalid IDs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Returns and Args sections, and every sentence adds value. It is slightly longer than strictly necessary but remains focused and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the lack of an output schema, the description provides adequate context by listing return contents and parameter provenance. It does not mention potential error cases or how to interpret statuses, but for a polling tool this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'job_id' as a string, but the description's Args section adds critical meaning: 'ID from reinvent_transfer_learning, reinvent_reinforcement_learning, or reinvent_register_job.' This tells the agent where to obtain the value, compensating for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Poll the status of any registered REINVENT4 TL or RL job', using a specific verb ('Poll') and a clear resource ('status'). This distinguishes it from sibling tools that launch, validate, or analyze jobs, making its purpose immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states 'Works for both agent-launched and user-registered jobs', providing useful context for when to use the tool. It does not explicitly name alternatives, but since it is the only status-polling tool among the siblings, the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_attach_pidA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds meaningful context beyond the obvious: the server uses os.kill(pid, 0) to check if the process is running, and the tool improves polling reliability. It could be more transparent about error handling or side effects (e.g., overwriting an existing PID), but the provided mechanism and rationale are strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose, a brief workflow explanation, and a small Args list. Every sentence adds value, and the first sentence front-loads the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple attach operation, the description covers the what, when, and why. It also gives the practical benefit (live polling reliability). It does not describe the return value or failure modes, which could be useful, but given the absence of an output schema and the tool's simplicity, the provided context is largely sufficient for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameters. The Args section defines job_id as 'Job ID from reinvent_register_job' and pid as 'Process ID of the running reinvent job', which adds meaning beyond the bare schema. It could be more detailed (e.g., formats or validation), but it clearly ties job_id to the upstream registration step.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Attach a process ID to a registered user job for live polling.' This is a specific verb (attach), resource (process ID to job), and purpose. It distinguishes itself from siblings by referencing the workflow with reinvent_register_job and focusing on polling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear 'when to use': after running a job manually and registering it with reinvent_register_job. It explains the benefit ('improving polling reliability') and mentions the server's os.kill mechanism. However, it does not explicitly state when not to use it or name alternative tools, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_analyze_moleculesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description clearly discloses side effects: writes analysis_report.json and per_molecule.csv to disk, and generates PNG plots when plots=True. It also mentions auto-skipping of evaluators without a reference file, which is a behavioral nuance. No annotations are provided, so the description carries the full burden, and it does so thoroughly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening, evaluator breakdown, physicochemical details, and an Args list. While it is lengthy, most details add value (e.g., listing exact metrics). Some evaluator metric lists could be condensed, but the overall structure is logical and front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 6 parameters and no output schema, the description covers nearly all relevant context: input formats, evaluators, metrics, plot outputs, defaults, and auto-skip behavior. It does not explicitly describe the structure of the returned summary, but this is a minor gap given the abundant detail elsewhere.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate entirely. It provides detailed meaning for every parameter in the Args section, including type, purpose, defaults, and examples (e.g., color_by lists 'qed', 'mw', etc.; output_dir default is shown). This fully bridges the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource combination: 'Evaluate a set of generated SMILES across multiple quality dimensions.' It clearly distinguishes this tool from siblings by noting it is designed for raw sampling output from reinvent_sampling, while other siblings handle sampling, RL, validation, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('Designed for raw sampling output from reinvent_sampling') and clarifies it works with any SMILES file. It also notes auto-skipping of reference-based evaluators if no reference file is provided, which guides usage. However, it does not explicitly state when not to use it or name alternative analysis tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_register_jobA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool only registers for status tracking (not execution) and provides path defaults. While it doesn't detail side effects or failure behavior, the registration-only nature is transparently conveyed, which is significant for a tool that could otherwise be mistaken as a job runner.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: a top-line purpose, a short usage directive, and a bullet-like Args list. Every sentence contributes new information—no filler, no redundancy, and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the full workflow: run the command first, register it, then poll or attach PID. It lacks an explicit note about return values or errors, but given the tool's simple registration nature and lack of output schema, the context provided is sufficient for an agent to act correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description's Args section adds essential meaning: run_type enum values, workdir purpose, job_id auto-generation, and explicit defaults for log_file and config_path. It compensates well for the missing schema-level descriptions, though it repeats the enum values already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Register a user-launched REINVENT4 job for status tracking' — a specific verb, resource, and scope. It also distinguishes from siblings by referencing polling with reinvent_job_status and PID attachment with reinvent_attach_pid, making the tool's unique role evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs 'Use after running the suggested_cmd yourself' and names the subsequent steps: poll with reinvent_job_status, optionally attach PID with reinvent_attach_pid. This provides clear when-to-use context and names alternatives/related tools, meeting the 5-level bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_validate_tomlA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses the validation method (preferred 'reinvent --validate' vs Pydantic model) and specifies the return structure. It does not cover side effects or failure modes outside the return schema, but for a validation tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized: a one-line summary, a sentence on method, then structured Args and Returns sections. Every sentence adds value, and it is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description compensates by detailing the return object's fields (valid, errors, warnings, config_summary). It covers the validation process and parameters, making it sufficiently complete for a 2-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must explain parameters. It does so via an Args section that clearly maps 'toml_path_or_content' to a path or TOML content string and 'is_string' to a boolean toggling content interpretation, adding meaning beyond the schema's bare titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a REINVENT4 TOML config before launch, specifying the resource (TOML config) and action (validate). It distinguishes itself from sibling tools by focusing on REINVENT4-specific TOML validation, although it doesn't explicitly contrast with reinvent_validate_input.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating 'before launch', indicating when this validation should occur. However, it does not explicitly mention alternatives or when not to use this tool, potentially leaving some ambiguity with sibling validators like reinvent_validate_input.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_rl_scoring_design_guideA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that schemas are introspected from the live registry (implying read-only, authoritative data) and explicitly scopes out component definitions, which helps set expectations. However, it doesn't explicitly state read-only behavior or other potential side effects, so a small gap remains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a front-loaded purpose sentence, then key notes, steps, and a call-to-action. Every sentence adds value, and the layout (with NOTE and numbered steps) enhances readability despite the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description is remarkably complete. It explains what the tool returns (transform types and schemas), what it excludes (components), provides a 5-step usage process, and specifies when to call it relative to the RL workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4 per the rubric. The description mentions that it returns lists like valid_transform_types and transform_parameter_schemas, but since there are no params to explain, this is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a scoring-function design guide with authoritative transform schemas from the live REINVENT registry. It distinguishes itself from siblings by explicitly noting it does not define component names/params, and by positioning itself as the source of truth for transforms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Call this before reinvent_reinforcement_learning to collect the scoring config.' It also states alternatives (component registry for components, /custom-component routing) and prohibitions ('Never use a transform type not in this list').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_custom_scoring_componentA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: atomic write operation, runtime discovery, TODO placeholder behavior, permission restrictions preventing later edits, automatic smoke testing, and the exact return payload. This goes well beyond what annotations would typically provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise purpose statement, a critical IMPORTANT callout, a bulleted Args list, and a Returns line. Every sentence adds value—no fluff or redundancy—and the most critical usage requirement is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 12 parameters, no output schema, and no annotations, the description fully covers inputs, outputs, side effects, and error behavior. It even explains the smoke test flow and says that test_result appears in the response, making the invocation contract complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage from description text, but the description's Args block documents all 12 parameters with meaning, defaults, and examples (e.g., test_params with scalar list-wrapping). This fully compensates for the schema's lack of descriptions, providing agents with the necessary context to fill parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-object statement: 'Generate a valid comp_*.py plugin file for REINVENT4.' This clearly states the tool's primary function and distinguishes it from sibling tools like reinvent_test_scoring_component or reinvent_sampling by focusing on plugin generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The IMPORTANT section provides explicit usage guidance: always provide implementation_code, write it as unindented Python, indentation is added automatically, and the file cannot be edited afterward. It also explains when the smoke test runs. However, it does not explicitly name alternatives or state when not to use this tool versus sibling tools, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_plot_rl_historyA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden of behavioral disclosure. It explains the absence of plot generation in inspect mode ('No plots are generated'), the standard set of plots generated in plot mode, output directory defaults, and return value (paths of saved PNGs). It also notes precedence behavior when both job_id and csv_path are supplied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and formatting. It is front-loaded with a one-line purpose, then the two-call protocol, then a concise parameter list. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and moderate complexity, the description is complete: it covers the two modes, parameter semantics, default paths, return values, and runtime applicability. It gives the agent enough to invoke the tool correctly in both inspect and plot scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description's Args section fully explains each parameter's meaning and usage: job_id for registry lookup, csv_path for direct file access, extra_columns for selecting additional columns, and output_dir for saving plots. This significantly compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Inspect columns or generate optimisation history plots for a REINVENT4 RL run.' It specifies the exact resource and verb, and distinguishes from siblings by describing its unique two-mode behavior (inspect vs. plot).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong usage context, including when it works ('mid-run and post-run') and the two-call protocol. It explicitly states input alternatives ('job_id OR csv_path') and precedence, but does not mention when to avoid this tool or name alternative tools for similar tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reinvent_test_scoring_componentA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully carries the burden. It discloses key behaviors: catching import errors, instantiating the component, testing valid and invalid SMILES, asserting the REINVENT4 plugin contract (returns ComponentResults with np.ndarray of floats, correct length, NaN for invalid inputs), and returning a structured dict. This is thorough and sets accurate expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a purpose statement, usage context, and an Args/Returns breakdown. It is dense but every sentence contributes relevant information. The front-loaded main purpose makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (plugin testing with contract assertions) and the lack of annotations or output schema, the description is remarkably complete. It covers the workflow, edge cases (invalid SMILES), return fields, and relationship to the sibling tool that auto-runs smoke tests.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides meaningful details for every parameter: file_path includes a path constraint, test_smiles explains default behavior, params gives an example, and class_name notes auto-detection. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool's function: 'Smoke-test a comp_*.py plugin on sample SMILES without running RL.' It uses a specific verb ('smoke-test'), identifies the resource (comp_*.py plugin), and distinguishes it from sibling tools like reinvent_custom_scoring_component and the RL-running tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('after manually editing a comp_*.py, or to re-verify with custom SMILES / parameters') and when not to ('For freshly generated files, the smoke test is already auto-run by reinvent_custom_scoring_component'). This provides clear guidance and an exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.