Server Quality Checklist

Tool Scores

• run_dc_opA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully bears the responsibility of disclosing behavior. It states that the tool 'injects a .op command into a temporary copy of the netlist (does not modify the original file)' and that results are small enough to return directly. This provides clear insight into side effects and performance characteristics, though it could also mention error handling or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with separate sections for purpose, behavior, arguments, and return value. Every sentence adds value without redundancy, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has only one parameter and an output schema is provided in the description, the description covers the necessary information: what the tool does, how it works, what is returned, and the format of the result. It is complete for its simplicity, though it could mention potential errors or limitations to achieve full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero description coverage, so the description must add meaning. The Args section clarifies that 'netlist_path' is an 'Absolute path to the netlist file', which provides essential semantic context beyond the schema's mere type definition. This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a DC operating point analysis on a SPICE netlist. The verb 'Run' and specific resource 'DC operating point analysis' are precise. The tool name and description differentiate it from siblings such as AC analysis, sweep, and transient, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool over alternatives like run_ac_analysis or run_transient. There is no explicit context about appropriate use cases, prerequisites, or conditions that would help an AI agent decide when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_transientA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool creates a temporary copy (no modification to original), injects a .tran command, saves results to .npz file, and provides loading instructions. This gives good insight into behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a concise summary sentence followed by behavioral details, an example usage, alternative tool reference, and clear parameter descriptions. No wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description covers return format (JSON summary with data_file, time range, trace names). It also addresses key behavioral aspects like file creation. For a moderate complexity tool (4 params, SPICE context), it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clearly explains each parameter: netlist_path, step_time, stop_time, and start_time (including default and format examples like '1n', '10u'). This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Run a transient simulation on a SPICE netlist' with a specific verb and resource. It distinguishes from sibling tool run_sweep by noting that run_sweep is for parameter sweeping across multiple runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use this tool (single transient analysis) and explicitly directs to run_sweep for sweeping parameters. It does not list explicit when-not scenarios but offers a clear alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_ac_analysisA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description discloses non-destructive behavior (temporary copy), output format (.npz), and how to load results. Could mention prerequisite netlist existence but still strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with paragraphs, front-loaded purpose, and an Args table. Every sentence provides essential information with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers parameters, non-modification, output file details, and return format. Output schema exists but description's Returns section adequately describes the result. Complete for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, description includes an Args section with clear explanations and examples for each parameter (e.g., SPICE suffix for frequencies), adding significant meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it runs an AC frequency sweep on a SPICE netlist, injects .ac command, does not modify original. Distinguishes from sibling run_sweep which is for parameter sweeps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use run_sweep instead for sweeping component values across multiple AC runs, providing clear when-to-use and alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sweepA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes internal use of SimRunner, concurrency, per-run .npz file creation, and default parallelism. Mentions output format (JSON list with run, data_file, values, traces). No annotations provided, so description fully carries the burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-organized: purpose sentence, implementation detail, usage recommendation, example, parameter descriptions, return description. Every sentence adds value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all inputs and output format despite having output schema. Explains output file naming and return structure. Even with good annotations, the description would be complete; without annotations, it is exemplary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All four parameters are explained with descriptions, types, and examples. netlist_path as absolute path, analysis_cmd as full SPICE line, runs as list of dicts with key-value pairs, parallel with default. Compensates for 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it runs multiple SPICE simulations in parallel sweeping component values. Distinguishes from siblings by explicitly advising use instead of calling run_ac_analysis/run_transient in a loop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use this tool vs alternatives: 'Use this instead of calling run_ac_analysis / run_transient in a loop — it's significantly faster for multi-run sweeps.' Provides a concrete example.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.