Server Quality Checklist

Tool Scores

• list_available_quantum_backendsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is clear. The description adds valuable context: each call reads the public list interface in real time and uses no cache, which explains why results may vary and aligns with openWorldHint. This is beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the main purpose, and provides essential parameter guidance and a behavioral note without waste. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with one optional parameter and an output schema, the description is complete: it explains the purpose, the parameter values, and the real-time behavior. No additional explanations of return values are needed since an output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema defines backend_type as an enum with values all, quantum_computer, and simulator, but lacks descriptive text. The description explicitly lists these possible values and ties them to the tool's purpose of filtering between quantum computers and simulators, adding meaning beyond the raw schema. However, it doesn't elaborate on the differences between the two backend types beyond their names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists currently running quantum computers and simulators on Tianyan Cloud. It uses a specific verb ('查询' / query) and resource, and distinguishes itself from siblings like get_quantum_backend (which likely retrieves a single backend) and submit_quantum_experiment (which submits jobs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing available backends, but does not explicitly state when to use this tool versus alternatives. It provides context about the backend_type parameter and real-time data fetching, but no clear exclusion or alternative comparisons. Since sibling tool names suggest different purposes, the usage is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_quantum_experimentA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate non-read-only, non-idempotent, and destructive behavior. The description adds specific context: it creates an experiment, consumes Tianyan cloud resources, may create duplicate tasks with repeated calls, and requires user confirmation. This goes beyond the annotations by detailing the exact side effects and the mandatory confirmation step, with no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at four sentences, with the primary action front-loaded in the first sentence. Each subsequent sentence adds essential context about side effects, duplicate risk, and user confirmation. No filler or repetition exists.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values need not be explained. The description covers the experiment creation, resource consumption, duplicate risk, and required confirmation. However, it omits any mention of the optional 'allow_hardware' parameter and the expected format of the 'circuit' string, which are relevant for correctly exercising the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions only some parameters indirectly: 'backend' (likely computer_code), 'circuit name' (likely name), 'lab_name', and 'shots'. It does not explain the 'circuit' parameter, the 'allow_hardware' boolean, or the exact meaning of 'computer_code'. This partial coverage leaves significant gaps for correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action: 'Submit and run one QCIS experiment circuit.' This clearly identifies the verb and resource, and it distinguishes the tool from siblings like list_available_quantum_backends and get_quantum_backend, which deal with backends rather than experiments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear prerequisite: the host must show the backend, circuit name, lab name, and shots to the user and obtain confirmation before submitting. It also warns that repeated calls may create duplicate tasks, offering guidance against redundant use. However, it does not explicitly mention alternatives or when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quantum_backendA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint. Description adds matching order and case-insensitivity details, plus confirmation of no fuzzy search, which is valuable context beyond the annotation safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, followed by precise matching rules. Every sentence contributes information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple single-parameter read operation with an output schema. Annotations cover safety profile, description covers matching semantics. No essential information is missing for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for the parameter, but the description compensates by explaining that 'identifier' can be a resource id or code and defines exact matching semantics. This adds meaning beyond the bare string type in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb '查询' (query) and identifies resource type '天衍云量子资源' (Tianyan Cloud quantum resources). It explicitly states exact matching by id or code, and excludes fuzzy/name search, distinguishing it from sibling list tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly states matching behavior (id first, then case-insensitive code) and explicitly excludes fuzzy search, implying use when exact id/code is known. However, it does not explicitly name alternative sibling tools like list_available_quantum_backends for browsing scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.