Server Quality Checklist

Tool Scores

• cortex_get_jobA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the call is 'Free' and enumerates return fields, but it does not explicitly state this is a non-mutating read operation, nor does it cover rate limits, error scenarios, or job lifecycle concerns. Some useful context is present, but gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler. The key verb and purpose are front-loaded, and each clause adds value: 'by id' specifies input, 'Free' notes cost, and the return list sets expectations. This is exemplary conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter polling tool without an output schema, the description is sufficiently complete. It identifies the source of the ID (via schema), states the return content, and fits well within the sibling workflow. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% because job_id is fully described as 'Job id from cortex_optimize'. The tool description adds no additional meaning about the parameter, so a baseline of 3 is appropriate since the schema carries the semantic load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Poll' with a clear resource ('optimization job by id') and explicitly lists what is returned (status, solution, objective, error). This distinguishes it from siblings like cortex_optimize (which creates jobs) and cortex_list_backends (which lists backends).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the tool is for checking the status/result of an optimization job, which is unique among siblings. It lacks an explicit 'when not to use' or direct alternative mention, but the polling purpose is self-evident from the description and schema reference to cortex_optimize.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cortex_list_backendsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds context about costs, the sellable flag, and being free, but it does not explicitly state whether the operation is read-only, whether any permissions are required, or if there are side effects—though a list operation is implicitly safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, followed by a concise usage hint. Every word earns its place; there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless list tool, the description provides a comprehensive overview of the returned data categories (availability, cost, price, sellable flag) and a practical usage context. Despite lacking annotations and an output schema, it is self-sufficient and covers the essential information an agent needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is trivially 100% since the properties object is empty. The description adds no parameter semantics because none are needed; it appropriately focuses on the output fields instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'List' with a specific resource ('solver backends'), enumerates the returned fields (availability, cost, price, sellable flag), and distinguishes itself from sibling optimization/job tools by its clear list-oriented scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage directive ('check this before choosing a quantum backend') and notes the tool is free, which indicates when it should be used. However, it does not explicitly contrast with sibling alternatives or state when not to use it, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cortex_optimizeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It transparently discloses pricing per mode, the charging formula (max(list, provider cost x 2.0)), and the asynchronous nature via job_id. It doesn't address failure handling or authentication, but the pricing and workflow transparency are strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no filler. The first sentence states purpose and return type, the second provides pricing, and the third gives usage guidance. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, pricing, mode guidance, and the need to poll with cortex_get_job. Although there's no output schema, the description clearly indicates the return of a job_id. It lacks details on error handling or response format, but for a compute submission tool, the essentials are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover all parameters (100% coverage), so baseline is 3. The description adds meaningful value by mapping prices to the mode enum and recommending auto unless quantum is needed, which helps users choose the correct mode.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool solves QUBO/Ising optimization problems and returns a job_id for polling. It distinguishes itself from siblings like cortex_get_job (polling) and cortex_estimate_optimization (estimation) by explicitly mentioning the solve and job_id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on mode selection ('Use mode=auto unless you specifically need quantum') and directs users to poll with cortex_get_job. However, it doesn't mention when to use cortex_estimate_optimization instead of this tool, leaving some ambiguity for cost estimation workflows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cortex_estimate_optimizationA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool is free, returns a specific decision block with listed components, and implies it is a pre-analysis (before paying). However, it does not explicitly state that it does not execute the optimization or alter any state, which could be inferred but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the core action and cost benefit ('for free'), followed by output expectations and usage guidance. Every phrase contributes value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having a complex nested parameter, no output schema, and no annotations, the description conveys what the tool does, what it returns (decision block with specific fields), and when to use it ('before paying for cortex_optimize'). This is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with detailed descriptions for the 'problem' parameter and its nested fields. The description does not need to add parameter details, and baseline 3 is appropriate. It adds no new parameter-level semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb 'Analyze' and clearly identifies the resource: 'an optimization problem.' It distinguishes itself from siblings by stating it returns a 'decision block' and positioning itself as the free pre-check before 'paying for cortex_optimize.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs the agent to 'Always call this before paying for cortex_optimize,' directly naming the alternative and establishing a clear ordering. This leaves no ambiguity about when to use this tool versus its paid sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.