Server Quality Checklist

Tool Scores

• solver_review_proposalB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states that the tool approves or rejects a proposal, but does not mention side effects, whether the decision is final, permission requirements, or what response to expect. This is a significant gap for a mutating action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core action and includes the endpoint for precision. There is no redundant filler, and the key distinction between approve and reject is clearly conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutating tool with no annotations and no output schema, the description is too thin. It fails to mention what happens after approval or rejection, whether reject requires a reason, possible error conditions, or the expected response format. An agent would likely need to inspect other sources to invoke this tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description should compensate for the undocumented proposal_id and decision parameters. It adds that the decision parameter selects which endpoint is used, but it does not explain what a proposal_id refers to, how to obtain valid IDs, or any constraints beyond the schema's enum and the optional reject-only reason.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: approve or reject a repair proposal, and includes the exact endpoint pattern. It is specific enough to distinguish from sibling tools like solver_proposals or solver_submit, which handle other proposal operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives such as solver_proposals or solver_submit. The intended context must be inferred from the tool name and endpoint, which is not enough for an agent choosing among many proposal-related operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_list_armsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does state the HTTP method (GET), the resource scope, and the return envelope, which implies a read-only paginated list operation. However, it does not explain pagination semantics, authentication needs, or behavior when no arms exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. It front-loads the core action and resource, then gives the endpoint and return shape. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given four optional parameters, no output schema, and no annotations, the description is not complete enough for reliable invocation. It provides the return shape but omits parameter explanations and pagination/usage context, forcing the agent to rely on parameter names and enums alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to compensate. It adds no explicit meaning for 'kind', 'limit', 'cursor', or 'status'. The mention of 'next_cursor' weakly implies that 'cursor' is a pagination token, but the other parameter purposes are left entirely to inference from the schema enums and names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('List'), a specific resource ('the tenant's registered arms'), and the HTTP endpoint. It is distinguishable from siblings like solver_register_arm and solver_enable_model_arm because it is explicitly a read-only listing operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives such as solver_list_model_deployments, solver_register_arm, or solver_enable_model_arm. An agent might infer it is for enumerating arms, but the description provides no explicit when-to-use or when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_proposalsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It usefully exposes the HTTP method GET and the read-only nature of listing/inspecting proposals. It does not disclose auth requirements, default status behavior, or what happens when proposal_id is combined with filters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one front-loaded sentence that puts the core operation first and adds the endpoint in parentheses. Every clause earns its place by contributing either an operation mode or routing information, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description needs to provide more operational context, but it omits the meanings of arm_id and status, the default list behavior, pagination, and the shape of a list response. It is sufficient for the proposal_id detail mode but incomplete for the full tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is low at 33%: only proposal_id has a schema description. The description adds no meaning for arm_id or status, and the word 'pending' only vaguely hints at a default status without mapping to the enum. It therefore fails to compensate for the undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource ('List pending repair proposals') and explicitly adds the inspect-one mode when proposal_id is set, including the REST path. This clearly identifies what the tool does and distinguishes it from mutation-oriented siblings like solver_review_proposal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The conditional 'or inspect one when proposal_id is set' provides clear guidance for choosing between this tool's two modes. However, it does not mention when to use this tool instead of solver_review_proposal or other proposal-related siblings, so external usage guidance is left implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_register_armA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure burden. It does add useful behavioral context by stating that the tool returns the created arm with arm_id, status, and artifact_hash for skill kind. However, it does not mention side effects beyond creation, permission needs, idempotency, or failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that wastes no words. It states the action, target, endpoint, and key return values compactly while remaining easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an 11-parameter schema with nested objects, no annotations, and no output schema, the description is too sparse to fully guide correct invocation. It omits the kind-dependent payload structure and does not clarify which optional fields apply to which arm kind, so an agent is left to infer key registration semantics from the schema alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 45%, so the description should compensate for the many undocumented parameters. It does not explain how kind selects among artifact, endpoint, or model_id, nor what lifecycle, cost_class, capability_tags, or data_class_grants mean. The only parameter-related hint is the artifact_hash mention for skill kind, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Register') with a clear resource ('a model/agent/skill arm') and even names the HTTP endpoint (POST /v1/arms). It also distinguishes the arm kinds, making the tool's purpose immediately obvious relative to siblings like solver_list_arms or solver_enable_model_arm.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operation itself implies usage: call this to create a new arm. However, there is no explicit guidance about when not to use it, no mention of prerequisites (e.g., needing a source connection), and no alternatives are named among the many sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_submitA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does real work: it discloses that this is a mutating POST operation and reveals the non-obvious idempotency contract (unique ID transmitted as the Idempotency-Key header, the call being impossible without it). It does not cover duplicate-key behavior, cost implications, or post-submission lifecycle, but the most critical behavioral trait is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, front-loaded with the core action and endpoint before the prerequisite. The closing clause 'an MCP client cannot construct a valid call without it' slightly repeats what 'Requires' already conveys and duplicates the schema's required flag, but the overall structure is economical and well-ordered.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool — 5 parameters, deeply nested objects, 4 required fields, no annotations, and no output schema — yet the description leaves major gaps: the meanings of policy and verifier_id, what the call returns (execution ID/receipt?), and the follow-up workflow (how to poll or retrieve results via siblings) are all unaddressed. The idempotency gotcha is handled, but the surrounding context an agent needs to invoke this correctly is largely missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20% — four of five parameters (task, policy, compose, verifier_id) have no semantic explanation in the schema or the description. The description clarifies only idempotency_key, whose tricky header semantics genuinely needed it. Since coverage is low, the description was expected to compensate, and it largely does not: what constitutes a policy, what verifier_id refers to, and what compose:'auto' means all remain opaque.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource — 'Submit a task for governed execution' — backed by the explicit endpoint (POST /v1/executions). This clearly differentiates it from siblings like solver_cancel, solver_status, and solver_result, which concern later stages of the lifecycle. An agent can identify this as the task-launching tool without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes the tool as the submission entry point for governed executions and stresses the hard prerequisite of a unique idempotency key. However, it never explicitly states when to use this tool versus alternatives, nor does it name any sibling or exclusion. Usage is implied by the verb and endpoint rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_cancelA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden of disclosing behavior. It only restates the cancel action and the endpoint, without explaining whether cancellation is asynchronous, idempotent, reversible, or what side effects it has on the execution and related resources. This is a meaningful gap for a mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. Every word earns its place: it names the action, the target resource, and the endpoint.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, this description covers the essential invocation details and the condition for use. However, the missing behavioral and post-condition information leaves the description incomplete for an agent that needs to reason about the consequences of cancellation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes execution_id as a required string, and the description provides minimal extra context by tying it to the execution resource in the endpoint. With 0% schema description coverage, the description slightly compensates but does not clarify format, source, or expected value conventions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Cancel') and resource ('running execution') and includes the exact HTTP endpoint. This clearly distinguishes it from sibling tools such as solver_status or solver_submit, which share the execution domain but not cancellation behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'running execution' gives a clear precondition: use the tool only when an execution is actively running. It does not name explicit alternatives or say when not to use it, but for a single-purpose cancel endpoint this is sufficient contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_list_sourcesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds meaningful behavioral context: sources run on the tenant's provider key/account, each entry contains protocol/auth profile versions and endpoint policy, and no credential material is involved. The read-only nature is implied by 'List' but not explicitly stated, which is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence delivers the core action and endpoint, and the second adds relevant behavioral detail. It is well-structured for quick scanning and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description is nearly complete: it gives the endpoint, tenant scope, and key behavioral context. It does not describe the full result format, but with no output schema that is a minor gap. The main missing piece is routing guidance relative to similarly named siblings, which was already factored into usage_guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is empty, so the baseline for this dimension is 4. The description adds no parameter-specific details, but none are needed; it does add value by explaining what each returned entry carries, which aids interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('List the model sources this tenant can connect') and includes the exact endpoint, making the resource and scope clear. It is distinguishable from sibling tools like solver_list_source_connections by the 'model sources' wording, though it does not explicitly name or contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives. The description implies a simple listing use case but does not explain when to prefer it over solver_list_source_connections or other siblings, and there are no exclusions or conditions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It usefully discloses that this is a GET polling operation, which indicates a read-only style behavior. It does not, however, describe whether the poll blocks, what lifecycle states exist, or what error responses look like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action, resource, and endpoint with no filler or redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter status-polling tool, the core call semantics are covered. However, with no output schema and no annotations, the description omits what the response looks like, what lifecycle values are possible, and how the agent should interpret the result, leaving the tool partially underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description maps the sole execution_id parameter to the {id} path segment in GET /v1/executions/{id}, which adds mild clarity beyond the bare schema. Still, it does not explain the expected format, source, or validation behavior of execution_id, leaving some room for interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Poll') and a clear resource ('an execution's lifecycle state'), and it pins the operation to an exact HTTP endpoint. This makes the tool's purpose immediately identifiable and distinct from sibling tools like solver_result or solver_submit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrasing implies this tool is for checking an execution's lifecycle state after submission. However, it does not explicitly state when to prefer this over similar siblings such as solver_result or solver_poll_source_handoff, nor does it mention any exclusions or polling frequency.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_enable_model_armA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals the HTTP endpoint, the required kind=model and model_deployment_id, the secretless template requirement, and the important restriction that wider claims are rejected. It does not mention auth, idempotency, or response behavior, but the key constraints for using it correctly are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with no filler. The endpoint is front-loaded, and the critical constraint about secretless template values and rejected claims is immediately actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core mechanics and a key limitation, but with no annotations and no output schema, an agent is left without guidance on response/return behavior, prerequisites, or how this tool relates to the very similar-sounding solver_register_arm. It is adequate but not fully complete for safe selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful guidance by pointing to the catalog entry's arm_registration_template as the source of values, and warns that claims wider than the template are rejected. However, it does not individually explain display_name, capability_tags, data_class_grants, or cost_class beyond the schema's enum definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: enabling a certification-backed model deployment as an arm. It also names the exact endpoint (POST /v1/arms with kind=model and model_deployment_id), which clearly distinguishes this from the more generic sibling solver_register_arm.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the wording 'certification-backed model deployment' and the endpoint details, but the description never explicitly says when to choose this tool over solver_register_arm or other sibling tools. It gives value-source guidance ('use the catalog entry's secretless arm_registration_template values') but not selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_poll_source_handoffA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It does disclose a read-only GET operation and that only the intent state is returned. However, it does not clarify whether a single call blocks or returns immediately, whether repeated polling is required, or what the possible state values are.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core action and resource. The endpoint is placed in parentheses as supporting detail, and the 'state only' note adds important clarification without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter polling tool with no output schema and no annotations, the description gives the main purpose and endpoint but omits operational essentials: how to obtain the handoff_intent_id, how polling terminates, and what state values will be returned. This leaves some necessary inference to the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does show handoff_intent_id being used in the endpoint path, which helps, but it does not explain where the ID comes from, such as the result of solver_start_source_handoff, or any format expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Poll a credential handoff intent.' The endpoint and the clarifying phrase 'Answers the intent state only' make it clear this is a status-read operation, not a mutation or creation tool, distinguishing it from siblings like solver_start_source_handoff.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'until the human completes it in the browser' clearly conveys when this tool is relevant: after a handoff intent has been started and the agent is waiting on user action. It does not explicitly name the prerequisite tool or provide when-not-to-use guidance, but the context is reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_list_source_connectionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It explicitly identifies GET /v1/source-connections, indicating a read-only list operation, and adds a valuable clarification that auth_binding_ref is an opaque handle and not a secret. Pagination or rate limits are not discussed, but for a zero-parameter read list the key behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences: the first states the action, resource, scope, and endpoint; the second adds a relevant security clarification. There is no filler or repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter read operation with no output schema, the description covers the operation, scope, endpoint, and a key response attribute. It could say more about the full return shape or the meaning of test/rotation state, but nothing essential for invoking it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already exhaustively covers the input surface. The description adds useful scoping ('this tenant's') and the endpoint, which is meaningful context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' plus a specific resource 'source connections' and clarifies the tenant scope and included state (test/rotation). It is immediately distinguishable from sibling tools like solver_list_sources or mutation-oriented tools without inspecting schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives, especially solver_list_sources. The listing intent is implied, but there are no exclusions, prerequisites, or decision criteria stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_resultA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden: it identifies the operation as a GET retrieval, conditions availability on completion/retention, and describes the response contents including the conditional provenance. It could add error or expiry behavior, but for a read-only fetch the key traits are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single information-dense sentence with no filler: action, endpoint, completion/retention condition, and return contents. The parenthetical endpoint placement keeps the main purpose front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple one-parameter read tool with no output schema or annotations, the description covers what the tool returns and when it is applicable. It could be more explicit about retention-period behavior and non-completed executions, but these are largely inferable from 'COMPLETED' and 'retention-bound.'

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the name and type for execution_id, so the description must add meaning. It does so by tying the parameter to the {id} in the endpoint and to a completed execution's result. It does not specify where to obtain the ID or its format, but the single self-named parameter makes this sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Fetch') and resource ('result of a COMPLETED execution'), and it lists the return contents: final text and model provenance. It is clearly distinct from status/receipt siblings in substance, though it does not name an alternative explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly scopes usage to COMPLETED executions and hints at a retention window, which gives strong context for when to call it. It does not explicitly name alternatives or say when not to use it, but the completed-execution condition is sufficient guidance for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_create_source_connectionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses important traits: the tool takes an opaque handoff_intent_id, never credential material, and newly created connections start disabled until a test passes. It does not cover idempotency, error cases, or response behavior, but the lifecycle and security context are valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences deliver the action, endpoint, key parameter constraint, security note, and workflow ordering with no filler or repetition. The most important decision-relevant information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The workflow context is strong, but with no output schema and no annotations, the description should also explain what the tool returns, prerequisites in more detail, and the meaning of the remaining parameters. It is adequate for a human who already knows the domain, but leaves notable gaps for an autonomous agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20%, so the description must compensate, but it only meaningfully explains handoff_intent_id ('opaque', 'never credential material'). The meanings of display_name, source_id, and auth_scheme are left to inference from names and enum values, and source_scope's schema description is the only detailed parameter guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Bind a source connection for your provider key/account from a COMPLETED handoff intent.' It also names the exact endpoint and makes the tool's role in the handoff-to-connection workflow clear, distinguishing it from list, start, poll, test, and sync siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when this tool is appropriate: from a COMPLETED handoff intent, not as a credential submission step. It also names the following steps ('disabled until solver_test_source_connection passes; deployments then come from solver_sync_source_deployments'), giving clear routing information across sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_list_model_deploymentsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It correctly implies a read-only GET operation, but it does not mention pagination, output format, or authorization requirements. The 'certification-backed' qualifier adds useful domain context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences front-load the core action and endpoint, then provide a cross-tool next step. Every word contributes, with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple parameterless list operation, the description supplies the resource, endpoint, and a downstream action. The absence of an output schema is not critical given the straightforward nature of a list call, though a brief note on returned fields would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the empty input schema is fully self-explanatory. The description adds no parameter details, but none are needed; the baseline of 4 for zero-parameter tools applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with a well-defined resource: the tenant's certification-backed model deployments, and names the API endpoint. It is clearly distinct from sibling tools like solver_list_arms and solver_list_sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence provides actionable context: after listing deployments, the agent can enable one as an arm via solver_enable_model_arm. This makes the tool's role in a workflow clear, although it does not explicitly enumerate alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_receiptA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the transparency burden. It adds a key behavioral guarantee — content-free, never returns task output — and clarifies the return payload categories. It doesn't cover failure or error behavior, but for a simple GET-style fetch this is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one compact sentence with the key facts front-loaded: action, resource, content list, endpoint, and critical caveat. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter fetch with no output schema or annotations, the definition covers purpose, return contents, and a critical limitation. It could be stronger by pointing to solver_result for task output, but it is not missing essential invocation information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only says execution_id is a required string with no description. The description adds semantic role by showing it in the path /v1/receipts/{execution_id} and referring to 'an execution', but it does not explain where the execution_id comes from or expected format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific resource ('receipt for an execution') and a clear verb ('Fetch'), then enumerates what the receipt contains (route rationale, per-slice evals, cost). The 'never returns task output' line implicitly distinguishes it from sibling result/status tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a clear when-to-use context: when you need execution metadata like route rationale, evals, or cost. It also supplies a when-not ('never returns task output'), but it does not explicitly name an alternative such as solver_result for task outputs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_test_source_connectionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It meaningfully discloses that a passing test activates the connection and that the wire reports test_state/test_error, giving the agent useful context beyond a simple 'test' label. It does not cover auth, rate limits, or error handling, so it stops short of 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences with no filler. It front-loads the action, adds the endpoint, explains the activation side effect, and tells the agent what the wire returns. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no annotations and no output schema, the description covers the operation, endpoint, side effect, and result signal. Minor gaps remain around failure behavior and how to find connection_id, but the tool is adequately specified for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates by showing the endpoint path /v1/source-connections/{id}/test, which clearly ties connection_id to the connection being tested. The parameter meaning is effectively conveyed even though format or where to obtain the ID is not spelled out.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Run the live credential test') on a specific resource ('source connection'), includes the exact endpoint, and is clearly distinct from sibling tools like list/create/sync. An agent can understand what this tool does without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when you need to validate credentials and possibly activate a connection), but it does not explicitly say when to prefer this over alternatives or mention related tools such as solver_create_source_connection or solver_start_source_handoff.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_start_source_handoffA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It explicitly states the tool never accepts or returns raw secrets, requires a human browser step, and returns a continue_url. It could add more detail about side effects or failure modes, but the core behavioral traits are well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences, each adding essential information: what the tool does, its secret-safety boundary, and the follow-up workflow. No filler words or redundant repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description is remarkably complete: it names the return value (continue_url), the required polling step, the terminal state (completed), and the next tool to call. An agent has enough context to invoke this correctly and proceed through the workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it does not explain the two parameters. 'For a source' hints at source_id, and auth_scheme's enum values are self-describing, but the description does not clarify how source_id is obtained or how auth_scheme affects the handoff. This is a notable gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start a hosted browser credential handoff'), the resource ('for a source'), and the exact endpoint (POST /v1/source-credential-handoffs). It is unambiguous and distinguishes this tool from sibling tools like solver_poll_source_handoff and solver_create_source_connection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit workflow guidance: poll with solver_poll_source_handoff until state=completed, then pass handoff_intent_id to solver_create_source_connection. It also clarifies the human-in-the-loop browser requirement, so an agent knows this is not a direct API call that completes autonomously.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solver_sync_source_deploymentsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well: it discloses persistence, the active-connection requirement, and the caveat that unevidenced identities are never persisted. It does not fully specify whether the sync removes previously persisted deployments or what happens on failure, so it is not a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences deliver purpose, endpoint, prerequisite, and a key caveat without redundancy. Front-loaded with the core action, then scoping constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema or annotations, the description covers purpose, side effect, prerequisite, and non-certification boundary. It is missing only explicit return/error detail, which is minor given 'answers the synced deployment set'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates by embedding connection_id in the endpoint and requiring it to be an ACTIVE tested connection. It adds meaningful semantic context beyond 'string'. It stops short of explaining how to obtain the ID or accepted formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a precise action ('Discover and persist') and resource ('connection-authorized, certification-backed deployments'), plus the exact endpoint. It also disambiguates from certification by noting discovery is not certification, making the tool's role clear among siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly states the prerequisite (ACTIVE/tested connection), which signals when this tool is appropriate relative to test/list flows. It also implies it should follow connection testing, but it doesn't explicitly name alternative tools or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.