query_backend

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It usefully discloses that backends abstract away session, prover selection and theory bookkeeping, and enumerates the possible verdict classes including Timeout and ResourceOut, which hints at failure behaviour. It says nothing about remote execution cost (starexec), latency, or what happens when a backend is unavailable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and result type, and every sentence is relevant. The trailing 'Call describe_szs' pointer is a small but legitimate addition rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A four-parameter, unannotated tool with no output schema: the description covers outcome classes and points to describe_szs for the ontology, which is helpful. It stops short of describing return shape for the raw flag, error/exception behaviour, or any backend prerequisites, which are needed for confident invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so backend, problem, raw and time_limit_sec are already documented, which sets the baseline at 3. The description adds only a general note that backends differ in ceremony and does not explain backend-specific behaviour differences beyond what the enum conveys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: run a TPTP problem through any AtpClient backend and return the normalised SZS verdict, and names the entry point query/2. It does not differentiate itself from run_prover or compare_provers, which occupy similar territory, so an agent still has to guess which one to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this versus run_prover, compare_provers or prove_isabelle, despite strong overlap. The only routing hint ('Call describe_szs for the full ontology') points at a helper for verdict semantics, not at an alternative execution path.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.