Server Quality Checklist

Tool Scores

• flux_get_app_statsB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does add two genuine behavioral facts beyond the name: the operation is 'live' (real-time sampling) and it requires the owner key. However, it says nothing about error behavior for stopped/missing apps, per-node scoping limits, or rate limits, so meaningful gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the resource being fetched, and the prerequisite placed second. Nothing is wasted, though the extreme terseness leaves gaps that the other dimensions penalize.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description must explain what the caller gets back and under what conditions it works. It never describes the stats payload shape, the per-node output structure, or what happens when the app is not running, leaving an agent under-equipped for a three-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% (only 'component' is documented). The description indirectly compensates: 'on one node' points at nodeIp and 'containers of an app' signals the component granularity. It still gives no syntax for the node IP or component naming and never clarifies the required 'name' or default-node behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (get) and resource (CPU/memory/network stats of containers for an app on a node), which clearly distinguishes it from siblings such as flux_get_app_logs, flux_get_app, or flux_get_network_info. It doesn't explicitly name an alternative, but the resource is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only usage guidance is an implied prerequisite ('Requires the owner key') and the title's hint that the instance must be running. There is no statement of when to use this versus flux_get_app, flux_get_network_info, or flux_get_app_logs, and no conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_get_appB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full behavioral burden. It usefully discloses that it works for 'any app on the network' (not just the caller's own) and enumerates the return contents, implying a safe read. However, it does not mention auth requirements, whether unpublished apps are excluded, or error behavior for unknown names, leaving real gaps for a zero-annotation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single well-formed sentence with the lookup action and return payload front-loaded, and no wasted words. It is efficient, though the brevity leaves the guidance gaps noted elsewhere unaddressed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read tool with no output schema, the description does describe the shape of the return (spec, expiry, instances, URLs), which is the main thing an agent needs. But with no annotations and no sibling routing, it stops short of fully orienting the agent for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Exactly one parameter with 0% schema description coverage, so the description must supply meaning. 'By name' identifies the parameter as an app name lookup key, but adds no format, case-sensitivity, or validation detail beyond what the schema's required string already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (looks up) and resource (app on the network by name) and enumerates the returned payload: published specification, expiry, running instances, and URLs. This distinguishes it from siblings like flux_get_app_logs and flux_get_app_stats by content, though it never names those alternatives explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance or alternatives are stated. An agent gets no direction on whether to call this versus flux_list_my_apps for discovery or flux_get_app_stats for metrics, and no preconditions are mentioned. Usage is only implied by the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_build_specB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden, and it does add real behavior: public ports are auto-picked and data replication is enabled by default. It also states the return content (spec plus local validation errors and warnings). It omits whether the tool has any side effects or persistence (build vs. deploy), which is the key behavioral question for this sibling family.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core transformation and followed by the auto-fill/return behavior; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 10-parameter tool with no annotations and no output schema, the description covers the return value well but leaves most parameters and the build-vs-deploy relationship unexplained. It is minimally adequate rather than complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 40% across 10 parameters, so the description should compensate more than it does. It loosely maps to components/images, ports, resources, and 'a term' (months/expireBlocks), but says nothing about name, instances, nodes, contacts, staticip, geolocation, or per-component env/commands/dataPath.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific transformation ('turns images, ports, resources and a term into a complete, correctly formatted v8 specification'), so the verb and output artifact are unambiguous. It does not, however, distinguish itself from siblings like flux_validate_spec or flux_quote_app, which an agent must choose between in the same workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit when-to-use guidance, no mention of prerequisites, and no routing to or away from flux_validate_spec, flux_quote_app, or flux_deploy_app. The only signal is the implied 'build before you deploy' ordering, which the agent must infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_get_network_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are supplied, so the description carries the full behavioral burden, yet it discloses nothing about permissions, whether the call is read-only (implied only by the 'get' name), rate limits, or whether values are live or cached. Listing the returned fields is useful but is content description rather than behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence that enumerates the return contents with no filler or preamble. Every phrase earns its place, and the key information leads.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description compensates by naming the four data points returned, which is enough for an agent to decide whether this tool answers its question. It falls short only on freshness/volatility of the values and any access requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, which sets the baseline at 4. The description appropriately spends no words on argument syntax, leaving the empty schema as the complete contract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The name and description together establish a specific read operation ('get') over a specific resource (network info), and the enumerated contents (node counts by tier, block height, FLUX/USD rate, payment address) make clear what the agent gets back. It is distinguishable from the app-deployment siblings like flux_deploy_app and flux_get_app, though it doesn't explicitly say it is not those.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no when-to-use guidance, no prerequisites, and no pointer to an alternative or complementary tool such as flux_get_pricing. The agent must infer that this is the tool to call for chain-level statistics simply from the content list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_list_my_appsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It does add useful context by naming what each entry includes (expiry and instance counts), but it says nothing about authentication requirements, pagination, rate limits, or what 'the configured Flux ID' resolves to.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with zero filler: verb, resource, scope, and return contents in one pass. Nothing redundant or padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list tool with two optional params, full schema coverage, and no output schema, the description is nearly sufficient — it even names the returned fields. Only the absence of any auth/pagination note keeps it from being fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so both parameters are already documented in the schema. The description's parenthetical ('or a given Flux ID') loosely maps to the owner parameter but adds no format or semantics beyond it, and nameContains goes unmentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Lists every app registered by the owner') with a clear scope ('every app') that distinguishes it from the single-app sibling flux_get_app. It does not name any sibling explicitly, but the plural/scoped framing makes the distinction inferable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance or explicit alternatives are given; the agent must infer that this is the bulk-listing counterpart to flux_get_app. The parenthetical about the Flux ID default is a parameter note, not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_validate_specA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does disclose useful behavior: a local check followed by a live FluxOS node verification with the specific dimensions checked. It stops short of stating side effects (does it persist or normalize anything?), failure/error reporting, or whether it is purely read-only, which matters for a tool that talks to a network node.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with the core action front-loaded and the verification details following. Every clause carries information; no filler or restated boilerplate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, and the description compensates by noting the success return (node-formatted spec). For a two-parameter tool with a richly documented nested schema, the definition is nearly complete, missing only failure-mode behavior and explicit sibling routing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%: the `network` flag and most nested spec fields are documented in the schema, but the description adds meaning by explaining what "verify on a node" actually entails. It does not clarify the interaction between `network` and the local-only path, so it only partially compensates for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ("Runs the local rule checks... verify the specification") and enumerates the actual checks performed (image reachable, architecture, ports, name availability). This clearly separates it from flux_build_spec and flux_deploy_app, though it never names those siblings explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase "exactly as it would at registration" implies this is a pre-deployment gate, so the use case is inferable. However, it never states when to choose this over flux_build_spec or before flux_deploy_app, nor when to skip it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_quote_appA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and largely succeeds: it discloses the conditional update path, the credit for the unused current term, and the USD/FLUX market-rate pricing. It still does not state permissions, side effects, or whether the call is strictly read-only, but the quote semantics and update-credit behavior are unusually well surfaced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two front-loaded sentences with no waste. The pricing behavior and conditional update logic are presented compactly and in the right order.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a quote tool with a complex nested spec and no output schema, the description gives enough return-value context by stating that the result is in USD with a FLUX amount at market rate, and it covers the update-credit behavior. It is less complete on auth or dry-run guarantees, but the core quoting contract is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is reported as 0% for the single top-level parameter, and the description adds no meaningful detail beyond calling it a 'given specification.' It does not explain required nested fields, defaults, or constraints, so it fails to compensate for the low schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it returns the Flux Cloud price for registering or updating a given specification. It clearly separates this quote operation from deployment or general pricing tools by tying the price to the app spec and the existing-app update case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but gives no explicit guidance on when to use it versus alternatives such as flux_get_pricing, flux_build_spec, or flux_deploy_app. The agent must infer that this is a pre-deployment quoting step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_wait_for_appA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full load and does reasonably well: it discloses the polling model, the 300s default / 600s cap, idempotent repeated invocation, and the shape of the result including a done flag. It omits what happens on timeout expiry (error vs partial result), which is the notable behavioral gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, each carrying distinct information (poll mechanism, timeout bounds, return payload, idempotency). Nothing is redundant and the key constraints are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter polling tool with no annotations and no output schema, the description covers the essentials of invocation and return values, but it leaves timeout-failure behavior, permission/auth needs, and the 'name' parameter's relationship to the deployed app unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%, and the description only reinforces timeoutSeconds semantics (already in the schema) plus the meaning of txid confirmations. The required 'name' parameter and the update-flow semantics of previousHash remain undocumented in the description beyond the schema's own text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the polling behavior and enumerates what it returns (payment confirmations, spec acceptance, running instances with URLs), so an agent can tell it is a wait/poll tool. It does not, however, contrast itself with siblings like flux_get_app or flux_deploy_app, which is the main remaining ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Safe to call repeatedly until done=true' implies a poll-until-complete usage pattern, and the default/max timeout hints at appropriate call scope. But it never states when to choose this over flux_get_app or that it should follow flux_deploy_app, so usage is implied rather than directed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_get_identityA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden and does reasonably well: it discloses the return contents, the missing-key edge case behavior ('explains what to configure'), and a security guarantee ('never returns private keys'). It does not cover permissions/authentication requirements, but for a zero-param local read tool this is solid disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, front-loaded with what is returned, followed by the missing-keys note and the private-key guarantee. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must describe the return values, and it names the exact fields (Flux ID, payment address, spendable balance, USD value). It also covers the missing-key scenario. It is essentially complete for a zero-param query tool, with only auth/permission context unaddressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters (zero-param schema), so per the rubric the baseline is 4. The description appropriately spends its words on return semantics instead of nonexistent inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: it returns the Flux ID, payment address, spendable FLUX balance, and USD value. This clearly distinguishes it from app-lifecycle siblings (flux_deploy_app, flux_get_app) and from flux_generate_keys. It stops short of explicitly naming a sibling to differentiate against, so it lands at a strong 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than stated: an agent can infer this is the identity/balance check, and the 'explains what to configure if keys are missing' clause supplies a conditional scenario. There is no explicit when-to-use versus when-to-use-something-else guidance or named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_get_app_logsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It usefully discloses the auth requirement (owner key) and that logs are pulled from one of possibly several nodes, but says nothing about whether the call is read-only, side effects, rate limits, or truncation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences, front-loaded with the core action and scope, with the prerequisite appended. Every clause carries information; nothing is padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-style fetch tool with no annotations and no output schema, the description covers the key context an agent needs: what is returned (log lines), the component/node scoping model, and the auth prerequisite. Remaining gaps (default node behavior, line cap) are handled by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50%, so the description must compensate. 'Last N log lines' maps to the undocumented `lines` parameter, 'one component of it' maps to `component`, and 'one of the nodes running it' maps to the schema-documented `nodeIp`; only `name` is left implicit. This adds real meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (fetches) and resource (last N log lines) plus scope (an app, or one component, from one of its nodes). This is clearly distinguishable from siblings like flux_get_app_stats or flux_get_app, which return different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states a prerequisite ('Requires the owner key') but gives no explicit when-to-use/when-not guidance or named alternatives. Usage is implied rather than stated, though the log-fetching purpose is unambiguous enough that no sibling competes for it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_deploy_appA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so well: it discloses the signing identity (Flux ID), the payment source (payment address), the FLUX spend, and precisely what the dry-run returns without spending. It stops short of covering failure/error behavior, idempotency, or gas/fee nuances, but the critical destructive-and-payment profile is fully and honestly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four tight sentences, front-loaded with the core action and the confirm gate, with the plan-vs-execute distinction and follow-up call each earning their place. Slightly dense but no wasted filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity, nested-object tool with no output schema and no annotations, the description covers the key behaviors an agent needs: the confirm gate, what the plan returns, the payment implication, and the next call. It could say more about the returned txid/plan fields or failure modes, but it is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%, so the schema already documents confirm and most nested spec fields (expire, repotag, containerData, etc.). The description reinforces confirm's semantics and adds the 'plan = price, balance, warnings' return shape, but adds no meaning for the many nested spec/enterprise fields beyond what the schema states. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the specific verb chain (validate, quote, sign, broadcast, pay) and resource (app specification), making it unmistakably distinct from siblings like flux_validate_spec, flux_quote_app, and flux_build_spec. It also states the register-new vs update-existing scope. An agent can identify exactly what this tool does without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly explains the confirm=true (execute and pay) vs confirm=false (dry-run plan, spends nothing) dichotomy, and prescribes the follow-up call to flux_wait_for_app with the returned txid. It does not explicitly contrast against flux_validate_spec or flux_quote_app as separate preflight steps, so the routing to alternatives is implicit rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_cancel_appA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden and does well: it discloses the destructive end state (expires in ~1 hour, then uninstalled everywhere), the cost implication (signs and pays the usually minimal update), and the dry-run behavior of confirm=false. It stops short of stating irreversibility, refundability, or required auth/scopes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, front-loaded with what happens, then the network consequence, then the confirm semantics. No filler and each clause adds decision-relevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, and the description does explain the divergent return behavior for confirm true vs false, plus the side effects. It lacks details on timing guarantees, irreversibility, and permissions, which matter for a destructive billing operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate, and it does for confirm: true means sign and pay, false returns the plan instead of mutating. The 'name' parameter is left to obvious inference, so the semantic gap is narrow but not fully closed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Ends an app early') and goes further to explain the mechanism (term update so it expires within about an hour, then network-wide uninstall). The title's 'stop it and stop paying' plus the term mechanics distinguish it from a generic control/stop sibling like flux_control_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through the confirm flag (commit-and-pay vs plan preview), which tells the agent how to use it safely, but there is no explicit statement of when to choose this over flux_control_app or other lifecycle tools, nor any stated prerequisites. Adequate but leaves sibling selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_generate_keysA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses the response contains secrets, that nothing is stored or sent anywhere by the tool, and the required follow-up (store in env vars, restart server). It omits whether regeneration overwrites existing keys or is irreversible.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences, front-loaded with what is produced, followed by where the secrets go and the side-effect guarantee. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters and no output schema, the description adequately covers what is returned (two named secrets) and what the caller must do with them. The only meaningful gap is a warning about irreversibility or repeated invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there is nothing for the description to clarify and the baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: creates two fresh WIF private keys, one for the Flux ID and one for the payment address. This distinguishes it cleanly from sibling read tools like flux_get_identity, which retrieves rather than generates credentials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than stated: the instruction to store keys in env vars and restart the server signals this is one-time setup. It never says when to call it versus when to use flux_get_identity to check for existing keys, nor what happens on a second call.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_get_pricingA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses that the returned rate is 'live' and distinguishes reference estimates from exact quotes, which is genuinely useful, but it says nothing about caching/refresh cadence, rate limits, or auth needs, and 'a few reference sizes' is vague about contents.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero filler. The payload of what is returned is front-loaded, and the sibling routing is placed last where it naturally reads as a caveat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does the necessary work of enumerating the return contents, and it covers the read-only nature implicitly through 'Returns' and 'live.' It falls slightly short on the shape of the estimates (which reference sizes, refresh timing), but is otherwise complete for a no-arg read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there is nothing to disambiguate and the baseline is 4. The description correctly implies no inputs are needed to obtain the rate card.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Returns') and enumerates the exact resources returned: the USD rate card, live FLUX/USD rate, pay-in-FLUX discount, and reference-size USD estimates. It also distinguishes itself from flux_quote_app, so an agent can tell the two apart without opening a schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly routes the agent to flux_quote_app 'for the exact price of a specific app,' which implicitly defines when to use this tool (general rate card / rough reference estimates) versus the alternative. It stops short of an explicit when-not or a stated prerequisite, but the routing is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• flux_control_appA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations the description carries the full load and does well: it discloses that hard=true destroys data, that remove does not truly delete (the network re-spawns it), and that the owner key is required for authorization. It omits any statement about disruption/gracefulness of restart, reversibility, or the shape of the response, which keeps it short of a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One dense, front-loaded paragraph keyed by action name, then scope, then auth. Every clause carries a distinct fact and nothing is repeated from the schema or title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an unannotated mutation tool with no output schema and 5 params, the description covers action semantics, scope, data-destruction, authorization and the correct sibling for stopping the app. It could still say what is returned or whether the operation is awaited, but nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 40%, but the description compensates by explaining what each action value means, the data-loss consequence of hard, and that scope is 'one node (nodeIp) or every node running the app (global)'. The remaining params (name) are self-evident; the nodeIp description in the schema is actually slightly richer than the prose, so no gap remains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb set (restart/redeploy/remove) against a specific resource (app containers/nodes) and even disambiguates the confusing case by naming the sibling flux_cancel_app. An agent can distinguish it from flux_cancel_app, flux_deploy_app and flux_get_app without opening any schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives per-action conditions ('redeploy: pulls the image again... hard=true also wipes data') and an explicit alternative for the wrong choice: 'remove... the registration stays and the network re-spawns it elsewhere, so use flux_cancel_app to stop paying.' That is an explicit when-not plus named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.