Server Quality Checklist

Tool Scores

• rag_searchC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the tool performs semantic search, but it does not disclose whether the operation is read-only, how results are returned, whether indexing is required, or any side effects or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short and front-loaded, but it is under-specified rather than concise. It contains only a phrase with no structure or elaboration, failing to provide necessary context in an efficient way.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 0% schema description coverage, no annotations, and no output schema, the description is incomplete. An agent cannot determine how to construct a valid query, what topK or namespace mean, or what the response looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining the parameters, but it mentions none. The tool has three parameters (query, topK, namespace), and the description adds no meaning beyond their raw names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'search' and the resource 'indexed documents', and specifies the retrieval type as 'semantic'. It conveys the tool's core purpose effectively, though it does not explicitly differentiate from sibling tools like files_list or inspect_storage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention conditions, prerequisites, or exclusions. Sibling tools include files_list and kvp_get, but no routing information is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kvp_deleteC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'Delete,' without addressing whether deletion is permanent, what happens when the key does not exist, or whether the namespace/key must already be present. This is minimal transparency for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler or repeated detail. It is efficient, though extremely terse; brevity is appropriate here, but the lack of supporting context prevents a higher score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and no parameter descriptions, the description is the only source of guidance. It fails to mention return behavior, error semantics, delete scope, or namespace roles, leaving important context unspecified for a delete operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description adds no meaning for the 'namespace' or 'key' parameters. An agent is left to infer that namespace scopes the store and key identifies the entry, but the description does not communicate this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('a KVP key'), and the deletion verb separates it from sibling tools like kvp_get, kvp_list, and kvp_put. It is slightly redundant with the tool name, but still gives an agent enough to understand the fundamental operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives, no mention of prerequisites, and no exclusions. The verb 'Delete' implies its use case, but the description does not explicitly contrast it with kvp_put, kvp_get, kvp_list, or keys_create.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kvp_listC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It only states that keys are listed and does not mention pagination via cursor, the effect of limit, ordering, namespace requirements, or failure behavior. These are meaningful gaps for a list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler and is appropriately front-loaded. It is efficient, but the brevity contributes to missing useful details such as limit and cursor behavior, so it is not maximally helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 schema parameters with 0% coverage, no annotations, and no output schema, the description is not sufficient for an agent to use the tool correctly. It omits pagination semantics, the meaning of limit, and any namespace-related edge cases or expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only clarifies that 'namespace' refers to a KVP namespace. The purpose of 'limit' and 'cursor' parameters is left entirely unspecified, so the description only partially compensates for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('keys in a KVP namespace'), which makes the core action clear and naturally distinguishes it from kvp_get, kvp_put, and kvp_delete. However, it does not explicitly differentiate itself from sibling tools or define the 'KVP' acronym, so it falls just short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives like kvp_get or kvp_put, nor are there any exclusions or contextual hints. An agent must infer usage solely from the tool name and the brief description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• files_listC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states the list operation is read-only in tone, but it does not explain return format, pagination, ordering, default behavior when namespace is omitted, or any error/edge cases. This leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every word adds information: the operation, the resource, and the filtering capability. This is appropriately concise for a one-parameter listing tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and a 0% schema description coverage, the description is too sparse. It fails to mention what the response looks like, whether the result is a list of metadata or contents, or the effect of omitting namespace. While the tool is simple, the available context is insufficient for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does clarify that namespace is an optional filter, which adds some meaning, but it does not explain valid namespace values, default behavior when omitted, or how filtering behaves. The meaning is barely more than the parameter name provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('List') and a specific resource ('uploaded files'), and notes an optional namespace filter. It does not explicitly contrast with sibling tools like file_upload or files_types, but the resource and action are specific enough to avoid major ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving uploaded files, optionally filtered by namespace, but it gives no explicit guidance about when to use it instead of related tools such as kvp_list, inspect_storage, or files_types. No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_uploadC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It adds an odd, likely typo'd hint, 'PrefLights rag_index capacity,' which might mean the tool preflights RAG index capacity, but this is unclear. It does not disclose persistence behavior, indexing timing, overwrite rules, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and the primary purpose is front-loaded. However, the second sentence is cryptic and appears garbled ('PrefLights rag_index capacity'), reducing clarity instead of earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a four-parameter tool with no annotations and no output schema, the description is incomplete. It fails to clarify the filename/contentType parameters, the meaning of the capacity preflight, or what happens after an upload, leaving an agent with significant uncertainty.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning around 'namespace' and 'UTF-8 text document,' but leaves 'filename' and 'contentType' unexplained, and provides no param-level detail beyond the schema's bare names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific action ('Upload a UTF-8 text document for RAG indexing into a namespace'), which clearly conveys the tool's function and resource. It does not explicitly distinguish itself from sibling tools, but its upload-for-RAG purpose is reasonably unique among the listed siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the phrase 'for RAG indexing into a namespace.' However, there is no explicit guidance on when to prefer this over alternatives such as files_list or rag_search, nor any exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kvp_getC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It says 'Read' which implies a non-mutating operation, but it does not disclose behavior on missing keys, return type, error semantics, or potential side effects. The description adds little beyond what the name 'kvp_get' already suggests, so it falls short of providing meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence: 'Read a value from namespaced KVP storage.' Every word contributes, and it avoids redundancy or filler. It is concise and structured appropriately, though it could have included more guidance without becoming bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter read tool, the description is minimally viable: it states the operation and hints at namespacing. However, with no output schema and no annotations, it does not explain the return value format or behavior when a key is absent. It covers the essence of the tool but leaves gaps that an agent might need to resolve through trial or additional knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The word 'namespaced' hints that 'namespace' acts as a scoping partition, which is a small useful signal. However, it says nothing about the 'key' parameter, value formats, or namespace lifecycle, leaving most parameter semantics to be inferred from the schema names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and identifies a tangible resource ('value from namespaced KVP storage'), making the core operation clear. It distinguishes itself from sibling tools like kvp_put, kvp_delete, and kvp_list through the singular 'a value' vs listing, though it never explicitly names alternatives or conditions. This is clear but not maximally explicit about sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as kvp_list for enumeration or kvp_put for writing. It only states the action, leaving the agent to infer usage from the tool name and parameter schema. There are no exclusions, prerequisites, or routing hints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• budget_getB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full behavioral burden. It implies a read-only operation with "Get" but does not clarify whether values are cached, whether the checkout URL is dynamically generated, what "soft thresholds" mean, or what side effects, if any, occur. Key behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single tight sentence that begins with the action and lists the three return categories. It contains no filler, redundancies, or irrelevant details, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, the description lists the primary return items, which covers the basics. However, it does not explain the meaning of "soft thresholds" or the circumstances under which the checkout URL is relevant. Given the sibling tools budget_estimate and billing_machine_pay, some contextual linkage would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty, with zero parameters and 100% schema coverage. There are no parameters to document, so the baseline is 4. The description appropriately focuses on the tool's output rather than non-existent inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pair: "Get remaining quotas, soft thresholds, and checkout URL for topping up." It identifies the budget resource and enumerates the specific outputs. It does not explicitly distinguish itself from siblings like usage_get or budget_estimate, but the listed outputs make its purpose reasonably identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no indication of when to use this tool versus alternatives such as budget_estimate or billing_machine_pay. It does not mention exclusions, prerequisites, or a preferred decision path. There is no guidance beyond the basic action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_storageB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description alone must disclose behavior. The word 'Summarize' implies a read-only aggregate operation, but it doesn't explicitly state whether this causes side effects, requires permissions, or may be expensive on large storage. The return format is also undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact phrase with no filler words. It front-loads the action and lists the scope efficiently. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, the description gives a reasonable idea of what is summarized, but there is no output schema and no behavioral context such as result shape, pagination, or cost implications. The description is minimally viable but leaves the agent unsure what response to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty with zero parameters, so there is nothing for the description to explain about parameters. The description's listed items are output categories rather than parameters, which is appropriate for a no-argument tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb, 'Summarize', and a resource scope: 'namespaces, key counts, file/vector inventory'. This clearly identifies what the tool does, though it doesn't differentiate it from similarly named siblings like files_list or kvp_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like files_list, kvp_list, or usage_get. An agent is left to infer that 'inspect_storage' is the aggregate/browse operation, but no explicit context or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kvp_putB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It does disclose one useful behavior: the response includes budget remaining, which suggests the operation consumes budget. However, it does not mention overwrite semantics, whether the key is created or updated, default expiration behavior, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the core operation and adds a relevant response detail (budget remaining) without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple key-value write tool, the description covers the primary action and the response's budget field, but it omits guidance on TTL, overwrite behavior, and why a budget-related response matters. Given the lack of annotations and output schema, a bit more context would help an agent invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It names namespace, key, and value and indicates value is a string, but it adds no detail about namespace/key constraints, value length limits, or the meaning of expirationTtl. The optional fourth parameter is entirely undocumented in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Store'), a concrete resource (a string value under a namespace/key), and clearly implies a write operation distinct from sibling tools like kvp_get, kvp_delete, and kvp_list. An agent can identify what this tool does without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use this tool versus alternatives such as kvp_get, kvp_delete, or keys_create. The sibling names imply a contrast, but the description itself does not state conditions, exclusions, or preferred alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• budget_estimateA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does convey a key safety trait: this tool estimates before spending and does not actually spend. However, it does not disclose whether the estimate is approximate, what the return value looks like, or any side effects or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler; the core purpose is front-loaded and the special-case parameter note earns its place. This is appropriately concise for the information it conveys.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% parameter coverage, the description is not complete enough for reliable invocation. It lacks per-kind parameter requirements, units for amount, and any indication of the shape or meaning of the estimate result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only explains that for rag_index, textLength/byteLength can be used instead of amount. It never defines amount, byteLength, textLength, or the meanings of kvp_ops, storage_bytes, and rag_query, leaving critical invocation details undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Estimate') and a specific resource ('whether an operation would hit 402/429 before spending'), making the tool's job unambiguous. This clearly distinguishes it from sibling tools like usage_get, budget_get, and billing_machine_pay, which handle actual usage, budgets, or payments rather than preflight estimation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'before spending' conveys the general timing and intent of the tool, and the rag_index note gives one concrete usage condition. However, it does not explicitly say when to prefer this over sibling tools or when not to use it, leaving that inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keys_createA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It reveals a prerequisite and the REST endpoint, but does not state that this is a mutating and potentially billing-relevant action, whether the created key secret is shown only once, or whether there are side effects like key invalidation or expiration behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler; both clauses add useful information about what is minted and what is required. It is appropriately sized for the core action, even though more context is needed elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is incomplete. It does not explain what the response looks like, whether the created key is sensitive or single-return, how the account-key prerequisite relates to the parameters, or what the operational consequences of minting a key are.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to compensate for the seven parameters. It only echoes the kind enum via 'run or sub' and leaves opBudget, namespaces, ttlSeconds, wipeOnExpire, tools, and name unexplained, so an agent cannot confidently configure them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a concrete action ('Mint'), the object ('run or sub API key'), and the source ('account key'), making what the tool does unambiguous even without the name. It also aligns with the required kind parameter and is clearly distinct from the unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly conveys the intended context: create a run or sub API key only when you have an account tp_ key. It does not explicitly list when-not-to-use or alternatives, but no sibling performs a similar key-creation function, so this is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agent_bootstrapA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the burden of behavioral disclosure. It usefully states 'no outbound mail' and implies the response contains a bootstrapToken. However, it does not disclose token lifetime, idempotency, authorization requirements, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The core action is front-loaded, and the second sentence adds valuable follow-up guidance without bloating the description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool, the description provides a usable workflow: create token from email, then use it with billing_machine_pay. But it omits label semantics, response format details, and any failure/edge-case context, so it is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It makes it clear that email is the input from which the token is created, but it says nothing about the optional label parameter, leaving its purpose entirely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'create bootstrap token from email'. It also distinguishes this tool from siblings by noting it has 'no outbound mail' and by naming the next step, billing_machine_pay, making the tool's role in the onboarding flow unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit usage context: 'Start zero-human onboarding'. It also tells the agent what to do next with the result, which is practical workflow guidance. It does not explicitly mention alternatives or when not to use this tool, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• billing_machine_payA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses that the tool returns a POST URL and amount and that the agent must call HTTP with the MPP Payment credential—key information about side effects and what the agent must do next. It stops short of discussing idempotency or failure modes, but covers the central behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences cover purpose, the parameter, and the required follow-up action with no filler. The most important operational detail—that the agent must call the returned URL with a credential—is stated directly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description is close to complete: it mentions the default SKU, the return shape, and the next action. It does not specify where the MPP Payment credential comes from, but this is a minor gap given the simple contract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds the default value (pro_prepaid_30d) and confirms sku is optional. The enum values themselves are self-descriptive enough that listing them again would add little; the main semantic gap is not explaining when each SKU is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: unlocking Pro or purchasing agent SKUs via Stripe MPP, and explains the concrete deliverable (POST URL and amount). It is not a tautology and is distinguishable from siblings like usage_get and budget_get, though it does not explicitly name an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The opening sentence makes the intended trigger explicit: use this when the agent needs to unlock Pro or buy agent SKUs. It does not provide exclusion criteria or name alternatives, but there are no close sibling payment tools, so the context is reasonably clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• support_contactA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It explicitly states that the subject and message are delivered to the Tillpad team and that replies go to the account email, which covers the main side effects of the operation. It does not mention failure modes or whether delivery is asynchronous, but the disclosed behavior is meaningful and appropriate for the tool's simplicity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences with no redundant content. It front-loads the core purpose, then adds delivery and eligibility details. Every sentence contributes useful information without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity of the tool, two string parameters, and no output schema, the description provides the essential context: what the tool does, where messages go, how replies are received, and the subscription requirement. A minor gap is the lack of any statement about immediate confirmation or behavior when the active-Pro condition is not met, but the description is otherwise adequate for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for both parameters. The description mentions 'subject and message' and connects them to the delivery behavior, but it adds little beyond the property names themselves and provides no constraints, length limits, or formatting guidance. This is minimally adequate for two self-explanatory string parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb and resource: 'Contact Tillpad support' from a Pro account. It further specifies the outcome (delivering subject/message to the Tillpad team) and the reply destination, making the tool's purpose unambiguous and distinct from the unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: it is the tool for contacting Tillpad support and requires an active Pro subscription. It does not explicitly name alternatives or exclusions, but no sibling tool appears to serve a similar support-contact purpose, so the usage guidance is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• usage_getA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. The 'Get' verb implies a read-only operation and 'authenticated account' signals an auth requirement, but there is no mention of period boundaries, quota reset timing, or response shape. For a zero-parameter read-only tool this is moderately adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundancy. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no annotations, and no output schema, the description is mostly sufficient for an agent to invoke the tool correctly. It clearly identifies the resource and scope, though it leaves return-value details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema has nothing to document and the description correctly adds no parameter details. The baseline 4 for no-parameter tools applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') with a specific resource ('current period usage and quotas') and a clear scope ('for the authenticated account'). This is more than a tautology and distinguishes it from siblings like budget_get and files_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by specifying 'current period' and 'authenticated account', which helps an agent decide this is the right tool for account-level usage/quota queries. It does not explicitly name alternatives or exclusions, but the resource is unambiguous enough to route selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• files_typesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of behavioral disclosure. It makes clear the tool is a read-only listing operation and specifies the categories of information returned. For a simple, non-mutating tool this is sufficiently transparent, though it could have explicitly stated it has no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is concise, front-loaded with the main verb and resource, and includes the key detail about what the list covers. There is no filler or repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless informational tool with no output schema, the description is complete: it states the action, the subject, and the expected content categories. An agent has everything it needs to decide to call this tool and interpret the response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics to clarify. The baseline of 4 applies, and the description enriches understanding by explaining what the returned list will contain. No additional parameter guidance is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a specific resource ('supported RAG upload file types'), and further clarifies the content as extensions, MIME types, and extract notes. This clearly distinguishes it from sibling tools like file_upload and files_list, which operate on actual files rather than type metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied rather than stated: an agent would naturally call this before doing a RAG file upload to check supported types. However, there is no explicit guidance about when to use this tool versus alternatives, nor any statement of when not to use it. The description is adequate but leaves the selection logic to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_finishA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears the full burden. It clearly discloses the destructive nature ('Wipe'), the scope (run-key namespaces), and the output (signed receipt). It does not explicitly warn of irreversibility or auth requirements, but for a zero-parameter tool this is substantive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the action and resource, then adds a necessary validity caveat. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description is complete: it says what will be destroyed, the scope, and what is returned. An agent has everything needed to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no properties, so the zero-parameter baseline of 4 applies. The phrase 'this run key' provides implicit execution context without requiring parameters, which is consistent with a parameterless invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific destructive action ('Wipe all namespaces bound to this run key') and the expected return ('signed wipe receipt'). It clearly distinguishes from sibling tools like kvp_delete, which targets a stored value rather than run-key namespaces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly constrains usage to run keys ('Only valid for run keys'), signaling it is not for other key types. It does not name alternatives, but no sibling tool performs a comparable namespace wipe, so the context is sufficient for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.