TokenTally by Harlo

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), so the description's job is to add behavior — and it does: boundary-aware splitting on paragraph/sentence boundaries, returned character offsets and token counts, and optional overlap. It omits default values and whether boundaries fall back to hard token cuts when no boundary exists, which is the main behavioral unknown.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence that front-loads the verb and resource and then packs in boundary policy and return fields with zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly spells out what comes back (offsets, token counts), which is the most important omission it could have made. But for a 5-parameter tool at 20% schema coverage, it says nothing about model/encoding selection, chunk_tokens defaults, or how overlap interacts with chunk size — an agent would have to guess at several call-shaping details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20%, so the description must carry more weight; it does explain overlap_tokens ('optional overlap') and chunk_tokens implicitly ('token-bounded'), and the text constraint (≤256 KB, no URL fetching) is in the schema. But model, encoding (an enum with two choices), and the defaults/relationship between chunk_tokens and overlap_tokens are undocumented anywhere, leaving real gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Split text into token-bounded chunks') plus the boundary policy and outputs, which clearly separates it from count_tokens and truncate_to_budget. It stops short of naming those siblings explicitly, so the differentiation is inferable rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the purpose and title ('Chunk text for RAG') — an agent can infer this is the pre-indexing step rather than the budget-check or truncation step. However, there is no explicit when-to-use, no condition for choosing chunking over truncate_to_budget, and no prerequisites (e.g., call count_tokens first to size chunk_tokens).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint false, and openWorldHint false, but the description adds substantial behavioral context beyond them: statelessness, no URL fetching, 256 KB input cap, possible 503 on pathological unbroken runs near the cap, exact vs approximate tokenizer caveats with variance estimates, and a dated cost-estimate caveat. These details materially help an agent invoke the tool correctly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and then adds precise caveats about exactness, approximations, cost estimates, statelessness, size limits, and timeout behavior. Every sentence provides useful operational information without repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a stateless, read-only token-counting tool with one fully described parameter and no output schema, the description covers what an agent needs: what is counted, which tokenizers are exact vs approximate, cost-estimate caveats, storage behavior, input limit, and a known failure mode. No critical context appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter, text, and schema description coverage is 100%, so the schema already carries parameter semantics. The description reinforces the 256 KB cap and says URLs are not fetched, but that information is also present in the schema, so it does not add significant meaning beyond structured data.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: count tokens for provided text across named tokenizers (o200k_base and cl100k_base) with additional approximate model counts. It clearly distinguishes exact OpenAI counts from approximate Claude/Gemini/Llama counts, so an agent can tell what the tool is for without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the capability description: use this for token counting across multiple tokenizer families. However, it does not explicitly say when to use this tool versus siblings such as estimate_cost, chunk_text, or truncate_to_budget, nor does it state exclusions or alternative-selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so safety is covered. The description adds real value by disclosing the dated price table (prices_as_of 2026-10-27) and that figures are estimates, not live billing. It does not describe the response shape or how per-model results are keyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two terse sentences, front-loaded with the core action and inputs, with the pricing caveat immediately after. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All parameters are optional (required=0) and there is no output schema, yet the description never explains default behavior when neither text nor tokens are supplied, nor the shape of the returned cost breakdown. The pricing-currency and freshness facts are good, but the call/return contract is under-specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 25%, so the description has to carry weight; it does clarify the two input modes ('text or input_tokens plus output_tokens'), which the schema does not explain for those bare integer params. It says nothing about the 'models' array — whether it is required, what happens if omitted, or how it controls per-model output.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Estimate USD cost per model') and names both input modes (text vs. token counts), so the purpose is unambiguous. It does not explicitly contrast itself with siblings like count_tokens, but the cost-orientation makes it distinguishable without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Estimates, not live billing' caveat gives useful usage context for when not to rely on the result for invoicing. However, there is no guidance on when to pick this over count_tokens, or on which input mode (text vs. tokens) to prefer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and closed-world, so safety needs no restatement. The description adds nothing about whether a marker is inserted, what the default keep direction is, or how encoding/model precedence is resolved, which matters for an operation that silently discards content.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with the action and constraints front-loaded and no filler. The compression is slightly at the cost of clarity around the encoding/model pairing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given four safety annotations and no output schema, the definition covers the essentials for a simple pure-transform tool. Remaining gaps (return value shape, default keep direction, tokenizer precedence) are modest but real for a 5-parameter tool at 20% schema coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 20% (only 'text' documented), yet the description does cover max_tokens, the start/end choice, and the encoding-or-model tokenizer notion, adding real meaning beyond the schema for three of five parameters. It still leaves 'model' vs 'encoding' precedence and the default for 'keep' unstated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (truncate) and resource (text) plus the governing constraint (max_tokens) and the keep-direction option, so an agent can distinguish it from count_tokens or chunk_text. It stops short of explicitly stating how it differs from the chunking sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for an encoding or model' implies the situation (fitting text into a token budget under a specific tokenizer) but there is no explicit when-to-use, when-not, or named alternative such as chunk_text for cases where content must be preserved rather than cut.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.