token-counter
Use when a user asks how many tokens a given text will consume, or needs to estimate prompt size before pricing a workload. Given text and tokenizer family, returns low/high token range and byte-level measurements.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text content to count tokens for | |
| tokenizer | No | Tokenizer family (default: default) | |
| expected_out_tokens | No | Expected output token budget (optional) |