count_tokens
Prevent token limit surprises by estimating token consumption locally before sending text to an LLM. Returns token, character, byte counts and cost estimate using byte-pair encoding compatible with modern models.
Instructions
Estimate the number of tokens a text will consume when sent to an LLM. Uses a byte-pair encoding approximation compatible with cl100k_base (GPT-4, Claude, and most modern models). Accurate to ±10% on English prose. Returns token count, character count, byte count, and a cost estimate footnote. Does NOT send text to any LLM — estimation is purely local. Has no side effects. Free. Use before sending long context to an LLM to avoid surprises. Do NOT use to count characters or words — use text_stats instead.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Text to count tokens for. | |
| model | No | Model name hint (default "cl100k_base"). Accepted: cl100k_base, gpt-4, gpt-4o, gpt-3.5-turbo, claude, claude-3, claude-sonnet, claude-haiku, claude-opus, text-embedding-ada-002. All currently use the same cl100k_base approximation. |