count_tokens
Count tokens in UTF-8 text using Algenta tokenizer models for prompt-size checks and token budgeting. Deterministic and read-only; returns token count and resolved model details.
Instructions
Count how many tokens a supported deterministic Algenta tokenizer model produces for UTF-8 text (default text.tokenizer; call list_models for every supported model id). Use this for prompt-size checks and token budgeting; call tokenize when you also need the token strings. Read-only and deterministic: the same input and model always return the same count, and nothing is stored. Returns the resolved model id, its tokenizer_kind, and token_count. An unsupported model id fails with model_not_supported.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| input | Yes | UTF-8 text whose tokens are counted. | |
| model | No | Tokenizer model id from list_models. | text.tokenizer |