README.md•6.41 kB
# Cost Tracking
Phoenix allows you to track token-based costs for LLM runs automatically. The costs are calculated from token counts and model pricing data, then rolled up to the trace and project level for comprehensive cost analysis.
In most cases it is simplest to let Phoenix handle cost calculation using its built-in model pricing table. When custom pricing is required, you can create custom cost configurations in Settings > Models.
{% embed url="https://storage.googleapis.com/arize-phoenix-assets/assets/videos/observe_cost_tracking.mp4" %}
## Send token counts
For Phoenix to accurately derive costs for LLM spans, you need to provide token counts in your traces:
* If you are using OpenInference auto-instrumentation with OpenAI, Anthropic, or other supported instrumentation, token counts and model information are automatically captured.
* If you are manually instrumenting your code, you should include the appropriate token count attributes in your spans.
* If you are using OpenTelemetry directly, ensure that your LLM spans include the OpenInference semantic conventions for token counts.
### Required span attributes for cost tracking
Phoenix uses the [OpenInference semantic conventions](https://arize-ai.github.io/openinference/spec/semantic_conventions.html) for cost tracking. The following attributes are required:
#### Basic token counts
| Attribute | Type | Description |
| ---------------------------- | ------- | ------------------------------------------------------- |
| `llm.token_count.prompt` | Integer | The number of tokens in the prompt |
| `llm.token_count.completion` | Integer | The number of tokens in the completion |
| `llm.token_count.total` | Integer | Total number of tokens, including prompt and completion |
#### Model information
| Attribute | Type | Description |
| ---------------- | ------ | ---------------------------------------------------------------- |
| `llm.model_name` | String | The name of the language model being utilized |
| `llm.provider` | String | The hosting provider of the llm (e.g., openai, anthropic, azure) |
### Detailed token breakdown (optional)
For more granular cost tracking, you can provide detailed token counts:
#### Prompt token details
| Attribute | Type | Description |
| -------------------------------------------- | ------- | -------------------------------------------------------- |
| `llm.token_count.prompt_details.cache_read` | Integer | The number of tokens read from previously cached prompts |
| `llm.token_count.prompt_details.cache_write` | Integer | The number of tokens written to cache |
| `llm.token_count.prompt_details.audio` | Integer | The number of audio input tokens presented in the prompt |
#### Completion token details
| Attribute | Type | Description |
| ---------------------------------------------- | ------- | ------------------------------------------------------- |
| `llm.token_count.completion_details.reasoning` | Integer | The number of tokens used for model reasoning |
| `llm.token_count.completion_details.audio` | Integer | The number of audio input tokens generated by the model |
## Model pricing configuration
Phoenix includes a comprehensive model pricing table with built-in support for popular models from:
* **OpenAI**: GPT-3.5, GPT-4, GPT-4 Turbo, GPT-4o, and newer models
* **Anthropic**: Claude 1.x, Claude 2.x, Claude 3.x, Claude 3.5 models
* **Google**: Gemini 1.0, Gemini 1.5, Gemini 2.0 models
* **Other providers**: Additional models as they become available
### Managing model prices
You can view and manage model pricing through the Phoenix UI:
1. Navigate to **Settings** → **Models** in the Phoenix interface
2. View existing models and their pricing information
3. Add custom models or override pricing for existing models
4. Set different prices for prompt (input) and completion (output) tokens
### Adding a custom model
To add pricing for a model not in the built-in table:
1. Click **Add new model** in the Models settings page
2. Fill in the model details:
* **Model Name**: Human-readable name for the model
* **Name Pattern**: Regex pattern to match the model name in traces
* **Provider**: Model provider (optional)
* **Prompt (Input) Cost**: Cost per 1M input tokens
* **Completion (Output) Cost**: Cost per 1M output tokens
* **Start Date**: When this pricing becomes effective (optional)
### Advanced pricing configuration
For models with complex pricing structures, you can configure detailed token pricing:
* **Prompt Price Breakdown**: Different rates for cache\_read, cache\_write, audio, image, video tokens
* **Completion Price Breakdown**: Different rates for reasoning, audio, image tokens
* **Provider Matching**: Match models by provider to avoid naming conflicts
## Viewing cost data
Once configured, Phoenix automatically displays cost information throughout the interface:
### Trace-level costs
* Total cost for the entire trace
* Breakdown by prompt vs completion costs
### Span-level costs
* Individual span costs with detailed breakdowns
* Token type-specific cost details
### Session-level costs
* Aggregated costs across all traces within a session
* Session-based cost analysis for multi-turn conversations
* Cost tracking for extended user interactions
### Experiment-level costs
Phoenix automatically tracks costs for traced experiments, providing detailed cost analysis across experiment runs:
* **Total experiment cost**: Sum of all LLM costs across all experiment runs
* **Cost per experiment run**: Individual cost for each dataset example run through an experiment Experiment costs are automatically calculated when you:
1. Run experiments on datasets through Phoenix
2. Include proper token count and model information in your traced LLM calls
3. Have model pricing configured for the models used in experiments
### Project-level analytics
* Total costs across all traces in a project
* Cost trends over time (coming-soon)
* Most expensive models (coming-soon)