@arizeai/phoenix-mcp

Get a list of all the prompts.

Prompts (templates, prompt templates) are versioned templates for input messages to an LLM. Each prompt includes both the input messages, but also the model and invocation parameters to use when generating outputs.

Returns a list of prompt objects with their IDs, names, and descriptions.

Example usage: List all available prompts

Expected return: Array of prompt objects with metadata. Example: [{ "name": "article-summarizer", "description": "Summarizes an article into concise bullet points", "source_prompt_id": null, "id": "promptid1234" }]

Get the latest version of a prompt. Returns the prompt version with its template, model configuration, and invocation parameters.

Example usage: Get the latest version of a prompt named 'article-summarizer'

Expected return: Prompt version object with template and configuration. Example: { "description": "Initial version", "model_provider": "OPENAI", "model_name": "gpt-3.5-turbo", "template": { "type": "chat", "messages": [ { "role": "system", "content": "You are an expert summarizer. Create clear, concise bullet points highlighting the key information." }, { "role": "user", "content": "Please summarize the following {{topic}} article:

{{article}}" } ] }, "template_type": "CHAT", "template_format": "MUSTACHE", "invocation_parameters": { "type": "openai", "openai": {} }, "id": "promptversionid1234" }

Get a prompt's latest version by its identifier (name or ID). Returns the prompt version with its template, model configuration, and invocation parameters.

Example usage: Get the latest version of a prompt with name 'article-summarizer'

Expected return: Prompt version object with template and configuration. Example: { "description": "Initial version", "model_provider": "OPENAI", "model_name": "gpt-3.5-turbo", "template": { "type": "chat", "messages": [ { "role": "system", "content": "You are an expert summarizer. Create clear, concise bullet points highlighting the key information." }, { "role": "user", "content": "Please summarize the following {{topic}} article:

{{article}}" } ] }, "template_type": "CHAT", "template_format": "MUSTACHE", "invocation_parameters": { "type": "openai", "openai": {} }, "id": "promptversionid1234" }

Get a specific version of a prompt using its version ID. Returns the prompt version with its template, model configuration, and invocation parameters.

Example usage: Get a specific prompt version with ID 'promptversionid1234'

Expected return: Prompt version object with template and configuration. Example: { "description": "Initial version", "model_provider": "OPENAI", "model_name": "gpt-3.5-turbo", "template": { "type": "chat", "messages": [ { "role": "system", "content": "You are an expert summarizer. Create clear, concise bullet points highlighting the key information." }, { "role": "user", "content": "Please summarize the following {{topic}} article:

{{article}}" } ] }, "template_type": "CHAT", "template_format": "MUSTACHE", "invocation_parameters": { "type": "openai", "openai": {} }, "id": "promptversionid1234" }

Create or update a prompt with its template and configuration. Creates a new prompt and its initial version with specified model settings.

Example usage: Create a new prompt named 'email_generator' with a template for generating emails

Expected return: A confirmation message of successful prompt creation

Get a list of all versions for a specific prompt. Returns versions with pagination support.

Example usage: List all versions of a prompt named 'article-summarizer'

Expected return: Array of prompt version objects with IDs and configuration. Example: [ { "description": "Initial version", "model_provider": "OPENAI", "model_name": "gpt-3.5-turbo", "template": { "type": "chat", "messages": [ { "role": "system", "content": "You are an expert summarizer. Create clear, concise bullet points highlighting the key information." }, { "role": "user", "content": "Please summarize the following {{topic}} article:

{{article}}" } ] }, "template_type": "CHAT", "template_format": "MUSTACHE", "invocation_parameters": { "type": "openai", "openai": {} }, "id": "promptversionid1234" } ]

Get a prompt version by its tag name. Returns the prompt version with its template, model configuration, and invocation parameters.

Example usage: Get the 'production' tagged version of prompt 'article-summarizer'

Expected return: Prompt version object with template and configuration. Example: { "description": "Initial version", "model_provider": "OPENAI", "model_name": "gpt-3.5-turbo", "template": { "type": "chat", "messages": [ { "role": "system", "content": "You are an expert summarizer. Create clear, concise bullet points highlighting the key information." }, { "role": "user", "content": "Please summarize the following {{topic}} article:

{{article}}" } ] }, "template_type": "CHAT", "template_format": "MUSTACHE", "invocation_parameters": { "type": "openai", "openai": {} }, "id": "promptversionid1234" }

Get a list of all tags for a specific prompt version. Returns tag objects with pagination support.

Example usage: List all tags associated with prompt version 'promptversionid1234'

Expected return: Array of tag objects with names and IDs. Example: [ { "name": "staging", "description": "The version deployed to staging", "id": "promptversionid1234" }, { "name": "development", "description": "The version deployed for development", "id": "promptversionid1234" } ]

Add a tag to a specific prompt version. The operation returns no content on success (204 status code).

Example usage: Tag prompt version 'promptversionid1234' with the name 'production'

Expected return: Confirmation message of successful tag addition

Get a list of all the experiments run on a given dataset.

Experiments are collections of experiment runs, each experiment run corresponds to a single dataset example. The dataset example is passed to an implied task which in turn produces an output.

Example usage: Show me all the experiments I've run on dataset RGF0YXNldDox

Expected return: Array of experiment objects with metadata. Example: [ { "id": "experimentid1234", "dataset_id": "datasetid1234", "dataset_version_id": "datasetversionid1234", "repetitions": 1, "metadata": {}, "project_name": "Experiment-abc123", "created_at": "YYYY-MM-DDTHH:mm:ssZ", "updated_at": "YYYY-MM-DDTHH:mm:ssZ" } ]

Get an experiment by its ID.

The tool returns experiment metadata in the first content block and a JSON object with the experiment data in the second. The experiment data contains both the results of each experiment run and the annotations made by an evaluator to score or label the results, for example, comparing the output of an experiment run to the expected output from the dataset example.

Example usage: Show me the experiment results for experiment RXhwZXJpbWVudDo4

Expected return: Object containing experiment metadata and results. Example: { "metadata": { "id": "experimentid1234", "dataset_id": "datasetid1234", "dataset_version_id": "datasetversionid1234", "repetitions": 1, "metadata": {}, "project_name": "Experiment-abc123", "created_at": "YYYY-MM-DDTHH:mm:ssZ", "updated_at": "YYYY-MM-DDTHH:mm:ssZ" }, "experimentResult": [ { "example_id": "exampleid1234", "repetition_number": 0, "input": "Sample input text", "reference_output": "Expected output text", "output": "Actual output text", "error": null, "latency_ms": 1000, "start_time": "2025-03-20T12:00:00Z", "end_time": "2025-03-20T12:00:01Z", "trace_id": "trace-123", "prompt_token_count": 10, "completion_token_count": 20, "annotations": [ { "name": "quality", "annotator_kind": "HUMAN", "label": "good", "score": 0.9, "explanation": "Output matches expected format", "trace_id": "trace-456", "error": null, "metadata": {}, "start_time": "YYYY-MM-DDTHH:mm:ssZ", "end_time": "YYYY-MM-DDTHH:mm:ssZ" } ] } ] }

Get a list of all datasets.

Datasets are collections of 'dataset examples' that each example includes an input, (expected) output, and optional metadata. They are primarily used as inputs for experiments.

Example usage: Show me all available datasets

Expected return: Array of dataset objects with metadata. Example: [ { "id": "RGF0YXNldDox", "name": "my-dataset", "description": "A dataset for testing", "metadata": {}, "created_at": "2024-03-20T12:00:00Z", "updated_at": "2024-03-20T12:00:00Z" } ]

Get examples from a dataset.

Dataset examples are an array of objects that each include an input, (expected) output, and optional metadata. These examples are typically used to represent input to an application or model (e.g. prompt template variables, a code file, or image) and used to test or benchmark changes.

Example usage: Show me all examples from dataset RGF0YXNldDox

Expected return: Object containing dataset ID, version ID, and array of examples. Example: { "dataset_id": "datasetid1234", "version_id": "datasetversionid1234", "examples": [ { "id": "exampleid1234", "input": { "text": "Sample input text" }, "output": { "text": "Expected output text" }, "metadata": {}, "updated_at": "YYYY-MM-DDTHH:mm:ssZ" } ] }

List experiments run on a dataset.

Example usage: Show me all experiments run on dataset RGF0YXNldDox

Expected return: Array of experiment objects with metadata. Example: [ { "id": "experimentid1234", "dataset_id": "datasetid1234", "dataset_version_id": "datasetversionid1234", "repetitions": 1, "metadata": {}, "project_name": "Experiment-abc123", "created_at": "YYYY-MM-DDTHH:mm:ssZ", "updated_at": "YYYY-MM-DDTHH:mm:ssZ" } ]

Add examples to an existing dataset.

This tool adds one or more examples to an existing dataset. Each example includes an input, output, and metadata. The metadata will automatically include information indicating that these examples were synthetically generated via MCP. When calling this tool, check existing examples using the "get-dataset-examples" tool to ensure that you are not adding duplicate examples and following existing patterns for how data should be structured.

Example usage: Look at the analyze "my-dataset" and augment them with new examples to cover relevant edge cases

Expected return: Confirmation of successful addition of examples to the dataset. Example: { "dataset_name": "my-dataset", "message": "Successfully added examples to dataset" }