List all Freeplay projects accessible to the authenticated user. This is a read-only operation.

Returns a formatted list of projects with their IDs and names. Start here for most workflows. Use a project ID from this list to scope most other Freeplay operations.

Search observability data in a Freeplay project. This is a read-only operation.

Returns sessions, traces, or completions matching the given filters.

• Sessions are top-level containers that group related traces and completions, representing a complete user interaction, conversation, or agent run.

• Traces group related completions and tool calls within a session, representing a functional unit of work such as a single agent run or workflow step.

• Completions are individual LLM calls with their inputs, outputs, and metadata.

Filter syntax

The filters parameter accepts a JSON string. A single filter has the form: {"field": "", "op": "", "value": }

Operators

eq        Equals
lt        Less than
gt        Greater than
lte       Less than or equal
gte       Greater than or equal
contains  Contains substring
between   Within numeric range

Available fields

Numeric fields (support eq, lt, gt, lte, gte): cost e.g. 0.003 latency e.g. 8 (seconds) start_time e.g. "2024-06-01 00:00:00"

Equality fields (support eq): environment e.g. "staging" prompt_template e.g. "my-prompt" prompt_template_id e.g. "uuid..." completion_id e.g. "uuid..." session_id e.g. "uuid..." model e.g. "gpt-4o" provider e.g. "openai" review_status e.g. "review_complete" agent_name e.g. "support-agent" trace_agent_name e.g. "my-agent" api_key e.g. "production-key" assignee e.g. "user@example.com" insight_name e.g. "Response Quality Issues"

Text search fields (support contains): completion_output e.g. "weather"

Nested JSON fields (field.* syntax, support contains): completion_inputs.* e.g. field="completion_inputs.topic", value="weather" completion_feedback.* e.g. field="completion_feedback.rating", value="positive" session_custom_metadata.* e.g. field="session_custom_metadata.user_type", value="premium" trace_custom_metadata.* e.g. field="trace_custom_metadata.workflow", value="onboarding" trace_input.* e.g. field="trace_input.query", value="weather" trace_output.* e.g. field="trace_output.response", value="sunny" trace_feedback.* e.g. field="trace_feedback.rating", value="positive"

Evaluation result fields (nested JSON, field.* syntax): completion_evaluation_results.* supports eq. e.g. field="completion_evaluation_results.Response Quality", value="4" completion_client_evaluation_results.* supports eq. e.g. field="completion_client_evaluation_results.score", value="85" trace_evaluation_results.* supports eq, gt, lt, gte, lte, contains. e.g. field="trace_evaluation_results.Quality Score", value=5 trace_client_eval_results.* supports eq, contains. e.g. field="trace_client_eval_results.confidence_score", value=0.95

Evaluation notes fields: evaluation_notes.content supports contains. e.g. "needs review" evaluation_notes.author supports eq. e.g. "user@example.com" evaluation_notes.created_at supports gt, lt, gte, lte. e.g. "2024-06-01 00:00:00"

Compound filters

Combine filters with "and", "or", "not" (nestable to any depth): {"and": [, , ...]} All conditions must match {"or": [, , ...]} At least one must match {"not": } Negates a condition

Example - expensive GPT-4o or Claude completions outside production: {"and": [{"field": "cost", "op": "gte", "value": 0.001}, {"or": [{"field": "model", "op": "eq", "value": "gpt-4o"}, {"field": "model", "op": "eq", "value": "claude-3-opus"}]}, {"not": {"field": "environment", "op": "eq", "value": "prod"}}]}

Args: project_id: The Freeplay project ID result_type: What to search for: "sessions", "traces", or "completions" filters: Optional JSON string with filter expression. Examples: Simple: '{"field": "environment", "op": "eq", "value": "production"}' Compound: '{"and": [{"field": "cost", "op": "gte", "value": 0.01}, {"field": "model", "op": "eq", "value": "gpt-4o"}]}' limit: Maximum number of results to return (default: 20) offset: Number of results to skip for pagination (default: 0)

List all prompt templates in a Freeplay project. This is a read-only operation.

Returns a list of prompt templates with their IDs, names, and latest version info. Use the template name or ID when creating new versions.

Args: project_id: The Freeplay project ID (required) limit: Maximum number of results to return (default: 50) offset: Number of results to skip for pagination (default: 0)

Get the full contents of a prompt template version. This is a read-only operation.

Returns the complete prompt template configuration including messages, model, provider, and LLM parameters. Use this to inspect a version before making changes.

Args: project_id: The Freeplay project ID template_id: The prompt template ID version_id: The version ID to retrieve

Create a new version of a prompt template. This is a write operation. Always confirm with the user before calling — describe which prompt template will be modified, the model and provider, and whether a new template will be created.

If the template doesn't exist and create_if_not_exists is True, the template will be created automatically.

Note: Deployment is not supported via MCP at this time to reduce risk of unintentional changes. Use the Freeplay UI to deploy versions to environments.

Args: project_id: The Freeplay project ID (required) template_name: Name of the prompt template (required) prompt_messages: JSON string of template messages array, e.g. '[{"role": "system", "content": "You are helpful."}]' (required) model: The model name, e.g. "gpt-4", "claude-3-opus" (required) provider: The provider name, e.g. "openai", "anthropic" (required) version_name: Optional name for this version version_description: Optional description for this version llm_parameters: Optional LLM parameters object, e.g. {"temperature": 0.7, "max_tokens": 1000} tool_schema: Optional tool definitions array for function calling output_schema: Optional output schema for structured outputs create_if_not_exists: If true, creates the template if it doesn't exist (default: true)

Get deployment status for a prompt template showing which versions are deployed to each environment. This is a read-only operation.

Returns which prompt template versions are currently deployed to all environments (dev, staging, production, and any custom environments).

Args: project_id: The Freeplay project ID (required) template_name: The name of the prompt template (required)

Analyze recent completions for a prompt template and identify missing logged fields. This is a read-only analysis operation.

Fetches recent completions and checks which important fields are not being logged. Returns a list of missing fields with explanations and fix suggestions. Use this to identify gaps in your observability instrumentation.

Args: project_id: The Freeplay project ID template_name: Optional prompt template name to filter by. If not provided, analyzes all completions and checks for missing prompt template associations. environment: Optional environment filter (e.g., "prod", "dev", "local") limit: Number of recent completions to analyze (default: 50)

List insights for a Freeplay project. This is a read-only operation.

Retrieve a paginated list of insights. Insights are automatically generated observations about patterns in your production data, surfacing actionable themes from auto-evaluation scores on logs and from human annotations. Filter by prompt template or agent to see specific insights for each component.

Args: project_id: The Freeplay project ID (required) page: Page number for pagination (default: 1) page_size: Number of insights per page (default: 30) prompt_template_id: Filter insights by prompt template UUID (optional) agent_id: Filter insights by agent UUID (optional)

Optimize a prompt template version using AI-powered analysis and automated experimentation. This is a write operation that creates a new prompt template version. Always confirm with the user before calling this tool — describe which prompt template will be optimized, the dataset being used, and that this may incur LLM costs.

Analyzes the prompt template version against scored examples from production logs and generates an improved version with guidance from the user on where to focus. The optimization considers human evaluation labels, customer feedback, and best practices based on the flags provided.

This is a long-running operation that may take several minutes. When run_test_after_optimization is True (the default), the job will also run baseline and optimized test runs and create a comparison, which incurs additional LLM costs.

Args: project_id: The Freeplay project ID prompt_template_version_id: The prompt template version ID to optimize dataset_id: The dataset ID containing examples to analyze user_instructions: Optional specific instructions for the optimization (e.g., "focus on reducing hallucinations") use_best_practices: Whether to apply general prompt engineering best practices (default: True) use_labels: Whether to use human evaluation labels from the dataset in analysis (default: True) use_customer_feedback: Whether to incorporate customer feedback data (default: True) run_test_after_optimization: Whether to run a comparison test after optimization (default: True)