Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@freeplay-mcp analyze production logs for errors in the last hour"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP (Model Context Protocol) server that enables AI agents to interact with Freeplay, the ops platform for AI engineering teams.
Use it to analyze production logs, identify quality issues, iterate on prompts and agents using real data, and run experiments to validate changes before deploying.
This MCP server is an experimental release and will change. Use at your own risk and keep an eye on what your agents are doing.
Current limitations:
Does not support deployment operations or destructive deletion actions — use the Freeplay UI
Uses your regular Freeplay API key (not specially scoped to limit access for agents)
Security warning: Because this uses your full API key, an agent could extract the key and formulate its own API calls outside the scope of the tools included with this MCP server, including destructive actions against your Freeplay account.
Additionally, all MCP servers share a security context within the host, enabling data exfiltration, prompt injection across tools, and cross-server data access.
Only use this with agents and MCP servers you fully trust.
Related MCP server: agent-debugger
If using Claude Code, it is recommended to use the freeplay-plugin, which includes skills and this MCP server: https://github.com/freeplayai/freeplay-plugin.
The simplest way to install only the Freeplay MCP server is via uvx:
claude mcp add freeplay -- uvx freeplay-mcpSet your API key in your MCP client process:
export FREEPLAY_API_KEY="your-api-key"
export FREEPLAY_BASE_URL="https://app.freeplay.ai"Start Claude Code and run /mcp to check installation.
Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
{
"mcpServers": {
"freeplay": {
"command": "uvx",
"args": [
"freeplay-mcp"
],
"env": {
"FREEPLAY_API_KEY": "your-api-key",
"FREEPLAY_BASE_URL": "https://app.freeplay.ai"
}
}
}
}For containerized deployments:
Clone and build:
git clone https://github.com/freeplayai/freeplay-mcp.git
cd freeplay-mcp
docker build -t freeplay-mcp .Set your environment variables (in .env, then source it).
export FREEPLAY_API_KEY="your-api-key"
export FREEPLAY_BASE_URL="https://app.freeplay.ai"Add to Claude Code:
claude mcp add --transport stdio freeplay-mcp -- docker run -i --rm -e FREEPLAY_API_KEY=$FREEPLAY_API_KEY -e FREEPLAY_BASE_URL=$FREEPLAY_BASE_URL freeplay-mcpFor production deployments, consider using a hardened base image such as Chainguard or Distroless.
Start Claude Code and run /mcp to check installation.
API key passed via environment variable FREEPLAY_API_KEY
All requests use Bearer token authentication
Base URL configurable via FREEPLAY_BASE_URL (default: https://app.freeplay.ai)
# Clone and install
git clone https://github.com/freeplayai/freeplay-mcp.git
cd freeplay-mcp
uv sync --group dev
# Lint (with auto-fix)
make lint
# Type check
make type-check
# Run both
make check{
"mcpServers": {
"freeplay": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"FREEPLAY_API_KEY",
"-e",
"FREEPLAY_BASE_URL",
"freeplay-mcp"
],
"env": {
"FREEPLAY_API_KEY": "your-api-key",
"FREEPLAY_BASE_URL": "https://app.freeplay.ai"
}
}
}
}Docs: https://docs.freeplay.ai
Security: security@freeplay.ai
Create a new version of a prompt template. This is a write operation. Always confirm with the user before calling — describe which prompt template will be modified, the model and provider, and whether a new template will be created.
If the template doesn't exist and create_if_not_exists is True, the template will be created automatically.
Note: Deployment is not supported via MCP at this time to reduce risk of unintentional changes. Use the Freeplay UI to deploy versions to environments.
Args: project_id: The Freeplay project ID (required) template_name: Name of the prompt template (required) prompt_messages: JSON string of template messages array, e.g. '[{"role": "system", "content": "You are helpful."}]' (required) model: The model name, e.g. "gpt-4", "claude-3-opus" (required) provider: The provider name, e.g. "openai", "anthropic" (required) version_name: Optional name for this version version_description: Optional description for this version llm_parameters: Optional LLM parameters object, e.g. {"temperature": 0.7, "max_tokens": 1000} tool_schema: Optional tool definitions array for function calling output_schema: Optional output schema for structured outputs create_if_not_exists: If true, creates the template if it doesn't exist (default: true)
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| provider | Yes | ||
| project_id | Yes | ||
| tool_schema | No | ||
| version_name | No | ||
| output_schema | No | ||
| template_name | Yes | ||
| llm_parameters | No | ||
| prompt_messages | Yes | ||
| version_description | No | ||
| create_if_not_exists | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully discloses behavior: it is a write operation requiring user confirmation. It explains the automatic creation behavior when create_if_not_exists is true. It does not detail versioning mechanics (e.g., numbering), but the overall behavior is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: starts with a clear purpose statement, followed by usage guidance, then a parameter block. It is relatively long but every part adds value. Minor redundancy could be trimmed (e.g., the parameter list repeats schema info), but it remains efficient for its complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 11 parameters (5 required), an output schema exists, and no annotations, the description covers all key aspects: purpose, required confirmation, automatic template creation, deployment limitation, and parameter details. It could mention default values (e.g., create_if_not_exists defaults to true) but these are in the schema. Overall, it is complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema description coverage is 0%, so the description must compensate. It does so by listing all parameters with examples (e.g., prompt_messages JSON format, model examples like 'gpt-4', provider examples like 'openai'). Optional parameters like llm_parameters and tool_schema are explained. Some parameter types (e.g., tool_schema as array of objects) could be more precise, but the examples help.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the core action: 'Create a new version of a prompt template.' It specifies it is a write operation. None of the sibling tools (e.g., get_prompt_version, list_prompt_templates) perform creation, so it is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs the agent to 'Always confirm with the user before calling' and to describe which template, model, provider, and whether a new template will be created. Also warns that 'Deployment is not supported via MCP' and directs to the Freeplay UI, guiding when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analyze recent completions for a prompt template and identify missing logged fields. This is a read-only analysis operation.
Fetches recent completions and checks which important fields are not being logged. Returns a list of missing fields with explanations and fix suggestions. Use this to identify gaps in your observability instrumentation.
Args: project_id: The Freeplay project ID template_name: Optional prompt template name to filter by. If not provided, analyzes all completions and checks for missing prompt template associations. environment: Optional environment filter (e.g., "prod", "dev", "local") limit: Number of recent completions to analyze (default: 50)
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| project_id | Yes | ||
| environment | No | ||
| template_name | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly states 'This is a read-only analysis operation,' indicating no destructive side effects. It also explains that it fetches recent completions and returns a list of missing fields with explanations and fixes. Without annotations, the description adequately discloses behavioral traits, though it could mention if any data is modified (but read-only is clear).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, a read-only clarification, and a bulleted list of arguments. It is slightly repetitive (e.g., first two sentences both describe the core action), but overall efficient and front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 params, no annotations, output schema exists), the description covers the purpose, read-only nature, parameter meanings, and return type. Since an output schema is present, it is not required to detail the return format. The description is complete enough for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds full meaning to all four parameters. It explains that project_id is required, template_name filters by prompt template (or checks all if not provided), environment filters environment, and limit defaults to 50. This compensates entirely for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: analyzing recent completions for a prompt template and identifying missing logged fields. It specifies the verb (analyze, identify) and the resource (recent completions for a prompt template), and distinguishes from siblings like get_prompt_version or list_insights by focusing on logging gaps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for use: 'Use this to identify gaps in your observability instrumentation.' It does not explicitly mention when not to use or list alternatives, but the purpose is distinct enough among siblings to imply appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get deployment status for a prompt template showing which versions are deployed to each environment. This is a read-only operation.
Returns which prompt template versions are currently deployed to all environments (dev, staging, production, and any custom environments).
Args: project_id: The Freeplay project ID (required) template_name: The name of the prompt template (required)
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | ||
| template_name | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the operation is read-only and indicates what it returns (versions deployed to environments like dev, staging, production, custom). However, it does not address behavior for invalid inputs, error cases, or rate limits. Since no annotations are provided, the description carries the full burden, but it provides basic safety information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two short paragraphs. The first paragraph introduces the purpose, and the second provides a parameter list. No redundant information is present. It could be slightly more streamlined, but overall it is well-structured and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only two required parameters and an output schema exists, the description covers the essential context: purpose, read-only nature, and environments. It does not explain edge cases or output structure, but the output schema compensates for the latter. For a simple tool, the description is adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter schema has 0% description coverage, so the description must compensate. It lists both parameters in an Args section and notes they are required, adding meaning beyond the schema's required specification. However, it does not provide details on valid formats, examples, or additional semantics, so compensation is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool retrieves deployment status for a prompt template, indicating which versions are deployed to each environment. It uses specific verbs ('Get') and identifies the resource ('deployed prompt versions'). It distinguishes itself from sibling tools like 'get_prompt_version' (which likely gets a specific version details) by focusing on deployment across environments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for checking deployment status but does not explicitly compare with sibling tools or provide when-not-to-use guidance. It mentions the tool is read-only, which gives some context, but lacks alternative tool references or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the full contents of a prompt template version. This is a read-only operation.
Returns the complete prompt template configuration including messages, model, provider, and LLM parameters. Use this to inspect a version before making changes.
Args: project_id: The Freeplay project ID template_id: The prompt template ID version_id: The version ID to retrieve
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | ||
| version_id | Yes | ||
| template_id | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It declares the operation as read-only and describes the return contents ('messages, model, provider, and LLM parameters'). This discloses the non-destructive nature and output structure, though it omits potential authentication or rate limit details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three clear sentences plus a structured Args list. The main purpose is front-loaded, every sentence adds value, and there is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and low complexity (3 required string params), the description adequately covers purpose, parameter semantics, and return content. It does not mention error conditions or prerequisites, but for a simple read operation this is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameter descriptions (0% coverage). The description compensates by listing Args with meanings: 'The Freeplay project ID', 'The prompt template ID', 'The version ID to retrieve'. This adds essential semantic context beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Get the full contents of a prompt template version', clearly identifying the verb (get) and resource (prompt template version). It distinguishes from siblings like create_prompt_version and get_deployed_prompt_versions by specifying it retrieves a specific version's full contents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'Use this to inspect a version before making changes', providing a clear when-to-use context. It does not mention alternatives or when not to use, but for a read-only tool this is sufficient guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List insights for a Freeplay project. This is a read-only operation.
Retrieve a paginated list of insights. Insights are automatically generated observations about patterns in your production data, surfacing actionable themes from auto-evaluation scores on logs and from human annotations. Filter by prompt template or agent to see specific insights for each component.
Args: project_id: The Freeplay project ID (required) page: Page number for pagination (default: 1) page_size: Number of insights per page (default: 30) prompt_template_id: Filter insights by prompt template UUID (optional) agent_id: Filter insights by agent UUID (optional)
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | ||
| agent_id | No | ||
| page_size | No | ||
| project_id | Yes | ||
| prompt_template_id | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description declares the operation as read-only, which is a key behavioral trait. However, it does not disclose error handling, rate limits, authentication requirements, or what happens when the project_id is invalid. Given no annotations, the description carries the full burden but is still somewhat opaque.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is fairly long and has some redundancy (e.g., 'List insights' appears twice). It is front-loaded with purpose and structured with a paragraph and Args list, but could be more concise without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description appropriately focuses on input semantics and what insights are. It covers pagination and filtering, which are key. Minor omissions like ordering or pagination field details prevent a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates well by documenting all five parameters in the Args section, including required/optional status, defaults, and filtering purposes. This adds significant meaning beyond the schema's type definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'insights' for a Freeplay project, and distinguishes it from sibling tools like 'search' by specifying it returns automatically generated observations. The purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to retrieve insights from a project) and mentions filtering by prompt template or agent, but does not provide explicit guidance on when not to use it or compare to alternatives like 'search' for more complex queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all Freeplay projects accessible to the authenticated user. This is a read-only operation.
Returns a formatted list of projects with their IDs and names. Start here for most workflows. Use a project ID from this list to scope most other Freeplay operations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It correctly states 'This is a read-only operation' and indicates the return format (list with IDs and names). However, it could detail pagination, ordering, or authentication requirements, which are absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: purpose, return format, usage guidance. Front-loaded with the main function. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple no-parameter tool with an output schema, the description is mostly complete: it covers purpose, read-only nature, output content, and usage context. Missing potential details like pagination, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, baseline is 4. The description adds value by describing the output (formatted list of projects with IDs and names), which is beyond the empty input schema. This helps the agent understand what to expect.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all Freeplay projects accessible to the user, with the verb 'List' and the resource 'projects'. It specifies read-only operation, and the context of siblings shows other list tools for different resources, so this tool is well-distinguished.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Start here for most workflows' and 'Use a project ID from this list to scope most other Freeplay operations', providing clear usage context. It does not explicitly exclude scenarios, but the guidance is practical and sufficient for a foundational list tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all prompt templates in a Freeplay project. This is a read-only operation.
Returns a list of prompt templates with their IDs, names, and latest version info. Use the template name or ID when creating new versions.
Args: project_id: The Freeplay project ID (required) limit: Maximum number of results to return (default: 50) offset: Number of results to skip for pagination (default: 0)
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No | ||
| project_id | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it is a read-only operation and returns a list with IDs, names, and latest version info. While no annotations are provided, the description adds context about pagination parameters and the nature of the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a clear purpose statement, followed by return info and an Args section. Every sentence adds value without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description adequately covers the tool's purpose, parameters, and return values. It is complete for a list operation with pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to all three parameters: project_id is required, limit defaults to 50, offset defaults to 0, explaining their roles in pagination. This compensates for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all prompt templates in a Freeplay project and specifies it is a read-only operation. This distinguishes it from sibling tools like create_prompt_version and get_prompt_version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to list templates) and hints at using the results for creating new versions. However, it does not explicitly mention when not to use it or point to alternatives for specific template details.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Optimize a prompt template version using AI-powered analysis and automated experimentation. This is a write operation that creates a new prompt template version. Always confirm with the user before calling this tool — describe which prompt template will be optimized, the dataset being used, and that this may incur LLM costs.
Analyzes the prompt template version against scored examples from production logs and generates an improved version with guidance from the user on where to focus. The optimization considers human evaluation labels, customer feedback, and best practices based on the flags provided.
This is a long-running operation that may take several minutes. When run_test_after_optimization is True (the default), the job will also run baseline and optimized test runs and create a comparison, which incurs additional LLM costs.
Args: project_id: The Freeplay project ID prompt_template_version_id: The prompt template version ID to optimize dataset_id: The dataset ID containing examples to analyze user_instructions: Optional specific instructions for the optimization (e.g., "focus on reducing hallucinations") use_best_practices: Whether to apply general prompt engineering best practices (default: True) use_labels: Whether to use human evaluation labels from the dataset in analysis (default: True) use_customer_feedback: Whether to incorporate customer feedback data (default: True) run_test_after_optimization: Whether to run a comparison test after optimization (default: True)
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | ||
| project_id | Yes | ||
| use_labels | No | ||
| user_instructions | No | ||
| use_best_practices | No | ||
| use_customer_feedback | No | ||
| prompt_template_version_id | Yes | ||
| run_test_after_optimization | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: it's a write operation, long-running, creates a new version, incurs LLM costs, and explains the effect of each boolean flag. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with key warnings upfront and bullet-like parameter list. Slightly verbose but every sentence adds value; could be tightened slightly without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given it has an output schema, return values are covered elsewhere. The description fully explains all parameters, behavioral nuances, prerequisites (user confirmation), and cost/performance implications for a complex 8-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It lists all 8 parameters with clear explanations, e.g., 'user_instructions: Optional specific instructions for the optimization (e.g., focus on reducing hallucinations)', adding significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Optimize a prompt template version' and specifies it is a write operation that creates a new version. It distinguishes from siblings like 'create_prompt_version' by focusing on AI-powered optimization.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to confirm with the user before calling, mentioning cost implications and what to relay. However, no direct exclusions or comparisons to alternatives are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search observability data in a Freeplay project. This is a read-only operation.
Returns sessions, traces, or completions matching the given filters.
Sessions are top-level containers that group related traces and completions, representing a complete user interaction, conversation, or agent run.
Traces group related completions and tool calls within a session, representing a functional unit of work such as a single agent run or workflow step.
Completions are individual LLM calls with their inputs, outputs, and metadata.
The filters parameter accepts a JSON string. A single filter has the form:
{"field": "", "op": "", "value": }
eq Equals
lt Less than
gt Greater than
lte Less than or equal
gte Greater than or equal
contains Contains substring
between Within numeric rangeNumeric fields (support eq, lt, gt, lte, gte): cost e.g. 0.003 latency e.g. 8 (seconds) start_time e.g. "2024-06-01 00:00:00"
Equality fields (support eq): environment e.g. "staging" prompt_template e.g. "my-prompt" prompt_template_id e.g. "uuid..." completion_id e.g. "uuid..." session_id e.g. "uuid..." model e.g. "gpt-4o" provider e.g. "openai" review_status e.g. "review_complete" agent_name e.g. "support-agent" trace_agent_name e.g. "my-agent" api_key e.g. "production-key" assignee e.g. "user@example.com" insight_name e.g. "Response Quality Issues"
Text search fields (support contains): completion_output e.g. "weather"
Nested JSON fields (field.* syntax, support contains): completion_inputs.* e.g. field="completion_inputs.topic", value="weather" completion_feedback.* e.g. field="completion_feedback.rating", value="positive" session_custom_metadata.* e.g. field="session_custom_metadata.user_type", value="premium" trace_custom_metadata.* e.g. field="trace_custom_metadata.workflow", value="onboarding" trace_input.* e.g. field="trace_input.query", value="weather" trace_output.* e.g. field="trace_output.response", value="sunny" trace_feedback.* e.g. field="trace_feedback.rating", value="positive"
Evaluation result fields (nested JSON, field.* syntax): completion_evaluation_results.* supports eq. e.g. field="completion_evaluation_results.Response Quality", value="4" completion_client_evaluation_results.* supports eq. e.g. field="completion_client_evaluation_results.score", value="85" trace_evaluation_results.* supports eq, gt, lt, gte, lte, contains. e.g. field="trace_evaluation_results.Quality Score", value=5 trace_client_eval_results.* supports eq, contains. e.g. field="trace_client_eval_results.confidence_score", value=0.95
Evaluation notes fields: evaluation_notes.content supports contains. e.g. "needs review" evaluation_notes.author supports eq. e.g. "user@example.com" evaluation_notes.created_at supports gt, lt, gte, lte. e.g. "2024-06-01 00:00:00"
Combine filters with "and", "or", "not" (nestable to any depth): {"and": [, , ...]} All conditions must match {"or": [, , ...]} At least one must match {"not": } Negates a condition
Example - expensive GPT-4o or Claude completions outside production: {"and": [{"field": "cost", "op": "gte", "value": 0.001}, {"or": [{"field": "model", "op": "eq", "value": "gpt-4o"}, {"field": "model", "op": "eq", "value": "claude-3-opus"}]}, {"not": {"field": "environment", "op": "eq", "value": "prod"}}]}
Args: project_id: The Freeplay project ID result_type: What to search for: "sessions", "traces", or "completions" filters: Optional JSON string with filter expression. Examples: Simple: '{"field": "environment", "op": "eq", "value": "production"}' Compound: '{"and": [{"field": "cost", "op": "gte", "value": 0.01}, {"field": "model", "op": "eq", "value": "gpt-4o"}]}' limit: Maximum number of results to return (default: 20) offset: Number of results to skip for pagination (default: 0)
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| offset | No | ||
| filters | No | ||
| project_id | Yes | ||
| result_type | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares read-only operation, explains result types, and details filter syntax comprehensively. With no annotations provided, the description fully discloses behavior: no side effects, authentication implied but not required beyond tool access, and pagination via limit/offset. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured: summary first, then key concepts, followed by filter syntax documentation, and parameter descriptions. Each section earns its place; no fluff. Front-loaded with core purpose and read-only nature.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 parameters, extensive filter system), the description covers all aspects: result types, filter syntax, all available fields with operators, examples, and parameter details. Output schema exists, so return values are not needed. Complete for agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% but description compensates fully with an 'Args' section explaining each parameter, plus extensive documentation of the 'filters' parameter including syntax, operators, available fields, and compound filter examples. Adds substantial meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States verb 'Search' and resource 'observability data in a Freeplay project', specifies read-only, and lists result types (sessions, traces, completions). Clearly distinguishes from sibling tools which focus on prompts, insights, and projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use: searching with filters for sessions, traces, or completions. Includes filter syntax and examples. Does not explicitly state when not to use, but sibling tools are sufficiently distinct so no ambiguity arises.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Each tool has a clearly distinct purpose: creating prompt versions, analyzing logging issues, retrieving deployment info, fetching specific versions, listing insights, projects, and templates, optimizing prompts, and searching observability data. No overlap in functionality.
All tool names follow a consistent verb_noun pattern (e.g., create_prompt_version, list_projects, search). Verbs like 'get', 'list', 'find', 'create', and 'search' are used consistently, and the naming is predictable and readable.
Nine tools is an appropriate scope for a prompt management and observability server. It covers the core workflows without being too sparse or overwhelmingly large.
The tool set covers essential CRUD-like operations (create, read) and search/analysis, but lacks delete operations for versions or templates. The missing delete is a minor gap, as most workflows are supported.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
AI agent observability for production traces, natural-language insights, and improvement loops.
Give your AI agents the tools to build, manage, and run automation workflows.
Build, validate, and deploy multi-agent AI solutions from any AI environment.
Create and manage AI agents that collaborate and solve problems through natural language interacti…
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/freeplayai/freeplay-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
