Prompt Lab MCP Server
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Prompt Lab MCP ServerStart a new prompt optimization workspace and show me the UI."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Prompt Lab MCP Server
Prompt optimization loops and regression test suites for Claude Code, with a companion web UI.
The agent runs inside your Claude Code session and owns all LLM work — scoring responses, proposing improved prompts, applying suggestions. The server holds workspace state and keeps the agent and the Prompt Lab UI in sync.
Quick start
Copy mcp-connect.json from this repo into your project as .mcp.json:
{
"mcpServers": {
"prompt-lab": {
"type": "http",
"url": "https://prompt-lab-mcp.up.railway.app/mcp"
}
}
}Claude Code connects automatically on next start. Verify with /mcp.
Related MCP server: Session Buddy
Example session
# 1. Open a workspace — agent shares the UI URL
start_web_app()
→ "Open https://prompt-lab-mcp.vercel.app?s=abc123 to follow along."
# 2. Register an API key
register_api_key(workspaceId, "sk-ant-...")
# 3. Set a system prompt and a test case
set_system_prompt(workspaceId, "You are a concise customer support agent...")
add_test_cases(workspaceId, [{
query: "How do I reset my password?",
targetAnswer: "Click 'Forgot password' on the login page and follow the email link."
}])
# 4. Run the optimization loop
loop_optimization(workspaceId, threshold=85)
→ Iteration 1 — score 58: response too long, no mention of email link
→ Iteration 2 — score 74: better, but missing the exact step
→ Iteration 3 — score 91: SUCCESS — prompt updated to require step-by-step answersThe UI shows each iteration's score, the agent's reasoning, and the revised system prompt in real time.
How it works
Prompt Lab UI (github.com/jurek-f/prompt-lab)
↕ HTTP
Prompt Lab MCP Server (Railway)
↕ MCP
Claude Code (your machine)API keys
API keys are never stored in the MCP server config. Instead, pass them to Claude Code as environment variables — the agent reads them and registers them with the server at the start of each session using register_api_key.
Set the key(s) for the provider(s) you want to use. The agent auto-detects the provider from the key prefix when calling register_api_key.
If they're already in your system environment, Claude Code inherits them automatically — nothing else to do. Otherwise add them to ~/.claude/env or your shell profile:
ANTHROPIC_API_KEY=sk-ant-...
GEMINI_API_KEY=AIza...
OPENAI_API_KEY=sk-...MCP tools
Setup
Tool | Description |
| Creates a workspace and returns the Prompt Lab UI URL. |
| Registers an API key for test runs. Provider is auto-detected from the key prefix. |
| Lists available models based on registered keys. |
| Sets the model for test runs. Syncs to the UI model selector. |
| Deletes a workspace and all its state. Irreversible. |
Templates
Templates are global and appear in the UI dropdowns as soon as they are pushed.
Tool | Description |
| Saves a test suite template. Appears in the UI "Load test suite…" dropdown. |
| Saves a system prompt template. Appears in the UI "Load template…" dropdown. |
Workspace state
Tool | Description |
| Reads the full workspace: system prompt, test cases, results, suggestions, model. |
| Sets the system prompt without incrementing the iteration counter. |
| Adds test cases. |
| Stores one scored test result. |
| Queues a revised prompt for review in the UI. |
| Applies a pending suggestion and increments the iteration counter. |
| Pass/fail summary across all test cases for the current system prompt. |
Optimization
Requires a workspace with at least one test case.
Tool | Description |
| Single pass — scores test cases, posts one suggestion, then waits for user review in the UI. |
| Automated loop — iterates until all scores meet the threshold or max iterations is reached. |
Regression
Tool | Description |
| Single pass — scores all test cases, no prompt changes. |
| Automated loop — repeats until every individual score meets the threshold. A high average that masks one failing case is not a pass. |
Archive
Tool | Description |
| Fetches all session summaries and regression runs pushed by the UI. |
Self-hosting
Deploy to Railway and set these environment variables:
Variable | Description |
| Upstash Redis URL for persistence |
| Upstash Redis token |
| URL of your Prompt Lab UI deployment |
npm install
npm run dev # starts on :3000MCP endpoint: http://localhost:3000/mcp
License
MIT — see LICENSE.
© 2026 Jurek Föllmer
Available Tools
19 toolsadd_test_casesA
Add test cases to this workspace.
Set replace: true to clear the existing suite and load a fresh one. Set replace: false (default) to append to the existing suite.
Each test case needs at least a query. targetAnswer is required for scoring. Omit targetAnswer only for exploratory runs where you score manually.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| testCases | Yes | Test cases to run | |
| replace | No | Replace all existing test cases. Default false (append). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the replace behavior and the requirement for targetAnswer for scoring. However, no annotations exist, and the description does not cover side effects like validation, error handling, or what happens if the workspace does not exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Five sentences, no redundant information, front-loaded with purpose. Each sentence provides unique value without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the replace parameter and test case requirements adequately, but fails to explain workspaceId or return behavior. Given no output schema and 3 parameters, the description is moderately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant meaning beyond the input schema: explains replace behavior ('clears existing suite') and the condition for targetAnswer (required for scoring, optional for exploratory). WorkspaceId remains undocumented in both schema and description, slightly reducing coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool adds test cases to a workspace, with specific verb and resource. However, it does not explicitly differentiate from sibling tools like run_regression_testsuite or loop_regression, which is a minor gap.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear guidance on when to use 'replace: true' vs 'false' and when to omit targetAnswer. Lacks explicit comparison to alternatives, but the parameter-level advice is helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
apply_suggestionA
Apply a pending suggestion: sets it as the active system prompt and increments the iteration counter.
Only call in fully automated loop mode (loop_optimization, loop_regression). In gated mode, wait for the user to approve via the UI.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| suggestionId | Yes | ID from post_prompt_suggestion response |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behavioral traits. It describes the primary effect but misses details like side effects (e.g., overwriting previous prompt), potential errors, or required permissions. It is adequate but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first defines action, second specifies loop mode, third handles gated mode. No redundancy, front-loaded with essential info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 required params and no output schema, the description covers purpose and usage mode. However, it lacks behavioral details (destructiveness, error conditions) which would be expected given no annotations. Adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%, covering only 'suggestionId' with 'ID from post_prompt_suggestion response'. The description adds no additional meaning for 'workspaceId' or any parameter usage guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: 'Apply a pending suggestion: sets it as the active system prompt and increments the iteration counter.' This clearly distinguishes it from siblings like 'post_prompt_suggestion' (creates suggestion) and 'set_system_prompt' (direct set).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call: 'Only call in fully automated loop mode (loop_optimization, loop_regression).' And when not: 'In gated mode, wait for the user to approve via the UI.' This provides clear context and exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_sessionB
Delete a workspace and all its state (test cases, results, suggestions, API keys). Irreversible.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description notes the action is 'Irreversible' and enumerates the state that is deleted (test cases, results, suggestions, API keys). However, it does not discuss permission requirements, consequences, or any side effects beyond the listed items.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, each sentence adds value (action + scope, irreversibility). No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple tool with one parameter and no output schema, the description provides purpose and irreversibility but lacks parameter details and usage context, leaving gaps for effective invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter (workspaceId) with no description. The tool description does not explain what workspaceId is, format, or how to obtain it. With 0% schema coverage, the description must compensate but fails to do so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a workspace and all its state' and lists specific items (test cases, results, suggestions, API keys), which distinguishes it from sibling tools that do not have delete functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no conditions under which deletion is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_regression_statusB
Pass/fail summary across all test cases for the current system prompt.
Call after running all test cases to decide: is the prompt good enough, or improve further? A test case passes if its most recent score >= threshold (default 70).
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| threshold | No | Minimum score to pass (default 70) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses pass/fail logic based on threshold and default value, but does not specify read-only nature, error conditions, or prerequisite state (e.g., tests must have been run).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose, then usage, then definition. No redundant words. Could be slightly more concise but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without output schema, description lacks output format details (e.g., counts, list). Also missing error handling and prerequisites. However, for a simple query tool, it covers core behavior adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (only threshold has description). Description adds context for threshold (default 70, pass condition) but not for workspaceId. With low coverage, description partially compensates but misses required parameter explanation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it provides a 'Pass/fail summary across all test cases for the current system prompt.' Differentiates from sibling tools like run_regression_testsuite by focusing on status retrieval rather than execution. Lacks explicit output format but is sufficient.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit timing advice: 'Call after running all test cases to decide: is the prompt good enough, or improve further?' but does not provide when-not-to-use or mention alternative tools like loop_regression for iterative improvement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_workspace_stateA
Read the full current state of a workspace.
Returns: system prompt, test cases, test results, suggestions, iteration counter, optimization goal, available models, selected model, and active query/target.
Call at the start of each session to recover state after a context break. Also call before running tests to get the latest test case IDs.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully defines the behavior: read-only operation returning structured state. It covers return fields and use cases but omits details like authentication requirements or rate limits, though not critical for a simple read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient, front-loads the purpose, and uses a bullet-like list for return fields, achieving high readability with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lists return items compensating for no output schema, and specifies usage context. However, missing parameter explanation and lack of prerequisites or state dependencies make it incomplete for a state-recovery tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the workspaceId parameter. It does not clarify what the ID represents or how to obtain it, leaving a significant gap for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Read') and resource ('full current state of a workspace'), and lists exact return fields, clearly distinguishing it from sibling tools that perform mutations or other queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises calling at session start to recover state and before running tests to get test case IDs, providing clear context for use. Lacks explicit when-not-to-use or alternatives, but the guidance is actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_modelsB
Register available AI models and API keys for this workspace.
Call once after start_web_app. Scan your environment for API keys and Ollama:
process.env.ANTHROPIC_API_KEY → add claude-haiku-4-5-20251001, claude-sonnet-4-6, claude-opus-4-8
process.env.GEMINI_API_KEY → add gemini-2.5-flash-lite, gemini-2.5-flash, gemini-2.5-pro
process.env.OPENAI_API_KEY → add gpt-4o-mini, gpt-4o
Ollama: fetch (process.env.OLLAMA_URL ?? "http://localhost:11434") + "/api/tags" → add each model.name; catch errors silently
Default model priority (first available wins): gemini-2.5-flash-lite → claude-haiku-4-5-20251001 → gpt-4o-mini
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| models | Yes | All model IDs available in your environment | |
| defaultModel | No | Override the priority-based default | |
| anthropicApiKey | No | ||
| geminiApiKey | No | ||
| openaiApiKey | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly discloses the tool's scanning behavior: checking environment variables for API keys and fetching Ollama models, with error handling for Ollama. It also specifies default model priority. However, it omits return value or side effects like overwriting existing models.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured as a numbered list, making steps easy to follow. However, it is somewhat verbose for the amount of information conveyed, repeating context about each API key. It could be tightened.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, no output schema, no annotations), the description covers detection logic and ordering but lacks clarity on input-output behavior, return value, and idempotence. It is partially complete but leaves gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 33%, and the description does not clarify the purpose of key parameters like 'anthropicApiKey' or 'models.' The description implies auto-detection, but the required 'models' parameter suggests user input, creating confusion. The description adds little value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Register available AI models and API keys for this workspace,' providing a specific verb and resource. However, the tool name 'list_models' conflicts with the described action of registration, causing slight confusion. The description differentiates from sibling 'register_api_key' by including model registration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises to 'Call once after start_web_app,' indicating a specific context. However, it does not explicitly state when not to use the tool or compare it to alternatives like 'register_api_key' for API-only registration. The guidance is present but minimal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_optimizationA
Run the full optimization loop until the threshold is met or max iterations reached.
Like start_optimization_session but auto-applies each suggestion and repeats.
Prerequisites: same as start_optimization_session.
Loop:
Run all test cases, score responses, call post_test_result for each.
Call get_regression_status.
If ALL scores >= threshold AND iteration >= 1 → SUCCESS.
If iteration >= maxIterations → EXHAUSTED. Report best result.
Analyse failures, write improved prompt (targeted — fix pattern, keep what works).
Call post_prompt_suggestion then apply_suggestion (auto authorised in loop mode).
Go to 1.
Do NOT stop after the first pass because it is passing — first pass is a baseline. Always run at least one improvement cycle.
After the loop: call pull_ui_history, save optimization results locally, call save_system_prompt_template with the best prompt found.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| threshold | No | Pass score 0–100 (default 70) | |
| maxIterations | No | Max loop iterations (default 5) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully describes the loop steps, success/exhaustion conditions, prompt improvement, and post-loop actions, ensuring complete behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed but well-structured with a list. It is front-loaded with purpose and each sentence adds value, though slightly verbose for a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a loop optimization tool and no output schema, the description covers the loop algorithm, conditions, and post-loop steps completely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes threshold and maxIterations; workspaceId lacks description. The description adds no further parameter meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a full optimization loop until threshold or max iterations, and distinguishes from sibling start_optimization_session by noting auto-application and repetition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies prerequisites same as start_optimization_session, explicitly warns not to stop after first pass, and requires at least one improvement cycle. Provides clear when and when-not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
loop_regressionA
Run the full regression loop: test all cases → score → improve → repeat.
Stops when BOTH conditions are met:
Overall pass rate >= threshold
Every individual test case score >= threshold Or when max iterations are exhausted.
Loop:
Run all test cases, score responses, call post_test_result for each.
Call get_regression_status.
If pass rate >= threshold AND all individual scores >= threshold → SUCCESS.
If iteration >= maxIterations → EXHAUSTED. Report best result.
Analyse failures, write improved prompt, call post_prompt_suggestion + apply_suggestion.
Go to 1.
After the loop: call pull_ui_history and save results locally.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| threshold | No | Pass score 0–100 (default: workspace goal or 70) | |
| maxIterations | No | Max iterations (default: workspace goal or 5) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It details the entire loop process, including calling other tools (post_test_result, get_regression_status, etc.) and end actions (pull_ui_history). It doesn't explicitly state whether the tool is destructive or requires specific permissions, but the step-by-step disclosure is thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured as a numbered list, front-loaded with the core loop summary. Some procedural details could be shortened, but the structure aids readability for an AI agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multi-step loop with multiple sub-calls and conditions) and the lack of output schema/annotations, the description is comprehensive. It covers all steps, stop conditions, and post-loop actions, enabling an agent to correctly invoke the tool and integrate with sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67% (2 of 3 parameters have descriptions). The description adds context on how 'threshold' and 'maxIterations' are used in the loop conditions, but doesn't add new semantic meaning beyond the schema. WorkspaceId lacks description in schema, and description doesn't compensate. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Run the full regression loop: test all cases → score → improve → repeat.' It uses a specific verb ('run') and resource ('full regression loop'), and distinguishes from siblings like 'run_regression_testsuite' (single run) and 'loop_optimization' (different optimization loop).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when the loop stops (BOTH conditions met or max iterations). It doesn't explicitly state when to use this tool vs alternatives like 'run_regression_testsuite' or 'loop_optimization', but the detailed loop behavior implies it's for full automated regression improvement. Lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
post_prompt_suggestionA
Queue a revised system prompt for the user to review.
Always explain in reasoning:
which test cases were failing and why
what specific change you made to the prompt
why you expect this change to fix those cases
In gated mode (start_optimization_session): user reviews in UI, then approves or rejects. In loop mode (loop_optimization, loop_regression): call apply_suggestion immediately after.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| prompt | Yes | The full revised system prompt | |
| reasoning | Yes | What changed and why | |
| expectedGain | No | e.g. "fixes failing classify queries by adding format instructions" |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description fully discloses behavioral traits: it queues for review, requires reasoning, and differentiates behavior between gated and loop modes. This is comprehensive for a queueing operation, though it lacks details on error states or auth requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with four sentences, front-loading the core purpose. Every sentence adds essential information, and there is no fluff. The structure is logical: purpose, reasoning requirement, mode-specific instructions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and moderate complexity (two modes, queueing), the description covers the key behaviors and flow. It could mention what happens in gated mode if user rejects, but for an agent the provided information is sufficient to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75% (3 of 4 parameters have descriptions). The description adds value by specifying the expected structure of the 'reasoning' parameter (test cases, change, expectation). For other parameters, it doesn't add much beyond the schema, but the overall guidance compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: queueing a revised system prompt for user review. It uses a specific verb 'queue' and resource 'prompt', and distinguishes from siblings like 'apply_suggestion' and 'set_system_prompt' by mentioning the two operational modes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use it: in gated mode (start_optimization_session) vs. loop mode (loop_optimization, loop_regression). It also tells the agent to call 'apply_suggestion' immediately after in loop mode, and mandates reasoning structure, which helps the agent understand proper usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
post_test_resultA
Store the scored result of one test case run.
Call after you run a test case against the model and evaluate the response. This makes the result visible in the UI and is used by get_regression_status.
Score 0–100 using this scale: 90–100: Correct, complete, well-structured — exceeds target. 70–89: Correct and complete — minor gaps or style issues. 50–69: Partially correct — key points present but missing important details. 30–49: Mostly wrong — one or two relevant points but fundamentally off. 0–29: Completely wrong, off-topic, or refused.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| testCaseId | Yes | ID from get_workspace_state testCases | |
| response | Yes | The full model response | |
| score | Yes | Quality score 0–100 | |
| reasoning | Yes | Why this score — what worked, what failed | |
| model | Yes | Model used, e.g. claude-haiku-4-5-20251001 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses that the tool stores a result, makes it visible in the UI, and is used for regression status. It includes a detailed 0-100 scoring scale, which adds behavioral context beyond a simple store operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded with purpose and usage. The scoring scale is integrated efficiently without unnecessary words. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (store operation, 6 required params, no output schema), the description covers purpose, usage, and scoring. It is nearly complete, though it could mention idempotency or overwriting behavior, but that is not essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 83% (5 of 6 params have descriptions). The description adds value by explaining the scoring scale for the 'score' parameter and specifying that 'testCaseId' comes from 'get_workspace_state testCases', enhancing understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'store' and the resource 'scored result of one test case run'. It also distinguishes from sibling tools like get_regression_status and add_test_cases by specifying it is for storing a single test result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Call after you run a test case against the model and evaluate the response.' and notes that the result is used by get_regression_status, providing clear context. It does not explicitly exclude alternatives, but the guidance is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pull_ui_historyA
Fetch all history entries the UI has pushed to this workspace.
The UI auto-pushes after every session summary ("Summarize & new") and every regression run. This gives you a record of what the user did in the UI between agent calls.
ALWAYS save the response to a local file: prompt-lab/workspaces//_ui_history.json
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It correctly indicates a read operation ('Fetch') and provides context about UI push events. However, it does not disclose potential behavioral traits such as response format, pagination, rate limits, or required permissions, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively short and front-loaded with the core purpose. Including an actionable instruction to save to a local file adds value but slightly deviates from describing the tool itself. Overall, it is concise with no unnecessary repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple schema (one parameter, no output schema), the description provides helpful context about UI push events but lacks details about the response structure or potential errors. It is marginally complete for a fetch tool but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter (workspaceId) with 0% schema description coverage. The description does not explain the parameter beyond its appearance in the file path instruction. No details about expected format or valid values are provided, so the parameter semantics are weak.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch all history entries the UI has pushed to this workspace', using a specific verb (Fetch) and resource (UI history). It distinguishes from sibling tools which include actions like add_test_cases or apply_suggestion, as it uniquely retrieves UI interaction records.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when the UI pushes history (after session summaries and regression runs), giving context for when to use the tool. It implies the tool is for reviewing user actions between agent calls, but does not explicitly state when not to use it or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
register_api_keyA
Register a provider API key for this workspace.
Use this when you need to register a key that was not passed to start_web_app. Specify provider explicitly: anthropic | google | openai.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| apiKey | Yes | ||
| provider | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only says 'Register' without elaborating on effects (e.g., overwrite, validation, auth requirements). The behavioral disclosure is minimal and insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, front-loaded with the main action, and every sentence adds necessary information. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation with no output schema, the description provides the core purpose and usage context. However, it lacks details on success/error responses and whether keys can be overwritten, leaving some gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It adds value for 'provider' by listing enum options, but 'workspaceId' and 'apiKey' are left entirely implicit. Only 1 of 3 parameters receives meaningful description, which is below the bar.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Register a provider API key for this workspace.' It uses a specific verb (register) and resource (API key), and distinguishes itself from sibling 'start_web_app' by noting this is for keys not passed there.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use this when you need to register a key that was not passed to start_web_app.' It also instructs to specify the provider explicitly with the options. No exclusions or when-not-to-use are given, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_regression_testsuiteA
Run all test cases against the current system prompt. Single pass — does not auto-improve.
Use this to verify an already-good prompt still passes all test cases. For automatic improvement loops, use loop_regression.
Steps to follow after this call:
Run each test case against the model, score the response, call post_test_result.
Call get_regression_status to see pass/fail summary.
Optionally: post_prompt_suggestion with an improvement (user reviews).
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| threshold | No | Pass score 0–100 (default 70) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavior: 'Single pass — does not auto-improve.' No annotations exist, so the description carries the burden. It omits details like whether the tool modifies state or requires permissions, but the single-pass constraint is well communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose, then usage, then steps. Three concise sentences plus bullet-style steps. Efficient, though the steps could be slightly streamlined. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 simple params, no output schema, and no nested objects, the description covers the core activity well and references sibling tools. However, it does not specify the tool's return value or whether it returns a session ID for later status checks, leaving some ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (only threshold has a description). The description adds no additional meaning for either parameter. workspaceId is undocumented in both schema and description, and threshold's schema description is already present. No value added beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Run all test cases against the current system prompt' – a specific verb+resource. It distinguishes itself from the sibling tool loop_regression by noting this is a single pass and not auto-improving.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use: 'Use this to verify an already-good prompt still passes all test cases.' And when not: 'For automatic improvement loops, use loop_regression.' Provides a clear step-by-step workflow after calling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
save_system_prompt_templateA
Save a named system prompt template so it appears in the UI "Load template…" dropdown.
Call at session startup for every .txt file in prompt-lab/system-prompts/: save_system_prompt_template(name=, content=)
Also call after a successful optimization loop to preserve the best prompt found.
Templates persist in Redis. Saving with the same name replaces the previous version.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Template name (shown in UI dropdown) | |
| content | Yes | System prompt text |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, description discloses persistence in Redis and overwrite behavior on same name. This is sufficient for a simple write operation. Could mention error handling or idempotency implications, but overall transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Six sentences, each adding unique value: purpose, usage patterns, persistence, replace behavior. No fluff or repetition. Well-structured for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has simple params, no output schema, but description covers purpose, usage, and persistence. Could mention return value or error scenarios, but missing info is minor given tool simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. Description adds moderate value by linking name to UI dropdown and providing example usage, but does not introduce new semantic details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Save' and resource 'system prompt template' with specific UI effect ('appears in the UI Load template… dropdown'). Also gives concrete use cases (session startup, after optimization loop), leaving no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call (session startup for .txt files, after optimization loop) and provides example call pattern. Lacks explicit 'when not to use' or alternatives, but usage context is clear and differentiated from siblings like save_template or set_system_prompt.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
save_templateA
Save a named test suite template so it appears in the UI "Load test suite…" dropdown.
Call at session startup for every .json file in prompt-lab/templates/: save_template(name=<file.name>, testCases=<file.testCases>)
Template format (matches what the UI exports as a downloadable JSON): { "name": "suite-name", "savedAt": "...", "testCases": [{ "label"?, "query", "targetAnswer"?, "passThreshold"?, "queryType"? }] }
Templates persist in Redis. Saving with the same name replaces the previous version.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Template name (shown in UI dropdown) | |
| testCases | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses persistence in Redis and that saving with same name replaces previous version. The template format matches UI export. No annotations provided so description carries burden adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, well-structured with purpose, usage pattern, and format details. No redundant sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, format, and persistence thoroughly. No output schema, but return value is implied. Lacks error handling, but adequate for context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description adds context beyond schema: explains testCases format matches UI export and gives usage example. Schema coverage is 50%, but description compensates with meaningful guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool saves a named test suite template for UI visibility, distinguishing it from siblings like save_system_prompt_template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage pattern: call at session startup for each .json file in a specific directory. Does not explicitly mention alternatives or when not to use, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_system_promptA
Set or update the system prompt for this workspace.
Does NOT increment the iteration counter — use this for initial setup or manual overrides. To record an optimization step, use apply_suggestion.
Load the current prompt from current.json or ask the user before overwriting.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| systemPrompt | Yes | The full system prompt text |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the transparency burden. It discloses the key behavioral trait of not incrementing the iteration counter. However, it does not mention other side effects like whether the change triggers any cascading state updates or requires specific permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at four sentences, front-loaded with purpose. It wastes no words, but could be slightly more structured with explicit parameter guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, no output schema) and the presence of sibling tools like apply_suggestion, the description adequately covers the core behavior and use case. It lacks details on return value or error conditions, but these are partially mitigated by the tool's straightforward nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (only systemPrompt has a description). The tool description does not add meaning for workspaceId, leaving it unexplained. The phrase 'for this workspace' provides implicit context, but insufficient compensation for the missing schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Set or update the system prompt for this workspace', using a specific verb and resource. It distinguishes from sibling tools by explicitly noting it does not increment the iteration counter, unlike apply_suggestion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'use this for initial setup or manual overrides' and directs to apply_suggestion for optimization steps. It also advises to load the current prompt or ask the user before overwriting.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_test_modelB
Switch the model used for test cases in this workspace. Updates the UI model selector.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| model | Yes | Model ID, e.g. gemini-2.5-flash-lite or claude-haiku-4-5-20251001 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions switching the model and updating the UI, but does not disclose persistence, side effects, required permissions, or what happens to existing test cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences, no unnecessary words, and the main purpose is front-loaded in the first sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 required params, no output schema), the description covers the basic action but lacks contextual details such as validation that the model must be from available models (implied by sibling 'list_models'). It is adequate but could be more helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (only 'model' has a description with examples). The tool description adds no information about the 'workspaceId' parameter, which remains undocumented. The description does not compensate for the missing schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool switches the model for test cases and updates the UI. The verb 'Switch' is specific to the resource 'test model', and it distinguishes itself from sibling tools like 'list_models' which only lists models.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any context about prerequisites or appropriate scenarios. The description simply states what it does without specifying usage conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_optimization_sessionA
Run one optimization pass on an existing workspace.
Prerequisites (do these first):
start_web_app → workspace URL + ID
set_system_prompt → starting prompt
add_test_cases → at least one case with targetAnswer
What this does:
Read system prompt and test cases from get_workspace_state.
Run each test case against the model (write + execute a temp Node.js script).
Score each response vs targetAnswer (LLM-as-judge, 0–100), call post_test_result.
Analyse failures, write improved prompt, call post_prompt_suggestion.
Present the suggestion — do NOT auto-apply. User reviews in the UI.
This is one iteration. After the user approves or rejects the suggestion, call start_optimization_session again or switch to loop_optimization.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | Yes | ||
| threshold | No | Pass score 0–100 (default 70) | |
| maxIterations | No | Goal iterations for tracking (default 5) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description details the entire internal process: reading state, running tests, scoring, analysing failures, writing prompts. It does not mention error handling or side effects on failure, but covers the main behavioral flow.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with prerequisites and numbered steps, front-loading the purpose. It is thorough but not excessively long; every sentence contributes value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity and no output schema, the description fully explains prerequisites, internal steps, and follow-up actions. It references sibling tools and integrates the workflow context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Two of three parameters have schema descriptions (67% coverage). The description adds default values for threshold (70) and maxIterations (5), enhancing understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Run one optimization pass' on 'an existing workspace'. It distinguishes from sibling tool 'loop_optimization' by noting this is one iteration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly lists three prerequisites (do these first) and provides a clear after-action: user reviews, then call again or switch to loop_optimization. Also warns 'do NOT auto-apply'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_web_appA
Open the prompt lab web UI. Returns the URL for the browser.
If no workspaceId is given, creates a new empty workspace. If workspaceId is given, connects to that workspace (must exist).
Always pass your environment API keys — they enable the UI Send button. Check each env var and pass it if set: anthropicApiKey: process.env.ANTHROPIC_API_KEY geminiApiKey: process.env.GEMINI_API_KEY openaiApiKey: process.env.OPENAI_API_KEY
After returning the URL, call list_models to register available models. Then set_system_prompt and add_test_cases before running optimization.
| Name | Required | Description | Default |
|---|---|---|---|
| workspaceId | No | Existing workspace ID. Omit to create a new workspace. | |
| label | No | Label for new workspace (ignored if workspaceId given) | |
| anthropicApiKey | No | ||
| geminiApiKey | No | ||
| openaiApiKey | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses workspace creation/connection behavior, URL return, and API key requirement. However, it doesn't mention if the tool modifies state beyond workspace creation or any rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with bullet-like list for API keys. Every sentence adds value. Slightly verbose in the step-by-step guidance, but overall efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and no output schema, description covers creation vs connection, API key usage, and subsequent workflow steps. Return value described as 'URL for the browser' suffices. Could be more complete on potential side effects, but adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 40% with descriptions for workspaceId and label. Description extensively covers the three API key parameters: explains they enable UI Send button, how to check env vars, and which keys to pass. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Open the prompt lab web UI' with a specific verb and resource. It distinguishes itself from siblings by focusing on UI launch, while sibling tools like list_models, set_system_prompt are for subsequent steps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use: create new workspace vs connect existing. Includes post-invocation steps: 'After returning the URL, call list_models... then set_system_prompt and add_test_cases'. Also instructs to always pass API keys, explaining why.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
TDQS
Most tools have distinct purposes, but pairs like loop_optimization/loop_regression and start_optimization_session/loop_optimization could be confused. Descriptions help differentiate, but the distinction between single-pass and loop modes might not be immediately clear.
All tool names follow a consistent verb_noun pattern in snake_case (e.g., add_test_cases, set_system_prompt). No mixing of conventions or ambiguous names.
19 tools cover the full prompt lab workflow without excess. Each tool addresses a distinct step in setup, testing, optimization, and management, making the count appropriate for the server's scope.
The tool set covers the main workflow end-to-end, but lacks granular control like editing or deleting individual test cases or canceling loops. Minor gaps, but core functionality is present.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Manage, version, and publish LLM prompts with blocks, variables, and evaluations.
Live SEO workflow tools for Claude Code, Codex, and AI agents.
Test and compare prompts across any AI provider. Bring your own keys.
Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceProvides a suite of tools like agent orchestration and token optimization for ClaudeMIT
- AlicenseAqualityBmaintenanceProvides comprehensive session management for Claude Code with automatic initialization/cleanup, quality checkpoints, and local conversation memory with semantic search for capturing learnings across coding sessions.62BSD 3-Clause
- AlicenseAqualityDmaintenanceAn MCP server that uses Claude 3.5 Sonnet to transform ordinary prompts into structured, professionally engineered instructions for any LLM. It enhances AI interactions by adding context, requirements, and structural clarity to raw user inputs.13MIT
- AlicenseCqualityCmaintenanceEnables AI-powered automated testing, security scanning, code review, and maintenance tasks directly within Claude Code or desktop.124MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/jurek-f/prompt-lab-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server