The livekit-agent-simulator server lets you simulate AI caller interactions against a LiveKit voice agent, record detailed behavioral logs, and analyze/compare results — without touching the agent's code.
init_project — Scaffold a .agent-sim/ directory (config + smoke scenario) in your target repo and add it to .gitignore.
list_scenarios — List all available scenarios (from .agent-sim/scenarios/*.jsonl) with their IDs, tags, and validity status.
validate_scenario — Validate a scenario file against its schema, checking for a required Persona brief and linting the PassCriteria.
export_scenario — Export a parsed scenario's full details (Persona, execution parameters, PassCriteria) as JSON.
run_scenario / execute_scenario — Run a simulation for a given scenario (the latter validates first), returning a run_id for tracking.
execute_scenarios — Execute multiple scenarios at once, optionally filtered by tag (e.g. smoke) or specific IDs.
get_run_status — Poll a simulation run's current state (running / done / failed), turn count, and duration.
get_run_log — Read the raw events.jsonl log for a run, with filtering by event kind (supports wildcards like tool.*), turn number, source, and time offset.
get_run_report — Retrieve a full report including summary, LLM judge verdict, suspicious turns, and paths to timeline/event files.
compare_runs — Diff two simulation runs side-by-side across duration, turn counts, tool errors, turn-taking percentiles, and judge verdicts.
list_runs — Browse run history (newest first) from SQLite, optionally filtered by scenario ID.
All tools operate on a project_root (the repo under test) and assume the target agent is already deployed.
Provides the AI simulated caller persona using Google's Gemini Live API for natural conversation testing.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@livekit-agent-simulator run the smoke-hello scenario"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Dial any LiveKit voice agent with an AI simulated caller — WebRTC room, inbound SIP, or outbound SIP — and keep a full forensic log.
Standalone MCP server + CLI (lks). Black-box testing: no imports from the agent under test, no edits to its code or .env.
curl -fsSL "https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/install.sh?$(date +%s)" \
| bash -s -- --verifyPaste into Claude Code, Cursor, Codex, AmpCode, Windsurf, or any coding agent from the repo you want to test:
Install and configure livekit-agent-simulator (CLI: lks) for this project by following the instructions here:
https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/docs/guide/installation.md
Target project root is this workspace. Use absolute --root paths. Install the portable CLI if missing, run lks init, help fill .agent-sim/config.yaml from my local env or ask me for LiveKit + active caller provider key (Gemini Live or OpenAI Realtime) + agent_name, ensure .agent-sim is gitignored, run preflight, and stop before execute if the voice agent worker is not running. Do not edit agent application source outside .agent-sim/.Same idea, one line:
Install and configure livekit-agent-simulator by following: https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/docs/guide/installation.mdVoice agents fail in ways unit tests never see:
Gap | What you miss |
No real caller | Scripts that never interrupt, stall, or switch language |
Chat-only evals | No room events, audio timing, or tool spans |
Manual QA calls | Not CI-reproducible, no structured PassCriteria |
Agent-coupled harnesses | Tests break when you refactor the worker |
livekit-agent-simulator drives an AI simulated caller — Gemini Live or OpenAI Realtime (per simulator.provider) — from scenario JSONL over one of three transport modes (Caller.mode), observes transcripts / tools / flow / room events, and writes a timestamped report you can play back.
Surface | What you get |
| init → preflight → execute → report → web |
MCP server | Same ops for Claude Code, Cursor, Codex, … |
Transport modes |
|
Reports |
|
Judge | Optional LLM PassCriteria scoring |
Feature | What it does |
Black-box dispatch | Only needs |
5 transport modes | WebRTC · inbound SIP · outbound human pickup · outbound sim callee · agent_dials |
Scenario JSONL | Persona, Caller, Telephony, Execute, Script, PassCriteria, Dispatch |
Forensic log | Per-turn events in SQLite + |
Report player | Local web UI: audio + transcript sync |
CLI ↔ MCP parity | One |
Portable packs | Download installer; no uv/pip required for users |
# Install once
curl -fsSL "https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/install.sh?$(date +%s)" \
| bash -s -- --verify
# In the repo you want to test (agent worker must already be running)
lks init --root /path/to/target
# edit /path/to/target/.agent-sim/config.yaml (LiveKit + active provider keys, agent_name)
lks preflight --root /path/to/target
lks execute smoke-hello --root /path/to/target
lks report <run-id> --root /path/to/target
lks web --root /path/to/target # Ctrl+C to stopRelated MCP server: Bolna MCP Server
The agent under test is a black box.
We never import or patch target application code. Dispatch metadata is opaque JSON.
Generic core, target-owned config.
Language, timezone, topics, and business strings belong in the target’s .agent-sim/ — not hardcoded in the package.
One ops layer for CLI and MCP.execute_* validates then runs. No “run vs execute” forks.
Forensics over vibes.
Every run produces structured events you can compare, log, and play back.
CI-friendly gates.
Hard fails on status / assert / script; optional strict judge for softer LLM scoring.
1. Read <target>/.agent-sim/config.yaml
2. Pick SimLeg from scenario Caller.mode (webrtc_sim | inbound_sip | outbound_human_pickup | outbound_sim_callee | agent_dials)
3. Connect leg → LiveKit room(s) / SIP hairpin as needed; the active caller provider (Gemini Live / OpenAI Realtime) stays WebRTC in the sim room
4. Bridge audio; observe transcripts, tools, timing, interruptions
5. Write reports/<run-id>/ + runs.sqlite
6. Optional LLM judge vs PassCriteria Caller.mode (scenario)
┌───────────────┬────────────────┬──────────────────┬────────────────────┐
│ webrtc_sim │ inbound_sip │ outbound_human_pickup │ outbound_sim_callee│
│ room audio │ sim dials DID │ human answers → │ Gemini SIP callee │
│ │ │ Gemini colocated│ (2-room hairpin) │
└───────┬───────┴────────┬───────┴────────┬─────────┴─────────┬──────────┘
│ │ │ │
└────────────────┼────────────────┼───────────────────┘
▼
┌──────────────────────────┐
│ Sim caller persona │
│ (Gemini Live / OpenAI │
│ Realtime) + LiveKit │
│ agent (black box) │
└────────────┬─────────────┘
│ observe
▼
reports/<run-id>/ · runs.sqlite · judgeMode details and config: docs/telephony.md. Templates: inbound-caller-sim, outbound-human-pickup, outbound-callee-sim.
Approach | Real room | AI caller | Forensic log | MCP | Black-box |
Manual phone QA | ✅ | ❌ | ❌ | ❌ | ✅ |
Unit / mock STT | ❌ | ❌ | Partial | ❌ | ❌ |
In-repo agent tests | ⚠️ | ⚠️ | Varies | ❌ | Often coupled |
lks | ✅ LiveKit | ✅ Gemini Live / OpenAI Realtime | ✅ Full | ✅ | ✅ |
When to use lks:
Regression suites for LiveKit voice agents
Agent-driven CI / coding-agent workflows (MCP)
Debugging turn-taking, tools, and silence without reading agent source
When it might not be ideal:
Pure text chatbots with no LiveKit room
Offline environments without LiveKit + an active caller provider API (Gemini Live or OpenAI Realtime)
Download only — no uv/pip/build on your machine. CI ships a portable pack (embedded Python + deps + report player).
# macOS / Linux
curl -fsSL "https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/install.sh?$(date +%s)" \
| bash -s -- --verify# Windows PowerShell
irm "https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/install.ps1" -OutFile "$env:TEMP\lks-install.ps1"
powershell -NoProfile -ExecutionPolicy Bypass -File "$env:TEMP\lks-install.ps1" -VerifyAlso available from a release asset:
curl -fsSL "https://github.com/quangdang46/livekit_agent_simulator/releases/download/v0.1.0/install.sh" \
| bash -s -- --verifyFlag | Purpose |
| Checksum verification |
| Pin release tag |
| Skip MCP registration into coding tools |
| Remove install |
By default the installer registers the MCP server livekit-agent-simulator (lks mcp) into detected tools: Claude Code, Cursor, Cline, Windsurf, VS Code Copilot, Gemini CLI, Amazon Q, OpenCode, Codex, Warp.
Agent-oriented install playbook (long form): docs/guide/installation.md
Raw URL for paste into agents:https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/docs/guide/installation.md
git clone https://github.com/quangdang46/livekit_agent_simulator.git
cd livekit-agent-simulator
uv sync --extra dev
uv run lks --helpRequires Python 3.10–3.13.
Users never build this — CI packs web/dist into the wheel as web_static. Source: web/.
pnpm --dir web install
pnpm --dir web build # → web/dist/ (attached by Hatch on uv build)
pnpm --dir web dev # HMR; proxy /api + /runs → lks web :8765See web/README.md.
lksr is the in-progress Rust full port of lks (single static-ish binary, no Python runtime). Source: src/livekit_agent_simulator_rust/. Same 22 CLI commands + MCP server + report player.
Status: data-plane ops (scenarios/validate/export/cues/plugins/runs/report/compare/optimize) and offline gates are at parity; the live-run path still has known gaps vs Python (script speak cues reach the room via the OpenAI bridge only, judge defaults to skip without judge.base_url, verify plugins need a build with --features python-plugins). For CI-critical runs use the Python lks; try lksr for quick local checks.
curl -fsSL "https://github.com/quangdang46/livekit_agent_simulator/raw/main/install-rust.sh" | bash -s -- --verify # once a v*-rust release exists
cd src/livekit_agent_simulator_rust && cargo build -p lks && cargo test --workspace # from source# Agent worker must be running and registered with LiveKit
lks guide
lks init --root /path/to/target
# fill .agent-sim/config.yaml
lks preflight --root /path/to/target
lks scenario-init smoke-hello --root /path/to/target # if needed
lks validate smoke-hello --root /path/to/target
lks execute smoke-hello --root /path/to/target
lks runs --root /path/to/target
lks report <run-id> --root /path/to/target
lks web --root /path/to/targetsmoke-hello)apiVersion: agent-sim/v1
kind: Scenario
metadata:
id: smoke-hello
locale: en-US
tags: [smoke]
persona:
name: Alex
brief: First-time caller; confirm you reached the right place, then end politely.
goals:
- Hear the agent
- Say you will call back
style: polite, brief
execute:
max_turns: 2
timeout_s: 90
first_speaker: user
pass_criteria:
criteria:
- The agent responded to the caller
- The agent responded in the caller's languageOptional multi-judge PassCriteria: judges[] + mode (all | majority | any). Assert highlights (tool_order, constraint_respected, recovery/latency): lks guide.
Full-line # comments in scaffolded YAML are guides — runtime ignores them. Legacy *.jsonl scenarios are still read.
Target-only data lives under <target>/.agent-sim/ (gitignored). Created by init.
Section | Required | Purpose |
| yes |
|
| yes | Server API credentials |
| yes | Must match worker dispatch name |
| no | Default opaque JSON string for all runs |
| yes | Key of the active caller provider ( |
| no | Caller brain: |
| no | Provider-neutral voice bag; defaults flash-live, Puck, |
| no | Named caller profiles — switch provider without editing the file |
| no | If set + PassCriteria → post-run LLM judge |
| no (default | Local stereo WAV (L=sim, R=agent); no Egress |
| no | Empty = all topics |
| no | Map data payloads → tool start/end/error |
See template: templates/config.yaml. Consumer-specific wiring: docs/portability.md.
--profileTo A/B test the same scenario against Gemini Live vs OpenAI Realtime (or
any set of provider/voice combos) without editing config.yaml between runs,
define named profiles under simulator.profiles: and select one with
--profile <name> on execute / execute-all / preflight.
simulator:
# legacy flat block = fallback (used when no --profile flag and no default profile)
provider: google
mode: realtime
api_key: "AQ.Ab8..." # Gemini Live key
# named profiles — switch with --profile <name>
profiles:
gemini:
default: true # auto-selected when no --profile flag
provider: google
api_key: "AQ.Ab8..." # Gemini Live key
openai:
provider: openai
api_key: "sk-..." # OpenAI key
voice:
model: "gpt-realtime-2.1-mini"
voice: "marin"lks execute smoke-hello # `gemini` (marked default: true)
lks execute smoke-hello --profile gemini # Gemini Live caller
lks execute smoke-hello --profile openai # OpenAI Realtime callerSelection (--profile absent): if exactly one profile has
default: true, it is used; otherwise the legacy flat simulator: block runs.
2+ profiles marked default: true is an error (no "first wins"). If profiles:
exist with no default and no flat-block credentials, config loading
errors loudly (no silent fallback). --profile <name> always wins regardless
of which profile is default. A missing profile name fails loudly (lists
available profiles) — no silent fallback. Profile names are case-sensitive.
Precedence: profile field → flat simulator: field → built-in default. A
profile inherits unspecified fields (voice, language, mode) from the flat
block, so openai above only overrides provider + api_key + voice, and
keeps mode: realtime. Presence of profiles: never changes what runs when
neither --profile nor a default: true profile is present — that is the flat
block (backward compatible).
⚠️ Gemini caller model note (observed 2026-08):
gemini-3.1-flash-live-preview— the historical default — is a preview model with known instability as the simulated caller: transient mid-call WebSocket drops (APIError 1006 / 1008, end reasongemini_socket_drop) in ~2/15 real runs, plus LiveKit-documented limits (send_client_contentrejected after the first model turn,update_instructions/generate_replyunsupported). If you see calls ending withgemini_socket_drop, switch the caller model to a stable release, e.g.:simulator: voice: model: "gemini-2.5-flash-native-audio-preview-12-2025" # or gemini-live-2.5-flash-native-audio (GA)Verified:
gemini-2.5-flash-native-audio-preview-12-2025connects and talks as the caller with 0 socket drops across real runs (the-12-2025date suffix is required —gemini-2.5-flash-native-audio-previewalone returnsAPI_KEY_INVALID).
CLI and MCP share the same public ops (ops.py). Prefer execute (validate then run).
CLI | MCP tool | Purpose |
|
| Scaffold |
|
| Setup/ops guide (markdown) |
|
| Local report player |
|
| Config + LiveKit connectivity |
|
| List |
|
| Verify plugins |
|
| Built-in + local PCM cues |
|
| Schema + lint |
|
| Parsed scenario JSON |
|
| Scaffold JSONL with |
|
| Validate then run one scenario |
|
| Batch (ids / tag) |
|
| In-memory scenario dict |
|
| SQLite run status |
|
| Filtered |
|
| Summary + verdict + paths |
|
| Diff two runs; |
|
| Run history |
| — | REST API (JSON over HTTP; same ops as CLI/MCP) |
|
| Offline persona-prompt optimizer (live benchmark loop) → |
| — | Start MCP server (stdio) |
lks execute smoke-hello --root /path/to/target
lks execute-all --tag smoke --root /path/to/target
lks serve --root /path/to/target # REST API on :8787 (same ops as CLI/MCP)
lks log <run-id> --root /path/to/target
lks compare <run-a> <run-b> --root /path/to/target
lks compare <baseline> <candidate> --baseline --root /path/to/target
lks optimize scen-a,scen-b --held-out scen-c --root /path/to/target # → optimized/<name>/
lks execute scen-a --optimized <name> --root /path/to/target # apply the winner
lks web --port 8765 --root /path/to/targetEvery MCP tool needs project_root except guide.
List/table-shaped commands (scenarios, runs, plugins, cues, validate,
preflight, execute, execute-all, execute-dict, compare, status,
report, log) print a human-readable rich table by default. Add
--json to any of them for the raw machine-readable payload — the same bytes
the MCP tools return. Single-dict commands (init, export, convert,
scenario-init, scenario-from-run, guide, web) always print JSON.
lks scenarios # human table
lks scenarios --json # raw JSON for scripts / CI / agents
lks execute-all --json | jq '.suite' # pipe JSON to jqAgents & CI: use --json — the default table is for humans.
Installer writes this when tools are detected. Manual Cursor:
{
"mcpServers": {
"livekit-agent-simulator": {
"command": "lks",
"args": ["mcp"],
"env": {}
}
}
}Dev checkout (package not installed globally):
{
"mcpServers": {
"livekit-agent-simulator": {
"command": "uv",
"args": ["run", "--directory", "/abs/path/livekit-agent-simulator", "lks", "mcp"]
}
}
}Equivalent one-shot entry: lks-mcp (same process as lks mcp).
src/livekit_agent_simulator/
├── cli.py / mcp_server.py # thin surfaces
├── ops.py # shared public ops
├── run_orchestrator.py # room lifecycle + run
├── scenario.py # JSONL parse / validate
├── config.py # .agent-sim/config.yaml
├── preflight.py
├── asserts.py / suite.py # CI gates
├── callers/ # Live caller (gemini / openai)
├── livekit/ # room, dispatch, observe
├── audio/ · script/ · plugins/
└── web/ # report player serverLayer | Role |
Target | Config, scenarios, reports, local plugins/cues |
Package | Scaffold defaults + built-in cues |
LiveKit | Room, dispatch, data topics, transcription |
Caller provider (Gemini Live / OpenAI Realtime) | Simulated caller voice (+ optional judge) |
Workflow | Trigger | What it does |
PR / push → | web UI build, | |
tag | pytest → wheel → portable packs (win/linux/mac) → GitHub Release |
# Local check
uv sync --extra dev
pnpm --dir web build
uv run pytest -q
# Release (pre-1.0 may force-retag a single 0.1.0)
git tag v0.1.0
git push origin v0.1.0preflight fails connectivitylks preflight --root /path/to/target
# Confirm livekit.url / api_key / api_secret and that the project is reachable.
# Skip API check while editing config:
lks preflight --no-connectivity --root /path/to/targetWorker process must be running and registered with the same livekit.agent_name.
Increase livekit.agent_join_timeout_ms if cold start is slow.
Check dispatch metadata is valid JSON string if your worker requires it.
Set simulator.api_key in .agent-sim/config.yaml for the active simulator.provider (google → Gemini Live, openai → OpenAI Realtime).
With observe.record_audio enabled (default true): reports/<run-id>/conversation.wav
lks web --root /path/to/targetlks mcp # must be what the host launches
# or reinstall without --no-mcp
curl -fsSL "https://raw.githubusercontent.com/quangdang46/livekit_agent_simulator/main/install.sh?$(date +%s)" \
| bash -s -- --verifylks validate my-case --root /path/to/target
lks scenario-init my-case --root /path/to/target # fresh scaffold with // guidesNot an agent framework — it tests agents; it does not implement business tools
Not offline-first — needs LiveKit + an active caller-provider API (Gemini Live or OpenAI Realtime)
Not a load generator — one simulated caller per run (batch via execute-all)
Capability | Current state | Notes |
Black-box dispatch | ✅ | Opaque metadata only |
Multi-caller rooms | ❌ | Single sim participant |
Caller backends | ✅ | Gemini Live and OpenAI Realtime are supported paths (per |
Pixel-perfect ASR scoring | ❌ | Use PassCriteria + judge / asserts |
Secrets in config | ⚠️ Paste in gitignored YAML | Do not commit |
Only scaffolds .agent-sim/ (gitignored). It does not edit agent source.
Same ops. Use CLI in terminals/CI; MCP inside coding agents. Prefer execute_* over ad-hoc run paths.
livekit.dispatch_metadata or scenario Dispatch.spec.metadata as an opaque JSON string. Core does not parse consumer keys. See docs/portability.md.
Yes — Assert.spec.tools, tool_order (required tool.start subsequence), observe.tool_event_patterns, Script/assert plugins, and/or PassCriteria + judge. See docs/plugins.md and lks guide.
<target>/.agent-sim/reports/<run-id>/ plus runs.sqlite under .agent-sim/.
No — lks web serves the prebuilt player from the install pack. Maintainers build from web/.
Doc | When |
Rules for AI agents working on this package | |
First end-to-end run | |
Consumer dispatch / observe setup | |
Verify plugins + Python API | |
SIP modes + outbound_sim_callee preflight | |
Barge / backchannel / noise authoring | |
| On-demand setup/ops guide (Assert, compare --baseline, PassCriteria) |
Please don't take this the wrong way, but I do not accept outside contributions for any of my projects. I simply don't have the mental bandwidth to review anything, and it's my name on the thing, so I'm responsible for any problems it causes; thus, the risk-reward is highly asymmetric from my perspective. I'd also have to worry about other "stakeholders," which seems unwise for tools I mostly make for myself for free. Feel free to submit issues, and even PRs if you want to illustrate a proposed fix, but know I won't merge them directly. Instead, I'll have Claude or Codex review submissions via gh and independently decide whether and how to address them. Bug reports in particular are welcome. Sorry if this offends, but I want to avoid wasted time and hurt feelings. I understand this isn't in sync with the prevailing open-source ethos that seeks community contributions, but it's the only way I can move at this velocity and keep my sanity.
Black-box LiveKit agent tests. Real rooms. Forensic reports.
Diff two runs: duration, turns, tool errors, turn-taking percentiles, verdicts.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id_a | Yes | ||
| run_id_b | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It lists comparison metrics but does not disclose side effects, authorization needs, rate limits, or whether it's read-only. The behavioral profile is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the action and lists key comparison points. However, it lacks any structural organization like sections or bullet points, which could improve scanability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 required parameters, 0% schema coverage, no annotations), the description fails to compensate. It does not explain how the diff is presented (despite an output schema existing), and parameter meaning is absent, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description does not explain any parameter (e.g., what 'project_root' is or that 'run_id_a' and 'run_id_b' must be valid). The parameter semantics are completely absent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('diff') and the specific aspects compared (duration, turns, tool errors, etc.), and the tool name 'compare_runs' is distinct from siblings like 'list_runs' or 'get_run_report'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for comparing two runs but does not explicitly state when to use or when to avoid, nor does it mention alternatives such as 'get_run_report' for single-run analysis. Usage is inferred rather than guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate then execute one scenario from .agent-sim/scenarios/*.jsonl. Returns validation + run result.
| Name | Required | Description | Default |
|---|---|---|---|
| scenario_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the two-step process (validate then execute) and the return type (validation + run result), but does not mention side effects, state changes, or logging. With no annotations, the description carries the full burden and provides only basic transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the tool's purpose and immediate result. Every word is necessary; no wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 2 parameters, no annotations, but an output schema (not shown). The description covers the core functionality and return type, but lacks details on prerequisites (e.g., project initialization), error cases, or how to interpret the validation result. Given the sibling set, more context on sequencing with other tools would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It specifies that scenario_id comes from .agent-sim/scenarios/*.jsonl files and project_root identifies the project, which adds value beyond the schema. However, it does not explain formats, constraints, or how to find valid scenario IDs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates then executes a scenario from a specific directory (.agent-sim/scenarios/*.jsonl) and returns both validation and run result. This distinguishes it from siblings like validate_scenario (validation only) and run_scenario (likely execution without validation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool performs validation before execution but does not explicitly state when to use this tool versus siblings like execute_scenarios (batch) or run_scenario. No guidance on prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute multiple scenarios. Omit scenario_ids to run all valid files; optional tag filter (e.g. smoke).
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | ||
| project_root | Yes | ||
| scenario_ids | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of disclosing behavioral traits. It only states what the tool does but does not mention side effects, destructive potential, authentication needs, or other behavioral aspects. For a tool that likely mutates state, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, efficiently front-loaded with purpose and key usage details. No superfluous words; every sentence adds necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of running multiple scenarios with optional filtering, the description covers the main choices but lacks specifics on error handling, behavior when no files are valid, or non-functional aspects. The presence of an output schema reduces the need to explain return values, but comparison with similar tools is minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains two of three parameters: scenario_ids (omit to run all) and tag (optional filter). However, the required 'project_root' is not described, relying on its self-explanatory name. This adds some value but is incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Execute multiple scenarios', which is a specific verb+resource. It distinguishes from the sibling 'execute_scenario' (singular) by the plural form, and further clarifies behavior by explaining that omitting scenario_ids runs all valid files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: 'Omit scenario_ids to run all valid files; optional tag filter (e.g. smoke).' This gives guidance on when to omit parameters. However, it does not explicitly compare with alternatives like 'run_scenario' or 'execute_scenario', though the plural name implies usage for multiple scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Export a parsed scenario (Persona, Execute run params, Dispatch flag, PassCriteria) as JSON.
| Name | Required | Description | Default |
|---|---|---|---|
| scenario_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, permissions needed, side effects, or state changes. It only states the export action but lacks deeper behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 15 words, concise and front-loaded with the key action. It earns its place with no wasted words, though could add structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has low complexity with 2 parameters and an output schema (assumed as JSON). However, the description fails to provide parameter details or usage context, which are needed for completeness. The description is too brief to fully inform an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The tool description provides no information about the parameters (scenario_id, project_root) beyond what is in the schema. The description does not add meaning or context for the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (export), the resource (parsed scenario), the output format (JSON), and lists the components (Persona, Execute run params, Dispatch flag, PassCriteria). It distinguishes this tool from siblings like execute_scenario or validate_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No explicit use cases, prerequisites, or exclusions. Only implied from sibling tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read events.jsonl with filters. kind supports trailing * prefix match (e.g. tool.*).
| Name | Required | Description | Default |
|---|---|---|---|
| kind | No | ||
| turn | No | ||
| limit | No | ||
| run_id | Yes | ||
| source | No | ||
| project_root | Yes | ||
| since_mono_ms | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavior. It only mentions that the tool reads a file and supports a prefix filter. It does not state that the tool is read-only, what side effects (if any) occur, or any auth requirements. The behavior of other parameters (e.g., limit, turn) is unaddressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (two sentences) and front-loads the core purpose. However, it could be more efficient by also covering key parameters without adding length. It does not waste words but is overly sparse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, 2 required, no schema coverage) and the existence of sibling tools, the description is insufficient. An output schema exists, so return values need not be explained, but the description fails to provide enough context for an agent to choose and configure the tool correctly without extensive schema exploration.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for only one parameter ('kind' describes prefix matching). The remaining six parameters (project_root, run_id, turn, limit, source, since_mono_ms) are not explained. The description does not compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Read events.jsonl') and describes a specific filter capability (prefix match on 'kind'). This gives a clear sense of the tool's function, but it does not differentiate from sibling tools like get_run_report or list_runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. There is no indication of when to use this tool over siblings (e.g., compare_runs, get_run_status). The description does not mention prerequisites, exclusions, or alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Full report: summary, judge verdict, suspicious turns, paths to timeline/events.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully disclose behavior. It mentions what the report contains but omits any side effects, prerequisites (e.g., run must be completed), or permissions. The minimal description leaves many behavioral aspects unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It quickly conveys the core output facets. Could be improved with structured formatting (e.g., bullet points) but is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return value details are covered elsewhere. The description explains what is in the report, filling that gap. However, it misses operational context: when to call this vs siblings, prerequisites, and whether the run must be completed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning for the two parameters. It does not. There is no explanation of what project_root or run_id represent, leaving the agent to infer from parameter names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description lists key components of the report (summary, judge verdict, suspicious turns, paths), making the output clear. However, it lacks an explicit verb stating the action (e.g., 'retrieves' or 'generates'), relying on the tool name. Distinguishes from siblings like get_run_log (log vs full report) but could be more explicit about scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_run_log, get_run_status, or compare_runs. The agent receives no decision-making support for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Status of a run from SQLite: running / done / failed, turn count, duration.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description correctly implies a read-only operation by stating it retrieves status. It lists the returned fields, though it does not explicitly mention that the tool has no side effects. Overall, it is sufficiently transparent for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that delivers essential information without superfluous words. It is front-loaded with the main purpose and outputs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no nested objects, output schema exists), the description covers the output but omits parameter details. It is adequate for a basic understanding but lacks completeness regarding input semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. However, it does not explain the parameters (run_id, project_root). While the parameter names are somewhat self-explanatory, an agent may need more context about formats or values. This is a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the status of a run from SQLite, listing possible statuses (running/done/failed) and additional data (turn count, duration). It is specific and distinguishes from sibling tools like get_run_log or get_run_report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_run_log or compare_runs. The description does not provide context for prerequisites or scenarios where another tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scaffold .agent-sim/ (config.yaml + smoke scenario) in the target repo and gitignore it.
| Name | Required | Description | Default |
|---|---|---|---|
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses main actions (scaffold directory, gitignore) but lacks detail on edge cases: what if .agent-sim exists? Does it overwrite or error? How does it modify .gitignore? No annotations provided to compensate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with all key points (verb, resource, location, gitignore). No filler, but could briefly mention prerequisite or idempotency without significant bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core purpose and action for a simple tool with one parameter and output schema. Missing details on error handling, idempotency, or prerequisite that project_root is a git repository.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% parameter description coverage, so description adds value by indicating 'project_root' is the target repo path. However, no specifics about expected format (absolute/relative) or validation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Scaffold' and resource '.agent-sim/' with specific files 'config.yaml + smoke scenario', and action 'gitignore it'. Clearly distinguishes from sibling tools which deal with scenarios and runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage as initialization step before other tools, but no explicit guidance on when to use, prerequisites (e.g., git repo), or when not to use (e.g., if already scaffolded).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run history from SQLite, newest first. Optionally filter by scenario_id.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| scenario_id | No | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description indicates it is a read-only operation (listing history) and notes ordering (newest first). It does not disclose potential effects, rate limits, or whether it modifies data, but the read behavior is implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the core purpose. No unnecessary words, but it could be slightly more structured for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain the required project_root parameter or the limit parameter. It omits important usage context, making it incomplete for a tool with multiple parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description only adds context for scenario_id (optional filter) but omits explanation of the required project_root parameter and the limit parameter (default 20). With 0% schema description coverage, this leaves gaps for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists run history from SQLite, ordered newest first, with optional filtering by scenario_id. However, it does not explicitly distinguish it from sibling tools like get_run_log or compare_runs, which may also relate to runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as compare_runs or get_run_log. The description only mentions optional filtering but lacks context on typical use cases or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all scenarios in .agent-sim/scenarios/*.jsonl with id, tags, and validity.
| Name | Required | Description | Default |
|---|---|---|---|
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It mentions the file path pattern and returned fields but omits important details such as error handling (e.g., what if the path doesn't exist), permissions required, or whether results are paginated. The behavior is partially transparent but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but missing key information about the parameter. It is front-loaded with the action and resource but fails to explain the parameter meaning. Slightly too terse given the need for parameter context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists (true) but is not shown; the description lists three fields (id, tags, validity) which is helpful. However, without parameter explanation or usage comparisons to siblings, the description feels incomplete. For a simple list operation, it is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter (project_root) with no description (0% coverage). The description does not explain what project_root is, its expected format, or how it relates to the file path. The agent must infer that project_root is a directory containing `.agent-sim/scenarios/`. This is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List'), the resource ('all scenarios'), and specifics about location (`.agent-sim/scenarios/*.jsonl`) and included fields (id, tags, validity). This distinguishes it from sibling tools like execute_scenario or validate_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like list_runs or validate_scenario. The description assumes the agent knows to use it for listing scenarios but does not exclude cases where filtering or validation might be needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a simulation (alias: prefer execute_scenario for validate-then-run).
| Name | Required | Description | Default |
|---|---|---|---|
| scenario_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full burden. It only says 'Run a simulation' without disclosing side effects, permissions, or behavioral traits. This is minimal transparency for a run/mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with zero redundancy. It is front-loaded and efficient, conveying its key point immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 2 required parameters, no schema descriptions, and no annotations, the description is too sparse. It lacks context about the simulation's nature, prerequisites, or output, leaving significant gaps for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% with no parameter descriptions. The description does not explain the meaning or format of 'scenario_id' or 'project_root', adding no value beyond the structural schema definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Run a simulation,' clearly indicating the action and resource. It distinguishes from the sibling 'execute_scenario' by noting that the latter is preferred for validate-then-run, providing differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a preference hint ('prefer execute_scenario for validate-then-run'), which implies when to use the alternative. However, it does not explicitly state when to use this tool or exclude other cases, leaving usage context somewhat implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate a scenario file: schema, required Persona brief, PassCriteria lint.
| Name | Required | Description | Default |
|---|---|---|---|
| scenario_id | Yes | ||
| project_root | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden but only lists validation checks. It does not disclose whether the tool is read-only, what happens on failure (e.g., errors), or any side effects. The presence of an output schema mitigates slightly, but behavioral context is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it is excessively terse and omits necessary details, balancing conciseness against completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 2 parameters and an output schema, the description should provide enough context for correct invocation. It specifies what is validated but omits the validation result format, prerequisites, and any constraints, leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning beyond the schema. The description mentions 'scenario file' but does not explain the two parameters (scenario_id, project_root) or their roles, leaving them underdefined.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates a scenario file with specific aspects: schema, required Persona brief, and PassCriteria lint. This distinguishes it from sibling tools like execute_scenario or list_scenarios, which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. While it can be inferred that validation should precede execution, the description lacks any when-to-use, when-not-to-use, or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Most tools have distinct purposes (scaffolding, scenario management, execution, analysis). However, 'run_scenario' is an alias for 'execute_scenario', creating redundancy and potential confusion. Otherwise, boundaries are clear.
All tool names follow a consistent verb_noun pattern using snake_case (e.g., 'list_scenarios', 'validate_scenario', 'compare_runs'). No mixing of conventions.
12 tools is well-scoped for a simulation framework: covering project initialization, scenario management, execution, validation, and results analysis. Each tool serves a clear purpose.
The tool surface covers the core workflow (init, validate, execute, analyze). Minor gaps: no direct tool for creating/editing scenarios (only export) and no deletion management. The redundant alias is a slight detraction.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Let AI agents place real phone calls from your verified number, with transcripts and recordings.
Give your AI agent a phone: place calls, navigate IVRs, wait on hold, get structured answers.
Give AI agents a phone: outbound AI calls that return a summary, transcript, and extracted fields.
AI voice agents that make real phone calls: single calls or campaigns, with transcripts and notes.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/quangdang46/livekit_agent_simulator'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
