The livekit-agent-simulator server lets you simulate AI caller interactions against a LiveKit voice agent, record detailed behavioral logs, and analyze/compare results — without touching the agent's code.

• init_project — Scaffold a .agent-sim/ directory (config + smoke scenario) in your target repo and add it to .gitignore.

• list_scenarios — List all available scenarios (from .agent-sim/scenarios/*.jsonl) with their IDs, tags, and validity status.

• validate_scenario — Validate a scenario file against its schema, checking for a required Persona brief and linting the PassCriteria.

• export_scenario — Export a parsed scenario's full details (Persona, execution parameters, PassCriteria) as JSON.

• run_scenario / execute_scenario — Run a simulation for a given scenario (the latter validates first), returning a run_id for tracking.

• execute_scenarios — Execute multiple scenarios at once, optionally filtered by tag (e.g. smoke) or specific IDs.

• get_run_status — Poll a simulation run's current state (running / done / failed), turn count, and duration.

• get_run_log — Read the raw events.jsonl log for a run, with filtering by event kind (supports wildcards like tool.*), turn number, source, and time offset.

• get_run_report — Retrieve a full report including summary, LLM judge verdict, suspicious turns, and paths to timeline/event files.

• compare_runs — Diff two simulation runs side-by-side across duration, turn counts, tool errors, turn-taking percentiles, and judge verdicts.

• list_runs — Browse run history (newest first) from SQLite, optionally filtered by scenario ID.

All tools operate on a project_root (the repo under test) and assume the target agent is already deployed.

1. Click on "Install Server".

2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.

3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@livekit-agent-simulator run the smoke-hello scenario"

That's it! The server will respond to your query, and you can continue using it as needed.

Here is a step-by-step guide with screenshots.