Skip to main content
Glama
okareo-ai

Okareo MCP Server

Official
by okareo-ai

Run Simulation

run_simulation

Launch a multi-turn conversation evaluation of your AI agent, using a simulated user and scenario to generate realistic interactions with quality checks, and retrieve results asynchronously.

Instructions

Run a multi-turn conversation evaluation of your AI agent.

    Combines a Target (the system under test), a Driver (the simulated user),
    and a Scenario (the test cases) to generate realistic multi-turn conversations
    and evaluate them with quality checks.

    Returns promptly so the call never times out on long runs. Short runs that
    finish within the buffer window return ``status: "finished"`` with results
    ready; longer runs return ``status: "running"`` with the ``test_run_id``,
    ``app_link``, and an ``estimated_runtime`` — the run continues to completion
    on its own. In both cases, poll get_test_run_results with the returned
    test_run_id for scores, and get_conversation_transcript for transcripts.

    To rerun a previous simulation — keeping its configuration but changing one or
    more parameters — pass based_on_run_id with the original run's ID and supply
    only the values you want to override. If scenario_name or target_name are
    omitted and based_on_run_id is provided, they will be resolved from the
    original run.

    For custom_endpoint Targets: an exception raised during the run (for
    example the endpoint erroring mid-conversation) FAILS the run — it is
    reported as a failed simulation, not silently skipped.

    **Voice augmentations** — for voice Targets, the `augmentation` parameter
    applies realistic acoustic and conversational effects. Six top-level keys:
    `cap`, `directed_speech`, `secondary_speaker`, `backchannel`, `barge_in`,
    plus the composable `noise`. **Composition rule**: at most one non-noise
    strategy may be active, optionally combined with `noise`. Augmentations
    apply only to voice Targets — calls against generation or custom_endpoint
    Targets with an augmentation block are rejected. Field-level errors
    (out-of-range probability, missing required field, swapped offsets, unknown
    strategy) are returned by the MCP before any backend call.

    Strategy required / optional fields (numeric ranges in brackets):
      - cap: probability [0.0, 1.0] required. pause_ms [0, 10000] optional.
      - directed_speech: probability [0.0, 1.0] required. lpf_cutoff_hz (>0),
        gain_db [-40.0, 0.0], sample_rate (>0), prompt, reverb_preset optional.
      - secondary_speaker: probability [0.0, 1.0] AND secondary_voice (non-empty
        string) required. inter_speaker_pause_ms [0, 5000], lpf_cutoff_hz (>0),
        gain_db [-40.0, 0.0], sample_rate (>0), secondary_prompt,
        secondary_voice_instructions, secondary_reverb_preset optional.
      - backchannel: utterance (non-empty string) required. probability
        [0.0, 1.0], min_offset_ms (>=0), max_offset_ms (>= min_offset_ms),
        seed optional.
      - barge_in: prompt (non-empty string) required. probability [0.0, 1.0],
        min_offset_ms (>=0), max_offset_ms (>= min_offset_ms), seed optional.
      - noise: noise_profile (non-empty string) AND noise_snr_db (number)
        required. seed optional.

    For copy-paste examples and the full reference, call
    `get_templates(["voice_augmentations"])`.

    Args:
        name: Human-readable name for this simulation run.
        scenario_name: Name of the scenario to use. Required unless based_on_run_id
            is provided and the original run's scenario can be resolved.
        target_name: Name of the target to evaluate. Required unless based_on_run_id
            is provided and the original run's target can be resolved.
        driver_name: Name of the driver persona. If omitted, the project default
            driver is used.
        checks: List of check names to apply (from list_checks). Pick from
            the list_checks category matching the task and modality —
            voice-specific categories for voice simulations, categories
            outside them for either modality; never chat-only checks for
            audio (or vice versa). Every simulation runs with at least one
            check: when omitted or empty, the benign code-based "latency"
            performance check is applied automatically and the response
            discloses the substitution via `default_check_applied`.
            Supplied checks are used unchanged.
        repeats: Number of times to run each scenario row, default 1.
        max_turns: Maximum conversation turns per simulation, default 5.
        first_turn: Who speaks first — 'target' or 'driver', default 'target'.
        based_on_run_id: ID of a previous simulation run to reuse parameters from.
            Explicitly supplied values override the original run's parameters.
        augmentation: (voice Targets only) Voice augmentation block. See the
            "Voice augmentations" section above for keys and ranges. An empty
            dict is treated as no augmentation.
        turn_transition_time: Milliseconds of pause between turns. Forwarded to
            the backend as-is; SDK default (1000) is used when omitted.
        silence_timeout_ms: The target reply timeout — how patient Okareo
            is before indicating that the target can't respond. Do NOT set
            or change this value unless the user specifically directs it;
            it should be 10000 ms in nearly all cases. It exists to
            accommodate untuned targets with very long tool calls, during
            which the Driver waits patiently. It does NOT change how fast
            Okareo responds, and lowering it does not speed anything up —
            a slow simulation is not a reason to change it. Forwarded to
            the backend; backend default is used when omitted.
        checks_at_every_turn: When True, checks are evaluated per turn (not
            only at end of run).
        stop_check: Early-stop config: `{"check_name": str, "stop_on": <value>}`.
            The run halts as soon as the named check returns `stop_on`.
    

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
nameYes
checksNo
repeatsNo
max_turnsNo
first_turnNotarget
stop_checkNo
driver_nameNo
target_nameNo
augmentationNo
scenario_nameNo
based_on_run_idNo
silence_timeout_msNo
checks_at_every_turnNo
turn_transition_timeNo

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
resultYes
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=false and openWorldHint=true, and the description adds substantial behavioral context: asynchronous execution ('Returns promptly so the call never times out... run continues to completion on its own'), status values ('finished' vs 'running'), and the failure mode for custom_endpoint targets ('exception raised during the run FAILS the run'). It also discloses default check substitution and field-level validation before backend calls, with no contradiction to annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads a clear one-sentence summary and is well-structured with sections and an Args list, making it easy to scan. However, it is quite long, and the voice augmentation section provides field-level detail that overlaps with the instruction to call get_templates for the full reference, introducing some redundancy that could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (14 parameters, async behavior, voice augmentation constraints), the description is remarkably complete. It covers all parameters, return statuses, the polling workflow, failure modes, and check handling. The presence of an output schema means return values need not be repeated, and the description provides all necessary context for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the full burden, and it does thoroughly. The Args section explains every parameter beyond schema types, e.g., 'repeats: Number of times to run each scenario row', 'silence_timeout_ms: Do NOT set... should be 10000 ms', and the augmentation block is fully detailed with required/optional fields and numeric ranges. This far exceeds minimal compensation for the schema gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Run a multi-turn conversation evaluation of your AI agent,' a specific verb+resource+modality that clearly states the tool's purpose. It distinguishes from siblings by emphasizing multi-turn conversation evaluation and pointing to follow-up tools like get_test_run_results and get_conversation_transcript, setting it apart from run_test and other simulation-related tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use and when-not-to-use guidance: silence_timeout_ms should not be changed unless requested, augmentation is rejected for non-voice targets, and checks are sourced from list_checks. It also directs users to get_templates for examples and get_test_run_results for polling, clearly naming alternatives and context for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/okareo-ai/okareo-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server