Skip to main content
Glama

run_test

Run a test against a voice agent you control. Two modes: (1) score a captured transcript offline — pass "transcript" (or "turns") plus "scenarioGoal"; the judge returns a scored run synchronously. (2) run a live synthetic call — pass "targetAgent" and "goal". A live "direct" (SIP/WebRTC) target returns a tokenized media WebSocket URL for your agent-side harness to dial; a "pstn" target places a real carrier call (pro+ plans) to a number you have verified — an unverified destination returns 403 NUMBER_NOT_VERIFIED and no call is placed. Poll get_run / GET /api/tests/{id} for the terminal scored state of a live run.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
goalNoWhat the synthetic caller should accomplish (live mode).
turnsNoPer-turn capture; formatted into a transcript when "transcript" is omitted.
recordNoRecord this call to YOUR OWN storage (BYOS, default false). Requires the byosRecording capability (pro/enterprise) and an enabled recording target; the customer-owned pointer comes back in the run’s recording_url. wordis-bond keeps only the pointer, never the audio.
bargeInNo
personaNoThe synthetic caller persona (live mode).
suiteIdNoLink this run to a suite (optional).
expectedNo
languageNoBCP-47 language tag, default "en".
transportNoForce a transport (live mode).
assertionsNoExtra pass/fail checks for the judge.
transcriptNoCaptured conversation to score (offline mode).
concurrencyNoNumber of parallel live calls.
targetAgentNoThe voice agent under test — a system you run. transport "direct" (SIP/WebRTC) has near-zero cost and works on every plan; "pstn" places a real carrier call (pro+ plans) and only to a number you have verified. The hosted demo target is { "transport": "direct", "peerId": "wb-demo-dental" }.
scenarioGoalNoWhat the conversation was meant to accomplish (offline mode).
recordTargetIdNoRecord to THIS target (else the most-recent enabled one).

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that offline runs return synchronously, live direct targets yield a tokenized media WebSocket URL, pstn calls require verified numbers and return 403 NUMBER_NOT_VERIFIED if unverified, and live runs must be polled for terminal state. This is substantial and exceeds typical annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a dense single paragraph but is front-loaded with the two-mode structure and every clause contributes essential info. It avoids fluff and waste. A little more visual structure (bullets or line breaks) would earn a 5, but it remains readable and appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 15 params and no output schema or annotations, the description covers modes, required inputs, transport behavior, error handling, and the polling mechanism. It tells the agent exactly what to pass and how to obtain results. The schema handles parameter details, while the description supplies the contextual glue needed for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 87%, so the schema already documents most parameters. The description adds value by grouping parameters by mode: transcript/turns/scenarioGoal for offline and targetAgent/goal for live. It also clarifies the behavioral implications of targetAgent transport options, including the 403 error and plan restrictions, going beyond the schema's field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear statement: 'Run a test against a voice agent you control.' It then disambiguates two concrete modes—offline transcript scoring and live synthetic calls—making the tool's function and scope immediately apparent. This distinguishes it from sibling tools like run_demo or test_flow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance for both modes: pass 'transcript' (or 'turns') plus 'scenarioGoal' for offline scoring; pass 'targetAgent' and 'goal' for live calls. It also explains transport-specific constraints (direct vs pstn, plan requirements, verified numbers) and refers to polling via get_run. It doesn't explicitly name alternative tools, but the mode guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4/5.0
Disambiguation4/5

Most tools are clearly distinct by resource (monitors, suites, flows, numbers, recordings), but run_test and test_flow could be confused since both execute tests, though their scopes differ. The descriptions help disambiguate them.

Naming Consistency5/5

All tool names follow a consistent verb_noun snake_case pattern (create_, get_, list_, run_, verify_, etc.), with no camelCase or mixed conventions. Even compound names like get_monitor_health and verify_number_confirm remain predictable.

Tool Count4/5

At 16 tools, the set is slightly above the optimal 3-15 range, but the breadth of the voice-agent testing/monitoring domain justifies each tool's existence. It feels well-scoped rather than bloated.

Completeness2/5

The tool set lacks update/delete operations for most entities (monitors, suites, flows) and omits a get_run tool to retrieve individual live test results, leaving significant gaps that agents cannot work around. This will cause failures in lifecycle management and live-run result retrieval.

Resources