List evaluation runs. Each run = agent + persona + test_set. Returns run_id, status, tags. Filter by tag: filter='tag="regression"'.

Get run status/results. Status: PENDING→RUNNING→COMPLETED. Completed runs include metrics (custom per org) and output_ids for transcripts.

Launch evaluation: agent + persona + test_set. Optionally add tags for filtering. Poll get_run until status=COMPLETED to see metrics.

List agents (AI systems to evaluate). Model types: VOICE, OUTBOUND_VOICE, SMS, WEBSOCKET, CHAT. Use agent_id when creating runs.

Get agent config: model_type, phone_number (voice), endpoint (websocket/chat), and display_name.

Create a new agent configuration. Specify the model type (voice, chat, SMS, websocket) and connection details.

Update an existing agent configuration. Only provided fields will be updated.

List test sets (collections of test cases). Each contains scenarios to run against an agent. Use test_set_id when creating runs.

Get test set details: display_name, description, and test case count. Use list_test_cases to see individual scenarios.

Create a test set to organize test cases. After creating, use create_test_case to add scenarios.

List test cases. Filter by test_set_id. Each has input_str (scenario text or JSON message array) and optional expected_behaviors.

Get test case details: input_str (the scenario), expected_behaviors, and metadata.

Create test case in a test set. input_str: single scenario message OR JSON array [{role,content},...] for multi-turn conversations.

Update test case input_str, expected_behaviors, or other fields.

List available evaluation metrics. Metrics define how agent performance is measured. Set include_builtin=true to see built-in metrics.

Get detailed information about a specific metric. Shows metric type, description, and configuration.

List personas (simulated users). Each has voice_name, language_code, background_sound (off/office/crowd/airport/etc), and behavior prompt. Required for runs.

Get persona details: voice_name, language_code (BCP-47), background_sound, persona_prompt (behavior), wait_seconds, conversation_initiation (speak_first/wait_for_user).