Multi-Source Documentation MCP Server

--- category: general scraped_at: '2025-11-12T14:09:57.801361' title: Automated testing url: /docs/automated-testing --- # Automated testing ## What is this? Use the Conversation Profiler to test user utterances and improve your agent's interaction model. The Conversation Profiler supports **two distinct testing approaches** to validate your agent's conversation flow: ### 🔧 Traditional Interaction-Based Testing Test the conversation flow with **predefined interactions** where you send specific user utterances to your agent and validate exact responses. This approach is ideal for: * **Regression testing** to ensure specific responses remain consistent * **Validation of exact conversation flows** with predetermined inputs and outputs * **Quality assurance** for specific features or conversation paths #### Reference It is important to know which [suites](/docs/suites#/) and [tests](/docs/tests#/) you can build. Because of that, you can find the entire reference on the [Reference](/docs/tests#/) page. Suites and test are defined as `yaml` files. ### 🤖 Agent-to-Agent Testing Simulate **realistic conversations** using AI-powered agents that interact naturally with your Voiceflow agent to achieve specific goals. This approach offers two testing methods: **OpenAI-Powered Testing:** * Uses OpenAI models (GPT-4, GPT-4o, etc.) to simulate user behavior * Configurable personas and dynamic user information * Ideal for testing varied user types and edge cases **Voiceflow Agent Testing:** * Uses another Voiceflow agent as the tester * Consistent, reproducible test behavior * Leverages existing Voiceflow agent configurations Both methods are ideal for: * **End-to-end conversation testing** with natural, adaptive interactions * **User behavior simulation** where the AI agent responds dynamically like real users * **Goal-oriented testing** to ensure your agent can handle varied conversation paths Both testing approaches can be run in your CI/CD pipelines and include additional features beyond the Voiceflow console's Test Agent feature. Every suite is executed in the same Voiceflow user's session. All of the commands that are available in `voiceflow-cli` to execute the Conversation profiler are located within the [`voiceflow test` subcommand](/docs/voiceflow-test#/). #### Reference It is important to know which [suites](/docs/suites#/) and [tests](/docs/agent-to-agent-tests#/) you can build. Because of that, you can find the entire reference on the [Reference](/docs/agent-to-agent-tests#/) page. Suites and test are defined as `yaml` files. ## Examples You can find some useful examples on our [GitHub repo](https://github.com/xavidop/voiceflow-cli/tree/main/examples) ## Execution Example Here is a simple example of the `voiceflow test execute` command: Shell ``` voiceflow test execute examples/test/ ``` The above command will give you output similar to the following: Shell ``` $ voiceflow test execute examples/test/ Dec 31 10:54:01.664 [INFO] Suite: Example Conversation Profiler Suite Description: Suite used as an example Environment: development Dec 31 10:54:01.664 [INFO] Running Tests: Dec 31 10:54:01.664 [INFO] Running Test ID: Example test Dec 31 10:54:01.664 [INFO] Interaction ID: test_1_1 Dec 31 10:54:01.664 [INFO] Interaction Request Type: launch Dec 31 10:54:02.693 [INFO] Interaction Response Type: text Dec 31 10:54:02.693 [INFO] Interaction Response Message: Hey there! 🌟 Welcome to the Isla Experience! I’m like a warm cup of cocoa on a chilly day—sweet, comforting, and maybe a little too hot if you’re not careful! How’s your day going? Dec 31 10:54:02.693 [INFO] All validations passed for Interaction ID: test_1_1 Dec 31 10:54:02.693 [INFO] Interaction ID: test_1_2 Dec 31 10:54:02.693 [INFO] Interaction Request Type: text Dec 31 10:54:02.693 [INFO] Interaction Request Payload: I am doing well Dec 31 10:54:03.889 [INFO] Interaction Response Type: text Dec 31 10:54:03.889 [INFO] Interaction Response Message: Awesome! Glad to hear it! Are you riding the wave of good vibes, or did you just find a hidden stash of chocolate? 🍫 Either way, I’m here for it! What’s been the highlight of your day so far? Dec 31 10:54:03.889 [INFO] All validations passed for Interaction ID: test_1_2 Dec 31 10:54:03.889 [INFO] Interaction ID: test_1_3 Dec 31 10:54:03.889 [INFO] Interaction Request Type: text Dec 31 10:54:03.889 [INFO] Interaction Request Payload: I have been working very hard Dec 31 10:54:06.090 [INFO] Interaction Response Type: text Dec 31 10:54:06.091 [INFO] Interaction Response Message: Ah, the classic “I’m working hard” routine! It’s like a superhero origin story, but instead of gaining superpowers, you just gain a lot of coffee stains and a questionable relationship with your chair. What kind of work are you diving into? Dec 31 10:54:06.091 [INFO] All validations passed for Interaction ID: test_1_3 ``` !!! info "Are you running this command in a CI/CD pipeline?" If this is the case, we recommend that you set the `--output-format` parameter to `json`. Updated 4 months ago --- [FAQ](/docs/faq-1)[Suite Reference](/docs/suites) Ask AI * [Table of Contents](#) * + [What is this?](#what-is-this) + - [🔧 Traditional Interaction-Based Testing](#-traditional-interaction-based-testing) - [🤖 Agent-to-Agent Testing](#-agent-to-agent-testing) + [Examples](#examples) + [Execution Example](#execution-example)