# Workflow System Documentation
**Last Updated:** 2025-12-29
**Status:** ✅ Implemented and Ready for Use
---
## Overview
The Workflow System enables users to create, manage, and execute preprogrammed multi-step flows that orchestrate operations across multiple MCP servers. Workflows automate complex robotics operations like avatar creation pipelines, virtual robot deployment, and physical-virtual robot synchronization.
## Key Features
- ✅ **CRUD Operations**: Create, read, update, delete workflows
- ✅ **Workflow Execution**: Execute workflows with custom variables
- ✅ **Step-by-Step Tracking**: Monitor execution progress in real-time
- ✅ **Built-in Templates**: Pre-configured workflows for common use cases
- ✅ **Variable Substitution**: Dynamic variable injection during execution
- ✅ **Error Handling**: Configurable error handling strategies (stop, continue, retry, rollback)
- ✅ **Search & Filtering**: Find workflows by category, tags, or search query
- ✅ **Import/Export**: Share workflows via JSON import/export
## Architecture
### Backend Components
**robotics-mcp** (MCP Server):
- `services/workflow_models.py` - Data models and types
- `services/workflow_storage.py` - SQLite database storage
- `services/workflow_executor.py` - Execution engine
- `tools/workflow_management.py` - MCP portmanteau tool
**robotics-webapp** (Backend API):
- `backend/workflow_service.py` - Service layer wrapping MCP calls
- `backend/main.py` - REST API endpoints
### Frontend Components
**robotics-webapp** (React Frontend):
- `src/services/workflowService.ts` - TypeScript service
- `src/app/workflows/page.tsx` - Workflow list and management UI
- `src/app/workflows/execute/page.tsx` - Workflow execution and debugging interface
- `src/components/WorkflowDebugger.tsx` - Debug UI component with breakpoints and step controls
- `src/components/WorkflowStepEditor.tsx` - Step editor with breakpoint toggle support
## Quick Start
### 1. Access Workflows Page
Navigate to **Workflows** in the sidebar (under "⚙️ AUTOMATION" section) or visit `/workflows` directly.
### 2. View Available Workflows
The workflows page displays all available workflows with:
- Name and description
- Category badge (avatar, vbot, sync, custom)
- Step count and variable count
- Tags
### 3. Execute a Workflow
1. Click **Execute** on any workflow card
2. Provide required variables (if any)
3. Monitor execution progress in real-time
### 4. Create a New Workflow
1. Click **New Workflow** button
2. Fill in workflow details:
- Name and description
- Category
- Steps (MCP tool calls, app launches, conditions, delays)
- Variables (user inputs, step outputs)
- Error handling configuration
3. Save the workflow
## Workflow Structure
### Basic Workflow
```json
{
"id": "workflow-123",
"name": "My Workflow",
"description": "Description of what this workflow does",
"category": "avatar",
"version": "1.0.0",
"author": "Your Name",
"tags": ["avatar", "vrchat"],
"steps": [
{
"id": "step1",
"order": 1,
"name": "Step Name",
"type": "mcp_tool",
"mcp_server": "unity3d",
"tool_name": "unity_vrm",
"arguments": {
"operation": "import_vrm",
"vrm_path": "${vrm_file_path}"
},
"required": true,
"output_variable": "import_result"
}
],
"variables": [
{
"name": "vrm_file_path",
"type": "file_path",
"required": true,
"description": "Path to VRM file"
}
],
"error_handling": {
"on_error": "stop",
"retry_count": 0,
"rollback_steps": [],
"error_notification": true
}
}
```
## Step Types
### MCP Tool Step
Execute a tool on an MCP server:
```json
{
"id": "step1",
"order": 1,
"name": "Import VRM",
"type": "mcp_tool",
"mcp_server": "unity3d",
"tool_name": "unity_vrm",
"arguments": {
"operation": "import_vrm",
"vrm_path": "${vrm_file_path}"
},
"required": true,
"output_variable": "import_result"
}
```
### App Launch Step
Launch an application:
```json
{
"id": "step2",
"order": 2,
"name": "Launch Unity3D",
"type": "app_launch",
"app_id": "unity3d",
"app_config": {
"desktop": 2,
"monitor": 2,
"project_path": "${unity_project_path}",
"fullscreen": false
},
"required": true
}
```
### Delay Step
Add a delay between steps:
```json
{
"id": "step3",
"order": 3,
"name": "Wait for App",
"type": "delay",
"arguments": {
"seconds": 5.0
},
"required": false
}
```
### Condition Step
Conditional branching:
```json
{
"id": "step4",
"order": 4,
"name": "Check Source",
"type": "condition",
"condition": {
"expression": "${vroid_source} == 'file'",
"true_branch": ["step5a"],
"false_branch": ["step5b"]
},
"required": true
}
```
## Variable Substitution
Workflows support variable substitution using `${variable_name}` syntax:
- **User Variables**: `${variable_name}` - From workflow execution variables
- **Step Outputs**: `${step_id.output.field}` - From previous step outputs
- **Built-in Functions**: `${timestamp}` - Current timestamp
- **Environment Variables**: `${env.VAR_NAME}` - System environment variables
### Example
```json
{
"arguments": {
"vrm_path": "${vrm_file_path}",
"output_dir": "${step1.output.directory}",
"timestamp": "${timestamp}"
}
}
```
## Built-in Templates
### VRoid to VRChat Avatar
Complete pipeline from VRoid creation to VRChat upload:
- **Category**: avatar
- **Steps**: Source selection → VRoid Studio → Unity3D → VRChat upload → OSC setup
- **Variables**: `vroid_source`, `vroid_file_path`
### Create Virtual Robot
Create vbot with 3D model and deploy to environments:
- **Category**: vbot
- **Steps**: Convert to VRM → Create virtual robot → Configure
- **Variables**: `robot_name`, `robot_type`, `model_file_path`
### VRM to VRChat with OSC Control
Import VRM from avatar-mcp models directory, process in Unity3D, upload to VRChat, and connect via OSC:
- **Category**: avatar
- **Steps**: 11 steps including VRM selection, Unity3D import, VRChat configuration, build, upload, OSC connection
- **Variables**: `vrm_name`, `unity_project_path`, `avatar_name`, `avatar_description`, `build_output_path`
- **Features**:
- Lists available VRMs from `D:\Dev\repos\avatar-mcp\models`
- Configures VRChat avatar descriptor, expression menu, gesture controller
- Connects OSC on port 9000
- Sends avatar selection via OSC
- **Estimated Duration**: 15-20 minutes
- **Complexity**: High
### VRM to Resonite with OSC Control
Import VRM from avatar-mcp, process in Unity3D, upload to Resonite, and connect via OSC:
- **Category**: avatar
- **Steps**: 10 steps including VRM import, Unity3D processing, Resonite configuration, build, upload, OSC connection
- **Variables**: `vrm_name`, `unity_project_path`, `avatar_name`, `build_output_path`
- **Features**:
- Configures Resonite avatar rig and components
- Connects OSC on port 9001
- Sends avatar selection via OSC
- **Estimated Duration**: 15-20 minutes
- **Complexity**: High
### Dual Platform Avatar Deployment
Import VRM once, deploy to both VRChat and Resonite simultaneously, then connect OSC to both:
- **Category**: avatar
- **Steps**: 16 steps with parallel execution for dual platform deployment
- **Variables**: `vrm_name`, `unity_project_path`, `avatar_name`, `build_output_path`
- **Features**:
- Single VRM import
- Parallel configuration for both platforms
- Parallel builds and uploads
- Dual OSC connections (ports 9000 and 9001)
- Launches both applications on separate monitors
- **Estimated Duration**: 20-25 minutes
- **Complexity**: Very High
- **Parallel Steps**: Builds and uploads execute in parallel
### WorldLabs.ai Splat to Resonite
Download Gaussian splat from worldlabs.ai, import into Unity3D, and export to Resonite:
- **Category**: avatar
- **Steps**: 12 steps including web search, download, extraction, Unity3D import, Resonite configuration, build, upload
- **Variables**: `splat_search_query`, `splat_url`, `unity_project_path`, `splat_name`, `splat_scale`, `splat_position`, `splat_rotation`, `world_name`, `splat_description`, `splat_tags`
- **Features**:
- Searches worldlabs.ai for splats
- Downloads and extracts splat archives
- Imports Gaussian splats to Unity3D with configurable transform
- Configures lighting, collision, and optimization for Resonite
- Uploads to Resonite world with metadata
- **Estimated Duration**: 25-30 minutes
- **Complexity**: Very High
- **Requirements**: Unity3D Gaussian Splat importer plugin, Resonite SDK
## API Endpoints
### Workflow Management
- `GET /api/workflows` - List workflows (with filters)
- `GET /api/workflows/{id}` - Get workflow details
- `POST /api/workflows` - Create workflow
- `PUT /api/workflows/{id}` - Update workflow
- `DELETE /api/workflows/{id}` - Delete workflow
### Workflow Execution
- `POST /api/workflows/{id}/execute` - Execute workflow
- `GET /api/workflows/executions/{execution_id}` - Get execution status
### Templates & Import/Export
- `GET /api/workflows/templates` - List templates
- `POST /api/workflows/import` - Import workflow from JSON
- `GET /api/workflows/{id}/export` - Export workflow to JSON
## MCP Tool Usage
The workflow system is accessible via the `workflow_management` MCP tool:
```python
# List workflows
result = await mcp_client.call_tool(
"robotics",
"workflow_management",
{
"operation": "list",
"category": "avatar"
}
)
# Execute workflow
result = await mcp_client.call_tool(
"robotics",
"workflow_management",
{
"operation": "execute",
"workflow_id": "workflow-123",
"variables": {
"vrm_file_path": "/path/to/file.vrm"
}
}
)
```
## Error Handling
Workflows support configurable error handling:
- **stop**: Stop execution on error (default)
- **continue**: Continue to next step on error
- **retry**: Retry failed step (with configurable retry count)
- **rollback**: Rollback to previous checkpoint
### Retry Configuration
```json
{
"retry": {
"max_retries": 3,
"retry_delay": 1.0,
"exponential_backoff": false
}
}
```
## Best Practices
1. **Use Descriptive Names**: Clear workflow and step names improve maintainability
2. **Add Descriptions**: Document what each workflow does
3. **Use Tags**: Tag workflows for easy filtering
4. **Test Variables**: Ensure all required variables are provided
5. **Handle Errors**: Configure appropriate error handling for critical steps
6. **Use Templates**: Start from templates and customize as needed
7. **Version Control**: Use version field to track workflow changes
## Debugging Workflows
The workflow system includes comprehensive debugging capabilities similar to a code debugger, allowing you to step through workflow execution and inspect MCP tool calls.
### Debug Mode
Enable debug mode when executing a workflow to pause after each step, allowing you to:
- Inspect variables and step outputs
- View MCP tool calls and responses
- Step through execution one instruction at a time
- Set breakpoints on specific steps
### Setting Breakpoints
1. **In Workflow Editor**: Click the bug icon (🐛) next to any step to toggle a breakpoint
2. **Breakpoint Indicator**: Steps with breakpoints show a yellow bug icon
3. **Execution Behavior**: Execution pauses before executing a step with a breakpoint
### Debug Controls
When a workflow is running in debug mode or paused at a breakpoint:
- **Step**: Execute the next instruction and pause again
- **Continue**: Run until the next breakpoint or end of workflow
- **Pause**: Pause execution at current step
- **Resume**: Continue execution from current step
### Debug Interface Features
**Step List View:**
- Current step highlighted in blue
- Status indicators (completed ✅, failed ❌, running ⏱️, pending ⏸️)
- Breakpoint indicators (🐛)
- Expandable step details
**MCP Tool Call Inspection:**
- View MCP server and tool name
- See original and substituted arguments
- Inspect tool response data
- Check for errors in tool execution
**Variable Inspection:**
- View all step outputs
- See variable values at current execution point
- Track variable changes across steps
### Example: Debugging a Workflow
```typescript
// Execute workflow in debug mode
const result = await workflowService.executeWorkflow(
workflowId,
{ vrm_name: 'my_avatar' },
true // debug_mode = true
)
// Step through execution
await workflowService.stepExecution(result.execution_id)
// Inspect execution status
const status = await workflowService.getExecutionStatus(result.execution_id)
console.log(status.step_results) // See all step results
console.log(status.step_outputs) // See variable values
```
### Debug UI Usage
1. **Navigate to Workflow**: Click "Execute" on any workflow
2. **Set Variables**: Enter required variables in the debug panel
3. **Enable Debug Mode**: Check "Debug Mode (Step-by-step)" checkbox
4. **Start Execution**: Click "Start Execution"
5. **Step Through**: Use "Step" button to execute one instruction at a time
6. **Inspect Results**: Expand steps to view MCP tool calls and responses
7. **Continue**: Use "Continue" to run until next breakpoint
## Troubleshooting
### Workflow Execution Fails
- Check MCP server connectivity
- Verify all required variables are provided
- Check step arguments for correct variable names
- Review error messages in execution status
- Use debug mode to identify which step fails
### Variables Not Substituting
- Ensure variable names match exactly (case-sensitive)
- Check variable scope (user variables vs step outputs)
- Verify variable is defined before use
- Use debug mode to inspect variable values at each step
### MCP Tool Calls Fail
- Verify MCP server is running
- Check tool name and server name are correct
- Review tool arguments format
- Check MCP server logs for errors
- Use debug mode to inspect tool call details and responses
## Future Enhancements
Planned features (from WORKFLOW_SYSTEM_PLAN.md):
- Visual workflow editor
- Workflow marketplace
- AI-powered workflow generation
- Scheduled execution
- Workflow analytics and metrics
- Parallel step execution
- Loop constructs
- User input steps
## Related Documentation
- [Workflow System Plan](../mcp-central-docs/docs/robotics/WORKFLOW_SYSTEM_PLAN.md) - Detailed planning document
- [MCP Integration](./MCP_INTEGRATION.md) - MCP server integration guide
- [Quick Start Guide](./QUICK_START.md) - Getting started with robotics-webapp
