create_workflow

Advanced low-level workflow creation. Prefer create_automation_workflow, create_monitor_workflow, or create_export_workflow for the three product workflows because those tools enforce product-specific inputs.

TYPES:

• "monitors": Aggregation + threshold monitoring with notifications (Monitors page)

• "automations": Triggered actions on log/trace events (Automations page)

• "exports": Scheduled, continuous request-log exports. Requires trigger_event_type="scheduled", schedule_cron, and an export task.

• "evaluators": Evaluator pipelines. Prefer create_evaluation_pipeline when wrapping graders.

• "reports" and "ingests": Backend-supported advanced workflow families.

TRIGGER EVENT TYPES:

• "request_log": Fires on every logged LLM request

• "trace_completed": Fires when a trace finishes

• "customer_budget_limit_reached": Fires on budget breach

• "credit_low_balance_threshold_reached": Fires on low credit balance

• "spend_cap_warning_threshold_reached": Fires at the spend-cap warning threshold

• "limit_policy_soft_triggered" / "limit_policy_hard_triggered": Fires for limit-policy events

• "on_eval_result_ingested": Fires when eval score is recorded

• "custom_event": Fires for a custom event

• "scheduled": Runs on a UTC cron schedule. Required for export workflows.

• "eval_only": No trigger, for evaluator pipelines used in experiments

TASK TYPES:

• condition: Filter/gate. Config: { condition_policy: { "event.": { operator, value } }, on_true: "continue", on_false: "stop" } Field paths use namespace: event.cost, event.model, event.status, state.. Operators: "in" (categorical), "gte"/"lte"/"gt"/"lt" (numeric), "icontains"/"startswith" (text)

• aggregation: Time-window metrics. Config: { time_step_minutes: 5, metrics: [{ field_name: "event.cost", aggregation_function: "sum", output_field_name: "cost_sum" }] }

• notification: Alert. Config: { severity: "high", message_template: "Cost: ${{state.agg.cost_sum}}" } Use {{variable}} for template variables.

• webhook: HTTP callback. Config: { webhook_url: "https://...", source: "event" }

• eval: Advanced callers must put generation_method at task root and supply evaluator_id plus the method-specific llm_config/code_config/human_config in config. Prefer create_automation_workflow for automatic evaluator hydration.

• ingest: Save to dataset. Config: { target_type: "dataset", target: { dataset_id: "" } }

• sampling: Random filter. Config: { rate: 0.1 } (10% of events)

• compute: Arithmetic on upstream outputs. Config: { function: "ratio", inputs: [{ source: "state.", field: "" }] }

• export: Append scheduled request logs to the export workflow output. Config: { filters?, include_fields?, is_inline_results?: boolean, sample_percentage?: number }.

• switch: Multi-branch routing. Config: { cases: [{ condition_policy: {...}, target: "" }], default: "" }

TASK CHAINING: The backend auto-chains sequential tasks when next is omitted. Set next explicitly for non-linear routing or an intentional target. Task ordering: gates (condition, sampling) → aggregation → actions (notification, webhook, eval, ingest).

EXAMPLE - Cost spike monitor: { "name": "Cost spike monitor", "type": "monitors", "trigger_event_type": "request_log", "tasks": [ { "id": "agg", "type": "aggregation", "label": "Cost sum (5m)", "next": "check", "config": { "time_step_minutes": 5, "metrics": [{ "field_name": "event.cost", "aggregation_function": "sum", "output_field_name": "cost_sum" }] } }, { "id": "check", "type": "condition", "label": "Cost >= $1", "next": "notify", "config": { "on_true": "continue", "on_false": "stop", "condition_policy": { "state.agg.cost_sum": { "operator": "gte", "value": 1 } } } }, { "id": "notify", "type": "webhook", "label": "Cost alert", "config": { "webhook_url": "https://example.com/respan-monitor-alerts", "source": "event" } } ] }

EXAMPLE - Hourly export workflow: { "name": "Hourly request-log export", "type": "exports", "trigger_event_type": "scheduled", "schedule_cron": "0 * * * *", "tasks": [ { "id": "export_logs", "type": "export", "label": "Export request logs", "config": { "include_fields": ["timestamp", "model", "input", "output", "cost"], "is_inline_results": false, "sample_percentage": 100 } } ] }