interact_with_agent
Runs one Convocore agent turn over WebSocket and returns the streamed reply aggregated into a single response, including text, UI-engine output, actions, metadata, and turn status.
Instructions
Run ONE agent turn over the Convocore /interact WebSocket and return the streamed result aggregated into a single response. Opens a WSS connection to wss://-gcp-api.vg-stuff.com/interact, sends one InteractObject (agentId + convoId + bucket + prompt + optional context), collects every chunk (text + UI-engine + actions + metadata + sync_chat_history) until the server closes (code 1000) or the timeout fires, then returns: { assistantText, uiEngineEnabled, uiEngineSnapshot, uiEngineSummary, actions, metadata, turns, closeCode, durationMs, timedOut, chunkCount, chunks? }.
USAGE:
Use the SAME
convoIdacross turns to keep history; use a fresh one to start a new conversation.prompt: "start"triggers the agent's initial greeting (no user message added).prompt: "@cancel:<reason>"cancels an in-flight turn (rarely useful from a single MCP call).prompt: "@rewind:<nodeId>"rewinds a node-based agent.For images, set messageType="visual" and pass
visualPayload.bucketis auto-derived from CONVOCORE_API_REGION; only override if you really mean to talk to the other region.Set
raw: trueto include every streamed chunk (debug frames, chunkIndex, every UI Engine snapshot, etc.) when you need the full trace. Default keeps the response token-cheap.TESTING: set
isTest: truefor test chats. PasstoolTest: { toolId, toolName, mode: validate|generate-and-test }to exercise one HTTP tool (or use dedicatedtest_agent_tool). PassvariablesOverrides/initNodesOptionsto trial variables without permanently changing them.For a direct HTTP dry-run of a tool's serverUrl use
test_agent_tool_request. For multi-turn suites userun_agent_auto_test.
INTERPRETING THE RESPONSE:
When
uiEngineEnabled: false, the agent streamed plain Markdown — readassistantText.When
uiEngineEnabled: true, the agent streamed UI Engine snapshots — readuiEngineSnapshot(parsed final TurnProps) anduiEngineSummary(compact per-message summary).assistantTextmay be empty in this mode.To force plain text on a UI-Engine-enabled agent for one turn, pass
disableUiEngine: true.
UI ENGINE — TWO LAYERS YOU MUST KEEP STRAIGHT:
LAYER 1 — AGENT LLM OUTPUT CONTRACT (what the tested agent is INSTRUCTED to emit):
The agent's system prompt forces its LLM to output ONE JSON ARRAY of message objects: [ { "type": "text"|"choice"|"visual"|"cardV2"|"carousel"|"iFrame"|"form"|"input", "payload": {...} }, ... ]
The very first character MUST be "[" and the very last MUST be "]". No prose, no ```json fences, no leading/trailing newlines.
Preferred message order in the array: text intro -> card / carousel / visual -> iFrame (if video URL exists) -> choice buttons (CTAs).
Buttons / image URLs / iFrame URLs MUST come from KB or tool output. NEVER invent URLs. If KB has nothing, omit the button or fall back to text.
select / radio fields MUST include a non-empty options[] of { value, label }. If real options are unknown, use type "text" or "textarea" instead.
form / input messages WEB CHANNELS ONLY (web-chat / text). On voice / telephony / WhatsApp / SMS / messaging the server strips them.
WhatsApp extra rule: only use visual / cardV2 / carousel when you have a verified image URL from KB or a tool — otherwise use text only.
LAYER 2 — WIRE FORMAT (what arrives over the /interact WebSocket):
When chunk.ui_engine === true, chunk.chunk is a JSON-stringified TurnProps SNAPSHOT of the bot turn so far.
SNAPSHOTS ARE OVERWRITING — replace the in-progress turn with the latest parse, never concatenate.
Parsed shape: { from: "bot", ts, modelId?, messages: ChatMessage[] }.
Each ChatMessage wraps the agent's array entry: { from: "bot", type, ts, mid?, item: { type, payload } }. So agent emits [{ type, payload }, ...] -> wire wraps each as { ..., item: { type, payload } } inside messages[].
PAYLOAD SHAPES (item.payload): text -> { message } (Markdown supported). choice -> { buttons: [{ name, request: { type: "open_url"|"text", payload: { url|message } } }] }. visual -> { image } (URL ending in .png/.jpg/.jpeg/.webp/.gif/.svg/.avif/.bmp). cardV2 -> { title, imageUrl?, description?: { text }, buttons?: UiEngineButton[] }. carousel -> { cards: CardPayload[] }. iFrame -> { layout: "vertical"|"horizontal", url } (vertical = video embeds, horizontal = general web). form -> { title?, description?, fields: UiEngineInputField[], submitButton?, onSubmit: { type: "submit_form", message? } }. Web channels only. input -> { field: UiEngineInputField, submitOnEnter?, onSubmit: { type: "submit_input", message? } }. Web channels only. UiEngineInputField.type: text|email|password|number|tel|url|textarea|select|checkbox|radio|date|time|datetime-local|file.
WHEN ui_engine === false (or omitted): chunk.chunk is plain Markdown text — append to the in-progress assistant message. TO FORCE PLAIN TEXT on a UI-Engine-enabled agent for one turn: pass disableUiEngine: true on the InteractRequest.
SERVER SANITIZERS YOU SHOULD KNOW (so you do not get surprised when a message disappears):
form messages with zero valid fields are DROPPED.
input messages with an invalid field are DROPPED.
select / radio fields with empty / duplicate / missing-label options are DROPPED.
any field missing id or label is DROPPED.
For the FULL UI Engine schema (every payload field, validation rules, allowed enums) call get_ui_engine_spec first.
WARNING: This consumes Convocore credits exactly like a real agent turn (LLM + voice + tools). It is NOT a dry-run.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| v2 | No | Force routing to the v2 (node-based) handler. | |
| raw | No | When true, include every raw streamed chunk in the response. Default false (token-cheap aggregated view only). | |
| bucket | No | Region bucket. Auto-derived from CONVOCORE_API_REGION when omitted (eu-gcp -> "voiceglow-eu", na-gcp -> "(default)"). | |
| isTest | No | Marks the turn as a test interaction. | |
| prompt | No | User message. Special values: "start" (initial greeting), "@cancel:<reason>", "@rewind:<nodeId>". | |
| agentId | Yes | The ID of the agent to interact with. | |
| convoId | Yes | Conversation ID. Reuse across turns to keep history; use a fresh ID to start a new conversation. | |
| replyTo | No | Reply context when the user is replying to a previous message. | |
| toolTest | No | Run a single tool in test mode. | |
| agentData | No | Optional agent override. When provided with at least an `ID`, the server skips loading the agent doc from Firestore. | |
| kbPreview | No | Knowledge-base preview mode (skips node routing). | |
| sessionId | No | Optional session id; defaults to convoId server-side. | |
| timeoutMs | No | How long to wait for the streamed turn to complete before closing the WebSocket. Default 120000 (120s), max 600000 (10min). | |
| isLLMStudio | No | Marks the turn as originating from LLM Studio. | |
| messageType | No | Type of input. Use "visual" together with visualPayload for images. | |
| turnsHistory | No | Optional override of conversation turns. When set, the server uses this instead of fetching from Firestore. | |
| visualPayload | No | Image/visual content for messageType: "visual" turns. | |
| workspaceData | No | Optional workspace override. | |
| actionMetadata | No | Includes mid (client-supplied message id) used for de-duplication. | |
| agentProfileId | No | Internal profile id (e.g. agency_plan_builder_v1). | |
| lightConvoData | No | Per-conversation user/context metadata (userName, userEmail, userPhone, origin, capturedVariables, ...). Surfaced to the agent system prompt where supported. | |
| disableUiEngine | No | Disable UI-engine JSON output for this turn. | |
| initNodesOptions | No | Optional overrides for tools / variables / messages history at session init. | |
| variablesOverrides | No | Create or override agent variables for this interact turn only (without permanently PATCHing variables). | |
| disableRecordHistory | No | Skip persisting this turn to Firestore. | |
| formSubmissionMetadata | No | Payload describing a UI-engine form / input submission. |