Submit a test job
submit_testQueue a new TestMyVibes job for a given URL. You explicitly choose the runner: AI agent (headless Chromium + GPT-4o vision, fastest, deterministic for well-specified goals) or human checker (slower, better for visual/UX judgment calls). Returns a jobId you can poll with get_test_status.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to test. Must be publicly reachable. | |
| goal | No | AI runner only. CONCRETE success criterion the agent stops on — e.g. 'Reach a URL containing /dashboard', 'See a Welcome banner on the page header', 'Receive an OTP email and submit the code'. Without a goal the AI runs out its step budget on exploration instead of completing a flow. | |
| runner | No | Who runs the test. 'ai' = headless browser + GPT-4o vision agent (default; use for deterministic flows, signup/login, regression checks). 'human' = real human checker on TMV's panel (use for visual/UX judgment, complex flows the AI can't drive, accessibility passes). | ai |
| jobType | No | Test category — defaults to 'General QA'. Affects credit cost when billed. | General QA |
| priority | No | Job priority. | normal |
| targetOS | No | Human-runner advisory string for OS (e.g. 'iOS 17', 'Android 14', 'macOS 14'). Surfaced on the checker's claim card. | |
| viewport | No | Explicit viewport for non-preset resolutions (e.g. {width: 2560, height: 1440} for a 27" desktop monitor). Wins over devicePreset only when devicePreset is NOT set. Use devicePreset for known phones/tablets and viewport for custom resolutions. | |
| projectId | No | Existing project to attach this job to (optional). | |
| offeringId | No | Marketplace offering id (browse via list_worker_offerings). When set, this job is priced + routed through the marketplace: the customer pays the offering's `creditsCharged` and the worker who fulfills the job earns the offering's pre-locked `workerPayoutCredits` (75% of charged). When omitted, the legacy personality/step-based pre-flight quote applies. | |
| slaMinutes | No | Target turnaround in minutes (human runner only; AI runs finish in ~1-5 min regardless). | |
| description | Yes | Plain-English description of what to test. The platform uses this to seed a checklist. | |
| mcpEndpoint | No | MCP Auditor only. URL of the MCP server to audit (e.g. https://api.example.com/mcp). Pair with `personalityOfferingId: 'mcp-smoke'` or `'mcp-full-audit'`. The MCP Auditor runs JSON-RPC against this endpoint instead of opening a browser at `url`. | |
| recordVideo | No | AI runner only. WebM video recording of the entire browser session. Defaults on; set false to opt out. When true, the worker captures a continuous screencast via Puppeteer and uploads it to Spaces; signed URL surfaced in get_test_results.aiReport.videoUrl. Free — no credit charge. 30-day retention same as step screenshots. Ops can globally disable the default with AI_RECORD_VIDEO_DEFAULT=false. | |
| useSmsInbox | No | AI runner only. When true, TMV provisions a throwaway phone number from Paradise's SMS test-number pool (US/CA available) bound to this run. The agent uses it for any phone field, and `wait_for_sms` blocks until verification SMS arrive. Required for phone+OTP signup flows. Pool is finite — release reserves the number for ~15min then auto-releases. | |
| devicePreset | No | Optional device emulation. Pass a Puppeteer KnownDevices name (e.g. 'iPhone 14 Pro', 'iPad Mini', 'Pixel 5', 'Galaxy S9+') and the AI agent runs the test as that device — proper viewport, touch events, user-agent, and DPR. No markup; this is the same Chromium with different emulation flags. Use list_device_presets to see the full 131-device catalog or the curated featured subset. For human runners this is advisory and surfaced on the checker's job card. | |
| identityMode | No | AI runner only. 'auto' infers when a signup/OTP flow needs a TMV inbox/persona; 'fresh' forces a new persona/inbox for this run; 'keep' creates a managed retained identity with a persistent inbox and saves credentials after a passing signup; 'reuse' signs in with testIdentityId/existingPersonaId; 'none' disables identity provisioning. | auto |
| mcpTransport | No | MCP Auditor only. Transport protocol the customer's MCP server speaks. Most servers built with @modelcontextprotocol/sdk use streamable-http; older ones use sse. No stdio support (we don't run customer code in TMV's sandbox). | streamable-http |
| projectLabel | No | Audit label naming which of your projects submitted this test (e.g. 'pm-claude-code', 'shiftsee-claude-code'). Not used for auth. | |
| targetDevice | No | Human-runner advisory string naming the device (e.g. 'iPhone 14 Pro', 'Pixel 7'). Surfaced on the checker's claim card so they know which device to test on. No effect for AI runners. | |
| useTestInbox | No | AI runner only. When true, TMV provisions a per-job inbox at `<job-prefix>-<random>@inbox.testmyvibes.com` bound to this run. The agent uses it for any email field, and `wait_for_email` blocks until verification emails arrive. Required for OTP / email-verify flows; pointless for read-only tests. | |
| mcpAuthHeader | No | MCP Auditor only. Optional auth header passed to the MCP endpoint (e.g. 'Bearer <token>', 'X-API-Key: <key>'). Format: 'HeaderName: value'. Used verbatim on every JSON-RPC request. | |
| targetBrowser | No | Human-runner advisory string for browser (e.g. 'Safari', 'Chrome', 'Firefox'). Surfaced on the checker's claim card. | |
| videoCallTest | No | AI runner only. Video/voice call testing (human↔AI calls, WebRTC flows): Chrome launches with a fake camera+microphone (auto-granted; synthetic pattern/tone media the far side really receives) and every RTCPeerConnection on the page is instrumented. The agent gains the check_call_media action, returning hard metrics — ICE state, time-to-first-frame, fps, resolution, packet loss, freezes, and whether remote audio is actually AUDIBLE. The raw metric timeline is surfaced in get_test_results.aiReport.callStats. Screenshots cannot distinguish a live call from a frozen frame; instruct the agent to start the call, then use check_call_media (~10s settle), then re-check later to confirm the call is sustained. For human↔human two-browser calls use submit_interaction_scene with videoCallTest on each role. | |
| sessionCookies | No | AI runner only. Session injection — pre-authenticated cookies planted on the browser BEFORE the first navigation, so the agent starts already signed in and skips the login/OTP gate. Purpose-built for gated flows (photoreal video calls, member dashboards) where driving an email-OTP login with the vision agent is slow and flaky. Obtain a real session however you like (server-to-server auth, a scripted OTP redeem) and pass the cookies here; they're domain-scoped to the test URL at inject time and never echoed back in results. Combine with videoCallTest to land a logged-in agent directly on a call surface. | |
| testIdentityId | No | AI runner only. Managed retained identity id from list_test_identities/create_test_identity. If it already has credentials the worker signs in as that returning user; if not, the worker uses its persistent email/persona for a fresh signup and saves credentials on PASS. | |
| useFakeProfile | No | AI runner only. Adds depth to the test persona beyond default username/displayName/bio. 'basic' (+1 credit): generates a physicalProfile JSON (age, height, hair color, eye color, etc.) so any open-ended profile fields are filled with consistent realistic values. 'full' (+2 credits): basic + 2 photorealistic Flux Schnell photos uploaded to TMV Spaces and exposed to the agent as signed URLs for avatar / profile-image uploads. Skip this for read-only tests; use 'basic' for profile-completion tests; use 'full' for photo-required signup flows. | off |
| keepTestAccount | No | AI runner only. When false (default), signup tests end by deleting the account they created so customer user tables don't accumulate orphan rows. Set true to KEEP the account alive after the test — the persona's credentials are persisted so a later submit_test with `existingPersonaId` can sign in as a returning user (repeat-testing offering). Costs more (persona retention fee) but saves signup steps on every subsequent run. | |
| smsInboxCountry | No | AI runner only. Used with useSmsInbox=true. Country code of the throwaway number to rent. US (default) covers most American/Canadian flows; CA needed for sites that gate by destination country. India is NOT available (Telnyx has no IN inventory). | US |
| syntheticVisitor | No | Paradise Modern Growth Kit structured input. When present, TMV queues a Synthetic Visitor Test: an AI visitor simulation focused on CTA/A-B conversion completion rather than general QA. | |
| targetScreenSize | No | Human-runner advisory string (e.g. '1920x1080', '390x844'). Stored on the job and surfaced on the checker's claim card. No effect for AI runners — use devicePreset or viewport instead. | |
| agentInstructions | No | AI runner only. Verbal step-by-step the vision agent follows. Pin exact field values here (e.g. 'When asked for a name use "QA Tester"; when asked for a password use "TestPass!2026"'). Without this the agent invents values and tests become non-reproducible. By default these are advisory — set strictAgentInstructions=true to enforce them as hard rules. | |
| existingPersonaId | No | AI runner only. Task #30 repeat-test. Set to the personaId of a previously-kept persona (from a job submitted with keepTestAccount=true). The worker skips provisioning + signup and instead reuses the persona's stored email + password to log straight in. Use this to exercise return-user flows (profile edits, dashboards, settings, follow-up actions) without paying for signup every time. Discounted -1 credit per run; persona retention itself costs 2 credits per 30-day window (first persona per project free). Call list_device_presets to see all device names. | |
| personalityOfferingId | No | Personality menu offering id (browse via list_personality_offerings). Locks the step budget, inbox provisioning, personality, and price to the offering. Mutually exclusive with offeringId — offeringId routes to a worker; personalityOfferingId is an AI-only packaged test priced by TMV. | |
| strictAgentInstructions | No | AI runner only. When true, agentInstructions are enforced with a stronger preamble + post-step self-check ("Did my last action violate any rule? If yes, reverse course before continuing"). Use for OTP / mid-form flows where one wrong click (extra OTP request, dropdown change after submit) invalidates state. Default false — instructions are advisory, the agent uses judgment. | |
| expectedEmailFromContains | No | AI runner only. Pin the wait_for_email fromContains filter (substring of the sender address). Use when the sender domain isn't the obvious test target (e.g. delivered from sendgrid.net but the site is acme.com). | |
| provisionTestCardFundedUsd | No | AI runner only. Mints a Stripe-Issuing test-payment card just-in-time when the worker picks up this job, funded to this USD amount. The PAN is held in-memory only — never touches the Job record, never returned to the caller. The AI agent receives it via the system prompt and types it at the customer's checkout. Card is frozen automatically at end of run (or 24h, whichever first). Billed at funded + 25% markup + $2 service fee. Currently sandbox-only — cards auth against Stripe test-mode merchants only until live activation lands. | |
| expectedEmailSubjectContains | No | AI runner only. Pin a case-insensitive substring the AI agent MUST use as wait_for_email's subjectContains filter. Useful when your customer's verification email subject doesn't match the site name (e.g. site is 'newvibecity.com' but email subject is 'Newvibecityhotel sign-in code'). Without this, the agent guesses from the URL/brand and can timeout on wrong filters. Surfaced in the system prompt with strict instructions. |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | No | Tool result payload (JSON object) |