Testsy

Store a sign-in account the tester can use. The password is encrypted, never returned, and never sent to the tester's AI: the tester types it. Tests refer to the account by name ("Sign in as the account "customer""). Through MCP the password passes through your AI agent: prefer adding passwords in the web app or over the REST API from a secret store.

Add a call the tester may make into your system. It sees only the name, the description and what the call returned. Using endpoints in a run needs a paid plan. bodyTemplate may contain {{param}} placeholders the tester fills in. Headers (for example Authorization) are encrypted and never returned.

Add a file the tester can upload during the test: up to 25 MB, up to 10 files per test. REST: send the raw bytes as the body with the file name in the X-File-Name header (or ?name=), or JSON {name, contentBase64}. MCP: name and contentBase64. A file with the same name replaces the old one.

Finish the upload with every part's etag. The build is checked (app id, version, whether our devices can run it) and returned. Use the returned build.id in run_test {buildId} or run_project {buildIds} to run the new build. Uploading alone does not start a run. The command from create_build_upload completes the upload itself; call this only after uploading parts to the signed links yourself. Also accepts the signed completeUrl instead of an API key.

Start a build upload and get signed links that upload the file without an API key: run the returned command in a shell, or PUT each part and POST complete to the links. The build is then checked and ready to run. For AI agents: tool arguments cannot carry a build, so hand the bytes to a shell command. command downloads the command-line tool and uploads the file (parts, then complete). Without it: PUT the raw bytes of part n (partBytes each, the last one shorter) to partUrlTemplate with {partNumber} replaced, then POST {parts: [{partNumber, etag}]} to completeUrl. The links expire at expiresAt and work only for this upload. Call find_build first to skip uploading a file that is already there.

Create a project, for example one per site or app.

Run a test by itself: on chosen weekdays, or monthly on a day, at a time in a time zone. Each scheduled run counts like any other run.

A read-only link to a run's result, replay and video that anyone can open without signing in, until it expires.

Create a test of a website (url) or of an app (platform android or ios, and appId; device phone or tablet and osVersion, default a phone on the latest OS). Pass plan (from plan_test, after the requester confirmed it): it is stored as the approved checklist and the tester follows it as written. Or pass requirement, plain-language text the tester plans itself at run time. Never put passwords in the text: name a sign-in account in credentials.

A view-only link to watch a run live and read its result, with no sign-in, valid while it runs and for some hours after. Runs started here get one automatically; this makes another, or one for a run started from the web app. The link shows the live screen, the steps and checks so far, then the read-only result. membersOnly is true when the workspace limits watch links to signed-in members (or the run came from a public repository). 403 when the workspace turned watch links off.

Delete a build. Runs that used it keep their results; a build a test pins cannot be deleted.

Delete a sign-in account. Tests that sign in with it stop running until it is added again or they are edited.

Delete an endpoint.

Archive a project: its tests stop running; past runs stay in the run history.

Delete a schedule.

Turn off a share link straight away.

Archive a test: it stops running; past runs stay in the run history.

Remove one of a test's files.

Copy a test under a new title: the test's info is copied over (start address, requirement, plan, options and files). Its run history and schedules stay with the original. The copy counts toward the plan's test limit. Without title it is named " (copy)". Pass projectId to put the copy in another project of the workspace.

Is this build already uploaded? Looks up the project's newest ready build by the file's sha256, else by a native fingerprint; build is null when neither matches. A sha256 match means the exact file is uploaded: run it with run_test {buildId} / run_project {buildIds} instead of uploading again. A fingerprint match (a hash of the native inputs the uploader stored with the build, for example @expo/fingerprint) means the native build is unchanged, so CI can skip building it. The command-line tool does this for files: testsy-ai build find / build ensure.

One build: its app id, version, status and any warnings.

How our browsers reach your sites: the static addresses to allow-list, the country they browse from, and the countries available.

One project.

A run: its status, and once final the verdict, the headline, each check's result with the reason, the values read, and links to the video, screenshots and the result page.

One test: its start address, requirement, plan, options and latest run.

Plan, runs used this billing period, included runs, extra-run packs and runs left.

The workspace this key belongs to: its id, name and plan, and this period's usage.

Fetch a build from an https link (an EAS, Bitrise, S3 or GitHub artifact link, up to the build size limit) into the project; it is checked and ready to run. Nothing to upload from your side. The link must be https to a public host name; up to 3 redirects are followed. headers (optional, at most 5: Authorization or X-… names) are sent to the first host only, for a private artifact store. The file name comes from Content-Disposition or the link; it must be an .apk or a zipped .app. Run the result with run_test {buildId} or run_project {buildIds}. Call find_build first when the file may already be there.

The app builds uploaded to a project (Android .apk, iOS Simulator .app zipped), newest first, with the largest size and part size for uploads.

The sign-in accounts stored for a project: names, usernames and sites. Passwords are never returned.

The phones and tablets app tests can run on, and their OS versions (latest first). Pass device and osVersion to create_test, update_test, run_test or run_project. A new app test runs on a phone with the latest OS unless you choose otherwise; the version is saved with the test, so it keeps its OS until you change it.

Calls into your own systems the tester may make during a run (for example "create a voucher"). Headers are never returned.

The projects in this workspace (a project groups the tests of one site or app).

Recent runs, newest first, optionally for one test or project or with one status; or the runs with the given ids.

When tests run by themselves: every schedule in the workspace, or those of one test or project.

The share links made for a run, with when they expire and how often they were opened.

Files a test can upload during its steps (for example an import file).

The tests of a project, each with its latest run.

Have the assistant draft a test from a plain-language description, or revise a plan, without saving anything. Returns the proposed steps, checks, values to record, assumptions and any questions only the requester can answer. Show the plan to the requester; to change it, call again with plan and change; to answer questions, pass answers. Once they confirm, pass the plan to create_test. When no question is open the plan has passed the planning review (review.ok). Pass projectId so the assistant knows the project's sign-in accounts and endpoints. Each call uses one assistant draft of the workspace's daily allowance.

Turn off every watch link of a run straight away.

Start a run of every test in a project (a suite). Each started run has a watchUrl to watch it live without signing in. buildIds: builds to use; each app test runs the one of its own app. Tests that could not start are listed in skipped with the reason. REST: send an Idempotency-Key header to make a retry safe.

Start a run of one test. Runs take minutes: poll get_run until final is true. watchUrl opens the run live without signing in. App tests run the newest build of their app unless buildId names another (see list_builds). watchUrl is a view-only link anyone can open (no sign-in) to watch the run live and read its result for a while after it ends; give it to the person waiting for the result. Absent when the workspace turned watch links off. REST: send an Idempotency-Key header so a retried request does not start a second run.

Choose the country our browsers and devices reach your sites and app servers from (web and app tests), or automatic: the site's own country (an app test then uses our default country).

Stop a queued or running run. A run you stop is not counted.

Rename a project or change its description.

Pause or resume a schedule.

Change a test: its title, start address, requirement, plan or options. Only the fields you pass change; within options, only the options you pass change and the others keep their values.

Wait up to seconds (default 25, at most 55) for a run to finish, then return it: one call instead of polling get_run every few seconds. Call it again while final is false. The answer comes back the moment the run is final (verdict, checks and evidence in), or when the time is up with final false — then call wait_run again. Runs take 2 to 20 minutes, so expect several calls for a long run; each costs nothing while it waits.