tarn-mcp is an MCP server that exposes Tarn's API testing framework to AI agents, enabling them to write, run, debug, and fix .tarn.yaml API tests programmatically.
Run API tests (tarn_run, tarn_run_agent): Execute .tarn.yaml test files with configurable environments, variable overrides, tag filters, and report modes. tarn_run_agent is the preferred entry point for iterative agent workflows, returning compact, root-cause-first output.
Validate test files (tarn_validate): Check .tarn.yaml files for YAML syntax and schema correctness without executing any HTTP requests.
List available tests (tarn_list): Discover and enumerate all tests and their steps across .tarn.yaml files in a project.
Generate fix plans (tarn_fix_plan): Analyze a Tarn JSON run report and return a prioritized, actionable remediation plan with next steps, evidence, and hints.
Inspect prior run failures (tarn_last_failures, tarn_last_root_causes): Read persisted failure groups from a prior run without re-executing tests; tarn_last_root_causes returns only root-cause groups for fastest failures-first planning.
Inspect run artifacts (tarn_get_run_artifacts, tarn_inspect): List artifact paths for a run, or drill into a specific run's archived report at run, file, test, or step granularity with optional failure category filtering.
Rerun only failed tests (tarn_rerun_failed): Replay only the failing (file, test) pairs from a prior run, enabling an efficient fix–rerun loop.
Render concise run reports (tarn_report): Generate a compact JSON summary with totals, exit code, and grouped failures from persisted artifacts—no test execution required.
Assess change impact (tarn_impact): Map changed source files, endpoints, or OpenAPI operations (or a git diff) to the .tarn.yaml tests most likely affected, with confidence tiers—read-only, no HTTP calls.
Scaffold new test files (tarn_scaffold): Generate .tarn.yaml skeletons from an OpenAPI operation ID, a raw curl command, an explicit method+URL, or a recorded fixture; optionally write the file to disk.
Pack remediation context (tarn_pack_context): Assemble a focused bundle of failing test information—including YAML snippets, request/response excerpts, capture lineage, and rerun hints—with truncation budgets for context-limited agents, in JSON or markdown format.
Integrates with Helix editor through tarn-lsp, providing language server support for Tarn test files.
Provides a Language Server Protocol (LSP) interface via tarn-lsp for Neovim, offering diagnostics, completions, and hover documentation for .tarn.yaml files.
Integrates with OpenAI Codex through an MCP server and skill, allowing Codex to perform API testing with Tarn.
Tarn is a CLI-first API testing tool written in Rust. Tests are .tarn.yaml files. Output is structured JSON with categorized failures and remediation hints, so an agent — Claude Code, Codex, opencode, Cursor, Windsurf, pi — can write a test, run it, read what broke, and fix it without scraping logs.
# tests/health.tarn.yaml
name: Health check
steps:
- name: GET /health
request:
method: GET
url: "{{ env.base_url }}/health"
assert:
status: 200$ tarn run
TARN Running tests/health.tarn.yaml
● Health check
✓ GET /health (4ms)
Results: 1 passed (15ms)When something breaks, --format json returns the same run as machine-readable data with failure_category, error_code, and the offending request/response. The tarn-mcp companion exposes a tarn_fix_plan tool that turns that report into actionable suggestions an agent can apply directly.
Structured failures, not log scraping — every failure carries a stable category, error code, and remediation hints. Agents branch on taxonomy, not regex.
MCP-native — tarn-mcp exposes list, validate, run, and fix_plan as structured tools for Claude Code, Codex, opencode, Cursor, and Windsurf (and pi via the skill + CLI). See AI agent integrations.
YAML the model already knows — no DSL to teach, no test framework to bootstrap. An LLM writes a .tarn.yaml and ships.
One static binary — curl | sh install, no runtime, drops into any CI image.
Batteries included — REST + GraphQL, captures, cookies, multipart, includes, polling, Lua, parallel execution, 7 output formats.
Related MCP server: mk-qa-master
# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/NazarKalytiuk/tarn/main/install.sh | sh
# from source
cargo install --git https://github.com/NazarKalytiuk/tarn.git --bin tarnPre-built binaries for macOS (Intel + Apple Silicon), Linux (amd64 + arm64), and Windows (amd64 zip) are on the releases page, each with a tarn-checksums.txt for SHA-256 verification and a generated tarn.rb Homebrew formula artifact. The installer also lays down tarn-mcp and tarn-lsp when present in the archive. Set TARN_INSTALL_DIR to install elsewhere. Container path: ghcr.io/<owner>/tarn:<tag> from the release workflow. Manual verification works with shasum -a 256 -c tarn-checksums.txt.
The 60-second path:
tarn init # scaffold tests/ + tarn.env.yaml + advanced templates
# edit tarn.env.yaml so base_url points at your API
tarn run # runs every .tarn.yaml under tests/Layer on the flags you actually need:
tarn run --format json --json-mode compact # structured output for agents and CI
tarn run --env staging # use a named environment
tarn run --only-failed # quiet down a noisy run
tarn run --watch # rerun on file changes
tarn run --parallel # run files in parallel
tarn list --tag smoke # what would run, without running
tarn fmt --check # canonical YAML, CI-gateableDefault to the failures-first loop — it keeps agents and humans off the megabyte-scale full report until they actually need it:
tarn validate <path> # syntax/config before running
tarn run <path> # writes .tarn/runs/<run_id>/
tarn failures # root-cause groups; cascades collapsed
tarn inspect last FILE::TEST::STEP # full context for ONE failure
# patch tests or application code
tarn rerun --failed # replay only failing (file, test) pairs
tarn diff prev last # confirm fixed / new / persistenttarn failures groups by root-cause fingerprint and collapses skipped_due_to_failed_capture cascades into their upstream entry — one failing step with five downstream skips surfaces as one entry with cascades: 5, not six. tarn inspect supports run-id aliases last / latest / @latest / prev and drills into one record via FILE[::TEST[::STEP]]. tarn rerun --failed stamps rerun_source onto the new report. tarn diff prev last buckets failure fingerprints into new / fixed / persistent so you can confirm a patch without re-reading the full report.
Reach for .tarn/runs/<run_id>/report.json only when failures + inspect cannot answer the question. See plugin/skills/tarn-api-testing/SKILL.md (Failures-First Loop) and docs/TROUBLESHOOTING.md for the canonical agent-facing guidance, including a worked example of a mutation endpoint whose response shape changed from {"uuid": "..."} to {"request": {"uuid": "..."}} and the $.uuid → $.request.uuid fix.
A fully local demo with no external network dependency:
PORT=3000 cargo run -p demo-server &
cargo run -p tarn -- run examples/demo-server/hello-world.tarn.yamlMore local scenarios — redirects, cookies, forms, error responses, authenticated CRUD — live in examples/demo-server/.
Full guides, CLI reference, AI workflow walkthroughs, and editor setup live on the docs site:
https://nazarkalytiuk.github.io/tarn/
In-repo docs to start with:
docs/INDEX.md — canonical map of all in-repo docs
docs/AI_WORKFLOW_DEMO.md — end-to-end agent loop walkthrough
docs/MCP_WORKFLOW.md — MCP server usage patterns
docs/TARN_PRODUCT_STRATEGY.md — product direction
docs/TARN_VS_HURL_COMPARISON.md and docs/HURL_MIGRATION.md — comparison and migration
docs/TARN_LSP.md — tarn-lsp Language Server (LSP 3.17) for Neovim, Helix, Zed, and other compatible clients
docs/VSCODE_EXTENSION.md — VS Code extension in editors/vscode
editors/zed/README.md — Zed extension, published via zed-industries/extensions
The reference sections below mirror what's on the docs site — useful when reading on GitHub directly.
Tarn drives any agent that speaks MCP or has a shell. tarn-mcp exposes list / validate / run / fix_plan (plus the failures-first tools); the tarn-api-testing skill teaches the loop. This table is the canonical supported-agents list — per-agent setup kits live under editors/.
Agent | How Tarn plugs in | Setup |
Claude Code |
| |
OpenAI Codex |
| |
opencode |
| |
pi |
| |
Cursor |
| |
Windsurf |
| |
Neovim / Helix / Zed / VS Code |
|
A reproducible write → run → read-failure → fix loop for any of these lives in examples/agent-loop/.
Test files use .tarn.yaml and can be organized in any directory structure.
name: Health check
steps:
- name: GET /health
request:
method: GET
url: "http://localhost:3000/health"
assert:
status: 200request.method accepts standard verbs and custom tokens such as PURGE or PROPFIND.
version: "1"
name: "User CRUD Operations"
description: "Tests complete user lifecycle"
tags: [crud, users, smoke]
env:
base_url: "http://localhost:3000/api/v1"
defaults:
headers:
Content-Type: "application/json"
timeout: 5000
retries: 1
tests:
create_and_verify:
description: "Create a user, then verify it exists"
tags: [smoke]
steps:
- name: Create user
request:
method: POST
url: "{{ env.base_url }}/users"
body:
name: "Jane Doe"
email: "jane.{{ $random_hex(6) }}@example.com"
capture:
user_id: "$.id"
assert:
status: 201
body:
"$.name": "Jane Doe"
"$.id": { type: string, not_empty: true }
- name: Verify user
request:
method: GET
url: "{{ env.base_url }}/users/{{ capture.user_id }}"
assert:
status: 200
body:
"$.id": "{{ capture.user_id }}"setup runs once before all tests. teardown runs after all tests even if tests fail.
name: "CRUD with auth"
setup:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "{{ env.admin_email }}"
password: "{{ env.admin_password }}"
capture:
auth_token: "$.token"
teardown:
- name: Cleanup
request:
method: POST
url: "{{ env.base_url }}/test/cleanup"
tests:
my_test:
steps:
- name: Authenticated request
request:
method: GET
url: "{{ env.base_url }}/users"
headers:
Authorization: "Bearer {{ capture.auth_token }}"
assert:
status: 200The most common case — POST (or PUT/PATCH) an endpoint with a JSON body. Write the body inline as YAML; Tarn serializes it as JSON and sets Content-Type: application/json automatically.
name: Create a user
steps:
- name: POST /users
request:
method: POST
url: "{{ env.base_url }}/users"
body:
name: "Jane Doe"
email: "jane@example.com"
role: "editor"
tags: ["content", "marketing"]
assert:
status: 201
body:
"$.id": { type: string, not_empty: true }body accepts any JSON value — object, array, string, number, boolean, or null. Templates resolve inside it, type-preservingly ("{{ capture.count }}" becomes the captured number, not a string):
request:
method: POST
url: "{{ env.base_url }}/orders"
body:
customer_id: "{{ capture.user_id }}"
idempotency_key: "{{ $uuid }}"
quantity: "{{ capture.qty }}" # stays a JSON numberbody_file)For larger or shared payloads, keep the JSON in its own file and reference it with body_file. The path resolves relative to the test file, the content is parsed as JSON, and it is interpolated exactly like an inline body (so {{ env }}, {{ capture }}, and builtins work inside the file). body_file is mutually exclusive with body.
request:
method: POST
url: "{{ env.base_url }}/users"
body_file: "payloads/create-user.json"// payloads/create-user.json
{
"name": "Grace Hopper",
"email": "grace.{{ $random_hex(6) }}@example.com",
"role": "admin"
}A missing file or invalid JSON fails that step with failure_category: parse_error (the run continues; it does not abort). Runnable end-to-end example: examples/post-json.tarn.yaml.
Sending a non-JSON body? Use
formfor URL-encoded data,multipartfor file uploads, orgraphqlfor GraphQL queries.
assert:
status: 200 # exact match
status: "2xx" # any 2xx status
status: # set of allowed codes
in: [200, 201, 204]
status: # range
gte: 400
lt: 500All body assertions use JSONPath expressions.
Equality:
body:
"$.name": "Alice" # string
"$.age": 30 # number
"$.active": true # boolean
"$.deletedAt": null # null
"$.field": { eq: "value" } # explicit
"$.field": { not_eq: "bad" } # inequalityNumeric comparisons:
body:
"$.age": { gt: 18, lt: 100 }
"$.count": { gte: 1, lte: 50 }String assertions:
body:
"$.email": { contains: "@example.com" }
"$.id": { starts_with: "usr_", matches: "^usr_[a-z0-9]+$" }
"$.name": { not_empty: true }
"$.notes": { empty: true }
"$.code": { length: 6 }
"$.msg": { not_contains: "error" }Format assertions:
body:
"$.request_id": { is_uuid: true }
"$.created_at": { is_date: true }
"$.client_ip": { is_ipv4: true }
"$.server_ip": { is_ipv6: true }Integrity assertions:
body:
"$": { bytes: 15 } # raw response body length
"$.payload": { sha256: "2cf24dba5fb0a30e..." } # matched value digest
"$.legacy": { md5: "5d41402abc4b2a76..." }Type checks:
body:
"$.name": { type: string }
"$.tags": { type: array, length_gt: 0 }
"$.meta": { type: object }Existence:
body:
"$.id": { exists: true }
"$.internal": { exists: false }Combined (AND logic):
body:
"$.id": { type: string, not_empty: true, starts_with: "usr_" }assert:
headers:
content-type: "application/json" # exact match
content-type: contains "application/json" # substring
x-request-id: matches "^[a-f0-9-]{36}$" # regexHeader names are case-insensitive.
assert:
duration: "< 500ms"
duration: "<= 1s"assert:
redirect:
url: "https://api.example.com/health"
count: 2redirect.url checks the final response URL after following redirects. redirect.count checks how many redirects were actually followed.
Priority | Source | Example |
1 (highest) | CLI |
|
2 | Shell env |
|
3 |
| (gitignored, for secrets) |
4 |
|
|
5 |
| default env file |
6 (lowest) | Inline | in the test file itself |
Capture values from responses to use in subsequent steps. Captured values preserve their original JSON types (numbers stay numbers, booleans stay booleans).
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "admin@example.com"
password: "password123"
capture:
token: "$.token" # JSONPath capture from body
user_id: "$.user.id" # nested path
- name: Use token
request:
method: GET
url: "{{ env.base_url }}/users"
headers:
Authorization: "Bearer {{ capture.token }}"Header capture — capture values from response headers with optional regex:
capture:
session_token:
header: "set-cookie"
regex: "session_token=([^;]+)"
request_id:
header: "x-request-id"Cookie capture — capture a response cookie by name from Set-Cookie:
capture:
session_cookie:
cookie: "session"Status capture — capture the HTTP status code as a number:
capture:
status_code:
status: trueFinal URL capture — capture the final response URL after redirects:
capture:
final_url:
url: trueJSONPath with regex — extract a sub-match from a body field:
capture:
user_id:
jsonpath: "$.message"
regex: "ID: (\\w+)"Whole-body regex — extract from the full response body string:
capture:
body_word:
body: true
regex: "plain (text)"Transform-lite in interpolation — reshape captured arrays and collections without dropping into Lua:
request:
form:
first_tag: "{{ capture.tags | first }}"
last_tag: "{{ capture.tags | last }}"
tag_count: "{{ capture.tags | count }}"
joined_tags: "{{ capture.tags | join('|') }}"
words: "{{ capture.message | split(' ') | count }}"
normalized: "{{ capture.message | replace(' response', '') }}"
status_code: "{{ capture.status_text | to_int }}"
payload: "{{ capture.user | to_string }}"first and last expect arrays. count works on arrays, objects, and strings. join(...) joins array items after converting each item to its string form. split(...) and replace(..., ...) operate on strings. to_int parses integer strings, and to_string stringifies any captured value.
# UUIDs
"{{ $uuid }}" # UUID v4 (alias for $uuid_v4)
"{{ $uuid_v4 }}" # random UUID v4
"{{ $uuid_v7 }}" # time-ordered UUID v7 (Unix-ms prefix)
# Random primitives
"{{ $random_hex(8) }}" # 8-char hex string
"{{ $random_int(1, 100) }}" # random integer in range
# Wall-clock
"{{ $timestamp }}" # unix timestamp
"{{ $now_iso }}" # ISO 8601 datetime
# Faker (EN locale)
"{{ $email }}" # random email
"{{ $first_name }}" "{{ $last_name }}" "{{ $name }}" "{{ $username }}"
"{{ $phone }}" # random phone number
"{{ $word }}" "{{ $words(3) }}" "{{ $sentence }}" "{{ $slug }}"
"{{ $alpha(8) }}" # n lowercase letters
"{{ $alnum(8) }}" # n lowercase alphanumerics
"{{ $choice(red, green, blue) }}"
"{{ $bool }}" # "true" or "false"
"{{ $ipv4 }}" "{{ $ipv6 }}"Reproducible runs. Set TARN_FAKER_SEED=<u64> (or faker.seed: <u64> in tarn.config.yaml) to freeze every RNG-backed built-in for the process. Wall-clock values ($timestamp, $now_iso, the timestamp prefix of $uuid_v7) stay real-time.
body:
"$.id": { is_uuid: true } # any UUID version
"$.legacy_id": { is_uuid_v4: true } # must be random v4
"$.event_id": { is_uuid_v7: true } # must be time-ordered v7Tarn automatically captures Set-Cookie headers and sends stored cookies on subsequent requests. This is enabled by default.
name: Auth flow
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "admin@test.com"
password: "secret"
# Set-Cookie from response is automatically stored
assert:
status: 200
- name: Access protected resource
request:
method: GET
url: "{{ env.base_url }}/profile"
# Cookie header is automatically sent
assert:
status: 200Disable automatic cookies per file:
cookies: "off"Or reset the default jar between named tests in a file so IDE subset runs and flaky suites never see session state from a prior test. Setup and teardown still share the file-level jar. Named jars (multi-user scenarios) are untouched.
cookies: "per-test"The --cookie-jar-per-test CLI flag forces per-test isolation regardless of the file's declared mode (except when the file sets cookies: "off", which always wins).
Tarn supports first-class bearer and basic auth helpers, while keeping explicit Authorization headers as the escape hatch:
request:
auth:
bearer: "{{ env.token }}"
headers:
X-API-Key: "{{ env.api_key }}"Basic auth:
request:
auth:
basic:
username: "{{ env.username }}"
password: "{{ env.password }}"You can also set defaults.auth once per file. If headers.Authorization is already present, Tarn leaves it unchanged.
Use cookies: false on a step to bypass the cookie jar entirely. No cookies are sent and no Set-Cookie headers are captured:
steps:
- name: Login
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "admin@test.com"
password: "secret"
assert:
status: 200
- name: Test unauthenticated access
cookies: false
request:
method: GET
url: "{{ env.base_url }}/profile"
assert:
status: 401For multi-user scenarios, use named jars to maintain separate cookie sessions:
steps:
- name: Login as admin
cookies: "admin"
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "admin@test.com"
password: "secret"
- name: Login as viewer
cookies: "viewer"
request:
method: POST
url: "{{ env.base_url }}/auth/login"
body:
email: "viewer@test.com"
password: "secret"
- name: Admin can manage users
cookies: "admin"
request:
method: GET
url: "{{ env.base_url }}/admin/users"
assert:
status: 200
- name: Viewer cannot manage users
cookies: "viewer"
request:
method: GET
url: "{{ env.base_url }}/admin/users"
assert:
status: 403Each named jar is independent — cookies captured in "admin" are never sent with "viewer" requests. Steps without a cookies: field (or with cookies: true) use the default jar.
Use tarn run --cookie-jar .tarn-cookies.json to preload jars from disk and write back the updated state after the run. The file stores named jars too, so multi-user sessions can survive across runs.
--cookie-jar currently works only with sequential execution. Combine it with --parallel only after jar sharing becomes deterministic.
When the cookie jar sends cookies automatically, frameworks with CSRF protection (e.g., Better Auth) may reject requests that lack an Origin header. Add it to defaults to fix:
defaults:
headers:
Content-Type: "application/json"
Origin: "http://localhost:3000"If your app derives the expected origin from the request URL, set Origin to match env.base_url:
defaults:
headers:
Origin: "{{ env.base_url }}"Send application/x-www-form-urlencoded payloads with form::
steps:
- name: Login form
request:
method: POST
url: "{{ env.base_url }}/auth/login"
form:
email: "user@example.com"
password: "{{ env.password }}"
assert:
status: 200Tarn URL-encodes the fields and auto-sets Content-Type: application/x-www-form-urlencoded unless you override it explicitly.
Note:
formcannot be combined withbody,graphql, ormultiparton the same step.
Send multipart form data for file uploads using the multipart: field:
steps:
- name: Upload photo
request:
method: POST
url: "{{ env.base_url }}/api/photos"
headers:
Authorization: "Bearer {{ capture.token }}"
multipart:
fields:
- name: "albumId"
value: "{{ capture.album_id }}"
- name: "title"
value: "Upload {{ $random_hex(8) }}"
- name: "description"
value: "A test upload"
files:
- name: "photo"
path: "{{ env.fixtures }}/test.jpg"
content_type: "image/jpeg"
- name: "thumbnail"
path: "./fixtures/thumb.png"
filename: "custom-name.png"
assert:
status: 201Every string in multipart — field names and values, plus each file's path, filename, and content_type — is interpolated exactly like a JSON body. {{ env.* }}, {{ capture.* }}, and builtins such as {{ $random_hex(8) }} all resolve before the request is sent, so multipart fields can chain captured IDs and randomized values for test isolation. (Capture values are coerced to strings, since multipart fields are inherently textual.)
Note:
multipartcannot be combined withbody,form, orgraphqlon the same step.
Reuse shared step sequences across test files with include: directives:
name: User tests
setup:
- include: ./shared/auth-setup.tarn.yaml
steps:
- name: Get users
request:
method: GET
url: "{{ env.base_url }}/users"
assert:
status: 200The included file's setup and steps are inlined at the include point. Includes work in setup, teardown, steps, and tests.*.steps. Circular includes are detected and rejected.
Includes also support lightweight parametrization and deep overrides for reusable step packs:
steps:
- include: ./shared/user-pack.tarn.yaml
with:
tenant: "acme"
user_id: 42
override:
request:
headers:
X-Tenant: "acme"Inside the included file, use {{ params.tenant }} and {{ params.user_id }} placeholders.
Native GraphQL support with the graphql: block. Automatically sets Content-Type: application/json and constructs the standard GraphQL JSON body.
steps:
- name: Get user
request:
method: POST
url: "{{ env.base_url }}/graphql"
graphql:
query: |
query GetUser($id: ID!) {
user(id: $id) {
id
name
email
}
}
variables:
id: "{{ capture.user_id }}"
operation_name: "GetUser"
assert:
status: 200
body:
"$.data.user.name": "Alice"
"$.errors": { exists: false }Re-execute a step until a condition is met. Useful for async workflows where you need to wait for a state change.
steps:
- name: Create export
request:
method: POST
url: "{{ env.base_url }}/exports"
capture:
export_id: "$.id"
assert:
status: 202
- name: Wait for completion
request:
method: GET
url: "{{ env.base_url }}/exports/{{ capture.export_id }}"
poll:
until:
body:
"$.status": "completed"
interval: "2s"
max_attempts: 15
assert:
status: 200
body:
"$.status": "completed"poll.until uses the same assertion syntax. The step re-executes every interval until the until condition passes or max_attempts is reached.
For logic that goes beyond declarative assertions, use inline Lua scripts. Scripts run after the HTTP response is received and have access to response and captures.
steps:
- name: Validate complex logic
request:
method: GET
url: "{{ env.base_url }}/users"
script: |
-- Access response
assert(response.status == 200, "Expected 200")
-- Work with the response body (Lua table)
local users = response.body.users
assert(#users > 0, "Expected at least one user")
-- Cross-field validation
for _, user in ipairs(users) do
assert(user.email:find("@"), "Invalid email for " .. user.name)
end
-- Set captures for subsequent steps
captures.first_user_id = users[1].id
assert:
status: 200Available in Lua:
response.status — HTTP status code
response.headers — response headers table
response.body — response body as Lua table (auto-parsed from JSON)
captures — read/write captures table
assert(condition, message) — assertion (collected, not thrown)
json.decode(string) — parse a JSON string into a Lua table
json.encode(value) — serialize a Lua value to a JSON string
Some test suites need to prep, stamp, or transform fixture files before running HTTP tests — for example, generating XML fixtures from a template or bumping a <ClassLibrary version> to a per-run unique value. A command: step runs an arbitrary shell command via sh -c (Unix) or cmd /C (Windows), with captured outputs feeding {{ capture.x }} like any HTTP step. Each step is either request: (HTTP) or command: (shell), never both.
setup:
- name: Bump fixture version
command:
run: "python3 bin/bump-version.py --version {{ $timestamp }}"
pass_env: [PATH, PYTHON]
workdir: "scripts/fixtures"
capture:
bumped_version:
stdout_regex: "version=([^\\s]+)"
exit_status:
exit_code: trueShell commands inside a .tarn.yaml could otherwise be a supply-chain attack vector — a malicious test file could exfiltrate secrets from the environment if cloning and running a freshly checked-out project executed shell automatically. Tarn refuses to spawn command: children unless one of these is set:
CLI flag: tarn run --allow-exec (preferred for ad-hoc runs, CI, and reruns)
Project config: allow_exec: true in tarn.config.yaml (for trusted, project-owned repos)
MCP: pass "allow_exec": true in the tarn_run / tarn_run_agent / tarn_rerun_failed tool params
Without an opt-in, every command: step is recorded with failure_category: skipped_by_policy and passed: true. The child process is never spawned. A freshly cloned repo therefore never executes shell on a default tarn run.
pass_envThe child process gets a tiny baseline env: PATH, HOME (USERPROFILE on Windows), and TMPDIR / TEMP / TMP. Tarn's own {{ env.x }} chain is never implicitly forwarded — secrets in tarn.env.local.yaml stay scoped to template interpolation. To forward an additional parent-process variable to the child:
command:
run: "deploy --token $API_TOKEN"
pass_env: [API_TOKEN]If you need a Tarn env variable in the shell, either interpolate it into command.run directly (run: "TOKEN={{ env.api_token }} deploy") or list the corresponding parent-process variable in pass_env.
Each entry under command.capture: must use exactly one of:
stdout_regex: "PATTERN" — capture group 1 (full match if no group). A miss fails the step under failure_category: command_failed unless optional: true.
exit_code: true — capture the literal exit code as an integer.
assert: and poll: are HTTP-only and rejected at parse time on command: steps.
failure_category: skipped_by_policy — benign skip; the run did not opt in. Step passed: true, exit code unaffected.
failure_category: command_failed — non-zero exit, signal kill (Unix), or a non-optional stdout_regex that did not match. Step passed: false, the run exits with 1.
tarn run [PATH] [OPTIONS] Run test files
tarn bench <PATH> [OPTIONS] Benchmark a step
tarn validate [PATH] [--format] Validate YAML (--format human|json)
tarn fmt [PATH] [--check] Normalize Tarn YAML
tarn list List all tests
tarn summary <PATH|-> Re-render a prior JSON report (llm/compact)
tarn import-hurl <PATH> Convert common-case Hurl files to Tarn
tarn init Scaffold a new project
tarn update Update to the latest version
tarn update --check Check for updates without installing
tarn completions <SHELL> Generate shell completionstarn run OptionsFlag | Description |
| Repeatable. Supports |
| For JSON outputs: |
| Filter by tag (comma-separated, AND logic) |
| Narrow execution to specific files, tests, or steps (repeatable; ANDs with |
| Override env variables (repeatable) |
| Load |
| Print full request/response for every step in the streaming progress |
| Include response body, headers, and captures in the report for every step (not just failed ones). Applies to |
| Cap the response body size embedded when |
| Show only failed tests and steps (summary counts stay accurate). |
| Disable streaming progress output; print the final report in one batch |
| Stream machine-readable NDJSON events to stdout (for editor integrations, MCP, structured CI) |
| Show interpolated requests without sending |
| Re-run on file changes |
| Run test files in parallel (see Parallel Execution) |
| Number of parallel workers (default: CPU count) |
| Suppress the |
tarn run # all tests
tarn run tests/auth.tarn.yaml # specific file
tarn run --tag smoke # filter by tag
tarn run --env staging # staging env
tarn run --var base_url=http://localhost:8080 # override var
tarn run --format json # JSON for LLM/CI
tarn run --format json --json-mode compact # smaller JSON for automation loops
tarn run --format html # HTML dashboard
tarn run --format curl # failed requests as curl
tarn run --format curl-all=reports/replay.sh # full suite replay script
tarn run --format human --format json=reports/run.json --format junit=reports/junit.xml
tarn run --format human,json=reports/run.json # comma-separated also works
tarn run --watch # re-run on changes
tarn run --parallel --jobs 4 # parallel execution
tarn run -v # verbose
tarn run --dry-run # preview only
tarn run --only-failed # hide passing tests, show failures only
tarn run --no-progress # disable streaming, batch dump at end
tarn run --only-failed --format json # CI-friendly: only failed items in JSON
tarn run --select tests/users.tarn.yaml::login # run just the "login" test in one file
tarn run --select tests/users.tarn.yaml::login::2 # run just step index 2 of login
tarn run --select "a.tarn.yaml::login" --select "b.tarn.yaml::checkout" # union across files
tarn fmt tests/ # rewrite a directory in place
tarn fmt tests/auth.tarn.yaml --check # CI-style formatting check--format llm)--format llm emits a grep-friendly summary line followed by only the
failed steps, each expanded with request, response, and the assertion
that failed. It strips ANSI colors automatically when stdout is piped
and omits boxed/colored headers entirely.
tarn run tests/ # emits llm format when piped (auto-selected)
tarn run tests/ --format llm # force llm format explicitly
tarn summary .tarn/last-run.json # re-summarize a prior run as llmtarn summary reads a prior JSON report (.tarn/last-run.json is
written after every run, or whatever you produced with tarn run --format json) and re-renders it as the llm format without re-running
the tests. Use - to stream from stdin:
tarn run tests/ --format json > run.json
tarn summary run.json # render run.json as llm
cat run.json | tarn summary - # same, piped
tarn summary run.json --format compact # render as compact insteadThe sibling --format compact format is a shorter human-ish variant
for quick console scanning — one line per file, inline expansion of
failed tests, and a trailing HTTP 500: 3 | JSONPath mismatch: 18
tally of failure categories. Both formats strip colors in non-TTY
output.
tarn validate --format json)tarn validate --format json emits a machine-readable report so editors and CI can surface parse errors inline. The schema:
{
"files": [
{
"file": "tests/users.tarn.yaml",
"valid": false,
"errors": [
{ "message": "found unexpected end of stream ...", "line": 14, "column": 7 }
]
}
]
}line and column are populated for YAML syntax errors (derived from serde_yaml's error location).
Parser semantic errors (unknown fields, shape mismatches) surface message only when the underlying error does not carry a location.
Exit code is 0 when every file is valid, 2 otherwise. The human format (--format human, the default) is unchanged.
tarn env --json)tarn env --json prints the project's named environments in a stable schema so editors can populate pickers and previews:
{
"project_root": "/path/to/project",
"default_env_file": "tarn.env.yaml",
"environments": [
{
"name": "staging",
"source_file": "tarn.env.staging.yaml",
"vars": {
"base_url": "https://staging.example.com",
"api_token": "***"
}
}
]
}Inline vars from tarn.config.yaml are redacted when the key matches redaction.env (case-insensitive), so tarn env --json never prints literal secrets. Environments are sorted alphabetically by name.
tarn run --parallel dispatches test files across multiple rayon workers. The parallelism unit is the file: all setup, teardown, captures, and cookie jars stay file-scoped, but files may execute concurrently. That's unsafe when tests share mutable state (DB rows, singletons, filesystem fixtures, rate-limited upstreams). Tarn ships three isolation primitives to close the gap:
serial_only: true on a TestFile (top-level) or on an individual named test under tests: pins the file onto a single worker that runs sequentially after every parallel bucket completes. A single serial_only test escalates its whole file to the serial bucket so per-file isolation (setup/teardown, cookie jars) stays intact.
group: "postgres" on a TestFile buckets files by resource name. Files sharing a group run on the same worker (serialized within the group), while different groups run in parallel. Use this to serialize "all the postgres tests" without giving up concurrency across unrelated resources.
parallel_opt_in: true in tarn.config.yaml silences the startup warning once the suite has been audited. While this flag is absent (or set to false), running tarn run --parallel emits a one-line stderr warning: warning: --parallel enabled without parallel_opt_in: true in tarn.config.yaml. Tests without serial_only may share state. Pass --no-parallel-warning on the CLI to suppress it for one-off CI runs.
# tarn.config.yaml
parallel: true
parallel_opt_in: true # opts in once the suite has been audited
# any .tarn.yaml that shares DB state
name: Users CRUD
serial_only: true
# or bucket by resource so postgres tests serialize while S3 tests run in parallel
name: Postgres integration
group: postgresBy default tarn run streams per-test output as each test finishes instead of dumping everything at the end. The behaviour adapts to how stdout is used:
Sequential (default) — each test is printed the moment it completes. You see progress live as the suite runs.
Parallel (--parallel) — each file is printed atomically when it completes, so output from concurrently running files never interleaves.
Stdout is human — streaming writes directly to stdout and the final emit prints only the summary line (no duplication).
Stdout is a structured format (json, junit, tap, html, curl) — progress streams to stderr so stdout stays pure and parseable.
Pass --no-progress to disable streaming entirely and restore the old "batch at end" behaviour (useful for CI logs that already capture per-line timestamps).
--ndjson)tarn run --ndjson streams machine-readable events to stdout, one JSON object per line. Designed for editor integrations (live Test Explorer updates), MCP clients, and CI pipelines that want structured progress without post-processing the final report.
Event types, in order:
file_started — a test file has begun running
step_finished — one step finished (with phase: "setup" | "test" | "teardown"). On failure, also carries failure_category, error_code, and assertion_failures[]
test_finished — a named test finished, with per-step counts
file_finished — a file finished, with its own summary
done — emitted once at the very end, carrying the aggregated summary for the whole run
--ndjson composes with file-bound --format targets, so you can stream live progress and write a final report at the same time:
# Stream NDJSON to stdout, final JSON report to disk
tarn run --ndjson --format json=reports/run.json | jq '.event'
# Pure NDJSON (default human output is silently dropped on stdout)
tarn run --ndjson--ndjson collides with any other structured format writing to stdout (e.g. --format json). Route the other format to a file, or pick one of the two streams.
In parallel mode (--parallel), each file's event stream is emitted atomically on file_finished so events from concurrently running files never interleave.
--only-failed works with both streaming and batch modes: passing tests and steps are omitted everywhere, but the final summary still reports total passed/failed counts.
Code | Meaning |
| All tests passed |
| One or more tests failed |
| Configuration/parse error |
| Runtime error (network, timeout, script) |
You can emit multiple formats in one run. Keep at most one bare non-HTML format for stdout and send the rest to files:
tarn run \
--format human \
--format json=reports/run.json \
--format junit=reports/junit.xml \
--format html=reports/run.html \
--format curl=reports/failures.sh \
--format curl-all=reports/replay.sh--format json)Structured JSON with versioned schema. Key design:
schema_version: 1 for forward compatibility
Full request/response included only for failed steps
failure_category on failures: assertion_failed, response_shape_mismatch, connection_error, timeout, parse_error, capture_error, unresolved_template, skipped_due_to_failed_capture, skipped_due_to_fail_fast
Stable error_code and remediation_hints are included on failed steps for automation-friendly diagnostics
response_status and response_summary on all executed steps (passed and failed) — AI agents can see what a passed step returned
captures_set on steps listing which capture variables were set; captures map on test groups showing all resolved values
--json-mode compact keeps the same top-level schema but drops passed assertion details and truncates response bodies to ~200 chars
Sensitive headers are redacted by default and can be customized per file with top-level redaction:
request is present for failed executed steps; response is omitted for connection/setup failures where no response exists
Schema files:
test files: schemas/v1/testfile.json
JSON report output: schemas/v1/report.json
{
"schema_version": 1,
"summary": { "status": "FAILED", "steps": { "total": 5, "passed": 4, "failed": 1 } },
"files": [{
"tests": [{
"captures": { "user_id": "usr_123", "token": "abc" },
"steps": [{
"name": "Create user",
"status": "PASSED",
"response_status": 201,
"response_summary": "201 Created: Object{3 keys}",
"captures_set": ["user_id"]
}, {
"name": "Update user",
"status": "FAILED",
"response_status": 400,
"response_summary": "400 Bad Request: name required",
"failure_category": "assertion_failed",
"error_code": "assertion_mismatch",
"remediation_hints": ["..."],
"assertions": {
"failures": [{ "assertion": "status", "expected": "201", "actual": "400", "message": "..." }]
},
"request": { "method": "POST", "url": "..." },
"response": { "status": 400, "body": { "error": "name required" } }
}]
}]
}]
}--format curl, --format curl-all)curl exports only failed executed requests. curl-all exports every executed request in run order, including setup and teardown.
tarn run --format human --format curl=reports/failures.sh
tarn run --format curl-all=reports/replay.shAlso supports: Human (colored terminal), JUnit XML, TAP, HTML (self-contained dashboard).
Example:
redaction:
headers:
- authorization
- x-session-token
env:
- api_token
captures:
- session_token
replacement: "[redacted]"Reuses your existing test files for benchmarking.
tarn bench tests/health.tarn.yaml -n 1000 -c 50
tarn bench tests/health.tarn.yaml -n 500 -c 25 --ramp-up 5s
tarn bench tests/health.tarn.yaml --format json # for CI thresholds
tarn bench tests/health.tarn.yaml --format csv --export json=reports/bench.json
tarn bench tests/health.tarn.yaml --fail-under-rps 200 --fail-above-p95-ms 80 TARN BENCH GET http://localhost:3000/health — 200 requests, 20 concurrent
Requests: 200 total, 200 ok, 0 failed (0.0%)
Throughput: 3125.0 req/s
Latency:
min 1ms
p50 2ms
p95 43ms
p99 45ms
max 45msTarn includes an MCP (Model Context Protocol) server for direct integration with AI coding tools.
The simplest approach is a project-level .mcp.json in the repo root (works with Claude Code and other MCP-compatible tools):
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}Alternatively, add to your Claude Code project settings (.claude/settings.json):
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}For Cursor, add to .cursor/mcp.json:
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp"
}
}
}Tool | Description |
| Run tests, returns structured JSON results |
| Validate YAML syntax without executing |
| List all tests and their steps |
| Analyze a Tarn JSON report and return prioritized next actions |
The MCP server lets your AI agent write .tarn.yaml tests, execute them, parse structured results, and iterate — all without leaving the editor.
Typical agent loop:
tarn_list to discover tests and steps
tarn_validate after generating YAML
tarn_run to get structured failures
tarn_fix_plan to turn the latest report into machine-friendly next steps
inspect failure_category, error_code, assertions.failures, and optional request/response
patch the test or application code
rerun until summary status is PASSED
See docs/MCP_WORKFLOW.md, docs/AI_WORKFLOW_DEMO.md, and docs/CONFORMANCE.md.
Tarn ships two Claude Code plugins from a single marketplace. They solve different problems and can be installed independently or together:
tarn (top-level plugin/) — bundles the tarn-mcp MCP server and the tarn-api-testing skill. Gives your agent structured API testing capabilities: tarn_run, tarn_validate, tarn_list, tarn_fix_plan.
tarn-lsp (editors/claude-code/tarn-lsp-plugin/) — registers the tarn-lsp language server with Claude Code's LSP plugin system so you get full .tarn.yaml language intelligence (diagnostics, hover, completion, code lens, code actions, quick fix, rename, go-to-definition, and the JSONPath evaluator) while editing in Claude Code.
Both plugins live in the same marketplace (the repo root .claude-plugin/marketplace.json). Register it once, then install either or both:
# 1. Register the marketplace (once)
claude plugin marketplace add NazarKalytiuk/tarn
# 2a. Install the MCP + skill plugin
claude plugin install tarn@tarn
# 2b. Install the LSP plugin (project scope — see caveat below)
claude plugin install tarn-lsp@tarn --scope projectAfter installing tarn, Claude Code can write, run, and debug .tarn.yaml tests directly via the bundled MCP server and skill. See MCP Server and Claude Code Skill for what each component provides.
tarn — MCP + skill pluginIf you prefer manual configuration, add the MCP server to a project-level .mcp.json in the repo root:
{
"mcpServers": {
"tarn": {
"command": "tarn-mcp",
"args": []
}
}
}This is equivalent to configuring the MCP server in .claude/settings.json but is portable across editors and tools that support MCP.
The tarn plugin configuration lives in .claude-plugin/:
plugin.json — name, version, description, author, and repository URL
marketplace.json — marketplace listing with owner info and the plugin registry (both tarn and tarn-lsp)
tarn-lsp — language server pluginSeparate install from the MCP plugin above (same marketplace though). This one registers tarn-lsp for .tarn.yaml / .yaml / .yml via Claude Code's LSP plugin system so every feature documented in docs/TARN_LSP.md is available while you edit in Claude Code.
Prerequisites:
Claude Code 2.0.74+
tarn-lsp binary available on $PATH — install with cargo install --path tarn-lsp from this repo, or symlink a workspace build (ln -s $(pwd)/target/release/tarn-lsp /usr/local/bin/tarn-lsp)
Install (from inside a Claude Code session):
/plugin marketplace add NazarKalytiuk/tarn
/plugin install tarn-lsp@tarn --scope project
/reload-pluginsAlready registered the marketplace for the tarn plugin? Skip the add line — both plugins share the same marketplace now. Substitute /absolute/path/to/repo for NazarKalytiuk/tarn if you want to install from a local checkout instead.
Compound-extension caveat: Claude Code's LSP plugin system only supports simple file extensions, so the tarn-lsp plugin claims all .yaml and .yml files in any project it is installed in (not just .tarn.yaml). Always install with --scope project in Tarn-focused repos only — do not install it globally if you also edit unrelated YAML in Claude Code.
See editors/claude-code/tarn-lsp-plugin/README.md for the full spec, troubleshooting, and the list of supported LSP features.
opencode supports Tarn through config only — there is no plugin installer or marketplace, so integration is three files checked into your repo:
your-repo/
├── opencode.jsonc # MCP + LSP registration
└── .opencode/skills/tarn-api-testing/ # agent-visible skill
└── SKILL.mdThis repo ships exactly this layout at opencode.jsonc and .opencode/skills/tarn-api-testing/ (the skill is a symlink to the canonical plugin/skills/tarn-api-testing/). Clone the repo, install tarn-mcp and tarn-lsp on $PATH, run opencode inside — MCP tools, .tarn.yaml diagnostics/hover/completion, and the tarn-api-testing skill light up immediately.
To mirror the setup in your own repo, copy the snippet from editors/opencode/opencode.example.jsonc into your own opencode.jsonc:
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"tarn": { "type": "local", "command": ["tarn-mcp"], "enabled": true }
},
"lsp": {
"tarn": { "command": ["tarn-lsp"], "extensions": [".yaml", ".yml"] }
}
}Compound-extension caveat: opencode's LSP matcher uses path.parse(file).ext, so the tarn LSP entry claims every .yaml / .yml file in the workspace (not just .tarn.yaml) — the same limitation as Claude Code. Keep this in project-level opencode.jsonc, not your global config.
See editors/opencode/README.md for prerequisites, troubleshooting, and the full skill-install flow.
The skills/tarn-api-testing/ directory contains a Claude Code skill that teaches AI agents how to write, run, debug, and iterate on Tarn tests. The skill is automatically loaded when an agent encounters API testing tasks.
What the skill provides:
Core workflow (write → validate → run → inspect → fix → rerun)
Complete command reference with all CLI flags
Test file format with minimal and full-featured examples
Environment variable resolution chain
Capture formats (JSONPath, headers, cookies, URL, status, body)
Assertion operator quick reference
JSON output schema and failure category taxonomy
Diagnosis loop for structured failure triage
MCP integration setup for Claude Code, opencode, Cursor, and Windsurf
Reference docs in skills/tarn-api-testing/references/:
File | Contents |
| Complete |
| Every assertion operator with examples |
| Structured JSON report schema and diagnosis algorithm |
| MCP server setup and tool reference |
The skill triggers on keywords like "API test", "tarn", ".tarn.yaml", "test this endpoint", "smoke test", and "integration test for API".
See docs/TROUBLESHOOTING.md for the full
guide, including the NestJS-style route ordering
trap that Tarn flags automatically.
Common cases:
connection_error: server is down, wrong host/port, DNS issue, TLS/connect failure
timeout: step timed out before receiving a complete response
assertion_failed: request succeeded, but status/header/body/duration check failed
capture_error: the step passed assertions, but extraction failed afterward
parse_error: invalid YAML, invalid JSONPath, or invalid config surface
Agent diagnosis loop:
run tarn validate first for syntax/config errors
run tarn run --format json
read failure_category before reading the message text
if a failed status assertion carries hints, follow the first hint before second-guessing the test
if response exists, inspect it before editing assertions
if request.url still contains {{ ... }}, fix env/capture interpolation before retrying
Non-JSON bodies:
Tarn preserves plain text / HTML responses as JSON strings in the structured report
use body: { "$": "plain text response" } to assert the whole root string when needed
Tarn does not aim for full Hurl parity. The main intentionally unclosed gaps are:
XPath / HTML assertions and captures
full Hurl-style filter DSL
exotic auth and libcurl-specific transport features
OpenAPI-first generation workflows
- uses: NazarKalytiuk/tarn@v1
with:
path: tests/
format: junit
env: stagingInputs:
Input | Default | Description |
|
| Test file or directory |
|
| Output format |
| — | Environment name |
| — | Tag filter |
|
| Tarn version |
| — | Variables (newline-separated |
tarn.config.yaml (optional)test_dir: "tests"
env_file: "tarn.env.yaml"
timeout: 10000
retries: 0
parallel: false
parallel_opt_in: false # set to `true` once tests have been audited for cross-file state sharing; silences the --parallel warning
defaults:
connect_timeout: 1000
follow_redirects: true
redaction:
headers: ["authorization", "cookie"]
environments:
staging:
env_file: "env/staging.yaml"
vars:
base_url: "https://staging.example.com"Behavior:
test_dir sets the default discovery directory for tarn run, tarn validate, and tarn list
env_file changes the root env file name; Tarn also checks .{name} and .local variants
defaults acts as project-wide request policy for headers/auth/timeouts/retries/redirects/delay
redaction provides a project-wide default report sanitization policy
environments makes named --env profiles first-class and powers tarn env
parallel: true makes parallel file execution the default for tarn run
parallel_opt_in: true acknowledges the isolation tradeoff (see Parallel Execution) and silences the --parallel warning; pair it with serial_only: and group: markers on files that share mutable state
defaults:
headers:
Content-Type: "application/json"
timeout: 5000
retries: 1
delay: "100ms" # default delay before each requestretries: 3 # retry up to 3 times on failure (exponential backoff)timeout: 30000 # 30 seconds for this stepdelay: "2s" # wait before executingMark an individual step so the report always records its response body,
response headers, and captures — even when the step passes. Equivalent
to running with --verbose-responses but scoped to a single step:
- name: fetch user
debug: true # keep response in the report for this step
request:
method: GET
url: "{{ env.base_url }}/users/42"
assert:
status: 200The global --verbose-responses flag plus --max-body <BYTES> give
the same behavior for every step in the run; debug: true is a
targeted override for one-off debugging. Bodies exceeding the
--max-body cap (8 KiB by default) are truncated with a
"...<truncated: N bytes>" marker.
Add to the top of your .tarn.yaml files for IDE autocompletion:
# yaml-language-server: $schema=https://raw.githubusercontent.com/NazarKalytiuk/tarn/main/schemas/v1/testfile.json
name: My test
steps: ...The schema is bundled at schemas/v1/testfile.json in the repository.
The structured report schema is bundled at schemas/v1/report.json.
A full-featured Tarn extension lives in editors/vscode and is published from tagged releases to both the VS Marketplace (nazarkalytiuk.tarn-vscode) and Open VSX via .github/workflows/vscode-extension-release.yml. Current version: 0.6.1.
Test Explorer discovery — .tarn.yaml files are indexed into a file → test → step tree. Run and Dry Run profiles, cancellable runs, and live streaming via tarn run --ndjson keep the UI in sync with long runs.
CodeLens above every test and step for Run, Dry Run, and Run step — no Test Explorer navigation required.
Rich failure peek view — on failure you get a unified diff of expected vs actual, plus the full request, response, remediation hints, failure_category, and error_code pulled straight from Tarn's JSON report.
Tag filter command and an "Install / Update Tarn" helper command, both surfaced in the command palette.
Output channel streams tarn stdout/stderr for each run.
Environment picker with a status-bar entry, persisted per workspace.
tarn.defaultEnvironment setting for the initial pick.
Status bar entries summarize active environment, tag filter, and last-run status.
Tarn file association for *.tarn.yaml and *.tarn.yml.
Full JSON schema validation for both test files and tarn-report.json via the redhat.vscode-yaml extension dependency.
Snippet library for test skeletons, polling, multipart, GraphQL, form requests, and includes. Prefix and coverage details live in editors/vscode/README.md.
Experimental LSP client (off by default) — the window-scoped tarn.experimentalLspClient setting spawns the tarn-lsp server alongside the extension's in-process providers. This is Phase V scaffolding; no feature has migrated to the LSP path yet, so leave it disabled unless you are testing the handoff.
Trusted / Untrusted workspace aware. In untrusted workspaces, Tarn features run in a read-only mode — discovery and schema validation still work, but commands that would execute tarn are disabled.
Remote Development audited end-to-end: Dev Container, GitHub Codespaces, WSL, and Remote SSH all work without additional configuration. See docs/VSCODE_REMOTE.md.
The extension exports a TarnExtensionApi for other extensions to consume:
const tarn = vscode.extensions
.getExtension('nazarkalytiuk.tarn-vscode')
?.exports as TarnExtensionApi | undefined;See editors/vscode/docs/API.md for the surface and stability guarantees.
Canonical user docs: editors/vscode/README.md
Design / internals: docs/VSCODE_EXTENSION.md
Changelog: editors/vscode/CHANGELOG.md
A Zed extension lives in editors/zed and is published to the zed-industries/extensions registry under the id tarn. The extension wraps the same tarn-lsp binary used by the VS Code extension — installing from Zed's Extensions panel auto-downloads the matching tarn-lsp release on first activation.
Coverage:
Syntax highlighting for .tarn.yaml / .tarn.yml, backed by tree-sitter-yaml.
Full tarn-lsp language intelligence: diagnostics, completion, hover, code actions, code lens, formatting, symbols, rename, references.
Snippet library ported from the VS Code extension (tarn-test, tarn-step, tarn-capture, tarn-poll, tarn-form, tarn-graphql, tarn-multipart, tarn-lifecycle, tarn-include).
Runnable tasks: tarn: run file, tarn: dry-run file, tarn: validate file, plus whole-workspace variants. Accessible from the task picker or the gutter runnable at the top of each file.
Settings passthrough via lsp.tarn-lsp.settings in Zed's settings.json, forwarded to tarn-lsp as workspace/configuration.
Zed has no custom UI surface for extensions, so the VS Code-only features (Test Explorer tree, environment picker, run-history panel, HTML report viewer, walkthrough) are not ported. Users who need them stay on VS Code; Zed users rely on LSP-driven feedback and the task runner.
See editors/zed/README.md for install and configuration.
tarn completions bash > /etc/bash_completion.d/tarn
tarn completions zsh > ~/.zsh/completions/_tarn
tarn completions fish > ~/.config/fish/completions/tarn.fishgit clone https://github.com/NazarKalytiuk/tarn.git
cd tarn
cargo build # build
cargo test --all # test suite
cargo clippy # lint
cargo fmt # format
bash scripts/ci/smoke.sh # release-path smoke test
# Run demo server + examples
PORT=3333 cargo run -p demo-server &
cargo run -p tarn -- run examples/ --var base_url=http://localhost:3333See docs/RELEASE_VERIFICATION.md for the broader release-candidate checklist, including watch-mode and installer verification.
Pipeline: parse YAML → resolve env → interpolate → execute HTTP → assert → report
Module | Role |
| Serde structs for |
| YAML loading + validation |
| 6-layer env resolution |
|
|
| Orchestrator (setup → tests → teardown) |
| HTTP client (reqwest) |
| JSONPath + header extraction |
| Automatic cookie jar |
|
|
| Built-in functions ( |
| Seedable RNG source for built-ins ( |
| Self-update mechanism |
| Status, body, headers, duration |
| Human, JSON, JUnit, TAP, HTML |
| Lua scripting engine (mlua) |
| File watcher (notify) |
| Performance testing (async) |
MIT
Analyze a Tarn JSON report and return a prioritized fix plan with next actions, evidence, and remediation hints. Accepts either a report object from tarn_run or the same inputs as tarn_run to execute first.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root used when `path` is provided. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| env | No | Environment name used when `path` is provided | |
| max_items | No | Limit the number of failing steps included in the plan | |
| path | No | Optional .tarn.yaml path or directory to run before planning. Relative paths resolve against `cwd`. | |
| report | No | Structured JSON report from tarn_run | |
| tag | No | Tag filter used when `path` is provided | |
| vars | No | Variable overrides used when `path` is provided |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It mentions the tool analyzes and returns a plan, but doesn't state whether it has side effects (e.g., creates files, modifies state) or if it is read-only. The name suggests planning, but confirmation is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the core purpose, the second explains the two input modes. No extraneous words or repetition. Every sentence adds essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and 7 parameters (including nested objects), the description is fairly complete about inputs but omits details about the output format (e.g., structure of the plan) and any prerequisites. It covers the two modes well but could describe what the plan contains for an agent to process the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The description adds significant value by clarifying the two invocation modes: either supplying a `report` or the same inputs as tarn_run (path, env, tag, vars). This contextual information is not captured in the individual parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool analyzes a Tarn JSON report to produce a prioritized fix plan with next actions, evidence, and remediation hints. It distinguishes from siblings like tarn_run (which runs tests but doesn't plan) by explicitly mentioning both modes: accepting a report or running tarn first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the two usage modes—passing a report object or the same inputs as tarn_run—which helps an agent decide how to invoke it. However, it lacks explicit guidance on when not to use it or comparison to alternatives like tarn_run_agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return artifact paths plus existence flags for a specific run. Does not load any artifact payload — just tells the agent what is on disk for the given run.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| run_id | No | Run identifier or alias. Defaults to `last`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description effectively discloses the key behavioral trait: the tool is read-only and does not load artifact payloads. This gives the agent a clear understanding of the tool's lightweight, non-destructive nature. Additional details about error handling or required privileges are omitted, but the core behavior is well communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two short sentences with no redundant information. The first sentence states the primary function, and the second clarifies a key behavioral constraint. This is highly efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list-style tool with two optional parameters and no output schema, the description adequately covers what the tool returns and its scope. It could be improved by briefly noting the format of the returned data, but given the tool's simplicity, the current description is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully describes both parameters (cwd and run_id) with detailed documentation including defaults and allowed formats. The tool description adds no further information about parameters beyond what is already in the schema, so the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Return' and the resource 'artifact paths plus existence flags for a specific run'. It also clarifies what it does not do ('Does not load any artifact payload'), which helps distinguish it from potential sibling tools that might load payloads. However, it does not explicitly differentiate from siblings by name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool by stating it only returns paths and existence flags and does not load payloads. This suggests using it when only metadata is needed. However, it lacks explicit guidance on when not to use it or direct references to alternative tools for loading payloads.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Map a change (files / endpoints / openapi ops / git diff) to the .tarn.yaml tests it most likely affects, with confidence tiers and run hints. Read-only: no HTTP and no test execution. Equivalent to: tarn impact --format json.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| diff | No | When true, run `git diff --name-only HEAD` under `cwd` and feed the result in as changed files. | |
| endpoints | No | Endpoints touched by the change. Each entry is either a `METHOD:/path` string or a `{method, path}` object. | |
| files | No | Changed source files as plain strings. | |
| min_confidence | No | Drop matches below this tier before returning. | |
| no_default_excludes | No | Disable the default discovery ignore rules (e.g. `.git`, `node_modules`). | |
| openapi_ops | No | OpenAPI `operationId`s whose behaviour changed. | |
| path | No | Restrict test discovery to this subpath (file or directory). Relative paths resolve against `cwd`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses read-only behavior (no HTTP, no test execution) and output characteristics (confidence tiers, run hints). While it lacks details on error handling or edge cases, it adequately describes the core behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. The first sentence front-loads the purpose and key features, and the second adds essential clarifications (read-only, CLI equivalence). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks an explicit output schema and only vaguely mentions output (confidence tiers, run hints). It does not cover error behavior or edge cases, leaving some gaps for an agent to guess the return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds limited value beyond what the schema already provides. It does not elaborate on parameter usage beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool maps a change to the tests it likely affects, with confidence tiers and run hints. It distinguishes itself from sibling tools like tarn_run and tarn_run_agent by noting it is read-only and does not execute tests.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for impact analysis without running tests, and mentions the CLI equivalent. However, it does not explicitly state when not to use it or provide direct comparisons to sibling tools for guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Inspect a prior run's archived report (NAZ-405) at run, file, test, or step granularity. Optional filter_category narrows the view to one FailureCategory. Response includes artifact paths for the run that seeded the view.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| filter_category | No | Narrow the run-level view to steps whose `failure_category` matches this value. | |
| run_id | No | Run identifier or alias (`last`, `prev`, etc.). Defaults to `last`. | |
| target | No | Address of the entity to inspect: `FILE`, `FILE::TEST`, or `FILE::TEST::STEP`. Omit for the run-level view. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the tool inspects archived reports and includes artifact paths in the response. However, with no annotations provided, it does not disclose whether the tool is read-only, requires permissions, or handles errors such as missing runs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loads the main purpose, and contains no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description mentions that the response includes artifact paths. However, it lacks details about error handling, pagination, or the full structure of the report, which would be helpful for a tool with multiple granularity levels.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for all four parameters. The description adds some context about 'filter_category' and 'target' granularity, but mostly repeats what the schema already provides, so it adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool inspects a prior run's archived report at multiple granularities (run, file, test, step). It mentions the optional filter_category and response contents, but does not explicitly differentiate from sibling tools like 'tarn_report' or 'tarn_last_failures'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used for inspecting archived reports at different levels, but provides no explicit guidance on when to use this tool versus alternatives, nor any conditions to avoid using it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the grouped failures (NAZ-402) for a specific run as structured JSON. Reads the persisted failures.json rather than re-running the tests. Useful for agents that want a failures-only view without loading the full report.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| run_id | No | Run identifier or alias (`last`, `prev`, `@latest`, or a literal `YYYYmmdd-HHMMSS-xxxxxx` id). Defaults to `last`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it reads a persisted file rather than re-running tests, implying non-destructive behavior and efficiency. No annotations provided, so description carries the burden well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no wasted words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, but the description specifies 'structured JSON' which is adequate for a utility tool. Could detail the JSON structure more, but context from sibling tools and schema covers essential needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline score is 3. Description does not add additional parameter details beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it returns grouped failures for a specific run as structured JSON, distinguishing itself from sibling tools like tarn_report by noting it provides a failures-only view without loading the full report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage context: 'Useful for agents that want a failures-only view without loading the full report.' Does not explicitly list alternatives or when not to use, but the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return only the root-cause failure groups (NAZ-402) for a run, without the wider failures envelope. The fastest failures-first read for an agent planning a fix. Equivalent to: tarn failures --format json (groups only).
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| run_id | No | Run identifier or alias (`last`, `prev`, `@latest`, or a literal id). Defaults to `last`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description bears full responsibility. It claims to be the 'fastest failures-first read' but does not disclose any side effects, authentication needs, rate limits, or error conditions. For a tool with no annotations, more behavioral detail is expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, use case, and equivalence. No filler, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only 2 parameters, no output schema, and no annotations, the description covers the core purpose and usage well. It lacks explicit output format details, which would be helpful since no output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers both parameters (cwd, run_id) with descriptions, achieving 100% coverage. The tool description adds no additional parameter-specific meaning beyond what the schema already provides, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Explicitly states it returns root-cause failure groups (NAZ-402), contrasting with the wider failures envelope. Includes a specific verb ('Return') and resource, and distinguishes from sibling 'tarn_last_failures' by mentioning it is the fastest failures-first read.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context for use ('agent planning a fix') and gives an equivalent CLI command. However, it does not explicitly state when not to use it or directly contrast with alternatives like tarn_last_failures.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all available tests in .tarn.yaml files. Returns file names, test names, and step counts.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| path | No | Path to directory (defaults to `cwd`). Relative paths resolve against `cwd`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden for behavioral disclosure. It partially fulfills this by stating it returns file names, test names, and step counts, implying a read-only operation. However, it does not explicitly confirm non-destructive behavior or mention any potential side effects, authentication needs, or performance impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, with the purpose stated first and the return value immediately following. Every word adds value, and there is no redundancy or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of a listing tool and the absence of an output schema, the description adequately covers the purpose and return format. It could be enhanced by noting that only tests in .tarn.yaml files are listed, which is already clear, but no significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of the two parameters (cwd and path) with clear descriptions. The tool description adds no additional parameter meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list', the resource 'tests', and the source '.tarn.yaml files'. It immediately distinguishes from sibling tools like tarn_run (execution) and tarn_inspect (detailed view) by specifying the output: file names, test names, and step counts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need to enumerate available tests, providing a clear context. However, it does not explicitly exclude when not to use it (e.g., when a filtered search is needed) nor compare to alternative siblings directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Assemble a remediation bundle (NAZ-414) from a prior run's artifacts: failing entries enriched with YAML snippets, request/response excerpts, captures lineage, and rerun hints. Supports truncation budgets for context-limited agents. Equivalent to: tarn pack-context.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| failed | No | Pack only failing entries. Defaults to true. | |
| files | No | Narrow entries to these files (path suffix match). | |
| format | No | Output shape: JSON bundle (default) or markdown. | |
| max_chars | No | Soft budget for the rendered output. Triggers structured truncation (see NAZ-414) when exceeded. | |
| run_id | No | Run identifier or alias. Defaults to the workspace-level `.tarn/` pointer. | |
| tests | No | Narrow entries to these test names (exact match). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses support for truncation budgets and the general assemble action. However, it does not mention side effects (e.g., no modifications), auth needs, or failure modes when artifacts are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first front-loads the core purpose and contents, second adds a feature and equivalent command. No unnecessary words, efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the what and high-level how, but lacking details on output shape (no output schema), prerequisites (must have prior run artifacts), and parameter interaction. With 7 parameters and no annotations, more context on error handling or typical usage would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds meaning beyond parameter names by contextualizing 'failing entries' (failed param), 'truncation budgets' (max_chars), and 'prior run' (run_id). This helps an agent understand what the parameters influence.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'assemble a remediation bundle (NAZ-414) from a prior run's artifacts', specifying the verb (assemble), resource (remediation bundle), and provenance (prior run). It lists contents (failing entries, YAML snippets, etc.) and differentiates from siblings like tarn_run or tarn_inspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage for creating a bundle from prior run artifacts, and mentions truncation budgets for context-limited agents. But no explicit guidance on when to use this tool versus alternatives like tarn_get_run_artifacts or tarn_report, and no when-not-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Render the concise report (NAZ-404) for a persisted run: a tiny JSON envelope with totals, exit code, and grouped failures. No HTTP, no test execution — purely reads summary.json + failures.json.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| run_id | No | Run identifier or alias. Defaults to `last`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides good behavioral transparency: it is read-only (purely reads files), no side effects, and no network calls. This suffices for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with purpose and essential details. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description appropriately describes the output format and input defaults. It covers all key aspects for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by stating defaults (cwd defaults to workspace root, run_id defaults to 'last'), going beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it renders a concise report for a persisted run, specifying the input files and output format. It distinguishes from siblings like tarn_run by noting no HTTP or test execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use (after a persisted run) and what not to expect (no HTTP, no test execution), effectively guiding usage without naming alternatives explicitly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Rerun only the failing (file, test) pairs from a prior run. Response shape matches tarn_run (run_id, artifacts, report) so agents can loop run → inspect → rerun without switching tool surfaces.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| env_name | No | Environment name to resolve for the rerun (loads tarn.env.{name}.yaml). | |
| report_mode | No | Which slice of the rerun's report to return inline. Defaults to `agent`. | |
| run_id | No | Source run identifier or alias to seed the selection from. Defaults to `last` (the workspace-level `.tarn/failures.json` pointer). | |
| vars | No | Variable overrides as key-value pairs. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While it states what the tool does, it does not discuss side effects (e.g., whether it triggers test execution, writes files), permissions, or read-only nature. This is insufficient for a tool that likely performs write operations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, both front-loaded with the core action and followed by valuable context about the response shape. No unnecessary words, every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no output schema, and nested objects, the description is relatively brief. It explains the core purpose and return shape but does not detail how 'failing pairs' are determined, error cases, or prerequisites (e.g., a prior run must exist). Adequate but with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema coverage is 100%, so the parameters are well documented in the schema. The description adds some context about the return shape but does not add meaning beyond what the schema provides for the parameters themselves. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool reruns only failing (file, test) pairs from a prior run, using the verb 'rerun' and specifying the resource. It distinguishes from siblings like tarn_run (which runs all tests) and tarn_run_agent (different purpose).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by mentioning that the response shape matches tarn_run, enabling a loop of run → inspect → rerun without switching tool surfaces. However, it lacks explicit when-not or alternative tool guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run API tests defined in .tarn.yaml files. Writes artifacts under .tarn/runs// and returns a compact agent report by default plus paths to the full artifacts so agents do not need to keep large JSON blobs in context.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. tarn.config.yaml, tarn.env.yaml, and relative paths are resolved against this directory. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| env | No | Environment name (loads tarn.env.{name}.yaml) | |
| path | No | Path to a .tarn.yaml test file or directory containing test files. Relative paths resolve against `cwd`. | |
| report_mode | No | Which slice of the run to return inline. `agent` (default) is the compact root-cause-first payload; `summary` and `failures` return the NAZ-401 artifacts; `full` returns the verbose JSON report. The run still writes every artifact regardless of the chosen mode. | |
| tag | No | Filter tests by tag (comma-separated) | |
| vars | No | Variable overrides as key-value pairs |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It does so by stating that artifacts are written to `.tarn/runs/<run_id>/` and that the tool returns a compact agent report plus paths to avoid large JSON blobs. It could mention if it is destructive or requires authentication, but the disclosed side effects and return format are valuable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, with the first sentence stating the primary action and the second providing key behavioral information. No superfluous text; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations), the description covers purpose, side effects, and return format reasonably well. It could elaborate on the structure of the 'compact agent report' or when to use different report modes, but it is sufficiently complete for most agents.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter well. The description adds context about the compact report and artifact paths but does not add new meaning beyond what the schema provides for individual parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Run API tests defined in .tarn.yaml files.' It specifies the action (run), resource (API tests), and distinguishes itself from siblings like tarn_fix_plan or tarn_rerun_failed by mentioning artifact handling and compact reports.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus siblings (e.g., tarn_run_agent, tarn_rerun_failed). The description implies it's for standard test runs that produce artifacts, but lacks when-not-to-use or alternative recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a suite with report_mode=agent pre-selected, surfacing the compact AgentReport (NAZ-412) plus artifact paths. Preferred entry point when the caller will iterate on failures. Equivalent to: tarn run --agent.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| env_name | No | Environment name (loads tarn.env.{name}.yaml). | |
| no_default_excludes | No | Disable the default discovery ignore rules. | |
| path | No | Path to a `.tarn.yaml` file or directory. Relative paths resolve against `cwd`. | |
| select | No | Explicit `FILE[::TEST[::STEP]]` selectors. Combine with `test_filter`/`step_filter` if desired. | |
| step_filter | No | Run only this step (name or zero-based index) within the filtered tests. | |
| tag | No | Comma-separated tag filter. | |
| test_filter | No | Run only this named test across every discovered file (wildcard selector). | |
| vars | No | Variable overrides as key-value pairs. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only describes the function (runs suite, returns report) without disclosing side effects, permissions, or state changes. Minimal behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences plus an equivalence line, all front-loaded with the core action and usage guidance. No unnecessary words; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 9 parameters, nested objects, and no output schema, the description adequately explains the tool's purpose and preferred use case. It mentions the AgentReport (NAZ-412) and artifact paths, though could be more detailed on return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 9 parameters have descriptions in the input schema (100% coverage). The description adds no additional parameter-level information beyond what the schema provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Run', the resource 'suite with report_mode=agent pre-selected', and distinguishes from tarn_run via agent report and failure iteration focus. It also provides the CLI equivalent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it is the preferred entry point when the caller will iterate on failures, giving clear context. Implicitly differentiates from tarn_run via the equivalence statement, though no explicit when-not guidance is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a minimal .tarn.yaml skeleton from one of four input modes (OpenAPI operation id, raw curl, method+url, or a recorded fixture). Returns the rendered YAML plus structured metadata (TODOs, inferred request, validation). Optional out writes the file to disk. Equivalent to: tarn scaffold --from-openapi / --from-curl / --method+--url / --from-recorded.
| Name | Required | Description | Default |
|---|---|---|---|
| curl | No | Payload for mode=curl. Provide either `command` (inline) or `file` (path to read). | |
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| explicit | No | Payload for mode=explicit. | |
| force | No | Allow overwriting an existing `out` path. | |
| format | No | When `out` is set, write YAML (default) or the JSON metadata block. | |
| mode | Yes | Input mode. Must match the payload object provided. | |
| name | No | Override the inferred top-level `name:` field. | |
| openapi | No | Payload for mode=openapi. | |
| out | No | Write the scaffold to this path (relative paths resolve against `cwd`). Refuses to overwrite unless `force=true`. | |
| recorded | No | Payload for mode=recorded. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It discloses core behavior (generating skeleton, returning metadata, optional file write) but omits details about side effects, permissions, or safety considerations. The `force` and `out` overwrite behavior is described in the schema but not reiterated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—three sentences covering all key aspects without redundancy. It prioritizes the main action, input modes, output, and optional file writing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and the tool's complexity (10 params, nested objects), the description provides a good overview of inputs and outputs (YAML + structured metadata). However, it could be more explicit about the exact return format or fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by summarizing input modes and providing an equivalent CLI command, but does not deepen understanding beyond what the schema already conveys.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: generating a minimal `.tarn.yaml` skeleton from four distinct input modes. It specifies the output (rendered YAML plus structured metadata) and optional file writing, distinguishing it from other tools on the server.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly guides when to use the tool by enumerating the four input modes (OpenAPI, curl, explicit, recorded). However, it does not explicitly state when not to use it or mention alternatives, leaving some ambiguity for the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate .tarn.yaml test files without executing them. Checks YAML syntax and schema validity.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Absolute path to the project root. Defaults to the workspace root captured during MCP `initialize`, or the server process's current directory. | |
| path | Yes | Path to a .tarn.yaml file or directory. Relative paths resolve against `cwd`. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description indicates read-only behavior (no execution) but lacks details on side effects, output format, or error handling. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the main action. No wasted words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, output schema, or complex parameters, the description is adequate but lacks details on return values, error messages, or examples. Could be more complete for an agent unfamiliar with the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. The tool description adds no extra meaning beyond what the schema already provides. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates .tarn.yaml files without executing, checking syntax and schema. It distinguishes from siblings like tarn_run by specifying no execution, but does not elaborate on the exact schema validation scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: validate before running tests. But no explicit guidance on when to use this vs alternatives like tarn_run or tarn_impact. No when-not-to-use or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v1.0.0tarn_fix_plantarn_get_run_artifactstarn_impacttarn_inspecttarn_last_failurestarn_last_root_causestarn_listtarn_pack_contexttarn_reporttarn_rerun_failedtarn_runtarn_run_agenttarn_scaffoldtarn_validateTools are largely distinct, but tarn_run and tarn_run_agent overlap in purpose despite differing modes, and tarn_last_failures/tarn_last_root_causes/tarn_report have subtle granularity differences that could cause minor confusion.
All tools follow a consistent tarn_verb_noun or tarn_noun pattern in snake_case, making the set predictable and easy to navigate.
14 tools cover the full lifecycle of testing with Tarn without being excessive; each tool serves a clear, non-redundant role.
The tool surface includes running tests, inspecting results in multiple granularities, impact analysis, scaffolding, validation, and fix planning—leaving no obvious gaps for the domain.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
AI-callable tools for API mocking, testing, monitoring, security, and automation.
MEOK MCP Test MCP — golden-file + schema-drift + tool-failure tests for any MCP server. Drop-in
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Build, validate, deploy — HTTP APIs, cron jobs, webhooks and MCP tools — from your AI client.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/NazarKalytiuk/tarn'
If you have feedback or need assistance with the MCP directory API, please join our Discord server