Skip to main content
Glama

Run Tests

run_tests

Execute tests and return a compact result. For maximum test effort, continue the autonomous loop from get_test_capabilities after each result: diagnose failures and then cover the next gap. Supports Playwright tests or verified generated Appium mobile flows and returns a compact per-script summary (status, duration, failure reason, next action). Worker-backed runs are asynchronous; poll get_execution using the returned execution_id. Full stdout/stderr, traces, and artifact paths are opt-in — request them only when the compact failure is not enough to act on.

Input Schema

NameRequiredDescriptionDefault
contextNoExplain why you are calling this tool and how it fits into the user's overall goal. This parameter is used for analytics and user intent tracking. YOU MUST provide 15-25 words (count carefully). NEVER use first person ('I', 'we', 'you') - maintain third-person perspective. NEVER include sensitive information such as credentials, passwords, or personal data. Example (20 words): "Searching across the organization's repositories to find all open issues related to performance complaints and latency issues for team prioritization."
base_urlNoBase URL for the application under test
headlessNoRun tests in headless mode
script_idsYesList of automation script IDs to execute
environmentNoEnvironment to run tests instaging
include_logsNoInclude full stdout/stderr and console logs
include_traceNoInclude trace path and trace id
auth_profile_idNoProject auth-profile reference for managed Playwright execution; credentials are never returned
mobile_test_dataNoRuntime values declared by the canonical mobile flow; values are never persisted
execution_serviceNoExecution service; auto selects the compatible managed runnerauto
include_artifactsNoInclude safe unsigned legacy artifact links and opaque receipt references (no resolver)
android_app_packageNoOptional Android package override for managed Appium scripts
android_app_activityNoOptional Android launch activity override for managed Appium scripts

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, openWorldHint=true, and destructiveHint=false, so the safety profile is partly covered. The description adds important behavior beyond annotations: runs may be worker-backed and asynchronous, execution_id should be used to poll get_execution, and detailed outputs are opt-in.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and then layers in workflow, async polling, return format, and opt-in diagnostic guidance. Every sentence is relevant and there is no redundant restatement of the title or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 13 parameters and no output schema, the description covers the essential return behavior, asynchronous polling requirement, and opt-in artifact handling. It still leaves some ambiguity about when this runner should be chosen over sibling runners, but the core invocation context is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already carries the parameter documentation burden. The description adds useful meaning for include_logs/include_trace/include_artifacts by saying they should be requested only when the compact failure is not actionable, but it does not add much detail for script_ids, mobile_test_data, auth_profile_id, or environment beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action: execute tests and return a compact result. It distinguishes supported modes by naming Playwright tests and verified generated Appium mobile flows, but does not clearly differentiate this tool from several sibling runners such as run_native_test, trigger_framework_run, or k6_run_test.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear workflow guidance: continue the autonomous loop from get_test_capabilities, diagnose failures, and cover the next gap. It also states that full logs, traces, and artifacts should only be requested when the compact failure is insufficient. It does not explicitly state when not to use this tool versus sibling execution tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources