Skip to main content
Glama
rongxianzhuo

aiTest MCP Server

by rongxianzhuo

aiTest MCP Server

AI-native browser testing, directly from your coding agent.

aiTest MCP Server connects AI coding tools (Cursor, Claude Desktop, Copilot) to the aiTest cloud testing platform. Describe what you want to test in plain English, and the AI engine handles the rest — planning, browser execution, root-cause analysis, and UX insights.

Quick Start

npx @aitest/mcp

Prerequisites

  • Node.js >= 18

  • An aiTest API key (get one here) or run against a local dev server

  • One of: Cursor, Claude Desktop, or any MCP-compatible client

Related MCP server: Assert

Configuration

Set these environment variables (or use your MCP client's config):

Variable

Required

Default

Description

AITEST_API_KEY

Your aiTest API key (aitest_sk_...)

AITEST_API_URL

http://localhost:8000/api/v1

aiTest REST API base URL

Cursor Setup

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "aitest": {
      "command": "npx",
      "args": ["@aitest/mcp"],
      "env": {
        "AITEST_API_KEY": "aitest_sk_your_key_here"
      }
    }
  }
}

Claude Desktop Setup

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "aitest": {
      "command": "npx",
      "args": ["@aitest/mcp"],
      "env": {
        "AITEST_API_KEY": "aitest_sk_your_key_here"
      }
    }
  }
}

Tools

aitest_run

Submit a new browser test job.

Parameters:
  url          — Target URL to test
  description  — What to test (natural language)
  credentials? — Login credentials { email, password }
  options?     — Test config { max_retries, screenshot_mode, timeout_seconds }

Returns: job_id + estimated completion time

Example: "Test that the login page accepts valid credentials, rejects invalid ones, and shows appropriate error messages."

aitest_status

Check the current status of a test job.

Parameters:
  job_id — The job ID from aitest_run

Returns: status (created/queued/executing/completed/failed) + timing

aitest_report

Get the full test report (only available when job is completed).

Parameters:
  job_id — The job ID from aitest_run

Returns: Summary, step-by-step results, root-cause analysis, UX insights

Development

# Clone
git clone git@github.com:rongxianzhuo/aitest-mcp.git
cd aitest-mcp

# Install
npm install

# Build
npm run build

# Run locally (against local aiTest API)
AITEST_API_URL=http://localhost:8000/api/v1 \
AITEST_API_KEY=aitest_sk_test \
  node dist/index.js

Architecture

AI Coding Agent (Cursor / Claude)
        │
        │  MCP Protocol (stdio)
        ▼
  aitest-mcp (thin proxy)
        │
        │  REST HTTPS
        ▼
  aiTest Cloud Platform
        │
        ├── FastAPI Server (Railway)
        ├── AI Engine (test planning + agent loop + analysis)
        ├── Supabase (auth + db + storage)
        └── Browser Sandbox (Playwright / Browserbase)

The MCP server is intentionally thin — zero AI logic, zero browser automation. It translates MCP tool calls into REST API requests and formats the responses for LLM consumption.

License

MIT — see LICENSE

Available Tools

3 tools
aitest_reportA

Get the complete test report for a finished job. Includes step-by-step results, pass/fail summary, root-cause analysis of failures, and UX improvement insights.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesThe job ID returned by aitest_run

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the tool returns a comprehensive test report for finished jobs, including specific details. No mention of side effects, but as a read operation, it is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second lists contents. No unnecessary words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema or annotations, the description is fairly complete for a simple 1-param tool. It covers what the tool returns and its prerequisite (finished job). Could mention that the job must exist, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter job_id with 100% schema coverage. The schema already describes it as 'The job ID returned by aitest_run.' The description adds context about the report contents but not about the parameter itself, meeting the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get the complete test report for a finished job,' specifying the verb, resource, and scope. It distinguishes from siblings (aitest_run and aitest_status) by focusing on completed jobs and detailed results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for finished jobs and lists report contents (step-by-step results, pass/fail summary, etc.), making it clear when to use this tool vs. alternatives. No explicit exclusions but context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

aitest_runA

Submit a browser test job to aiTest. Provide a URL and a natural-language description of what to test. The AI engine will plan, execute, and analyze the test, then return a report. Example: 'Test that the login form accepts valid credentials and shows the dashboard.'

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe target URL to test (e.g., https://example.com/login)
optionsNoOptional test configuration.
credentialsNoOptional login credentials if the test requires authentication.
descriptionYesNatural-language description of what to test. Be specific about the expected behavior.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries the burden. It mentions that the AI engine plans, executes, analyzes, and returns a report, but does not disclose potential behavioral traits such as being asynchronous, duration, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences plus an example, front-loaded with purpose. Every sentence earns its place; no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No annotations or output schema, and nested objects exist. Description is adequate for a simple tool but lacks details about result format, async behavior, or prerequisites.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3. Description adds minimal value beyond schema, just reiterating that description is a natural-language test description. No explanation of nested object parameters beyond what schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool submits a browser test job to aiTest, taking a URL and natural-language description. It distinguishes itself from sibling tools (aitest_report, aitest_status) by focusing on running tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an example and mentions what to provide, but does not explicitly state when not to use the tool or suggest alternatives for querying test status or results. Sibling names imply alternatives but are not referenced.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

aitest_statusA

Check the status of a previously submitted test job. Returns the current status (created, queued, executing, completed, failed) and timing info.

ParametersJSON Schema
NameRequiredDescriptionDefault
job_idYesThe job ID returned by aitest_run

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but the description clearly states it returns status and timing info, implying a read-only operation. It is transparent about what to expect, though it doesn't disclose potential side effects (none expected).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the purpose and quickly mentions return values. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the main purpose and return values (status, timing info) but lacks details on the exact structure of 'timing info' or whether additional fields are returned. Without an output schema, slightly more detail would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'job_id' is fully described in the schema with 100% coverage. The description adds no additional meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks the status of a submitted test job, using a specific verb ('Check') and resource ('status of test job'). It distinguishes from siblings 'aitest_run' (submits) and 'aitest_report' (likely retrieves full report).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use after submitting a job via 'previously submitted', but lacks explicit when-not-to-use or comparison with sibling tools like 'aitest_report'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedaitest_report
    • First observedaitest_run
    • First observedaitest_status

TDQS

A4.1/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a distinct purpose: run submits tests, status checks progress, report retrieves results. No overlap or ambiguity.

Naming Consistency5/5

All tools follow the consistent pattern 'aitest_<verb>', with clear imperative verbs (run, status, report).

Tool Count4/5

Three tools cover the essential workflow (submit, monitor, retrieve results) with no redundancy. Slightly minimal but appropriate for the focused scope.

Completeness4/5

Covers the full lifecycle of testing: submission, status polling, and report retrieval. Missing operations like cancellation or listing are minor gaps.

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers