Skip to main content
Glama

setup_playwright_tests

Generate Playwright end-to-end test setup for documentation sites, including containers and CI/CD integration, to automate testing workflows.

Instructions

Generate Playwright E2E test setup for documentation site (containers + CI/CD)

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
repositoryPathYesPath to documentation repository
ssgYes
projectNameYesProject name for tests
mainBranchNomain
includeAccessibilityTestsNo
includeDockerfileNo
includeGitHubActionsNo

Implementation Reference

  • The main handler function that implements the logic for setting up Playwright E2E tests, including generating config files, tests, Dockerfiles, GitHub Actions workflows, and updating package.json based on the SSG type.
    export async function setupPlaywrightTests(
      args: unknown,
    ): Promise<ToolResponse> {
      const {
        repositoryPath,
        ssg,
        projectName,
        mainBranch,
        includeAccessibilityTests,
        includeDockerfile,
        includeGitHubActions,
      } = inputSchema.parse(args);
    
      try {
        const config = SSG_CONFIGS[ssg];
        const templatesDir = path.join(__dirname, "../templates/playwright");
    
        // Create directories
        const testsDir = path.join(repositoryPath, "tests/e2e");
        await fs.mkdir(testsDir, { recursive: true });
    
        if (includeGitHubActions) {
          const workflowsDir = path.join(repositoryPath, ".github/workflows");
          await fs.mkdir(workflowsDir, { recursive: true });
        }
    
        // Read and process templates
        const filesCreated: string[] = [];
    
        // 1. Playwright config
        const configTemplate = await fs.readFile(
          path.join(templatesDir, "playwright.config.template.ts"),
          "utf-8",
        );
        const playwrightConfig = configTemplate.replace(
          /{{port}}/g,
          config.port.toString(),
        );
    
        await fs.writeFile(
          path.join(repositoryPath, "playwright.config.ts"),
          playwrightConfig,
        );
        filesCreated.push("playwright.config.ts");
    
        // 2. Link validation tests
        const linkTestTemplate = await fs.readFile(
          path.join(templatesDir, "link-validation.spec.template.ts"),
          "utf-8",
        );
        const linkTest = linkTestTemplate.replace(/{{projectName}}/g, projectName);
    
        await fs.writeFile(
          path.join(testsDir, "link-validation.spec.ts"),
          linkTest,
        );
        filesCreated.push("tests/e2e/link-validation.spec.ts");
    
        // 3. Accessibility tests (if enabled)
        if (includeAccessibilityTests) {
          const a11yTemplate = await fs.readFile(
            path.join(templatesDir, "accessibility.spec.template.ts"),
            "utf-8",
          );
    
          await fs.writeFile(
            path.join(testsDir, "accessibility.spec.ts"),
            a11yTemplate,
          );
          filesCreated.push("tests/e2e/accessibility.spec.ts");
        }
    
        // 4. Dockerfile (if enabled)
        if (includeDockerfile) {
          const dockerTemplate = await fs.readFile(
            path.join(templatesDir, "Dockerfile.template"),
            "utf-8",
          );
          const dockerfile = dockerTemplate
            .replace(/{{ssg}}/g, ssg)
            .replace(/{{buildCommand}}/g, config.buildCommand)
            .replace(/{{buildDir}}/g, config.buildDir);
    
          await fs.writeFile(
            path.join(repositoryPath, "Dockerfile.playwright"),
            dockerfile,
          );
          filesCreated.push("Dockerfile.playwright");
        }
    
        // 5. GitHub Actions workflow (if enabled)
        if (includeGitHubActions) {
          const workflowTemplate = await fs.readFile(
            path.join(templatesDir, "docs-e2e.workflow.template.yml"),
            "utf-8",
          );
          const workflow = workflowTemplate
            .replace(/{{mainBranch}}/g, mainBranch)
            .replace(/{{buildCommand}}/g, config.buildCommand)
            .replace(/{{buildDir}}/g, config.buildDir)
            .replace(/{{port}}/g, config.port.toString());
    
          await fs.writeFile(
            path.join(repositoryPath, ".github/workflows/docs-e2e-tests.yml"),
            workflow,
          );
          filesCreated.push(".github/workflows/docs-e2e-tests.yml");
        }
    
        // 6. Update package.json
        const packageJsonPath = path.join(repositoryPath, "package.json");
        let packageJson: any = {};
    
        try {
          const existing = await fs.readFile(packageJsonPath, "utf-8");
          packageJson = JSON.parse(existing);
        } catch {
          // Create new package.json
          packageJson = {
            name: projectName.toLowerCase().replace(/\s+/g, "-"),
            version: "1.0.0",
            private: true,
            scripts: {},
            dependencies: {},
            devDependencies: {},
          };
        }
    
        // Add Playwright dependencies
        packageJson.devDependencies = {
          ...packageJson.devDependencies,
          "@playwright/test": "^1.55.1",
          ...(includeAccessibilityTests
            ? { "@axe-core/playwright": "^4.10.2" }
            : {}),
        };
    
        // Add test scripts
        packageJson.scripts = {
          ...packageJson.scripts,
          "test:e2e": "playwright test",
          "test:e2e:ui": "playwright test --ui",
          "test:e2e:report": "playwright show-report",
          "test:e2e:docker":
            "docker build -t docs-test -f Dockerfile.playwright . && docker run --rm docs-test",
        };
    
        await fs.writeFile(packageJsonPath, JSON.stringify(packageJson, null, 2));
        filesCreated.push("package.json (updated)");
    
        // 7. Create .gitignore entries
        const gitignorePath = path.join(repositoryPath, ".gitignore");
        const gitignoreEntries = [
          "test-results/",
          "playwright-report/",
          "playwright-results.json",
          "playwright/.cache/",
        ].join("\n");
    
        try {
          const existing = await fs.readFile(gitignorePath, "utf-8");
          if (!existing.includes("test-results/")) {
            await fs.writeFile(
              gitignorePath,
              `${existing}\n\n# Playwright\n${gitignoreEntries}\n`,
            );
            filesCreated.push(".gitignore (updated)");
          }
        } catch {
          await fs.writeFile(gitignorePath, `# Playwright\n${gitignoreEntries}\n`);
          filesCreated.push(".gitignore");
        }
    
        return {
          content: [
            {
              type: "text" as const,
              text: JSON.stringify(
                {
                  success: true,
                  filesCreated,
                  nextSteps: [
                    "Run `npm install` to install Playwright dependencies",
                    "Run `npx playwright install` to download browser binaries",
                    "Test locally: `npm run test:e2e`",
                    includeDockerfile
                      ? "Build container: `docker build -t docs-test -f Dockerfile.playwright .`"
                      : "",
                    includeGitHubActions
                      ? "Push to trigger GitHub Actions workflow"
                      : "",
                  ].filter(Boolean),
                  configuration: {
                    ssg,
                    buildCommand: config.buildCommand,
                    buildDir: config.buildDir,
                    port: config.port,
                    testsIncluded: {
                      linkValidation: true,
                      accessibility: includeAccessibilityTests,
                    },
                    integrations: {
                      docker: includeDockerfile,
                      githubActions: includeGitHubActions,
                    },
                  },
                },
                null,
                2,
              ),
            },
          ],
        };
      } catch (error: any) {
        return {
          content: [
            {
              type: "text" as const,
              text: JSON.stringify(
                {
                  success: false,
                  error: error.message,
                },
                null,
                2,
              ),
            },
          ],
          isError: true,
        };
      }
    }
  • Zod schema defining the input parameters for the tool, including repository path, SSG type, project name, and optional flags for accessibility tests, Dockerfile, and GitHub Actions.
    const inputSchema = z.object({
      repositoryPath: z.string().describe("Path to the documentation repository"),
      ssg: z.enum(["jekyll", "hugo", "docusaurus", "mkdocs", "eleventy"]),
      projectName: z.string().describe("Project name for tests"),
      mainBranch: z.string().optional().default("main"),
      includeAccessibilityTests: z.boolean().optional().default(true),
      includeDockerfile: z.boolean().optional().default(true),
      includeGitHubActions: z.boolean().optional().default(true),
    });
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions generating 'containers + CI/CD', implying file creation or configuration changes, but doesn't specify whether this is a read-only setup generation or if it modifies the repository, what permissions are needed, or any side effects like overwriting existing files. For a tool with 7 parameters and no annotations, this lack of detail is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It directly states what the tool does and includes key scope details in parentheses. Every part of the description earns its place by conveying essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (7 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what the tool outputs (e.g., generated files, configuration details), how it interacts with the repository, or any behavioral constraints. For a setup generation tool with multiple configuration options, more context is needed to guide effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is low at 29%, with only 2 out of 7 parameters having descriptions in the schema. The description doesn't add any parameter-specific information beyond the tool's overall purpose. It doesn't explain what 'ssg' means, how 'repositoryPath' is used, or the implications of boolean flags like 'includeAccessibilityTests'. This fails to compensate for the poor schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Generate Playwright E2E test setup for documentation site (containers + CI/CD)'. It specifies the verb ('Generate'), resource ('Playwright E2E test setup'), and scope ('documentation site'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'test_local_deployment' or 'simulate_execution', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing, or comparisons to sibling tools such as 'test_local_deployment' or 'simulate_execution'. The agent must infer usage from the purpose alone, which is insufficient for optimal tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/tosin2013/documcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server