Skip to main content
Glama

🌐 Language: English | Español

QA Testing Blueprints (QA-11X v3.0)

Zero-Infra Agentic QA, AI Evals & DevSecOps Governance Blueprint

License: MIT Rule Zero: Ephemeral Docker Trivy Scan Modular AI Skills AI Coding Agents FINOS CALM

QA-11X v3.0 turns both AI Coding Agents (Google Antigravity, Claude Code, Cursor, Windsurf, Codex, Copilot, Hermes Agent, and 20+ more) and human engineering teams into deterministic Staff-level QA & DevSecOps operators. Every check runs inside ephemeral, non-root Docker containers (Rule Zero) with zero host contamination, enforcing >= 95% coverage, real database/contract parity, dual-mode AI evaluations, and DISA ASD STIG V6R5 + OWASP v4.2 security controls from day one.


⚑ 30-Second Zero-Friction Quickstart

Bootstrap any repository in 30 seconds without cloning this monorepo:

# Auto-detect stacks & vibe-coded markers, configure .qa-11x.json, AGENTS.md, ACTIVE_TASK.md & PR workflow:
npx -y @sinfallas/mcp-qa-11x@latest quickstart

# Explicit tier, output language policy, agent selection, or dry-run preview:
npx -y @sinfallas/mcp-qa-11x@latest quickstart --tier standard --lang en --agents auto
npx -y @sinfallas/mcp-qa-11x@latest quickstart --tier enterprise --lang es --dry-run

Add Proportional PR Protection to GitHub Actions

Use the hardened composite action .github/actions/qa-11x-pr-check/action.yml in your pull-request workflow:

name: QA-11X PR Check
on: [pull_request]
permissions:
  contents: read
jobs:
  qa-11x:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
        with:
          fetch-depth: 0
      - uses: tecno-consultores/qa-testing-blueprints/.github/actions/qa-11x-pr-check@v2
        with:
          tier: standard
          lang: en

Related MCP server: QE-MCP

🧭 Differential Architecture (QA-11X v3.0)

flowchart TD
    UserReq["Developer or PR Trigger"] --> Quickstart["npx @sinfallas/mcp-qa-11x quickstart"]
    Quickstart --> Router{"3-Path Execution & Tiers"}
    Router -->|"DOCS_ONLY"| FastDocs["make sync-governance + make lint-docs"]
    Router -->|"MICRO_CHANGE / standard"| BlastAST["AST Blast Radius & Tiered CI"]
    Router -->|"FULL_11X / enterprise"| FullPyramid["13-Layer QA Pyramid + FINOS CALM"]
    BlastAST --> RuleZero["Rule Zero Non-Root Docker sinfallas/*"]
    FullPyramid --> RuleZero
    RuleZero --> Shields["AI Evals Dual-Mode + DISA/OWASP + Release Gate"]
    FastDocs --> LocalCommit["Certified Local Commit QA-GIT-GATE-01"]
    Shields --> LocalCommit

Progressive Adoption Tiers

Tier

Target Adoption

Included Gates & Guarantees

starter

Day-1 Adoption (30s)

Zero-plain-text secret scanning, Rule Zero container hygiene, .qa-11x.json, AGENTS.md connector across 27+ AI coding agents, and DOCS_ONLY fast-path

standard

Daily PR Workflows

Everything in starter + >= 95% unit coverage, strict types (mypy --strict, tsc, PHPStan max), AST Blast-Radius CI partitioning (40%–60% compute savings), and UI anti-pattern linting (QA-UI-ANTI-001..005)

enterprise

Regulated Production

Full 13-layer FULL_11X pyramid, blocking FINOS CALM architecture validation, PostgreSQL 17 RLS + WORM seq_num audit chains, DISA STIG V6R5 / OWASP v4.2, 3-phase clock assertions (t < starts_at, starts_at <= t <= ends_at, t > ends_at), Pact + k6 load, and multi-signal Release Readiness Gate (GO / NO_GO / SKIPPED)

Zero-Infra Ephemeral Docker Matrix (sinfallas/*)

All runtime, lint, mutation, contract, and BDD suites execute inside non-root containers (-u "$(id -u):$(id -g)" -e HOME=/tmp) with Trivy HIGH/CRITICAL = 0:

Ephemeral Image

Role in QA-11X v3.0

Certified Tags & Pre-Baked Tooling

sinfallas/base-python-uv

Python, FastAPI, AI Evals & Vibe-Rescue

3.11, 3.12, 3.13, 3.14, 3.15, latest (uv, ruff, mypy, pytest, bandit, radon, mutmut, deepeval, pact-python, tesseract-ocr)

sinfallas/base-node-ionic

MCP Server, Node, Vue 3, React & CALM

22, 24, 26, latest (pnpm, promptfoo, @finos/calm-cli, playwright, @axe-core/playwright, @pact-foundation/pact)

sinfallas/base-mobile-appium

Native & Hybrid Mobile W3C Bridge (Android/iOS)

26, latest (Node.js 26, pnpm, OpenJDK 21, Android SDK/ADB, Appium 3, uiautomator2, xcuitest, WebdriverIO v9)

sinfallas/basephpcomposer

PHP 8.2–8.5 APIs & Services

8.2, 8.3, 8.4, 8.5, latest (pcov, phpstan, psalm, pest, phpunit, infection)

sinfallas/karatelabs

API & HTTP/Edge/OSHP/XML BDD Contracts

1.5.1 / 11x-v2-local (Java + Maven + Karate JUnit5; proportional to stateful multi-service FULL_11X flows)

sinfallas/base-bash-qa

DevOps & POSIX Shell Scripts

latest (ShellCheck, shfmt, BATS-core, bats-mock, kcov)

sinfallas/remote-graphify

AST Context Optimization & Blast Radius

latest (graphify Tree-sitter AST graph & cluster extraction)

Infra & Load Engines

Live DB, Cache & Smoke Load

postgres:17-alpine, eqalpha/keydb:alpine, grafana/k6:2.4.0


πŸ—‚οΈ Full-Stack, Mobile, AI Evals & Vibe-Rescue Blueprints

Directory

Domain

Key v2.0 Capabilities

fastapi/

Python Async APIs

Ruff, Mypy strict, PostgreSQL 17 RLS + Alembic parity, Pact V4 consumer (test_pact_consumer.py), k6 smoke load (smoke.k6.js), Karate BDD

python/

Python Libraries & CLIs

UV, Ruff (CC <= 5), Mypy strict, Pytest >= 95%, Hypothesis, Mutmut (MSI >= 80%), Bandit

node-backend/

Node.js / TS Microservices

ESLint --max-warnings 0, Vitest >= 95%, Pact V4 provider verifier (provider.pact.test.ts), Stryker, ZAP

vue3/

Vue 3 + Vite Frontends

Screenplay Actor.ts, Self-Healing DOM semanticLocator.ts, Vitest >= 95%, @axe-core, Lighthouse CI

react-frontend/

React + Vite Frontends

Screenplay Actor.ts, Self-Healing DOM semanticLocator.ts, RTL, Playwright, @axe-core

php/

PHP 8.2–8.5 Services

PHP-CS-Fixer, PHPStan Level Max, Psalm Taint Analysis, Pest/PHPUnit + PCOV >= 95%, Infection

bash-scripts/

POSIX / Bash Automation

ShellCheck, shfmt, BATS-core, bats-mock, kcov >= 95%

mobile-appium/

Native & Hybrid Mobile

Appium 3 / W3C + WebdriverIO (wdio.conf.ts), MobileActor.ts (Android, iOS, Ionic, RN)

ai-evals/

Dual-Mode AI & Multimodal QA

RAG faithfulness, Promptfoo OWASP LLM Top 10, WebRTC voice latency (test_voice_websocket_latency.py), PDF/OCR Decimal (test_pdf_table_fidelity.py), golden_from_traces.py

vibe-rescue/

Vibe-to-Enterprise Hardening

audit_vibe_prototype.py (VIBE-SEC-001..QA-005), PostgreSQL 17 RLS/WORM template (rls_hardening.sql)

architecture/

Architecture-as-Code

Blocking FINOS CALM topology (qa-11x.calm.json) & 7 mandatory 11X controls (make validate-calm)

mcp-server/

@sinfallas/mcp-qa-11x

Node.js MCP server (stdio), quickstart, install-skill, init-agents, reconcile-agents, tms-sync & A2A agent card


πŸ›  Testing Tools Matrix

Below is the corporate validation stack summarized per environment and test type. N/A indicates test categories that do not apply due to the intrinsic nature of the environment (e.g. a standalone Python CLI, Bash script, or declarative architecture topology does not expose an HTTP server, so DAST and Chaos Engineering do not apply).

Environment

Linting / Formatting

Unit Tests / Mocking

Coverage

Dependency Audit

Static Security (SAST)

Mutation

BDD / Black-Box

Concurrency / Stress

Profiling / Complexity

DAST (Dynamic)

Chaos Engineering

Observability (Metrics)

IaC / Docker Audit

Local Git Hooks

Python (python/)

Ruff + Mypy (Strict)

Pytest + Hypothesis + pytest-mock / respx

Pytest-Cov (>= 95%)

pip-audit + CycloneDX SBOM

Bandit

Mutmut (>= 80%)

N/A

N/A

py-spy / memray + Radon (CC <= A)

N/A

N/A

Pushgateway + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

FastAPI (fastapi/)

Ruff + Mypy (Strict)

Pytest + pytest-asyncio + respx + PostgreSQL 17 + KeyDB + Alembic

Pytest-Cov (>= 95%)

pip-audit + CycloneDX SBOM

Bandit

Mutmut (>= 80%)

Karate Labs + Schemathesis + Pact V4

Locust (Headless) + k6

py-spy / memray + Radon (CC <= A)

OWASP ZAP Baseline

Pumba

prometheus-fastapi-instrumentator + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

Node Backend (node-backend/)

ESLint (--max-warnings 0) + Prettier + tsc

Vitest + Supertest + Nock

@vitest/coverage-v8 (>= 95%)

pnpm audit + .npmrc (ignore-scripts=true)

eslint-plugin-security

Stryker (>= 80%)

Karate Labs + Pact V4

Artillery + k6

0x + Clinic.js + SonarJS (<= 15)

OWASP ZAP Baseline

Pumba

express-prom-bundle + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

Vue 3 (vue3/)

ESLint (--max-warnings 0) + Prettier + vue-tsc

Vitest + Vue Test Utils + Screenplay + Playwright + @axe-core

@vitest/coverage-v8 (>= 95%)

pnpm audit + .npmrc (ignore-scripts=true)

eslint-plugin-security

Stryker

Karate Labs (HTTP/OSHP) + Playwright UI

N/A

Lighthouse CI + SonarJS (<= 15)

OWASP ZAP Baseline

N/A

nginx-prometheus-exporter + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

React (react-frontend/)

ESLint (--max-warnings 0) + Prettier + tsc

Vitest + React Testing Library + Screenplay + Playwright + @axe-core

@vitest/coverage-v8 (>= 95%)

pnpm audit + .npmrc (ignore-scripts=true)

eslint-plugin-security + react-hooks

Stryker

Karate Labs (HTTP/OSHP) + Playwright UI

N/A

Lighthouse CI + SonarJS (<= 15)

OWASP ZAP Baseline

N/A

nginx-prometheus-exporter + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

PHP 8.2–8.5 (php/)

PHP-CS-Fixer (@PER-CS2.0) + PHPStan (Level Max)

Pest / PHPUnit 11.5+ + MockHandler + PostgreSQL 17 + KeyDB

PCOV (>= 95%)

composer audit + roave/security-advisories + CycloneDX SBOM

Psalm (--taint-analysis)

Infection (MSI >= 80%)

Karate Labs + Swagger-PHP / Schemathesis

Locust / k6

tomasvotruba/cognitive-complexity (<= 9)

OWASP ZAP Baseline

Pumba

promphp/prometheus_client_php + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

Bash (bash-scripts/)

ShellCheck + shfmt

BATS-core + bats-support + bats-assert + bats-file + bats-mock

kcov (>= 95%)

N/A

ShellCheck

N/A

N/A

N/A

hyperfine + time -v

N/A

N/A

Pushgateway + Prometheus

Trivy + Hadolint + Gitleaks

Lefthook

Mobile Appium (mobile-appium/)

tsc --strict + Prettier

WebdriverIO v9 + MobileActor Screenplay (W3C Bridge, Android/iOS/Ionic/RN)

node:test / C8 (>= 95%)

pnpm audit + .npmrc + CycloneDX SBOM

eslint-plugin-security + Deep-Link / Biometry guards

N/A

Appium 3 (uiautomator2/xcuitest) + Karate BFF

N/A

CC <= 5 + Fail-Verbose /tmp Traces

N/A (delegated to BFF OWASP ZAP)

N/A (Offline Sync & Biometry toggle)

Allure JSON Fail-Verbose (/tmp)

Trivy + Hadolint + Gitleaks (sinfallas/base-mobile-appium:26)

Lefthook

AI Evals (ai-evals/)

Ruff (CC <= 5) + Mypy (Strict)

Pytest asyncio + DeepEval + WebRTC Voice + PDF/OCR Decimal

Pytest-Cov (>= 95%)

pip-audit + CycloneDX SBOM

Bandit + Promptfoo OWASP LLM Top 10

N/A

DeepEval + Promptfoo (Offline + Live)

WebRTC TTFB <= 500ms

Radon (CC <= A) + golden_from_traces.py

Promptfoo LLM Red-Team

N/A

JSON Eval Trajectory Trace

Trivy + Gitleaks

Lefthook

Vibe Rescue (vibe-rescue/)

Ruff (CC <= 5) + Mypy (Strict)

Pytest + audit_vibe_prototype.py (VIBE-SEC-001..QA-005)

Pytest-Cov (>= 95%)

pip-audit + rls_hardening.sql

Bandit + RLS / AES-256-GCM / WORM

N/A

QA-11X 10/10 Rescue Gate

N/A

Radon (CC <= 5)

OWASP ZAP (post-rescue)

N/A

Finding State Tracker JSON

Trivy + Hadolint + Gitleaks

Lefthook

Architecture (architecture/)

AJV CLI / CALM (validate_calm_architecture.sh)

CALM Schema & 11-Control Verification (qa-11x.calm.json)

N/A

CALM Control Integrity

11 Blocking Architecture Controls (RLS/WORM/mTLS)

N/A

FINOS CALM Architecture Gate

N/A

Graphify AST Coupling / Layer Check

N/A

N/A

CALM Validation Report

Trivy + Gitleaks

Lefthook

MCP Server (mcp-server/)

ESLint (--max-warnings 0) + Prettier + tsc

Vitest + run-example-e2e.mjs (Clean-Room stdio)

@vitest/coverage-v8 (>= 95%)

pnpm audit + .npmrc (ignore-scripts=true)

eslint-plugin-security + promptGuard + audit_security_stig_owasp

Stryker

Karate Labs (mcp_cli_test.feature)

N/A

SonarJS (<= 15)

N/A

N/A

Pass-Quiet Telemetry & PR_VERDICT SHA-256

Trivy + Gitleaks

Lefthook


πŸ›‘οΈ Dual-Mode AI Evals, Local LLMs & Security Shield

  1. Mode 1 β€” Zero-Cost Deterministic Offline Contracts (make test-ai-offline):

    • Runs hermetic RAG, agent-trajectory, WebRTC voice (TTFB p95 <= 500ms), and OCR Decimal table checks with stubbed judges (>= 95% coverage, CC <= 5, mypy --strict) at $0 API cost.

    • Protects MCP tool execution fail-closed via promptGuard.ts (NFKC canonicalization, zero-width/Trojan Source sanitization, prompt-injection & QA-GIT-GATE-01 bypass prevention, PII/secret redaction using <EXAMPLE_REDACTED> placeholders).

    • Compresses long transcripts deterministically with .agents/scripts/compact_context_11x.py and supports air-gapped Ollama / vLLM profiles plus A2A Agent Card discovery (agentCard.ts, docs/specs/14_LOCAL_LLM_AND_A2A_PROFILES.md).

  2. Mode 2 β€” On-Demand Probabilistic & Red-Team Audits (make audit-llm):

    • Executes DeepEval (GEval, TaskCompletionMetric, faithfulness) and Promptfoo OWASP LLM Top 10 red-teaming against configured cloud providers or local Ollama/vLLM endpoints (SKIPPED_NO_LLM_SECRETS_CONFIGURED when credentials are absent in CI).


πŸ€– 10 Modular Skills (< 20 KB), MCP Server & 27+ AI Agents

  • Progressive Skill Routing (< 20 KB budget): Led by .agents/skills/qa-11x-architect/SKILL.md (< 12 KB), loading 9 specialized sub-skills on demand (qa-11x-devsecops-hardener, qa-11x-systematic-debugging, qa-11x-screenplay-builder, qa-11x-llm-evaluator, qa-11x-vibe-rescue, qa-11x-docker-certifier, docker-trivy-keydb-hardening, graphify-ast-navigator, refactoring-ui).

  • Public MCP Surface (@sinfallas/mcp-qa-11x v3.0.0): Exposes 18 Tools and 12 Resources (blueprint://*) over stdio β€” combining the 12 core v1 tools + 6 v2.0 evaluators (FINOS CALM, Release Gate, Agent Trajectory Evals, OWASP LLM Red Team, Vibe-Rescue, Enterprise TMS Sync) protected end-to-end by promptGuard & TaskStateGuard.

  • Human-Confirmed Git Closure (QA-GIT-GATE-01): Enforced by .agents/rules/git_closure_gate.md β€” agents certify locally via make certify and stop at a local git commit; automatic git push or npm publish is strictly forbidden.

  • Setup & Catalog Guides: See docs/agent-setup.en.md (English), docs/agent-setup.md (Spanish), RESOURCES.md, llms.txt (GEO index), catalogs/ (QASkills, SkillsLLM), docs/01_ARQUITECTURA_PIRAMIDE_11X.md, TESTING.md, SECURITY.md, and ROADMAP.md.

# Install global skills, operational scripts, and thin AGENTS.md connector across detected IDEs:
npx -y @sinfallas/mcp-qa-11x@latest install-skill --init-agents --include-scripts --reconcile

  • Author: Jesus Palencia (<sinfallas@gmail.com>)

  • Copyright: Copyright (C) 2026 Jesus Palencia <sinfallas@gmail.com> (Creado en Tecno Consultores)

  • License: Distributed under the MIT License.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      D
      maintenance
      A Python-based MCP server that enforces configurable Quality Assurance automation strategies for AI coding agents, ensuring generated tests adhere to strict architectural and testing standards.
      -
    • A
      license
      A
      quality
      B
      maintenance
      Keyless, local MCP server bringing ISTQB / OWASP / IEEE / ISO / EU AI Act QA standards into your AI client. Standards-grounded retrieval, deterministic QA effort estimation, automated QA document quality review (0-100 rubric), and JUnit/CSV test-results flakiness analysis.
      6
      102 PyPI
      7
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      A local, MIT-licensed AI test harness with 10 MCP tools and 11 prompts for risk-based testing, accessibility checks, quality maps, evidence, and interactive reports. Runs over stdio against an explicitly configured project folder; uses the host AI coding agent, with no hosted backend or runtime telemetry.
      2
      MIT