[English](README.md) | [Русский](README_RU.md)

# TestCraft Engine

TestCraft Engine is a Python 3.9+ library for deterministic constrained test-design calculations and traceability diagnostics. The base installation contains only Pydantic. LangGraph, model providers, and dotenv loading are optional.

## What the library guarantees

The pairwise and N-way APIs validate their complete input before generating cases. A report is complete only when every feasible target has a generated full assignment. A target that cannot be extended to any assignment satisfying all constraints is reported in `infeasible_targets`; it is not silently counted as covered.

The legacy functions remain available:

```python
from testcraft.combinatorics import generate_pairwise_combinations

combinations = generate_pairwise_combinations(
    {"os": ["Windows", "Linux"], "browser": ["Chrome", "Firefox"]},
    constraints=[{"os": "Windows", "browser": "Firefox"}],
)
```

Use the structured API when diagnostics are needed:

```python
from testcraft.combinatorics import generate_pairwise_report

report = generate_pairwise_report(
    {"os": ["Windows", "Linux"], "browser": ["Chrome", "Firefox"]},
    constraints=[{"os": "Windows", "browser": "Firefox"}],
)
print(report.is_complete)
print(report.coverage_percentage)
print(report.infeasible_targets)
print(report.to_dict())
```

`generate_nway_report` accepts `n_way=3` and higher. If the requested arity is larger than the number of parameters, the effective arity is the number of parameters and the report says so in diagnostics. Exact enumeration and a minimum-size proof are attempted only for bounded small spaces. `optimal` is `true` only when that proof completed; otherwise it is `None` and the strategy is reported. A greedy fallback is still checked for complete feasible coverage, but is not described as optimal.

Constraints are prohibited conjunctions. A constraint may mention one parameter or a high-arity subset. For a lower-arity target, at least one complete feasible continuation is required before the target is considered feasible.

## Other combinatorics

`analyze_boundary_values` accepts `[min, max]` ranges or mappings with `min`, `max`, optional `step`, and explicit `inclusive_min` and `inclusive_max` flags. Values must be finite real numbers; booleans, strings, non-finite values, invalid ranges, and invalid steps are rejected. The report separates valid and invalid points according to the declared step and endpoint semantics.

`generate_decision_table` validates condition and action schemas. Multiple actions or more than three conditions require explicit rules so that mutually exclusive business actions are not invented. A small synthetic table is marked `is_complete=false`; the CLI requires `--allow-unresolved` before persisting or returning success for it. `allow_unresolved=True` returns a clearly marked condition-only diagnostic table; it is not a completed business decision table.

## Traceability and pipeline

`calculate_traceability_matrix` returns the historical AC-to-TC mapping and coverage statistics. The statistics also contain `tc_to_ac`, duplicate diagnostics, unknown references, and an explicit `coverage_ok` result. `calculate_bidirectional_traceability` returns both directions directly.

The pipeline has two execution backends with the same node functions:

- LangGraph when the `ai` extra is installed and the backend imports successfully.
- A dependency-free sequential runner otherwise.

Both backends use the same coverage gate. A package is `complete` only when link coverage passes and no validation or generation errors remain. Link coverage means that every criterion ID is referenced by at least one test case. Semantic coverage is not inferred from an ID reference and is reported as `not_assessed` unless an external process supplies that assessment.

Without a model, the pipeline creates a deterministic draft test case for each criterion. This provides a measurable link-coverage draft, not a semantic quality guarantee. A requested model is not silently replaced by a fallback: partial JSON, unknown references, duplicate IDs, invalid fields, or missing criteria produce diagnostics and an incomplete package.

Each BDD scenario has `covers_ac` links. Feature serialization strips control characters, normalizes tags, and keeps scenario steps on single Gherkin lines. The generated feature also emits safe `@ac-...` tags when possible.

## Installation

```bash
pip install .
pip install .[dev]
```

Optional extras are separate:

```bash
pip install .[ai]
pip install .[openai]
pip install .[ollama]
pip install .[env]
pip install ".[ai,openai,env]"
```

The `ai` extra installs LangGraph and core LangChain support, `openai` installs the OpenAI-compatible provider used by OpenAI, DeepSeek, and custom endpoints, and `ollama` installs local model support. The `dev` extra does not install model or LangGraph packages, so the core test suite is intended to run without them.

## Command line

```bash
testcraft --version
testcraft pairwise parameters.json --constraints constraints.json --output pairwise.json --json
testcraft bva ranges.json --output boundaries.json --json
testcraft decision-table table.json --output table-result.json --json
testcraft design criteria.json --title "Example Suite" --output package.json --gherkin feature.feature --json
```

`decision-table` accepts an object containing `conditions`, `actions`, and optional `rules`. Constraints passed to `pairwise` must exist; a missing constraints file is an error. Existing output files are protected unless `--force` is supplied. Output and Gherkin paths must be different. Writes use a same-directory temporary file followed by an atomic replace. JSON input is read as UTF-8 with optional BOM support and is size-limited.

The design command can use an explicitly requested model:

```bash
testcraft --env-file .env design criteria.json --llm --provider openai --output package.json
```

Environment files are not loaded at import time. `--env-file` is explicit. Provider keys are isolated: an OpenAI key is not reused for DeepSeek or a custom OpenAI-compatible endpoint. DeepSeek uses `DEEPSEEK_API_KEY`, custom uses `CUSTOM_API_KEY`, and Ollama uses its host configuration. Unknown providers raise `ValueError`. Timeout, token, retry, and Ollama-specific options are available through the Python factory.

When a model is used, acceptance criteria and descriptions are sent to the selected external provider. The CLI enables best-effort redaction of common API keys, bearer tokens, password assignments, and private-key blocks before building the architect prompt. This redaction is not a complete data-loss-prevention system; review sensitive data before enabling an external provider. No API key is placed in package provenance. When available, provenance records provider, model, backend, and whether redaction was requested.

## Exit codes and diagnostics

- `0`: command completed and any required coverage gate passed.
- `1`: invalid input, JSON, model configuration, missing file, or I/O error.
- `2`: command-line usage error from argparse.
- `3`: design package is incomplete or its link-coverage gate failed, or an unresolved decision table was rejected.
- `4`: output-path collision or overwrite-protection failure.

Use `--json` to receive machine-readable results and errors. The design command always returns a non-zero incomplete status by default and has no success bypass for a failed link-coverage gate.

## Pluggable Combinatorial Backends & Zero-Trust Audit

TestCraft supports pluggable generation backends:
- `BuiltinPairwiseBackend`: Fast, pure-Python deterministic solver (zero dependencies).
- `CoverTableBackend`: Adapter for CoverTable AETG algorithm (`pip install testcraft-engine[covertable]`).
- `PictBackend`: Adapter for Microsoft PICT CLI (`pict.exe` / `pict`).

Regardless of backend, all generated suites are validated by an independent Zero-Trust auditor:
```python
from testcraft.combinatorics import verify_pairwise_coverage

audit = verify_pairwise_coverage(parameters, combinations, n_way=2, constraints=constraints)
print(audit.is_valid)              # True if 100% feasible tuples covered & 0 constraints violated
print(audit.coverage_percentage)   # Exact mathematical coverage
```

## BDD & Test Execution Exporters

TestCraft is an orchestration layer, exporting directly to test runners:
- **Behave / Cucumber:** `package.to_behave()` (Gherkin `.feature`) and `package.to_behave_bundle()` (includes Python step definitions).
- **Robot Framework:** `package.to_robot()` (standard `.robot` test suites).
- **Pytest:** `package.to_pytest()` (parametrized test module using `@pytest.mark.parametrize`).

## Model Context Protocol (MCP)

TestCraft provides native MCP tools for AI agents:
- `generate_pairwise`: Combinatorial N-way / pairwise test generator.
- `analyze_bva`: Valid and invalid boundary value analyzer.
- `build_traceability`: Bidirectional matrix and gate calculator.
- `design_test_package`: End-to-end requirement-to-test-suite pipeline.

Run as an MCP tool provider:
```bash
python -m testcraft.mcp.server
```

## Development


```bash
ruff check src tests examples
mypy
pytest -p no:cacheprovider
pytest --cov=testcraft --cov-report=term-missing -p no:cacheprovider
```

The project keeps a `src` layout and supports Python 3.9 and newer. The included CI workflow runs core tests on Linux and Windows across Python 3.9 and newer, and performs a package build/install smoke test where the environment permits it.

## License

MIT. See `LICENSE`.