# owasp-wstg-mcp

The substantive content this server exposes — every test case, every remediation note, every category structure — is the [OWASP Web Security Testing Guide](https://owasp.org/www-project-web-security-testing-guide/latest/), authored and maintained by the OWASP Foundation and the WSTG project's contributors. The living source lives in the [OWASP/wstg master tree](https://github.com/OWASP/wstg/tree/master/); read it there. This server is derivative tooling: a retrieval-optimized index over their work that makes the guide queryable from MCP-aware clients. The credit for the underlying knowledge belongs to them.

`owasp-wstg-mcp` is a local Knowledge Graph RAG (KG-RAG) server that indexes the full OWASP Web Security Testing Guide and exposes it as MCP tools. It combines two retrieval modes: semantic search for finding the right test by symptom, and graph traversal for following the relationships already encoded in the source material. Five tools sit on top: semantic search, direct lookup by canonical WSTG ID, category enumeration, neighbor traversal across related tests, and per-section extraction. A single WSTG test can run almost 8 KB of markdown, most of which is surrounding context. The section tool returns just the requested H2 block ("How to Test", "Remediation", and four others), so callers can consume the part of a test that answers the question they have rather than the whole document.

KG-RAG fits the OWASP guide because the knowledge graph is already in the source. Session-testing entries cite other session-testing entries, input-validation tests branch into sub-IDs, every test carries a category anchor. The server extracts those relationships with regex during a one-shot build step, stores them as directed edges in SQLite alongside the dense embeddings, and walks them at query time. A query like "what else does broken session handling touch?" returns a concrete list of related tests in one call. The OWASP guide remains the authority; the server just makes it faster to follow.

The package ships four submodules under `owasp_wstg_mcp/`:

| Submodule | Role |
|-----------|------|
| `extraction/` | Pure-stdlib pipeline: unpack the WSTG tarball, walk the corpus, parse canonical WSTG-IDs, emit deterministic JSONL chunks. |
| `embedding/` | Local CPU-only embedding (ONNX MiniLM-L6-v2) + sqlite-vec vector store; exposes `search` / `get` / `list_category`. |
| `graph/` | Structural-graph retrieval: regex-extracted edges in SQLite (`chunk_edges` table), section-anchor index; exposes `neighbors` / `section`. |
| `server/` | MCP stdio server thin-wrapping the embedding + graph query APIs as five tools. |

The `skills/` directory at the repo root contains bundled agent skills for the MCP tool surface. See the "Skills" section near the end of this file.

The MCP server is registered with Claude Code (or any MCP-compatible client) over stdio.

---

## Relationship to OWASP

Purrly Digital is not affiliated with, endorsed by, or sponsored by the OWASP Foundation. This is independent derivative tooling built on top of the publicly available OWASP Web Security Testing Guide, which the OWASP WSTG project contributors maintain under CC BY-SA 4.0.

We built this to fill a gap: the guide is excellent reference material but was not queryable from MCP-aware clients. `owasp-wstg-mcp` makes it retrievable. The underlying knowledge, the corpus, and the credit all belong to OWASP and the WSTG contributors.

If the OWASP WSTG team wants this repo, the name, or the published package transferred to them, we will hand it over. Open an issue on this repo to start that conversation.

---

## Deploy from a fresh clone

The full happy path from `git clone` to a working tool call in Claude Code. The repo ships a pre-built index, so no rebuild is required on first run — `pip install` is enough.

**Prerequisite:** Python 3.10 or newer (`python3 --version`).

```bash
# 1. Clone the repo and enter it.
git clone <repo-url> owasp-wstg-mcp
cd owasp-wstg-mcp

# 2. Create a venv and install the package.
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

# 3. Register the server with Claude Code at user scope.
claude mcp add owasp-wstg --scope user -- "$(pwd)/.venv/bin/owasp-wstg-mcp-serve"

# 4. Verify the registration.
claude mcp list        # expect: owasp-wstg ✓ Connected
```

From a fresh Claude Code session, confirm the tools are live with a single call:

```
wstg_get("WSTG-ATHN-04")
# → {"wstg_id": "WSTG-ATHN-04",
#    "title":   "Testing for Bypassing Authentication Schema",
#    "chunk_index": 0, "section_anchor": null, …}
```

That round-trips the registration, the stdio transport, the sqlite-vec store, and the embedding metadata. If it returns the record, the install is good.

The sections below cover each step in more detail and the build-from-tarball path for users who want to regenerate the index.

---

## Install

The install step from the section above produces a self-contained venv with the package, its runtime dependencies, and four console scripts on the path.

The extraction layer is pure stdlib. The embedding and server layers add runtime dependencies declared in `pyproject.toml` (`onnxruntime`, `tokenizers`, `huggingface-hub`, `sqlite-vec`, `numpy`, `mcp`). `pytest` is the only dev dep.

The package installs four console scripts:

| Command | Purpose |
|---------|---------|
| `owasp-wstg-extract` | Extract + chunk the WSTG tarball into JSONL. |
| `owasp-wstg-build-index` | Build the sqlite-vec vector store from JSONL. |
| `owasp-wstg-build-graph` | Build the `chunk_edges` graph table from JSONL. |
| `owasp-wstg-mcp-serve` | Start the MCP stdio server. |

## Configuration

### Test environment variables

The opt-in corpus-integrity tests under `tests/test_corpus_integrity.py` and the cross-version derivation tests under `tests/test_chunker_category_derivation.py` need real WSTG tarballs. Two env vars select which tarball satisfies which test:

| Env var | Purpose |
|---------|---------|
| `OWASP_WSTG_TARBALL` | Path to a WSTG tarball (release or master archive — e.g. `v4.2.tar.gz` or `wstg-master.tar.gz`). Opts in the corpus-integrity tests (corpus-version-agnostic; they assert against the lockfile at `tests/data/corpus_integrity_lockfile.json`, so any tarball that matches the current lockfile works) and the v4.2-pinned derivation test (`test_derived_map_matches_hardcoded_v42`), which specifically needs the v4.2 release tarball. |
| `OWASP_WSTG_MASTER_TARBALL` | Path to a WSTG master-HEAD tarball (e.g. archive-from-master). Opts in the master derivation test. |

Either or both may be unset; tests that need a missing tarball are skipped, so the suite stays runnable in minimal environments.

### Cache location

`owasp-wstg-extract` unpacks the tarball into a cache directory and skips re-extraction on a hit. The cache root is resolved with the following precedence (highest first):

1. The `--cache-dir` CLI flag, when passed.
2. The `OWASP_WSTG_CACHE_DIR` environment variable, when set.
3. The repo-relative default `<repo-root>/.cache/owasp-wstg-mcp/` (resolved via `Path(__file__).resolve().parents[2]`, so the default moves with the clone).

Inside the resolved cache root the CLI uses a single shared bucket `<cache-root>/cli/` for the unpacked corpus. This matches the cache posture of tools like `pip` and `npm`: the CLI does not know which variant a given tarball is, so it does not partition the bucket by variant. Switching tarball variants against the same cache root requires removing the bucket between runs:

```bash
rm -rf <cache-root>/cli
```

Direct callers of `unpack(...)` (test fixtures, downstream pipeline code) can pass `subdir=` to keep multiple variants side by side under one cache root without the manual blow-away step.

`.cache/` is gitignored, so the default location stays out of source control.

---

## Running the extraction

```bash
# Write JSONL to a file (uses the repo-relative default cache):
owasp-wstg-extract \
    --input path/to/wstg.tar.gz \
    --output chunks.jsonl

# Or stream JSONL to stdout (useful for piping / inspection):
owasp-wstg-extract \
    --input path/to/wstg.tar.gz \
    --output - \
    | head -3

# An explicit --cache-dir still works and wins over the env var / default:
owasp-wstg-extract \
    --input path/to/wstg.tar.gz \
    --output chunks.jsonl \
    --cache-dir .cache/
```

Re-running against the same cache directory is a no-op for the unpack step. The JSONL output is byte-stable across reruns.

## JSONL output schema

The output file is JSON Lines. Line 1 is a category-map header record; every subsequent line is one chunk. Keys are emitted in sorted order (`sort_keys=True`) so output diffs cleanly across runs.

### Header record

```json
{"type": "category_map", "map": {"ATHN": "Authentication Testing", "...": "..."}}
```

### Chunk record

Each test-case file emits one `#parent` chunk (full document body) plus one chunk per canonical H2 section anchor present in the file. Framework and overview files with no canonical H2s emit only a `#parent` chunk.

```json
{
  "chunk_id": "document__4-Web_Application_Security_Testing__07-Input_Validation_Testing__05-Testing_for_SQL_Injection.md#parent",
  "wstg_id": "WSTG-INPV-05",
  "parent_wstg_id": null,
  "title": "Testing for SQL Injection",
  "parent_category": "INPV",
  "category_name": "Input Validation Testing",
  "source_path": "document/4-Web_Application_Security_Testing/07-Input_Validation_Testing/05-Testing_for_SQL_Injection.md",
  "chunk_index": 0,
  "section_anchor": null,
  "chunk_text": "# Testing for SQL Injection\n\n|ID          |\n..."
}
```

A corresponding section chunk (e.g. the `remediation` section of the same file):

```json
{
  "chunk_id": "document__4-Web_Application_Security_Testing__07-Input_Validation_Testing__05-Testing_for_SQL_Injection.md#remediation",
  "wstg_id": "WSTG-INPV-05",
  "parent_wstg_id": null,
  "title": "Testing for SQL Injection",
  "parent_category": "INPV",
  "category_name": "Input Validation Testing",
  "source_path": "document/4-Web_Application_Security_Testing/07-Input_Validation_Testing/05-Testing_for_SQL_Injection.md",
  "chunk_index": 3,
  "section_anchor": "remediation",
  "chunk_text": "Use parameterized queries..."
}
```

### Field semantics

| Field | Type | Notes |
|-------|------|-------|
| `chunk_id` | string | Deterministic PK; `<sanitized-source-path>#<section>`. `section` is `parent` for the full-document chunk, or one of the six canonical anchor names for section chunks. Stable across rebuilds. |
| `wstg_id` | string \| null | Canonical ID from the in-file table (e.g. `WSTG-INPV-05`). `null` for framework / sub-ID / overview files. |
| `parent_wstg_id` | string \| null | Populated for sub-ID files (e.g. `05.1-*.md`) by looking up the sibling parent file's WSTG-ID. `null` otherwise. |
| `title` | string | First H1 heading, or the filename stem if none. |
| `parent_category` | string \| null | Four-letter code (`INPV`, `ATHN`, …) from the WSTG-ID, or transitively via `parent_wstg_id`. |
| `category_name` | string \| null | Full category name from the Testing_Checklist.md map. |
| `source_path` | string | Forward-slash relative path from the corpus root. |
| `chunk_index` | int | `0` for the `#parent` chunk (full document body); `1..N` for section chunks in canonical anchor order (`summary`, `objectives`, `how_to_test`, `remediation`, `tools`, `references`). |
| `section_anchor` | string \| null | `null` for the `#parent` chunk; the canonical H2 anchor name for section chunks (e.g. `"remediation"`). |
| `chunk_text` | string | Full markdown body for `#parent` chunks; the body text of that H2 section for section chunks. |

### `chunk_id` strategy

`chunk_id` is `<sanitized-source-path>#<section>`:

- Slashes in `source_path` become `__`.
- Characters outside `[A-Za-z0-9._-]` become `_`.
- The section name (`parent`, `summary`, `objectives`, `how_to_test`, `remediation`, `tools`, `references`) is appended after a literal `#`.

This is deterministic (no hashes, no timestamps) and human-readable — the ID names both the source file and the section it represents, so provenance is visible without a separate lookup.

### Category map: header record vs sidecar

Emitted as the first line of the JSONL output (a header record with `"type": "category_map"`). Downstream consumers stream the JSONL file once; a header record keeps everything in one artifact and avoids the need to read a second file. Consumers that only care about chunk records can skip records where `type == "category_map"` (chunk records have no `type` field, so a single if-check suffices).

---

## Walker allowlist

The walker processes exactly these paths (relative to the corpus root):

- `document/**/*.md` — the testing guide proper, including framework, appendix, and README files under each category.
- `checklist/Testing_Checklist.md` — consumed as the category-map source (not emitted as a retrievable chunk).
- `README.md` — top-level framing.
- `Testing_for_APIs.md` — the APIs chapter.

Anything outside that list (`.github/`, images, PDFs, `package.json`, other `checklist/*` files, translations) is silently ignored.

## Corpus-reality notes

### Sub-ID files have no in-file WSTG-ID table

Sub-ID files (e.g. `05.1-Testing_for_Oracle.md`) do not carry their own WSTG-ID in their own markdown table — direct inspection of the corpus shows the in-file ID table only appears on the parent test-case file (`05-Testing_for_SQL_Injection.md`).

The extraction handles that reality by:

- Emitting sub-ID files as their own chunks (they are independent content).
- Setting `wstg_id` to `null` on those chunks (no table to parse).
- Deriving `parent_wstg_id` by reading the sibling parent file (`05-*.md`) and taking its WSTG-ID.
- Deriving `parent_category` / `category_name` transitively from the parent's WSTG-ID.

Downstream consumers that want to group a sub-ID chunk under its parent test case should key on `parent_wstg_id` rather than on the child chunk's `wstg_id`.

### README files under each category are framework chunks

Each category directory under `document/4-Web_Application_Security_Testing/` carries a `README.md` that acts as a category overview. Those files have no WSTG-ID table, so the walker emits them as framework chunks (`wstg_id=null`, `parent_wstg_id=null`). Category context for those files is available through the separately emitted category map.

---

## Embedding + vector store (`embedding/` submodule)

A local, CPU-only, cloud-free embedding + retrieval layer over the JSONL chunks. Exposes three query primitives and a build-index CLI; consumed by the MCP server.

### Embedding model — ONNX runtime + MiniLM-L6-v2

- **Model:** `sentence-transformers/all-MiniLM-L6-v2`, pinned at revision `c9745ed1d9f207416be6d2e6f8de32d1f16199bf`. Only the `onnx/model.onnx` variant plus its tokenizer are used — no torch weights pulled.
- **Runtime:** `onnxruntime` (CPU execution provider) + `tokenizers` for WordPiece encoding.
- **Pooling:** attention-mask-weighted mean pool over the last hidden state, then L2 normalisation — matches sentence-transformers' default for this model. Produces 384-dim unit-length vectors.
- **Rationale:** total install footprint is ~200 MB (onnxruntime + tokenizers + 90 MB ONNX weights) vs ~1 GB for a torch sentence-transformers install, with faster cold start on CPU. No GPU required; no cloud calls.
- **Fallback:** a native torch `sentence-transformers` pipeline is the documented alternative if the ONNX path ever proves blocking. Swap `Embedder` in `owasp_wstg_mcp/embedding/model.py` — the public `embed_texts` and the build/search paths treat it as a black box.

### Vector store — sqlite-vec

- **Store:** single sqlite file at `owasp_wstg_mcp/data/wstg_index.sqlite` (path is overridable). Two tables:
  - `chunk_vectors` (vec0 virtual table) — `chunk_id` PK, embedding column (384-dim float32), plus `wstg_id` + `parent_category` partition columns for KNN-with-filter.
  - `chunk_metadata` (regular table) — scalar metadata + `chunk_text`, indexed on `wstg_id` and `parent_category`.
- **Why sqlite-vec:** single-file, vendorable extension that loads cleanly on common Linux/macOS hosts. Uses the idiomatic vec0 KNN surface (`WHERE embedding MATCH :q AND k = :k`). `chunk_text` is kept in a regular joined table rather than a vec0 auxiliary column — WSTG test-case bodies can run tens of kilobytes, and vec0 auxiliaries are intended for short filter values.
- **Fallback:** if sqlite-vec ever fails to load on a target host (extension-loading disabled, libsqlite3 mismatch), the documented fallback is Chroma in single-file persistent mode. Swap `owasp_wstg_mcp/embedding/store.py` — the public `vector_search` / `upsert_chunks` / `lookup_by_wstg_id` / `lookup_section` / `list_by_category` signatures are the isolation boundary.

### Idempotence — upsert-by-chunk_id

`chunk_id` values are stable across reruns, so `build_index` is "insert-or-replace by `chunk_id`" inside a single transaction. A second build against identical JSONL produces zero net row delta on both `chunk_metadata` and `chunk_vectors`, exits 0, and leaves the row set byte-identical. (The sqlite file's on-disk byte length can change by a single page due to free-list churn — this is cosmetic, not a row-level change.)

### Schema version and duplicate-wstg_id enforcement

`owasp_wstg_mcp/embedding/store.py` carries a `SCHEMA_VERSION` constant stored in the sqlite `PRAGMA user_version`. **Current version: 5.**

- **v2:** `chunk_metadata.wstg_id` carries a `UNIQUE` constraint. Default SQLite semantics apply (each `NULL` is distinct), so framework / appendix / README chunks with `wstg_id=NULL` still coexist. Two rows sharing a non-null `wstg_id` raise `DuplicateWstgIdError` — a corpus defect that should fail the build loudly rather than silently upsert-replace.
- **v3:** adds the `chunk_edges` table backing the structural-graph retrieval layer. The addition is schema-additive — `chunk_metadata` and `chunk_vectors` are unchanged — but the `user_version` bump still forces a rebuild on upgrade because `ensure_schema_current` rejects any `user_version != 3`.
- **v4 → v5:** per-H2-section chunking. Each test-case markdown file now emits a synthetic `#parent` chunk (full document body) plus one chunk per canonical H2 section anchor present in the file (`summary`, `objectives`, `how_to_test`, `remediation`, `tools`, `references`). This semantic granularity change required: (a) a `section_anchor TEXT` column on `chunk_metadata` (NULL for parent rows, anchor name for section rows); (b) composite UNIQUE INDEX rework — the v4 single-column partial index is replaced by two partial indexes: `ux_chunk_metadata_wstg_id_section` on `(wstg_id, section_anchor)` for section rows, and `ux_chunk_metadata_wstg_id_parent` on `(wstg_id)` for parent rows (SQLite treats each NULL as distinct, so a single composite index cannot enforce the one-parent-per-wstg_id invariant); (c) a `lookup_section(wstg_id, anchor)` primitive for primary-key-style indexed reads over the composite.
- **Rebuild-only migration.** Stale stores (any `user_version != 5`) are rejected by `ensure_schema_current` with an error pointing at the build CLIs. No in-place `ALTER TABLE` migration is provided or supported.

### Rebuild procedure

```bash
# 1. Produce JSONL chunks from the tarball:
owasp-wstg-extract \
    --input path/to/wstg.tar.gz \
    --output chunks.jsonl \
    --cache-dir .cache/

# 2. Build the vector store from those chunks:
owasp-wstg-build-index --input chunks.jsonl

# 3. Build the structural-graph edges from the same chunks:
owasp-wstg-build-graph --input chunks.jsonl

# Pipeline form (skip the intermediate file for step 2):
owasp-wstg-extract --input path/to/wstg.tar.gz \
                   --output - --cache-dir .cache/ \
    | owasp-wstg-build-index --input -
```

**Steps 2 and 3 are order-independent.** `owasp-wstg-build-index` and `owasp-wstg-build-graph` write to disjoint tables in the same sqlite file and share the schema-init sequence (`ensure_schema_current` → `initialize_schema`); either CLI can run first against an empty store. Both must complete successfully before the MCP server is started — `wstg_search` / `wstg_get` / `wstg_list_category` read from the embedding tables, `wstg_neighbors` / `wstg_section` read from the `chunk_edges` table plus `chunk_metadata`.

The default output location is `owasp_wstg_mcp/data/wstg_index.sqlite` inside the installed package; override with `--output PATH` or the `OWASP_WSTG_MCP_STORE_PATH` environment variable. Override the model cache with `--model-cache-dir` or `OWASP_WSTG_MCP_MODEL_DIR`. The MiniLM snapshot is downloaded on first run (~90 MB); subsequent runs reuse the cache.

**Build time (observed):** seconds on CPU once the model cache is warm. The second (idempotent) build runs in similar time — same work, no model reload.

For the procedure to re-target this package at a new WSTG corpus tarball (download → regen → integrity-test → commit → collision resolution), see [docs/version-bump-runbook.md](docs/version-bump-runbook.md).

### Query API

```python
from owasp_wstg_mcp.embedding import search, get, list_category, build_index
```

#### `search(query: str, top_k: int = 5) -> list[dict]`

Semantic search over the chunk body. Returns up to `top_k` hits ranked by descending cosine similarity. Each hit:

```
{
  "text":             str,            # chunk body (full document for parent chunks; H2 section body for section chunks)
  "wstg_id":          str | None,     # canonical ID (None for framework chunks)
  "parent_wstg_id":   str | None,     # parent test-case ID for sub-ID chunks
  "title":            str,            # first H1 or filename stem
  "parent_category":  str | None,     # 4-letter category code (INPV, ATHN, …)
  "category_name":    str | None,     # full category name
  "source_path":      str,            # relative path inside the corpus
  "section_anchor":   str | None,     # H2 anchor for section chunks (e.g. "how_to_test"); None for parent/framework
  "score":            float,          # cosine similarity, ∈ [-1.0, 1.0]
}
```

#### `get(wstg_id: str) -> dict | None`

Exact lookup by canonical WSTG-ID. Returns `None` for unknown IDs and for framework / appendix chunks whose `wstg_id` is `None` by design — those are retrievable via `search` only.

#### `list_category(category_code: str) -> list[dict]`

Enumerates test-case chunks under a 4-letter category code. Returns `[{wstg_id, title}, …]` sorted by `wstg_id`. Returns `[]` for unknown codes, for the empty string, and for categories whose chunks all have `wstg_id = None`.

#### `build_index(jsonl_path, *, store_path=None, ...) -> dict`

Programmatic equivalent of the CLI. Useful when an embedding consumer needs to rebuild without shelling out.

### Null-handling semantics

| Chunk shape | `search` | `get(wstg_id)` | `list_category(code)` |
|-------------|----------|----------------|------------------------|
| Test-case chunk, `wstg_id` set | surfaces | exact hit | listed under its `parent_category` |
| Sub-ID chunk, `wstg_id=None`, `parent_wstg_id` set | surfaces | not retrievable | not listed (no `wstg_id`) |
| Framework / README / appendix, both null | surfaces | not retrievable | not listed |

`get` and `list_category` are keyed on `wstg_id`; framework content has no such key. Consumers who want framework context go through `search`. The unit tests in `tests/test_embedding_build.py` pin this contract.

### Smoke-test procedure (real corpus)

With `chunks.jsonl` produced from the real tarball and the store built:

```python
from owasp_wstg_mcp.embedding import search, get

search("session fixation test steps", top_k=3)
# Expected top-3 to include WSTG-SESS-03 chunks (parent or section chunks).
# Each hit now carries a ``section_anchor`` field — e.g. "objectives",
# "how_to_test", or null for parent/framework chunks.

search("how to test for SSRF", top_k=3)
# Expected top-1: WSTG-INPV-19 (parent or section chunk).

get("WSTG-ATHN-04")
# Expected: {"wstg_id": "WSTG-ATHN-04",
#            "title":   "Testing for Bypassing Authentication Schema",
#            "chunk_index": 0, "section_anchor": null, …}
# Returns the #parent chunk (full document body; chunk_index=0, section_anchor=null).
```

### Tests

The unit tests under `tests/test_embedding_store.py` and `tests/test_embedding_build.py` inject a `KeywordStubEmbedder` so the suite runs fast (< 2 s) and deterministically, independently of the HuggingFace model download. The real ONNX pipeline is exercised by the smoke-test commands above, not in the pytest suite.

---

## MCP server (`server/` submodule)

The server is a stdio-transport MCP server exposing the embedding and graph query APIs as five tools. Two equivalent entrypoints:

```bash
# Console script (installed with the package):
owasp-wstg-mcp-serve

# Module form (equivalent):
python -m owasp_wstg_mcp.server
```

You do not normally invoke it by hand — an MCP-compatible client (e.g. Claude Code) spawns it via its registration.

The server reads from the same sqlite-vec store that `owasp-wstg-build-index` and `owasp-wstg-build-graph` write to (`owasp_wstg_mcp/data/wstg_index.sqlite`, overridable via `OWASP_WSTG_MCP_STORE_PATH`). If the store is missing or stale, rebuild it with the "Rebuild procedure" above — both build CLIs must succeed before the server starts. The server itself does not build the index or the graph; the build steps are separate one-shots.

### Registering with Claude Code

The "Deploy from a fresh clone" section above covers the user-scope `claude mcp add` command. Swap `--scope user` for `--scope project` to register the server only for the current workspace instead of globally.

`claude mcp get owasp-wstg` prints the registered command line and transport for a deeper check than `claude mcp list`. A fresh Claude Code session picks up the five tools (`wstg_search`, `wstg_get`, `wstg_list_category`, `wstg_neighbors`, `wstg_section`) on startup.

### Tools

| Tool | Input | Output | Notes |
|------|-------|--------|-------|
| `wstg_search` | `query: string (non-empty)`, `top_k: int (1..25, default 5)` | `[{text, wstg_id, parent_wstg_id, title, parent_category, category_name, source_path, section_anchor, score}]` | Semantic retrieval over chunk text. `section_anchor` is the H2 anchor for section chunks (e.g. `"how_to_test"`, `"remediation"`) or `null` for parent and framework chunks. `score` is cosine similarity, or `null` for framework chunks. |
| `wstg_get` | `wstg_id: string (pattern `^WSTG-[A-Z]{4}-\d{2}$`)` | Full chunk record or `null` | Well-formed unknown ID → `null`. Malformed ID → protocol error. |
| `wstg_list_category` | `category: string (pattern `^[A-Z]{4}$`)` | `[{wstg_id, title}]` | Unknown well-formed code → `[]`. Malformed → protocol error. Valid codes: APIT, ATHN, ATHZ, BUSL, CLNT, CONF, CRYP, ERRH, IDNT, INFO, INPV, SESS. |
| `wstg_neighbors` | `wstg_id: string`, `edge_types: list[string] \| null`, `direction: "out"\|"in"\|"both"` (default `"both"`), `depth: 1\|2` (default `1`) | `[{neighbor, neighbor_kind, title, edge_type, direction, hops}]` | Structural-graph neighbours. See "Structural-graph retrieval" below. |
| `wstg_section` | `wstg_id: string`, `section: string` | `{wstg_id, section, text}` or `null` | One canonical section of a test. See "Structural-graph retrieval" below. |

Invariant violations (empty query, `top_k` out of `[1, 25]`, malformed WSTG-ID, non-4-letter category, unknown `edge_types` entry, bad `direction`, bad `depth`, unknown `section` anchor) are raised as exceptions and surface as MCP protocol errors (JSON-RPC error responses / `isError=true` tool results per the SDK convention). "Unknown but well-formed" is not an error — `wstg_get` / `wstg_section` return `null`, `wstg_list_category` / `wstg_neighbors` return `[]`.

### Example tool calls

For the full sample-query reference, see [docs/sample-queries.md](docs/sample-queries.md). One illustrative call:

```
wstg_get("WSTG-ATHN-04")
# → {"wstg_id": "WSTG-ATHN-04",
#    "title":   "Testing for Bypassing Authentication Schema",
#    "chunk_index": 0, "section_anchor": null, …}
# Returns the #parent chunk (full document body).
```

---

## Structural-graph retrieval (`graph/` submodule)

The graph layer adds a structural-graph retrieval layer on top of the same sqlite file the embedding layer uses. It exposes two MCP tools — `wstg_neighbors` and `wstg_section` — that let a caller walk between related WSTG test cases and pull one canonical section out of a test without reading the whole markdown body.

### Section-anchor vocabulary

The `graph/extract.py` module walks each chunk's markdown, finds H2 headings, normalizes them (lowercase, whitespace collapsed, "Test Objectives" → `objectives`, etc.), and keeps exactly six anchors. Any H2 whose normalized form is not in this table is **dropped silently** — no error, no warning, no chunk rejection. The set is intentionally fixed so `wstg_section` has a closed vocabulary and tools can enumerate valid anchors:

| Anchor | Typical WSTG H2 text it matches |
|--------|---------------------------------|
| `summary` | "Summary" |
| `objectives` | "Test Objectives", "Objectives" |
| `how_to_test` | "How to Test" |
| `remediation` | "Remediation" |
| `tools` | "Tools" |
| `references` | "References" |

These six anchors are also the valid `section` argument values for `wstg_section` and the possible `neighbor` values for `wstg_neighbors` rows where `neighbor_kind == "section_anchor"`.

### `chunk_edges` table and edge types

The `chunk_edges` table holds one row per directed relationship between chunks. Four edge types are emitted:

| `edge_type` | Meaning | `neighbor_kind` in the tool response |
|-------------|---------|--------------------------------------|
| `has_subtest` | Parent test → child sub-ID test (e.g. `WSTG-INPV-05` → `WSTG-INPV-05.1`) | `test` |
| `references` | One test cites another test in its body | `test` |
| `in_category` | Test belongs to a 4-letter category (e.g. → `INPV`) | `category` |
| `has_section` | Test exposes a canonical section anchor (e.g. → `how_to_test`) | `section_anchor` |

Rebuilding `chunk_edges` is idempotent by design — the store's `ux_chunk_edges_identity` unique index folds duplicate (src, dst, edge_type, edge_target) tuples on re-run.

### `wstg_neighbors` tool contract

**Signature:**

```
wstg_neighbors(
    wstg_id: str,
    edge_types: list[str] | None = None,
    direction: str = "both",       # "out" | "in" | "both"
    depth: int = 1,                # 1 or 2
) -> list[dict]
```

**Validation posture.**

- `wstg_id` must match `^WSTG-[A-Z]{4}-\d{2}$`. Malformed → protocol error.
- `edge_types` must be `None` (all four) or a list that is a subset of `{"has_subtest", "references", "in_category", "has_section"}`. Unknown entries → protocol error.
- `direction` must be one of `"out"`, `"in"`, `"both"`. Anything else → protocol error.
- `depth` must be exactly `1` or `2` (reject, don't clamp; `bool` is rejected even though Python treats it as `int`).
- Well-formed but unknown `wstg_id` → `[]` (not an error).

**Response shape.** Each row is:

```
{
  "neighbor":      str,          # the other endpoint (WSTG-ID, category code, or section anchor)
  "neighbor_kind": str,          # "test" | "category" | "section_anchor"
  "title":         str | None,   # test title, category display name, or None for section anchors
  "edge_type":     str,          # one of the four edge types above
  "direction":     str,          # "out" or "in" (never "both" — "both" on input fans out to two row sets)
  "hops":          int,          # 1 for direct edges, 2 for transitive (only when depth=2)
}
```

**Example:**

```
wstg_neighbors("WSTG-INPV-05", edge_types=["has_subtest"], direction="out")
# → [{"neighbor": "WSTG-INPV-05.1", "neighbor_kind": "test",
#     "title": "Testing for Oracle", "edge_type": "has_subtest",
#     "direction": "out", "hops": 1}, ...]

wstg_neighbors("WSTG-ATHN-04")
# → all four edge types, both directions, depth=1.
```

### `wstg_section` tool contract

**Signature:**

```
wstg_section(wstg_id: str, section: str) -> dict | None
```

**Validation posture.**

- `wstg_id` must match `^WSTG-[A-Z]{4}-\d{2}$`. Malformed → protocol error.
- `section` must be one of the six anchors listed above. Anything else → protocol error.
- Well-formed but unknown `wstg_id` → `null` (matches `wstg_get`).
- Valid ID + valid section, but no section chunk exists for that anchor (H2 not present in the file, or the file is a stub that emits only its `#parent` chunk) → `null`.

**Implementation note.** `wstg_section` is an indexed lookup over the composite `(wstg_id, section_anchor)` index in `chunk_metadata` (SCHEMA_VERSION 5). It returns the `chunk_text` of the matching section chunk directly — no body-walking or substring extraction.

**Response shape on success:**

```
{
  "wstg_id": str,   # echo of the input
  "section": str,   # echo of the input anchor
  "text":    str,   # the body of that H2 section chunk (markdown)
}
```

**Example:**

```
wstg_section("WSTG-ATHN-04", "how_to_test")
# → {"wstg_id": "WSTG-ATHN-04", "section": "how_to_test",
#    "text": "Test for ...\n\n..."}
```

### Tests

Graph-layer unit tests live in `tests/test_graph_*.py` and the server-integration tests for the two new tools live in `tests/test_server_graph.py`. The `chunk_edges` table is populated for every test run from the committed `wstg_index.sqlite` artefact or from fixtures.

---

## Tests

```bash
.venv/bin/python3 -m pytest
```

Four fixture files live under `tests/fixtures/`, one per case the chunker needs to handle: a test-case file with a WSTG-ID table, a sub-ID file, a framework/overview file, and a checklist snippet.

See Configuration → Test environment variables for the full opt-in surface that gates the corpus-integrity and cross-version derivation tests.

---

## Skills

The `skills/` directory at the repo root contains bundled skills for MCP-aware agents. Skills are markdown files an agent reads to adopt a structured workflow over the MCP tools.

### `skills/security-qa-tester/`

Encodes the security-qa-tester workflow: how to scope a test run, which MCP tools to call in which order, how to walk related tests via `wstg_neighbors`, and how to cite WSTG findings so downstream reviewers can verify them against the source.

Three files:

| File | Purpose |
|------|---------|
| `SKILL.md` | Model-agnostic base — tool surface, workflow shape (scope → search → neighbor-walk → cite), citation convention. Load this first. |
| `SKILL-opus.md` | Claude Opus 4.7 variant — adds XML-wrapped worked examples, output-format controls, and phrasing calibrated for Opus 4.7. Load after `SKILL.md`. |
| `SKILL-gpt-5.5.md` | GPT-5.5 variant — structures the workflow as an outcome-first prompt block following OpenAI's prompt-guidance conventions. Load after `SKILL.md`. |

To use: locate `skills/security-qa-tester/` in the repo, read `SKILL.md`, then read the variant file that matches your model family.