# graph-mcp-java-gen

[![CI](https://github.com/rajendarmuddasani/02-graph-grounded-genai-test-generation/actions/workflows/ci.yml/badge.svg)](https://github.com/rajendarmuddasani/02-graph-grounded-genai-test-generation/actions/workflows/ci.yml)
[![Python](https://img.shields.io/badge/Python-3.10%20%7C%203.12-4ea5d9)](pyproject.toml)
[![Evidence](https://img.shields.io/badge/evidence-validated-18745a)](evidence/claims.json)
[![License](https://img.shields.io/badge/license-MIT-f3c969)](LICENSE)
[![MCP](https://img.shields.io/badge/MCP-stdio-7b4ea0)](src/graph_mcp/server.py)
[![Neo4j](https://img.shields.io/badge/Neo4j-5.26-008CC1)](evidence/neo4j_integration.json)

**Graph-grounded MCP server that converts bounded requests and public TOML
specifications into validated, compilable Java test artifacts.**

A natural-language or structured request enters an official Model Context Protocol (MCP) stdio server. A versioned graph catalog (Neo4j or JSON fixture) supplies the only symbols the generator may cite. A multi-layer validator checks syntax, framework contract, grounding, and forbidden-API rules before any source is returned. Two optional LLM agents — an intent normaliser and a post-generation reviewer — extend the pipeline to free-form input without compromising the deterministic safety envelope.

---

## Architecture

```mermaid
flowchart TD
  NL[Bounded or free-form request] --> MCP[FastMCP stdio server]
  FIELDS[Structured fields] --> MCP
  TOML[Public TOML specification] --> MCP
  MCP --> INTENT[Validated generation intent]
  MCP --> SPEC[Validated public test specification]
  INTENT --> GRAPH[Framework catalog]
  GRAPH --> JAVA[Deterministic Java generator]
  SPEC --> SDK[Public SDK catalog]
  SDK --> BUNDLE[TestMethod, TestCase, and DataHandler generator]
  JAVA --> VALIDATE[Syntax, contract, grounding, and safety gates]
  BUNDLE --> VALIDATE
  VALIDATE -->|Pass| ACCEPT[Accepted Java artifacts]
  VALIDATE -->|Fail| REJECT[Typed rejection]
```

---

## Multi-Agent Pipeline

```mermaid
sequenceDiagram
    autonumber
    actor User
    participant MCP as FastMCP Server
    participant A1 as LLMIntentParser
    participant GDB as Graph Catalog
    participant GEN as Generator and Validator
    participant A2 as ReviewAgent

    User->>MCP: generate_java_test_nlp(free-form NL)
    MCP->>A1: extract intent fields
    A1-->>MCP: {class, package, module, config, version}
    MCP->>GDB: get versioned symbols
    GDB-->>MCP: 7 cited GraphSymbol objects
    MCP->>GEN: render Java + validate
    GEN-->>MCP: validated Java source
    MCP->>A2: review(source, class, package)
    A2-->>MCP: {approved, checklist, issues}
    MCP-->>User: {status, source, citations, review}
```

---

## End-to-End Pipeline

Ingestion builds the graph once; generation runs per request. Every generated class
may import **only** symbols the graph returned — the grounding gate makes a hallucinated
import structurally impossible.

```mermaid
flowchart LR
    SOURCE[Versioned synthetic metadata] --> FIXTURE[JSON fixture]
    FIXTURE --> NEO4J[Optional Neo4j materialization]
    FIXTURE --> SERVER[FastMCP server with seven tools]
    NEO4J --> SERVER
    PUBLIC[Public SDK fixture] --> SERVER
    SERVER --> CORE[Original workflow generation]
    SERVER --> SMT[Public TOML bundle generation]
    CORE --> GATES[Validation gates]
    SMT --> GATES
    GATES --> RESULT[Java artifacts or typed rejection]
```

### Pipeline reference card

| Group | Count | Members |
|---|:---:|---|
| **Intent fields** | **5** | `class_name` · `package_name` · `module_name` · `config_path` · `version` |
| **MCP tools** | **7** | Metadata, search, three original generation tools, public bundle generation, and source validation |
| **Required symbols** | **7** | `BaseTestMethod` · `ConfigBlock` · `ConfigLoader` · `LevelChangeAction` · `TestCaseBase` · `TestList` · `TestListManager` |
| **Review checklist** | **6** | `extends_base_class` · `has_javadoc` · `uses_config_loader` · `defines_test_sequences` · `no_hardcoded_strings` · `correct_package` |

**Validation gates (all must pass):** Tree-sitter syntax · framework contract · grounding = `|imports ∩ cited| / |imports| = 1.0` · source-safety (no `Runtime`/`ProcessBuilder`/`System.exit`/`java.io`/`java.net`/`native`).

---

## Evidence Dashboard

All measurements use independently generated CC0-licensed synthetic fixtures.
Results are from the accepted `strict_graph_v2` policy on the held-out confirmation split.

| Surface | Result | Artifact |
|---|:---|---|
| Benchmark scale | 96 CC0 intents — 32 dev / 32 val / **32 confirmation** | [task_evaluation.json](evidence/task_evaluation.json) |
| Confirmation task success | **32 / 32** bounded tasks | [evaluation_trace.json](evidence/evaluation_trace.json) |
| Generated-source validation | **24 / 24** supported intents — syntax + contract + grounding + safety | [task_evaluation.json](evidence/task_evaluation.json) |
| Safe adversarial rejection | **8 / 8** — zero false accepts | [task_evaluation.json](evidence/task_evaluation.json) |
| Citation precision | **100%** — only graph-cited symbols imported | [task_evaluation.json](evidence/task_evaluation.json) |
| Required-symbol recall | **100%** — every required symbol present | [task_evaluation.json](evidence/task_evaluation.json) |
| Live Neo4j integration | **Neo4j 5.26.29** — 8 symbols, 12 methods materialized | [neo4j_integration.json](evidence/neo4j_integration.json) |
| Official MCP benchmark | **120 / 120** expected outcomes — zero protocol errors | [mcp_benchmark.json](evidence/mcp_benchmark.json) |
| MCP warm latency (p50 / p95 / p99) | **29.13 / 48.61 / 54.23 ms** at concurrency 1 | [mcp_benchmark.json](evidence/mcp_benchmark.json) |
| Java compilation | **8 / 8 class files** via Eclipse ECJ 3.21 | [java_compile.json](evidence/java_compile.json) |
| External model calls (deterministic path) | **0 calls · $0.00** | [mcp_benchmark.json](evidence/mcp_benchmark.json) |

> Latency figures are single-process local Windows measurements, not production SLOs.

---

## Policy Selection

Four generation policies were evaluated. The selection objective was declared before opening the confirmation split: maximise validation task success among candidates passing **all** safety gates. Confirmation was opened exactly once for the selected candidate.

| Candidate | Task success | Gen valid | Safe reject | Citation prec | Decision |
|---|:---:|:---:|:---:|:---:|---|
| `no_graph_v0` | 21.9% | 0% | 87.5% | 0% | Rejected — no grounding |
| `lenient_repair_v1` | 75.0% | 100% | 0% | 100% | Rejected — 8 false accepts |
| **`strict_graph_v2`** | **100%** | **100%** | **100%** | **100%** | **Selected** |
| `wide_context_v3` | 96.9% | 100% | 87.5% | 87.5% | Rejected — irrelevant context + 1 false accept |

---

## MCP Tools

| Tool | Type | Behaviour |
|---|---|---|
| `get_fixture_metadata` | Read | Returns fixture identity, provenance, license, backend, symbol count |
| `search_graph` | Read | Parameterised name/method search; max 20 results |
| `generate_java_test` | Generate | Typed fields → graph lookup → Java → all validation gates |
| `generate_java_test_from_intent` | Generate | Bounded 3-form grammar → same strict policy |
| `generate_public_smt8_bundle` | Generate | TOML → validated TestMethod, TestCase, and DataHandler Java bundle |
| `validate_java_source` | Validate | Checks up to 20 000 chars; never writes or executes source |
| `generate_java_test_nlp` | **Multi-agent** | LLM intent parser → generator → LLM reviewer; requires `OPENAI_API_KEY` |

The Neo4j adapter uses fixed parameterised Cypher, rejects credentials in URIs, and refuses fixture identity collisions.

### Public synthetic SMT8-style bundle

`generate_public_smt8_bundle` accepts an original, product-neutral TOML schema. It supports
`voltage`, `leakage`, and `functional` method families:

```toml
[test]
name = "CoreVoltage"
package = "publicdemo.tests"
method = "voltage"
pin = "VCORE"
low_limit = 0.75
high_limit = 0.85
force_value = 0.8
samples = 16
```

The tool returns three Java sources named `*TestMethod.java`, `*TestCase.java`, and
`*DataHandler.java`. Before release, every source must pass tree-sitter syntax parsing,
artifact-contract checks, forbidden-API scanning, and 100% import grounding against
`fixtures/public_smt8_graph.json`. See
[`examples/public_smt8/voltage_test.toml`](examples/public_smt8/voltage_test.toml).

This profile is **SMT8-style public synthetic output**. It is not compatible with, derived
from, or represented as any proprietary test framework.

---

## Quick Start

```bash
python -m venv .venv
# Windows
.\.venv\Scripts\Activate.ps1
# Linux / macOS
source .venv/bin/activate

pip install -r requirements-dev.txt
pip install --no-deps -e .

# Run the offline smoke test (no database needed)
python scripts/container_smoke.py python -m graph_mcp.server
```

### MCP client configuration (VS Code / Claude Desktop)

```json
{
  "mcpServers": {
    "graph-java-gen": {
      "command": "/absolute/path/to/.venv/bin/python",
      "args": ["-m", "graph_mcp.server"],
      "cwd": "/absolute/path/to/repo"
    }
  }
}
```

### Enable the multi-agent NLP tool

```bash
# Add to your environment or .env file
OPENAI_API_KEY=sk-...
GRAPH_BACKEND=neo4j   # optional; defaults to local JSON fixture
```

---

## Reproduce Evidence

```bash
# Build the CC0 benchmark fixture
python scripts/build_evaluation_fixture.py

# Run all four candidate policies and select strict_graph_v2
python scripts/evaluate_workflow.py

# Validate the claims ledger and evidence privacy rules
python scripts/validate_evidence.py

# Full test suite
pytest --cov=src --cov-report=term-missing --cov-fail-under=75

# Lint and security
ruff check src tests scripts
bandit -r src scripts -q -ll
pip-audit -r requirements.txt --progress-spinner off
```

### Live Neo4j path

```bash
# Start a local Neo4j Community instance (Docker)
docker compose up -d neo4j
python scripts/wait_for_neo4j.py

# Seed the synthetic graph fixture and verify retrieval
python scripts/seed_graph.py
python scripts/verify_neo4j.py   # writes evidence/neo4j_integration.json

# Full MCP benchmark over stdio with live graph
python scripts/benchmark_mcp.py  # writes evidence/mcp_benchmark.json
```

### Java compilation

```bash
# Requires JDK 17 or newer on PATH
python scripts/compile_generated.py --require-compiler
# Writes evidence/java_compile.json
```

---

## Security Design

- **No raw Cypher on the MCP surface** — all graph queries are parameterised.
- **Strict field allowlists** — class names, package names, module names, versions, and config paths are checked against compiled regex patterns before any graph lookup.
- **Source safety scanner** — generated Java is rejected if it references `Runtime.getRuntime`, `ProcessBuilder`, `System.exit`, `java.io`, `java.nio.file`, or `java.net`.
- **Path traversal prevention** — absolute paths and `..` segments are rejected in config path fields.
- **Grounding enforcement** — framework imports must correspond to graph-retrieved
  symbols; the original workflow additionally permits only its explicit JDK import
  allowlist.
- **LLM output re-validated** — fields extracted by the LLM intent parser pass through the same `GenerationIntent.from_mapping()` validation as direct API calls.
- **Neo4j credentials** — loaded only from environment variables; never logged or returned in evidence artifacts.
- **XML preflight** — `defusedxml` prevents entity-expansion attacks in project-structure scanning.
- **Container** — pinned Chainguard Linux image, non-root UID/GID 65532; CI performs an MCP-over-container stdio smoke test.

See [SECURITY.md](SECURITY.md) for the full threat boundary.

---

## Repository Map

```text
src/graph_mcp/
  workflow.py            intent parsing · graph lookup · Java generation · validation
  smt8_public.py         public TOML schema · three-artifact generation · validation
  graph_store.py         Neo4j catalog adapter (parameterised Cypher)
  llm_intent_parser.py   Agent 1 — LLM free-form NL → GenerationIntent
  review_agent.py        Agent 2 — LLM post-generation checklist reviewer
  server.py              FastMCP stdio server (7 tools)
  evaluation.py          candidate scoring and selection harness

fixtures/
  synthetic_graph.json   CC0 versioned framework symbol catalog (SHA-256 bound)
  public_smt8_graph.json CC0 public synthetic SDK symbol catalog
  evaluation_cases.json  96 CC0 natural-language intents (32/32/32 split)
  java_framework/        independently generated Java compilation fixtures

evidence/
  claims.json            machine-readable claims ledger (14 public claims)
  evaluation_protocol.json  pre-declared selection rules and safety gates
  task_evaluation.json   per-candidate, per-split, per-case results
  evaluation_trace.json  confirmation case-level trace
  neo4j_integration.json live Neo4j integration result
  mcp_benchmark.json     MCP protocol benchmark (120 calls)
  java_compile.json      ECJ compilation result

scripts/
  build_evaluation_fixture.py   generate benchmark from seed
  evaluate_workflow.py          run and score all four candidates
  validate_evidence.py          verify claims ledger and privacy rules
  benchmark_mcp.py              official MCP stdio latency benchmark
  verify_neo4j.py               live graph integration check
  compile_generated.py          Java compilation gate
  seed_graph.py                 materialise fixture into Neo4j

tests/
  test_generation_loop.py       generation + validation unit tests
  test_smt8_public.py            TOML, bundle validation, and optional javac compile tests
  test_graph_store.py           Neo4j adapter unit tests
  test_mcp_protocol.py          official MCP protocol conformance
  test_evaluation.py            evaluation harness tests
  test_evidence.py              claims ledger integrity tests
  test_neo4j_live.py            opt-in live graph tests (NEO4J_* env required)

docs/
  ARCHITECTURE.md        component design and data flow
  POLICY_CARD.md         candidate selection details
  DATA_CARD.md           fixture provenance and license
  MCP_INTEGRATION.md     client configuration guide
  DEPLOYMENT.md          Docker and container notes

templates/               MCP prompt templates for VS Code Copilot
examples/                sample project preflight scanner
```

---

## Boundaries

The following are **not** claimed by this repository:

- Free-form intent parsing quality independent of model version — the LLM pipeline is opt-in and its results are not captured in the frozen evaluation artifacts.
- Compatibility with any proprietary or confidential Java test framework.
- Generation of vendor-specific test-program assets, APIs, names, or product data.
- Production latency SLO — all measurements are single-process local sequential benchmarks.
- Concurrent, distributed, or high-availability operation.
- Automatic execution of generated Java against hardware or a test instrument.
- Any productivity, cost, yield, or test-time saving — this repository contains only generation and validation evidence.

The full machine-readable boundary is in [evidence/claims.json](evidence/claims.json).

---

## License

Repository code: **MIT**.
Graph fixture, intent cases, and Java stubs: **CC0-1.0** (labelled in fixture metadata).