Verified Regex Generator

MCP → Verified Regex Generator. The user describes a pattern in plain English. An LLM generates the regex. An agent generates real sample strings (positive + negative examples), tests the regex against them with the actual re engine, and iterates if it's wrong. Regex is infamous for looking right while being subtly wrong — verification is the whole value here, not the generation.

Runs on Groq (openai/gpt-oss-120b by default) — fast and free-tier friendly, so it's cheap to demo publicly.

This isn't "ask an LLM for a regex and hope." It's a closed verification loop:

description  ──►  LLM writes a candidate regex
                        │
description  ──►  LLM writes real test strings (incl. tricky near-misses)
                        │
                        ▼
              Python's `re` engine checks the candidate
              against every test string — ground truth,
              not the model's opinion of itself
                        │
              ┌─────────┴─────────┐
          all pass             something failed
              │                     │
              ▼                     ▼
           done               feed the exact failures
                               back to the model, try again

Demo

A real run: "US phone number" → the agent writes 16 test strings, proposes a candidate regex, checks it against Python's real re engine, and converges on a verified pattern in 2 iterations.

Related MCP server: EvalView

What's in here

• regex_agent/core.py — the actual agent loop (model-agnostic of transport). Everything else is a thin wrapper around this.

• mcp_server/ — a real MCP server exposing generate_verified_regex as a tool, so it can be used directly from Claude Desktop or Claude Code.

• web/ — a FastAPI + vanilla-JS demo with two parts:

  1. A live, animated view of the agent's reasoning: test cases, each candidate regex, and the pass/fail table per iteration.

  2. A regex101-style live tester underneath — an editable pattern field with flags (g/i/m/s/full-match), live match highlighting against your own test string, a match list with capture groups, and a plain-English token-by-token breakdown of the regex (all client-side, no API calls). It auto-fills with whatever the agent just verified, but works standalone for any regex you paste in — useful even if you already know regex and just want to test one.

  No MCP client required — runs in a browser.

Setup

pip install -r requirements.txt
cp .env.example .env   # then add your GROQ_API_KEY

Run the web demo

cd web
python server.py

Open http://127.0.0.1:8000.

Run the MCP server

Add this to your MCP client config (e.g. Claude Desktop's claude_desktop_config.json, or .claude/settings.json for Claude Code):

{
  "mcpServers": {
    "verified-regex-generator": {
      "command": "python",
      "args": ["/absolute/path/to/2 project/mcp_server/server.py"]
    }
  }
}

Then ask Claude something like "Use the verified regex generator to build me a regex for a US phone number." — it will call the tool, which runs the full generate → test → verify loop server-side and returns a JSON report.

Why this is a good showcase

Most "AI writes code" demos stop at generation. This one treats the LLM's first answer as a hypothesis, not an answer — and only claims success once it's checked against ground truth (the real regex engine, on real strings, including adversarial near-misses the model itself proposes). That loop — generate → verify → revise — is the core pattern behind reliable agentic tools, and it's small enough to read end-to-end in regex_agent/core.py.