eda-pilot-orfs

EDA Pilot AI is a closed-loop optimizer for semiconductor physical design. A LangGraph agent searches OpenROAD Flow Scripts (ORFS) settings for the smallest clock period that still signs off, while a guarded tool layer, published as an MCP server, stands between the agent and the flow and refuses anything that is not on an allowlist. Every change is validated in code before GNU Make starts. The flow then runs Yosys, OpenROAD and KLayout (through WSL on Windows), and the metrics come back from the flow's own metadata.json.

The project is complete. It was measured on three SkyWater SKY130 HD designs (gcd, aes, ibex) against default ORFS and seeded random search, with a 10-run budget per method.

Signoff in this project means the flow finished, DRC count = 0, setup WNS ≥ 0 and hold WNS ≥ 0. A faster clock that misses setup counts as a failed run, not a win.

✨ Results at a glance

• ✅ The agent signed off 3 of 3 designs. Default ORFS signed off 0 of 3.

• ⏱️ Tightest passing clock on every design, 14.5–30.3% below the best random-search result.

• ⚖️ The trade-off is stated up front. The tighter clocks cost 7–23% more area, 30–71% more power and 1.5–5.5× the runtime of random search's looser passing points. Pooled signoff rates are about equal (agent 7/19, random 11/30).

• 🧪 52 scored ORFS runs (~21 hours of flow wall-clock) and 48 unit tests that run without OpenROAD.

Related MCP server: Cybersec Toolkit

🧭 Why this exists

Closing timing in a physical-design flow is a search problem. Each attempt is a full place-and-route run that takes minutes to hours, and the knobs interact. Language-model agents are a natural fit for proposing the next experiment, but an agent that can type arbitrary make arguments, edit platform Tcl, or invent a metric is a liability.

EDA Pilot answers one question honestly: can a guarded agent loop find a smaller signoff clock than the default flow and random search, under the same run budget, while never letting the model touch the launch path?

🚀 Features

🏗️ Architecture

Sources: diagrams/architecture.svg · architecture.mmd · architecture.drawio · full write-up in ARCHITECTURE.md.

Workflow: one agent iteration

sequenceDiagram
    autonumber
    participant H as Benchmark harness
    participant A as LangGraph agent
    participant V as validate (code)
    participant M as MCP tools
    participant F as ORFS in WSL
    H->>A: design, platform, budget
    loop until budget spent or passing-clock gap < 0.05 ns
        A->>A: analyse: read last metrics and report tails
        A->>A: propose: up to 3 changes, each with a reason
        A->>V: proposal
        alt outside the allowlist
            V-->>A: rejected (no process started)
        else legal
            V->>M: run_flow(design, platform, params)
            M->>M: allowlist, argv (shell=False), budget lock
            M->>F: make finish, then make metadata-generate
            F-->>M: run_id, status, runtime
            A->>M: get_metrics(run_id), read_report(run_id, timing)
            M-->>A: WNS / TNS, power, area, DRC
            A->>A: check: signoff? best run? update clock bounds
        end
    end
    A-->>H: best passing run and decision log

📦 Installation

Requirements

• Python 3.11+

• An OpenROAD-flow-scripts checkout, built with OpenROAD and Yosys. This is only needed for real runs.

• GNU Make. On Windows, ORFS runs inside WSL Ubuntu, and the server reaches it through the \\wsl$\<distro>\… path.

git clone https://github.com/NileshModlier/EDA-Pilot-AI.git
cd EDA-Pilot-AI
python -m pip install -e ".[dev]"     # mcp>=2, pydantic>=2, langgraph>=1, pytest
python -m pytest                      # 48 tests, no OpenROAD needed

Point the server at the flow

export ORFS_ROOT=/path/to/OpenROAD-flow-scripts         # Windows: $env:ORFS_ROOT = "\\wsl$\Ubuntu\home\<user>\OpenROAD-flow-scripts"
export EDA_PILOT_TIMEOUT_SECONDS=7200                     # per finish run
export EDA_PILOT_MAX_RUNS=10                              # must be >= the agent budget

🛠️ Usage

1. Serve the tools over MCP (stdio)

python -m eda_pilot.server                 # or the console script: eda-pilot-orfs
mcp dev eda_pilot/server.py                # MCP Inspector (pip install "mcp[cli]")

2. Run the agent

python -m agent.agent --design gcd --platform sky130hd --budget 10

It writes one JSON line per iteration to runs/decision_log.jsonl. It stops when the budget is spent or when the passing clock has been narrowed to within 0.05 ns. With no EDA_PILOT_MODEL, analyse and propose use the deterministic offline clock search. The measured benchmark used that mode (see Benchmark methodology).

3. Run the benchmark

python -m benchmarks.benchmark --designs gcd aes ibex --platform sky130hd --budgets 10 --seed 1
python -m benchmarks.compare_results benchmarks/results/*.json      # re-rank saved results
python scripts/make_release_plots.py                               # re-draw plots/ from results/final_benchmark_results.json

📊 Benchmark results

sky130hd · gcd, aes, ibex · budget 10 per search method · seed 1 · ORFS through WSL Ubuntu. The full analysis is in BENCHMARK_RESULTS.md and reports/final_benchmark_report.md. Every scored trial is in results/final_benchmark_results.json.

Best signoff per method

OpenROAD AutoTuner was scheduled in the same harness and produced no scored trials. The executable was not installed for gcd and aes, and on ibex Ray could not import HyperOpt before the first sample. No AutoTuner numbers are reported or estimated.

Signoff rate

🔍 Key findings

1. Signoff vs default. Default ORFS finished every design with DRC = 0 and still failed setup, by −1.460 ns (gcd), −0.081 ns (aes) and −0.551 ns (ibex). The agent signed off all three. Its passing clocks are slower than those default targets, so the gain over default is a legal signoff, not a faster clock.

2. Clock vs random search. The agent's best passing clock is 30.3%, 23.0% and 14.5% tighter than random search's best.

3. The price of a tighter clock. At their best passing points random search uses less area (−7% to −19%), less power (−23% to −41%) and less runtime. A PPA-weighted score would rank the methods differently.

4. Signoff rate. The agent's rate is clearly higher on aes (3/10 vs 1/10). It is about equal pooled (36.8% vs 36.7%) and slightly lower on gcd. The ibex 1/1 is a single run, not a rate.

5. Failure modes differ. Random search failed on placement density (FLW-0024), detailed placement (DPL-0033) and aes routing congestion (GRT-0116). The agent never hit a flow error. It missed timing near the boundary, and two aes runs hit the flow timeout.

6. Search shape matters more than "intelligence". The measured agent used the deterministic offline advisor, a guarded bisection on _SDC_CLK_PERIOD. That is the result the evidence supports. An LLM advisor plugs into the same two nodes and has not been benchmarked yet.

🖼️ Screenshots

🗂️ Repository layout

EDA-Pilot-AI/
├── eda_pilot/            MCP server (eda-pilot-orfs) and settings
├── mcp_tools/            validation · run_flow (+ WSL bridge) · metrics · reports
├── agent/                LangGraph loop, state, Pydantic proposal models
├── benchmarks/           default · random · AutoTuner · agent harness + compare_results
├── configs/              allowed_parameters.json (sky130hd gcd / aes / ibex)
├── tests/                48 pytest tests with a make stand-in
├── scripts/              environment probes, ORFS discovery, campaign resume, release plots
├── tools/                Windows launcher for the AutoTuner installed in WSL
├── results/              final_benchmark_results.json, campaign checkpoints, decision logs
├── reports/              final benchmark report, summary, validation reports
├── plots/                release figures (regenerated from results JSON)
├── diagrams/             architecture (.svg / .mmd / .drawio), technology stack
├── docs/                 executive summary, stack, lessons, roadmap, interview guide
├── release/              v1.0 release notes, recruiter and hiring-manager summaries
├── assets/               logos, banner, screenshots
└── .github/              CI (pytest on Python 3.11–3.13), issue and PR templates

📚 Documentation

🛣️ Roadmap

• Now: repair the AutoTuner environment (Ray + HyperOpt) and run it at the same budget and seed. Record failures as failures.

• Next: repeat random search and the agent over more seeds, and publish the distribution of best passing clocks.

• Then: benchmark an LLM advisor in analyse / propose against the offline bisection on the same allowlist.

• Later: a PPA-ranked view of the same trials, wider clock floors for ibex, a second PDK (nangate45) and larger sky130hd designs.

Details: docs/roadmap.md.

📖 Citation

If this project is useful in your work, please cite it and the tools it builds on (citations.bib):

@software{raghavan_eda_pilot_ai_2026,
  author  = {Raghavan, Nilesh P},
  title   = {{EDA Pilot AI}: A Guarded LangGraph and MCP Agent for Timing Closure in OpenROAD},
  year    = {2026},
  version = {1.0.0},
  url     = {https://github.com/NileshModlier/EDA-Pilot-AI}
}

🤝 Contributing

Issues and pull requests are welcome. Please read CONTRIBUTING.md and the Code of Conduct first. The one rule that is never relaxed: new knobs enter through configs/allowed_parameters.json, never through a prompt.

📄 License

Released under the MIT License. OpenROAD, OpenROAD Flow Scripts, Yosys, KLayout and the SkyWater SKY130 PDK are separate projects under their own licenses.

Built on OpenROAD, OpenROAD Flow Scripts, Yosys, KLayout, the SkyWater SKY130 PDK, LangGraph and the Model Context Protocol.