<p align="center">
  <img src="https://raw.githubusercontent.com/jayaprakash2207/Portage-_MCP/main/assets/portage-banner.svg" alt="Portage — every CLI, one command from your AI assistant" width="100%">
</p>

<p align="center">
  <b>Turn any command-line tool into an MCP server — by reading its own <code>--help</code>.</b><br>
  <sub>One generic pipeline for every CLI. No hand-written wrappers. Default-deny execution.</sub>
</p>

<p align="center">
  <a href="https://github.com/jayaprakash2207/Portage-_MCP/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/jayaprakash2207/Portage-_MCP/actions/workflows/ci.yml/badge.svg"></a>
  <a href="https://pypi.org/project/portage-mcp/"><img alt="PyPI" src="https://img.shields.io/pypi/v/portage-mcp?label=pypi&color=2ea44f"></a>
  <img alt="tests" src="https://img.shields.io/badge/tests-368%20unit%20%2B%2016%20integration-2ea44f">
  <img alt="coverage" src="https://img.shields.io/badge/coverage-~91%25-2ea44f">
  <img alt="typing" src="https://img.shields.io/badge/mypy-strict-1f6feb">
  <img alt="lint" src="https://img.shields.io/badge/lint-ruff-d7ff64">
  <img alt="python" src="https://img.shields.io/badge/python-3.10%20%E2%80%93%203.12-1f6feb">
  <img alt="mcp" src="https://img.shields.io/badge/MCP-SDK%202.x-7C3AED">
  <img alt="license" src="https://img.shields.io/badge/license-MIT-2ea44f">
</p>

<p align="center">
  <a href="#-quick-start">Quick start</a> ·
  <a href="#-see-it-in-action">In action</a> ·
  <a href="#-how-it-works">How it works</a> ·
  <a href="#-security-model">Security</a> ·
  <a href="#-cli">CLI</a> ·
  <a href="#-configuration">Config</a> ·
  <a href="#-remote-deployment">Remote</a> ·
  <a href="#-roadmap">Roadmap</a>
</p>

---

## What is this?

Thousands of command-line tools — `ffmpeg`, `jq`, `ripgrep`, `git`, `curl`, your
own scripts — and almost none have a
[Model Context Protocol](https://modelcontextprotocol.io) server, so an AI
assistant can't drive them. Hand-writing one per CLI is repetitive busywork.

**Portage generates it from the CLI's own documentation:**

```
 ┌──────────┐   --help / -h   ┌────────┐   ┌──────────┐   ┌──────────────┐   ┌─────────────┐
 │  <tool>  │ ───(+ man)────▶ │ parse  │─▶ │  CLI IR  │─▶ │ JSON Schema  │─▶ │  MCP server │─▶ AI
 └──────────┘                 └────────┘   └──────────┘   │ + arg specs  │   │ stdio / HTTP│
                                                          └──────────────┘   └──────┬──────┘
                                                                                    ▼
                                              validate → value policy → allow-list → sandbox → run
```

One code path handles every CLI — there is **no** `if tool == "git"` anywhere, and
a test enforces it.

> **`portage-mcp` 0.1.0 is on PyPI.** Discovery, `--help` **and** man-page
> parsing, schema generation, the stdio/HTTP MCP server, the execution engine and
> the full safety layer are done and CI-verified — 368 unit tests + 16
> integration against real `jq` / `ripgrep` / `curl` / `git` / `ffmpeg`, ~91 %
> coverage, `ruff` + `mypy --strict` clean. Docker + Fly.io deploy recipes
> included; `fly deploy` / `wrangler deploy` are the only manual steps left.

---

## ⚡ Quick start

```bash
pip install portage-mcp
```

```bash
# Claude Code
claude mcp add portage -- portage serve jq ripgrep curl git

# Claude Desktop — claude_desktop_config.json
{ "mcpServers": { "portage": { "command": "portage",
                               "args": ["serve", "jq", "ripgrep", "curl", "git"] } } }
```

Restart the client — the generated tools show up. Out of the box a tool call
returns a **structured preview** (validation + authorization + the exact `argv`)
and runs nothing; real execution is a per-CLI opt-in
([Configuration](#-configuration)).

<sub>From source: <code>git clone https://github.com/jayaprakash2207/Portage-_MCP && cd Portage-_MCP && pip install -e ".[dev]"</code></sub>

---

## 🎬 See it in action

**`ripgrep` describes itself:**

```text
$ rg --help
    -A, --after-context <NUM>   Show NUM lines after each match.
    -e, --regexp <PATTERN>      A pattern to search for. This option can be provided multiple times...
    -i, --ignore-case          When this flag is provided, all patterns will be searched case insensitively.
    ...
```

**Portage turns that into an MCP tool — no config, no code:**

```jsonc
$ portage generate rg
{
  "name": "rg",
  "description": "ripgrep 15.x  recursively searches for lines matching a regex",
  "input_schema": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "after_context": { "type": "integer", "description": "Show NUM lines after each match." },
      "regexp":        { "type": "array", "items": { "type": "string" },
                         "description": "A pattern to search for. ...provided multiple times..." },
      "ignore_case":   { "type": "boolean", "description": "...searched case insensitively..." },
      "pattern":       { "type": "string" }
    },
    "required": ["pattern"]
  }
}
```

Integers stay integers, `--regexp` (documented as repeatable) becomes an array,
enums keep their choices, and every field carries the CLI's own prose so the
model uses it correctly. `portage inspect rg` shows the whole chain — discovered
help, parsed IR, generated tools, and a dry-run — without executing anything.

---

## 🤔 Why Portage?

Auto-generating MCP servers for existing apps splits three ways:

| Approach | Ecosystem (2026) | |
|---|---|---|
| **OpenAPI → MCP** | mature, crowded — FastMCP, AWS Labs, many others | not this |
| **Browser automation → MCP** | mature, crowded — Playwright MCP, Skyvern, Stagehand | not this |
| **CLI (`--help` / man) → MCP** | wide open — prior attempts are tiny, one-language, or need hand-written YAML per tool; none parse man pages; no reference implementation | **this** |

Portage fills that gap with a safety model built in from the start, not bolted on.

---

## 🧭 How it works

```mermaid
flowchart TD
    A["CLI on your machine"] -->|"--help / -h"| B["discovery"]
    A -.->|"man tool"| B
    B --> C["parser<br/>sections · usage · options<br/>commands · type inference"]
    C --> D["normalized CLI IR (CliProgram)"]
    D --> E["schema generator"]
    E --> F["MCP tool defs<br/>JSON Schema + arg_specs"]
    F --> G["stdio / HTTP MCP server"]
    G <--> H["AI assistant"]
    H -->|"tools/call"| I["safety pipeline"]
    I --> J["schema validation"]
    J --> K["per-value policy"]
    K --> L["allow-list — default deny"]
    L --> M["safe argv build — no shell"]
    M --> N["sandbox + rlimits + timeout"]
    N --> O["subprocess"]
    O --> P["structured result + audit event"]
```

Each stage is its own module and depends only on the shared data model — parsing,
schema generation, protocol handling and execution never import one another.

| Layer | Module | What it does |
|---|---|---|
| **Discovery** | [`discovery.py`](src/portage/discovery.py) | Resolve a CLI on `PATH`, capture `--help` → `-h` safely (timeouts, help-on-stderr, non-zero exit, truncation). |
| **Parse** | [`parser/`](src/portage/parser/) | Layered `--help` **+ man-page** parser → `CliProgram` IR. Reports `ParseConfidence`; keeps anything it can't classify as an `UnknownConstruct` instead of guessing. |
| **Merge** | [`merge.py`](src/portage/merge.py) | Deterministically fold the man-page IR into the `--help` IR — richer descriptions win, `--help` stays authoritative for structure. |
| **Schema** | [`schema.py`](src/portage/schema.py) | IR → draft-2020-12 JSON Schema (`additionalProperties: false`), deterministic names (`git remote add` → `git_remote_add`), collision-safe, plus `arg_specs` reconstruction metadata. |
| **Serve** | [`service.py`](src/portage/service.py) · [`server.py`](src/portage/server.py) | Transport-agnostic registry + an `mcp` 2.x adapter. stdio and streamable HTTP with optional bearer auth. |
| **Execute** | [`executor.py`](src/portage/executor.py) | Structured `argv` builder + `shell=False`, `stdin`-closed, timeout-bounded runner. POSIX `setrlimit`. |
| **Safety** | [`validation`](src/portage/validation.py) · [`value_policy`](src/portage/value_policy.py) · [`authorization`](src/portage/authorization.py) · [`sandbox`](src/portage/sandbox.py) · [`audit`](src/portage/audit.py) · [`pipeline`](src/portage/pipeline.py) | The one path from a tool call to a process. |

---

## ✨ Features

<table>
<tr>
<td width="50%" valign="top">

**Parsing that doesn't lie**
- GNU / POSIX / BSD option styles, `--opt=VAL`, `--opt[=WHEN]`, `--[no-]flag`
- usage-line alternations (`[-p | --paginate | -P]`) → individual options
- enums from `{a,b,c}` / `<a|b|c>` / quoted "one of" lists
- repeatable options → arrays, `--arg NAME VALUE` → arity-2
- documented defaults, positionals, variadics, nested subcommands
- `ParseConfidence` per program / command / option
- unclassifiable fragments preserved, never invented

</td>
<td width="50%" valign="top">

**Execution you can trust**
- no code path builds a command string — ever
- every value → distinct `argv` elements; flag & value never joined
- **default deny**: nothing runs without an explicit allow-list
- schema validation → per-value policy → allow-list → sandbox
- `bubblewrap` / `firejail` / `docker` sandbox (fail-closed)
- POSIX rlimits + wall-clock timeout + output cap
- audit events record argument *names*, never values
- dry-run preview of the exact `argv`

</td>
</tr>
<tr>
<td width="50%" valign="top">

**Two documentation sources**
- `--help` first, `man <tool>` where available
- overstrike / ANSI cleanup, boilerplate-tail trimming
- deterministic merge; conflicts surfaced, not dropped
- recursive subcommand discovery (`git remote add` → `git_remote_add`)

</td>
<td width="50%" valign="top">

**Local first, remote ready**
- stdio for Claude Desktop / Code
- `portage serve --http` → Starlette/uvicorn, bearer-auth on `/mcp`, open `/healthz`
- Cloudflare Worker front door + Docker/Fly.io Engine recipes ([`deploy/`](deploy/))

</td>
</tr>
</table>

---

## 🔒 Security model

Portage lets an AI assistant run real commands, so the execution path *is* the
product. Every `tools/call` passes through, in order:

| # | Gate | Guarantee |
|---|---|---|
| 1 | **Schema validation** | Arguments checked against the generated draft-2020-12 schema. Unknown fields, wrong types, bad enums, missing required → structured rejection. |
| 2 | **Per-value policy** | Optional `value_rules`: max length, required / forbidden regex, "path must resolve under". Broken patterns fail closed. |
| 3 | **Allow-list — default deny** | Nothing runs unless a `CliConfig` sets `execution_enabled: true` **and** an `allowed_commands` prefix matches **and** every emitted flag is in `allowed_options`. An empty rule never matches. |
| 4 | **Safe `argv` construction** | Values become individual `argv` elements from `arg_specs`. No shell. No `--flag=value` joining. No string interpolation. Verified inert against `;` `\|` `&&` `$()` backticks newlines quotes redirection path-traversal. |
| 5 | **Sandbox** *(opt-in)* | `bubblewrap` / `firejail` / `docker`: read-only root, private `/tmp`, no network by default. `mode: require` refuses to run if no launcher is present. |
| 6 | **Resource limits + timeout** | POSIX `setrlimit` (CPU / memory / file size / nproc); every run bounded and killed on overrun; output capped. |
| 7 | **Audit** | A structured `AuditEvent` per call — timestamp, tool, executable, command path, validation & authorization results, mode, exit code, duration. Argument **names only**. |

The executable is chosen by Portage from the tool definition and passed to the
engine as an absolute path — an MCP caller cannot select or redirect it.

<details>
<summary><b>What is <i>not</i> yet covered</b></summary>

- Container / restricted-user isolation on the deployed engine (sandbox wrappers
  exist; a permitted command still runs as the engine's user).
- Cross-argument policy (`--output` and the positional must share a dir).
- rlimits are POSIX-only; on Windows only the timeout + output cap apply.
- `--version`-style flags that bypass a required positional can't be modelled in
  JSON Schema, so such a call is rejected as missing-required.

Full list in [`TEST_REPORT.md`](TEST_REPORT.md).
</details>

---

## 🖥 CLI

| Command | Does |
|---|---|
| `portage doctor [tool]` | Environment check — interpreter, `mcp` SDK, `man`, optional CLI lookup. |
| `portage discover <tool>` | Capture the CLI's help text; print the structured `DiscoveryResult`. |
| `portage parse <tool>` | Discover + parse → the normalized `CliProgram` IR as JSON. |
| `portage generate <tool>` | Discover + parse → the generated MCP tool definitions + schemas. |
| `portage inspect <tool>` | **One-shot debug view:** discovery + man status + IR + tools + an optional dry-run. Nothing executes. |
| `portage call <tool> <cli> --json '{…}'` | Run one tool through the safety pipeline (dry-run unless `--execute` **and** policy permits). |
| `portage serve <cli>… [--config F] [--http] [--http-token …]` | Run the MCP server (stdio, or streamable HTTP at `/mcp`). |

---

## ⚙ Configuration

`portage serve --config portage.json`. Everything not listed stays denied.

```jsonc
{
  "server_name": "portage",
  "clis": [
    {
      "name": "git",
      "use_man_page": true,
      "discover_subcommands": true,
      "subcommand_depth": 2,
      "policy": {
        "execution_enabled": true,
        "allowed_commands": [["git", "status"], ["git", "log"], ["git", "show"]],
        "allowed_options": ["--oneline", "--stat", "--short", "-n"],
        "timeout_seconds": 20,
        "value_rules": [{ "json_name": "max_count", "pattern": "\\d{1,4}" }],
        "resource_limits": { "cpu_seconds": 15, "memory_mb": 512, "max_processes": 64 },
        "sandbox": { "mode": "auto", "backend": "bubblewrap", "allow_network": false }
      }
    },
    { "name": "jq", "policy": { "execution_enabled": false } }   // discovery-only
  ]
}
```

Ready-to-adapt: [`deploy/portage.example.json`](deploy/portage.example.json) ·
[`deploy/engine/portage.json`](deploy/engine/portage.json).

---

## 🌐 Remote deployment

Cloudflare Workers can't run native binaries, so the design is two parts:

```
MCP client ──HTTP──▶ Portage-Protocol (Cloudflare Worker: bearer auth + reverse proxy)
                          │  HTTPS
                          ▼
                     Portage-Engine (portage serve --http)  ──▶  the real CLI
```

- **Engine** — `pip install portage-mcp`, then `portage serve --config … --http`.
  `PORTAGE_HTTP_TOKEN` turns on bearer auth on `/mcp`; `/healthz` stays open.
  Ready-made [`Dockerfile` + `fly.toml`](deploy/engine/) — `fly deploy` and you're up.
- **Worker** — [`deploy/cloudflare-worker/`](deploy/cloudflare-worker/), type-checked
  with `tsc` and verified end-to-end via `wrangler dev`
  (`test-local.ps1` / `test-local.sh`): health route, `401` without the token,
  a real MCP `initialize` proxied through.

Full recipes in [`deploy/README.md`](deploy/README.md).

---

## 🧪 Development

```bash
ruff check .                                   # lint
mypy                                           # type-check (strict)
pytest -q                                      # 368 unit tests — no network, no CLIs
pytest -q --run-integration -m integration     # + real jq/rg/curl/git/ffmpeg (+ bubblewrap)
pytest -q --cov=portage                        # coverage
```

CI runs all of the above on Python 3.10 / 3.11 / 3.12. See
[`CONTRIBUTING.md`](CONTRIBUTING.md) and [`SECURITY.md`](SECURITY.md).

---

## 🗺 Roadmap

- [x] Discovery, layered `--help` **+ man-page** parser → normalized IR
- [x] JSON Schema / MCP tool generation — deterministic, collision-safe, `arg_specs`
- [x] stdio **and** streamable-HTTP MCP server
- [x] Execution engine — structured `argv`, no shell
- [x] Safety layer — validation · value policy · **default-deny** allow-list · sandbox · audit · dry-run
- [x] End-to-end against 5 real CLIs, no per-tool code
- [x] POSIX rlimits + bubblewrap/firejail/docker sandbox (fail-closed)
- [x] Recursive subcommand discovery · usage-line alternation decomposition
- [x] Cloudflare Worker front door + Docker/Fly.io Engine recipes (verified locally)
- [x] **Published to PyPI** — `pip install portage-mcp`
- [ ] `fly deploy` the Engine + `wrangler deploy` the Worker
- [ ] Container / restricted-user isolation exercised on a Linux host
- [ ] `tbl`-formatted man tables · submit to the MCP server registries

---

## Why "Portage"?

A *portage* is carrying a boat overland between two waterways — bridging things
that don't otherwise connect. Portage carries CLI functionality across into the
MCP waterway so AI assistants can use it.

## License

MIT — see [LICENSE](LICENSE). Contributions welcome.