# comind-mcp

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)

[![comind-mcp MCP server](https://glama.ai/mcp/servers/comind-pro/comind-mcp/badges/card.svg)](https://glama.ai/mcp/servers/comind-pro/comind-mcp)

Repository: **https://github.com/comind-pro/comind-mcp**

**MCP gateway** — connects various MCP servers and REST APIs, lets you curate and combine tools, organize them into **groups** (each = a separate virtual MCP server with a single endpoint) and hand them out to agents. An agent sees only the narrow set of tools assigned to it and can **schedule its own crons** through MCP.

Self-hosted: a single Node service + Postgres. Multi-user with per-account isolation.

```
Source (mcp │ openapi │ http) ──import──▶ Tool (native │ composite, curated)
                                              │
Group = virtual MCP ◀──toolset[]──────────────┘   + built-in self-cron tools
   └─▶  /g/:groupId/mcp   (Streamable HTTP, single endpoint)
            └─▶ Agent (Bearer key) — only granted V-MCPs, schedules itself
Vault (${secret.X}) · Scheduler · CallLog / Metrics
```

---

## Quick start

Prerequisites: Node 20+, pnpm 9 (`corepack enable`), Docker (local Postgres).

```bash
make setup        # install deps, start Postgres, apply migrations
make dev          # Postgres + server :8787 + web :5173
```

- **Web UI** — http://localhost:5173 (register an account, then sign in)
- **Gateway + Control API** — http://localhost:8787 (`GET /healthz`)
- **Postgres** — runs in Docker (`docker compose`); repo `.env` maps host port `5434`

See `make help` for all targets. Underlying pnpm scripts (`pnpm dev`, `pnpm dev:server`, `pnpm dev:web`) still work but don't manage the Postgres container.

### Database modes

The store is selected by the `DATABASE_URL` scheme — same schema, same migrations:

| `DATABASE_URL` | Mode | Use for |
| --- | --- | --- |
| `postgres://…` | External **Postgres** | Production, multi-instance (horizontal scale). |
| `file:/data/comind` | Embedded **Postgres (PGlite)** | Zero-infra self-host, single container, demos, Glama. |
| `memory:` | Embedded, in-memory | Throwaway / CI smoke. |

PGlite *is* Postgres (WASM), so everything (jsonb, `percentile_cont`, migrations) runs unchanged — no external DB process. **Persistence:** the `file:` directory is a real Postgres data dir; mount it as a volume (e.g. `/data`) to keep data across releases. Migrations are additive and idempotent, so an upgrade never wipes existing data. Embedded mode is single-node (no multi-instance — one writer).

```bash
# zero-infra: no Docker/Postgres needed
DATABASE_URL=file:/data/comind SERVER_ENV=dev pnpm --filter comind-server start
```

---

## End-to-end scenario

1. **Sources** → add a source (MCP proxy, OpenAPI, or HTTP) → **Test** → **Import tools**.
2. **Tools** → rename / hide the unnecessary / assemble a **composite** (an intent tool from several calls).
3. **Groups** → create a group → mark the toolset (checkboxes) → (optional) add a schedule.
4. **Agents** → create an agent in the group → get an **API key (once)** + MCP **endpoint**.
5. Connect any MCP client to `http://localhost:8787/g/<groupId>/mcp` with `Authorization: Bearer <key>`. The client sees only the group's toolset (+ self-cron tools).
6. **Logs** → calls, metrics, errors.

---

## Concepts

| Term | What it is |
|---|---|
| **Source** | Upstream: another MCP server (proxy), a REST API (OpenAPI 3.x → tools), or an HTTP service with explicit endpoints |
| **Tool** | A single call. `native` (proxied from a source), `composite` (a saved multi-step intent), `virtual` (an HTTP request template) or `python` (a sandboxed script) |
| **Composite** | Deterministically runs several calls and assembles a single result (output template, `$.input.*`/`$.steps.ID.*`) |
| **Python tool** | A body of Python run in a WASM sandbox — no network, no filesystem. Reaches other tools via `await call(...)`. Off by default (see below) |
| **Group** | A virtual MCP server: a curated set of tools, exposed as a single endpoint `/g/:groupId/mcp` |
| **Agent** | A consumer bound to a group via an API key. Sees only the group's toolset |
| **Self-cron** | MCP tools `schedule_task` / `list_schedules` / `cancel_schedule` inside a group — the agent schedules itself. Switch it off per workspace (Workspaces → Schedules): the tools disappear from the agent's `tools/list`, calls are refused, and the crons it already created are **paused** until it is back on. Your own schedules in that workspace keep running |
| **Secret** | An encrypted credential (AES-256-GCM) or an env reference. Substituted at runtime via `${secret.NAME}`; the agent never sees it |
| **Project / Team** | Nodes of a per-account tree. Every resource lives in one node; a node sees its own subtree plus what was shared with it (`project_access`). Pass `?project=<id>` to any control-plane call to act as that node; omit it to see everything. Each account starts with a default team and a default project. |
| **COO / Lead agents** | An agent's `role`. A **Lead** (default) only sees its workspaces' tools. A **COO** additionally gets the `project_*` admin tools on its MCP endpoint and administers its project's subtree: connections, secrets (names only — you fill the values), staged tools, workspaces, child projects, grants and new agents. Everything a COO does is audited in Activity with its arguments. |

---

## API (Control Plane, REST on :8787)

```
GET  /healthz
# sources
POST/GET /sources          GET/PATCH/DELETE /sources/:id
POST /sources/:id/test     POST /sources/:id/import
# tools
GET /tools  (?sourceId&kind&visible)   GET/PATCH/DELETE /tools/:id
PATCH /tools/:id {outputFields:string[]|null, outputLimit:int>0|null}   (owner projection defaults)
# composites
POST/GET /composite-tools  GET/DELETE /composite-tools/:id   POST /composite-tools/:id/run
# python tools (gated — see "Python tools")
POST /python-tools         GET/PATCH/DELETE /python-tools/:id
POST /python-tools/test    POST /python-tools/:id/run
GET  /features
# groups
POST/GET /groups           GET/PATCH/DELETE /groups/:id   (body: ..., compactResults?: bool, default true)
GET/PUT /groups/:id/tools
# projects (tree + sharing)  — every route above accepts ?project=<nodeId>
GET /projects              POST /projects {kind:'team'|'project', name, parentId}
GET /projects/:id/summary
DELETE /projects/:id       (409 default_node for the default team/project)
GET/POST /projects/:id/access {resourceType:'source'|'tool'|'secret', resourceId}
DELETE /projects/:id/access/:type/:resourceId
POST /projects/:id/move {resourceType:'source'|'tool'|'secret'|'group'|'agent', resourceId}
                           (a source carries its tools and secrets; 409 native_tool — move the source;
                            a moved workspace drops tools it can no longer use → {unlinked})
# agents
POST/GET /agents           GET/DELETE /agents/:id            POST /agents/:id/rotate-key
PATCH /agents/:id {role:'lead'|'coo'}
# schedules
POST/GET /groups/:id/schedules    DELETE /schedules/:id
POST /schedules/:id/run           GET /schedules/:id/runs
# secrets (metadata only; value/ciphertext is NEVER returned)
POST/GET /secrets          DELETE /secrets/:id
# observability
GET /logs (?groupId&agentId&toolName&status&limit)   GET /metrics (totals/byTool carry saved_bytes)
GET /agents/:id/inspect    POST /agents/:id/invoke
```

## Gateway (for agents, MCP)

```
POST /a/mcp            — agent-wide endpoint: union of tools across the agent's groups
POST /g/:groupId/mcp   — Streamable HTTP endpoint (Authorization: Bearer <agent-key>)
```
SSE transport — planned.

**Connect from Claude / ChatGPT (web):** step-by-step guide with screenshots — [docs/connect.md](docs/connect.md).

---

## COO agents

To make a COO agent: create an agent in a project (`POST /agents?project=<id>`), then set its role to `coo` (via the Agents tab in the web UI, or `PATCH /agents/:id {role:'coo'}`), and connect it to `/a/mcp` with its API key. A COO sees the `project_*` admin tools on the MCP endpoint for its project's subtree and can manage all resources within it.

| tool | what it does |
|---|---|
| `project_list` | nodes in the COO's subtree |
| `project_create_child` | new team/project under a node of the subtree |
| `project_delete` | delete a descendant node; without `confirm:true` returns a preview of what is removed |
| `project_access_create` | share a source/tool/secret the COO can see with a descendant project |
| `project_access_revoke` | revoke a grant; without `confirm` lists workspaces that would lose tools; with it also unlinks them |
| `project_connections_list` | visible connections (never the config) |
| `project_connection_describe` | cached objects (GA properties, tables, mailboxes) + tools with schemas |
| `project_connection_create` | new connection; config may reference `${secret.NAME}` only |
| `project_connection_test` | health check |
| `project_connection_import` | import/refresh tools; `force` without `confirm` previews removals |
| `project_secret_declare` | declare a secret by name → shows as *pending* in Secrets until you set its value |
| `project_tool_candidate_create` | virtual / composite / python tool in staging (in no workspace yet) |
| `project_tool_candidate_test` | run a tool with arguments |
| `project_tool_candidate_promote` | add a tool to a workspace |
| `project_workspace_create` | new workspace (virtual MCP server) with an optional tool set |
| `project_workspace_set_tools` | replace a workspace's tool set |
| `project_agent_mcp_provision` | new Lead (or COO) agent + key; the key is returned once |
| `project_agent_revoke` | archive an agent's keys; preview without `confirm` |

Isolation: a COO acts as its own project node — every id it passes is resolved inside its subtree, invisible ids read as `not_found`. It cannot change its own role, delete its own node or an ancestor, or revoke itself. Secret values never pass through a COO: it declares names, you fill them in Secrets.

Agents: a COO provisions COOs only into descendant nodes (never a peer at its own node), and revokes agents in descendant nodes or ones it provisioned itself. Revoking an agent archives its keys and ends its OAuth sessions. Custom tool names starting with `project.` are reserved.

Trust boundary: A COO never receives secret values through the API or audit, but any tool it creates (an HTTP connection or a virtual tool) can reference `${secret.NAME}` for every secret its node may use and send the request to any host. Treat a COO as trusted with the secrets you share with its project; share only what it needs. Per-secret allowed hosts are a planned follow-up.

---

## Projects tab

A tree of teams and projects for the account, with per-node counts of connections, tools, secrets, and workspaces. Each node's **Allowlist** lists what has been shared *into* it; "Share a resource" offers any connection, tool or secret from outside the node's own subtree. The node card lists the COO agents that administer it; the role itself is set on the Agents tab. Revoking a grant warns which workspaces would lose tools before unlinking them. "Move here" re-homes a connection, tool, secret, workspace or agent into the node (native tools move with their connection). The tree card creates a team or project under any parent; each node offers "Delete node" (hidden for the default team and project). A project selector in the header filters all tabs and resource rows; each row shows the node it belongs to, whether it was shared from elsewhere, and which agent created it (resources only).

Pending secrets declared by a COO agent appear in the Secrets tab with a `pending` badge and a password field to set the value — once set, the secret is encrypted at rest and the `pending` badge disappears.

---

## Python tools

A tool whose body is Python. Useful where the composite engine runs out of road:
loops, arithmetic, parsing, folding many calls into one table.

```python
rows = []
for tok in args["tokens"]:
    book = await call("market.get_order_book", {"token_id": tok})   # any tool you own
    if book["is_error"]:
        continue
    rows.append(book["structured"])

output = {"count": len(rows), "rows": rows}
```

- In scope: `args` (the tool's input), `await call(name, args)` → `{"text", "structured", "is_error"}`,
  and `steps` when the code is a step inside a composite (`{"id": "x", "python": "..."}`).
- The result is whatever you assign to **`output`**. If the script defines `main`, `main(args)`
  is called instead (sync or async). Neither one → an explicit error, never a silent empty result.
- A top-level `return` is a Python `SyntaxError` and kills the whole script — assign to `output`,
  or wrap the logic in `def main(args)`.
- `print()` is captured and shown in the tool editor.

**Sandbox.** Pyodide (CPython → WASM) in a worker thread: no network, no filesystem, no `process`.
Node's network modules are blocked in the worker before Pyodide loads, so Python sockets fail too —
the only way out of a script is `call(...)`, which goes through the normal tool runtime (auth, SSRF
guard, call log). A runaway script is killed by terminating the worker.

**Cost.** One worker per nesting level, booted lazily and kept warm: first run after start ≈ 1s,
subsequent runs ≈ 10ms. Runs at the same level are serialised, so a long script delays other python
tools (native/virtual tools are unaffected). A python tool calling a python tool calling a python
tool is the limit — deeper nesting is refused.

**Off by default.** Either set `PYTHON_TOOLS=1` (opens the feature to every account on the
instance — local dev / single-user self-host), or grant it per user:

```sql
INSERT INTO user_features (id, user_id, feature, enabled)
VALUES (gen_random_uuid()::text, '<user-id>', 'python_tools', true);
```

Revoking the row stops existing tools too — the ACL is re-checked on every call, not just at
authoring time. Tuning: `PYTHON_TOOL_TIMEOUT_MS` (30000), `PYTHON_TOOL_MAX_CALLS` (100),
`PYTHON_TOOL_MAX_CODE_BYTES` (65536).

## Source check

Every `SOURCE_CHECK_MINUTES` (default 10, `0` = off) the server pings every connection and records what a re-import would change (`sources.drift`: `new` / `changed` / `gone` tool names, plus `driftCheckedAt`). The Connections page shows an "update available" badge on such rows; the row's "Import tools" adds the new ones, "Force re-import" also replaces changed tools and removes gone ones. An import records the drift it leaves behind, so the badge clears as soon as the source and the import agree. "Refresh all" (`POST /sources/check`) runs the pass now for the connections in scope.

## Compact results

Tool results cost the agent's context, so the gateway shrinks them before they leave. Level 1 applies
only to agent calls; owner projection applies to every top-level call. Composites and Python tools
calling other tools always get the raw result.
Errors are never touched.

**Level 1, auto-compaction** (results of 2 KiB or more, per-workspace toggle, default on). JSON becomes
flat `key: value` lines (nulls and empty strings dropped, numbers kept exact, base64 payloads replaced
by a placeholder), and arrays of 3+ similar objects become a `|` table. HTML becomes text (scripts, styles
and tags removed, links as `text (url)`). Logs collapse repeated lines to `line (×N)`. It is skipped
when the output would not be smaller. If the flat text is still 8 KiB or more, a hint line suggests
`_fields` / `_limit`.

**Level 2, projection** (always on). Every tool in `tools/list` accepts two extra optional arguments,
stripped before the upstream sees them: `_fields` (dot paths, arrays traversed implicitly) and `_limit`
(max items per array):

```json
{ "query": "open bugs", "_fields": ["items.title", "items.url"], "_limit": 5 }
```

If no path matches, the original result comes back with a note, so a typo never blanks it.

**Owner defaults.** Set `outputFields` / `outputLimit` on a tool (`PATCH /tools/:id`, or the Tools tab)
to project every call by default; the agent's `_fields` / `_limit` can only narrow that further.
The per-workspace switch is `compactResults` on `POST/PATCH /groups` (Workspaces tab). Projection stays on.

**Savings** show in Activity: a `saved` total, a `saved` column per tool, and in/out KB per log row
(`GET /metrics`: `saved_bytes`).

**Trade-off.** Level 1 returns flat text, not JSON, in the `text` block. A programmatic consumer that
parses that block should read `structuredContent` instead or turn the workspace toggle off. For tools
with an `outputSchema`, `structuredContent` is always the raw upstream value (MCP clients validate it);
for others it stays JSON, cleaned and projected, and is dropped when still large.

---

## Structure

| Path | Purpose |
|---|---|
| `server/` | Node service (Fastify + MCP SDK + Drizzle/Postgres) — control API + gateway |
| `server/src/connectors/` | MCP proxy · OpenAPI→tools · HTTP connectors |
| `server/src/composite/` | Composite engine (intent tools) |
| `server/src/runtime/` | `invokeTool` — shared runtime (gateway / composite / scheduler) + the Pyodide sandbox |
| `server/src/gateway/` | Group's virtual MCP server + agent auth |
| `server/src/scheduler/` | node-cron registry + JobRun + self-cron |
| `server/src/secrets/` | Vault (AES-256-GCM) + `${secret.X}` injection |
| `server/src/routes/` | REST endpoints |
| `server/src/db/` | Drizzle schema + pg client (Postgres) |
| `web/` | Web UI (Vite + React) — Sources / Tools / V-MCP / Agents / Secrets / Logs |

Development details — [DEVELOPMENT.md](DEVELOPMENT.md).

---

## Security

- Secrets are encrypted at-rest (AES-256-GCM); the agent/config see only the `${secret.NAME}` placeholder, the value is substituted at runtime.
- An agent gets only its group's toolset; calls are gated by toolset on every request.
- API keys are stored as an sha256 hash, the token is shown once.
- A failure of one upstream does not bring down the endpoint (fault isolation in the runtime).

## Modules & features

Built iteratively, module by module. Everything below is implemented and working.

### Core gateway
- ✅ **Connectors** — proxy an existing MCP server, import a REST API from OpenAPI 3.x (own parser → tools), or wire an HTTP service with explicit endpoints.
- ✅ **Tool registry & curation** — import tools, rename, edit descriptions, toggle visibility, per-owner unique names.
- ✅ **Composite engine** — intent tools that run several calls in sequence; conditional `when`; templating (`$.input.*`, `$.steps.ID.text`); output template; per-step **trace** for tuning.
- ✅ **Shared runtime** (`invokeTool`) — one dispatcher for gateway, composites and scheduler; native→connector, composite→recursion (depth-limited); fault isolation (a bad upstream never crashes the caller).
- ✅ **Groups = virtual MCP** — bundle curated tools into a single MCP endpoint `/g/:groupId/mcp` (Streamable HTTP).
- ✅ **Agents** — consumer identities with one API key (sha256-hashed, shown once) + key rotation.
- ✅ **Agent ↔ V-MCP grants (M2M)** — grant/revoke access per group; one agent can reach many group endpoints; the key only works for granted groups.

### Scheduling
- ✅ **Scheduler** — cron registry (node-cron), JobRun log, run-now, loaded on boot.
- ✅ **Self-cron over MCP** — built-in `schedule_task` / `list_schedules` / `cancel_schedule` tools inside a group; a connected agent schedules itself.

### Secrets & auth-to-upstreams
- ✅ **Vault** — credentials encrypted at rest (AES-256-GCM); injected at runtime via `${secret.NAME}`; agents/config never see the value.
- ✅ **Source-scoped secrets** — same name can exist per source; scoped overrides global.
- ✅ **Static auth** — bearer/api-key/custom headers, basic (username/password).
- ✅ **Dynamic token flows** — `oauth2_client_credentials`, `token_request` (login→JSON-path), `oauth2_refresh` (cached + auto-refresh).
- ✅ **User OAuth** — `oauth2_authorization_code` (Connect flow) and **MCP-native OAuth** (`mcp_oauth`: SDK discovery + DCR + PKCE + refresh, with optional pre-registered `clientId`).

### Accounts & isolation
- ✅ **Auth** — email/password (scrypt) + HS256 session JWTs; register / login / me.
- ✅ **Multi-user isolation** — every resource is owned by a user; all routes scoped by owner; tools resolve only within the owner's namespace. No cross-account access.

### Observability
- ✅ **Call logs** — who/which tool/status/duration/token estimate per invocation.
- ✅ **Metrics** — totals + by-tool + by-agent.
- ✅ **Inspector & test-invoke** — see what an agent sees per granted V-MCP; run any tool to view the raw response.

### Web UI (Vite + React)
- ✅ **Auth** — login / register, token gating, logout.
- ✅ **Form ⟷ JSON builders** for sources and composites (edit a form or the raw JSON, two-way).
- ✅ **Inline secrets** in the source wizard (scoped to the source).
- ✅ **Grouped, collapsible, searchable** tool picker & registry (scales to large imported APIs).
- ✅ **Connect snippets** per V-MCP (`claude mcp add …`, curl) with copy buttons.
- ✅ Tabs: Sources · Tools · V-MCP · Agents · Secrets · Logs.

### Infrastructure
- ✅ **Postgres** via Drizzle (migrations auto-applied on boot).
- ✅ **Docker Compose** for local Postgres + **Makefile** (`make setup` / `make dev` / `make db-*`).
- ✅ `.env` loading, generated dev secrets.

### Not yet (optional next)
- ⬜ Org / project layer (teams, sharing).
- ⬜ SSE transport on the gateway (Streamable HTTP only today).
- ⬜ Hot-reload `tools/changed` notifications.
- ⬜ OpenAPI endpoint for a toolset; traces.

---

## Roadmap

- [ ] Rate-limit `/auth` (password brute-force), the gateway, and per-agent quotas.
- [ ] Make the scheduler multi-replica safe (Postgres advisory lock or a dedicated worker) — today in-memory cron fires N times with N instances.
- [ ] Move migrations to a separate deploy step (they run on every instance boot → race with multiple replicas).
- [ ] JWT revocation — short-lived access + refresh tokens (a leaked 7-day token can't be invalidated; logout is local-only).
- [ ] Secret management — KMS + rotation for `VAULT_KEY` / `JWT_SECRET`; tighten CORS (defaults to `*`); document the TLS reverse proxy.
- [ ] Serve the web UI for production (build & serve `dist` behind a CDN/proxy; Vite dev only today).
- [ ] Pagination on list endpoints (tools, logs).
- [ ] Scheduler retry / backoff / alerting.
- [ ] OpenAPI parser — handle complex specs (`allOf`, deep `$ref`).
- [ ] Password reset / email verification; user audit log.

---

## Distribution

Packaged as an **OCI image** (`ghcr.io/comind-pro/comind-mcp`) and listed in the
**official MCP Registry** (`registry.modelcontextprotocol.io`) — the canonical
source that downstream catalogs (PulseMCP, Smithery, Docker Hub, …) consume. The
metadata lives in [`server.json`](./server.json) under the GitHub-verified
namespace `io.github.comind-pro/comind-mcp`.

Run the image (zero-infra, embedded Postgres):

```bash
docker run -p 8787:8787 -v comind-data:/data \
  -e SERVER_ENV=dev ghcr.io/comind-pro/comind-mcp:latest
# prod: drop SERVER_ENV=dev and set VAULT_KEY + JWT_SECRET
```

Releasing is automated — push a version tag and CI ([`release.yml`](./.github/workflows/release.yml))
builds & pushes the image to GHCR, then publishes `server.json` to the registry
via GitHub OIDC (no tokens):

```bash
git tag v0.2.0 && git push origin v0.2.0
```

> Note: ComindMCP is a multi-tenant **gateway** (HTTP MCP at `/g/:slug/mcp`, agent-key auth),
> not a single stdio server — registry clients self-deploy it and connect their own agents.

---

## Contributing

**comind-mcp is open source** (MIT) and contributions are welcome — bug reports, features, docs, tests.

1. Fork & branch from `main` (`feat/...`, `fix/...`).
2. Set up locally — see [DEVELOPMENT.md](./DEVELOPMENT.md). TL;DR: `corepack enable && pnpm install`, then `pnpm dev`.
3. Before opening a PR: `pnpm typecheck` and `pnpm -r test` must pass.
4. Use [Conventional Commits](https://www.conventionalcommits.org/) for messages (`feat:`, `fix:`, `docs:`, `chore:`).
5. Open a PR against `comind-pro/comind-mcp` with a clear description; link any related issue.

Questions or ideas? Open an [issue](https://github.com/comind-pro/comind-mcp/issues). See [CONTRIBUTING.md](./CONTRIBUTING.md) for details.

---

## License

[MIT](./LICENSE) © comind — open source, free to use, modify, and distribute anywhere, including commercially.

Repository: https://github.com/comind-pro/comind-mcp