# MCP Tool Manager

> **A production-hardened, AI-native tool registry and agent management system built on the Model Context Protocol (MCP).**

[![Node.js](https://img.shields.io/badge/Node.js-%3E%3D16.0-brightgreen)](https://nodejs.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![MCP](https://img.shields.io/badge/Protocol-MCP%20v1.0-purple)](https://modelcontextprotocol.io)
[![Security](https://img.shields.io/badge/Security-Hardened-red)](mcp-server-project/security/)

---

## 📖 Table of Contents

1. [What Is This?](#what-is-this)
2. [Architecture Overview](#architecture-overview)
3. [Project Structure](#project-structure)
4. [Quick Start](#quick-start)
5. [Configuration Reference](#configuration-reference)
6. [API Reference](#api-reference)
7. [Implementation Plan](#implementation-plan)
8. [Security Model](#security-model)
9. [Monitoring & Observability](#monitoring--observability)
10. [Roadmap](#roadmap)
11. [Contributing](#contributing)

---

## What Is This?

MCP Tool Manager is a **dual-server platform** that solves the hardest operational problems in AI-integrated tool systems:

| Problem | Solution |
|---|---|
| LLMs burning context windows on huge API responses | Per-tool byte budget with graceful truncation signalling |
| Upstream API failures cascading to the LLM | Per-tool circuit breaker (CLOSED → OPEN → HALF-OPEN) |
| All data lost on server restart | Automatic periodic disk snapshots, restored on startup |
| Brute force / injection attacks on the API gateway | 10-family threat detector + tiered rate limiting + IP auto-block |
| No way to trace a request end-to-end | X-Trace-ID header propagated across all layers and to upstream APIs |
| Tools/agents registered in volatile memory only | File-persisted call log + agent JSON configs + state snapshots |

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────────────┐
│                        MCP Tool Manager Platform                        │
│                                                                         │
│  ┌──────────────────────┐        ┌────────────────────────────────────┐ │
│  │   Manager Server      │        │    Hardened MCP Server             │ │
│  │   src/server          │        │    mcp-server-project              │ │
│  │                       │        │                                    │ │
│  │  • REST API (CRUD)    │        │  • MCP Protocol endpoint           │ │
│  │  • JWT + API key auth │        │  • Agent API key auth + expiry     │ │
│  │  • Tool registry      │        │  • Circuit breaker per tool        │ │
│  │  • Agent management   │        │  • Retry + exponential backoff     │ │
│  │  • Credential vault   │        │  • Response cache (TTL per tool)   │ │
│  │  • Audit log          │        │  • Context window limiting         │ │
│  │  • State snapshots    │        │  • File-persisted call log         │ │
│  │  • WebSocket events   │        │  • 10-family threat detection      │ │
│  │  • Response cache     │        │  • Admin /metrics endpoint         │ │
│  └──────────┬───────────┘        └──────────────┬─────────────────────┘ │
│             │                                    │                       │
│  ┌──────────▼───────────┐        ┌──────────────▼─────────────────────┐ │
│  │   React Dashboard     │        │    Claude Desktop / LLM Agent      │ │
│  │   src/dashboard       │        │    (connects via MCP SDK)          │ │
│  └──────────────────────┘        └────────────────────────────────────┘ │
│                                                                         │
│  ┌──────────────────────────────────────────────────────────────────┐   │
│  │  Cross-cutting: X-Trace-ID · Rate Limiting · Helmet CSP ·        │   │
│  │  Structured Logging · Connection Limit · Compression             │   │
│  └──────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## Project Structure

```
mcp/
├── .env.example                    # Template — copy to .env and fill in values
├── .gitignore                      # Excludes .env, node_modules, logs, snapshots
├── package.json                    # Root scripts — start both servers, CLI, tests
├── README.md                       # This file
├── REPORT.md                       # Full technical capability report
├── CHANGELOG.md                    # Version history
│
├── src/
│   ├── server/                     # Manager Server (REST API)
│   │   ├── index.js                # Entry point — snapshot restore + server start
│   │   ├── app.js                  # Express app — all middleware wired
│   │   ├── routes/
│   │   │   ├── tools.js            # CRUD + test execution for tools
│   │   │   ├── agents.js           # Agent management + tool discovery
│   │   │   ├── auth.js             # Login, register, API key management
│   │   │   ├── credentials.js      # Encrypted credential vault
│   │   │   └── monitoring.js       # Stats, audit log, cache, snapshot status
│   │   ├── middleware/
│   │   │   ├── auth.js             # JWT + API key auth + RBAC
│   │   │   └── error-handler.js    # Typed errors + global handler
│   │   ├── storage/
│   │   │   ├── in-memory-store.js  # All in-memory Maps + operations
│   │   │   ├── seeder.js           # Initial data (skipped if snapshot exists)
│   │   │   └── state-snapshot.js   # Periodic disk snapshots (JSON files)
│   │   ├── utils/
│   │   │   ├── trace.js            # X-Trace-ID middleware
│   │   │   ├── context-limit.js    # Response byte budget + pagination guard
│   │   │   ├── response-cache.js   # node-cache wrapper + TTL presets
│   │   │   ├── encryption.js       # AES-256-CBC for credential vault
│   │   │   └── logger.js           # Levelled logger (error/warn/info/debug)
│   │   └── websocket.js            # Real-time events via WebSocket
│   │
│   ├── dashboard/                  # React + Vite management UI
│   │   ├── src/
│   │   │   ├── pages/              # Dashboard, Tools, Agents, Monitoring, Settings
│   │   │   ├── components/         # Sidebar, Topbar, ToastContainer
│   │   │   ├── services/api.js     # Axios client for Manager Server
│   │   │   └── styles/             # global.css, sidebar.css
│   │   └── vite.config.js
│   │
│   ├── sdk/
│   │   └── index.js                # Developer SDK — npm-publishable client
│   │
│   └── cli/
│       └── index.js                # Admin CLI (17 commands)
│
├── mcp-server-project/             # Hardened MCP Server
│   ├── package.json
│   ├── src/
│   │   ├── server.js               # Boot sequence — all 7 security layers
│   │   ├── mcp-protocol.js         # MCP spec endpoint (/mcp/tools, /mcp/invoke)
│   │   ├── routes/
│   │   │   ├── invoke.js           # Tool invocation (retry + CB + cache + limit)
│   │   │   ├── info.js             # Tool discovery per agent
│   │   │   └── metrics.js          # Admin monitoring endpoint
│   │   ├── middleware/
│   │   │   ├── auth.js             # Agent auth + expiry + scope + disabled check
│   │   │   ├── trace.js            # X-Trace-ID attachment
│   │   │   └── context-limit.js    # Response byte budget
│   │   ├── state/
│   │   │   ├── call-log.js         # Disk-persisted call log (NDJSON)
│   │   │   ├── circuit-breaker.js  # Per-tool CLOSED/OPEN/HALF state machine
│   │   │   └── response-cache.js   # TTL cache with auto-eviction
│   │   ├── loaders/
│   │   │   ├── registry.js         # Central tool+agent in-memory registry
│   │   │   ├── tool-loader.js      # Loads *.json from /tools/
│   │   │   ├── agent-loader.js     # Loads *.json from /agents/
│   │   │   └── credential-loader.js # Merges .env + JSON credentials
│   │   └── watcher.js              # chokidar hot-reload on /tools/ and /agents/
│   ├── security/
│   │   ├── middleware/
│   │   │   ├── security-headers.js # Strict Helmet CSP + CORS
│   │   │   ├── rate-limiter.js     # 3-tier rate limiting + IP auto-block
│   │   │   └── threat-detector.js  # 10-family injection/attack detector
│   │   └── logger/
│   │       └── security-log.js     # Structured security event log (5 levels)
│   ├── tools/                      # Tool definition JSON files
│   ├── agents/                     # Agent definition JSON files
│   ├── credentials/                # .env and JSON secrets (gitignored)
│   ├── logs/                       # Security log + call log (gitignored)
│   └── security-tests/             # Attack simulation suite + benchmark
│
├── snapshots/                      # Manager server state snapshots (gitignored)
└── examples/                       # Example tool/agent JSON files
```

---

## Quick Start

### Prerequisites

| Requirement | Version |
|---|---|
| Node.js | ≥ 16.0.0 |
| npm | ≥ 7.0.0 |
| Git | any |

### 1. Clone

```bash
git clone https://github.com/YOUR_USERNAME/mcp-tool-manager.git
cd mcp-tool-manager
```

### 2. Install dependencies

```bash
# Root (Manager Server + CLI + SDK)
npm install

# Dashboard
cd src/dashboard && npm install && cd ../..

# MCP Server
cd mcp-server-project && npm install && cd ..
```

### 3. Configure

```bash
# Manager Server
cp .env.example .env
# Edit .env with your JWT_SECRET, ENCRYPTION_KEY, etc.

# MCP Server
cp mcp-server-project/credentials/.env.example mcp-server-project/credentials/.env
# Edit credentials/.env with your agent keys and tool API keys
```

### 4. Run

```bash
# Terminal 1 — Manager Server (port 5000)
npm run dev:server

# Terminal 2 — React Dashboard (port 3000)
npm run dev:dashboard

# Terminal 3 — MCP Server (port 5001 by default)
cd mcp-server-project && npm start
```

### 5. Access

| Interface | URL |
|---|---|
| Dashboard | http://localhost:3000 |
| Manager API | http://localhost:5000 |
| Manager Health | http://localhost:5000/api/monitoring/health |
| MCP Server | http://localhost:5001 |
| MCP Health | http://localhost:5001/health |
| MCP Metrics | http://localhost:5001/metrics/health |

### Default Login (Manager)

```
Email:    admin@mcp-tool-manager.dev
Password: admin123
```

> ⚠️ Change this immediately in production via `ADMIN_USERNAME` / `ADMIN_PASSWORD` env vars.

---

## Configuration Reference

### Manager Server (`.env`)

```dotenv
# Core
NODE_ENV=development
MCP_SERVER_PORT=5000
MCP_SERVER_HOST=localhost
LOG_LEVEL=info

# Auth
JWT_SECRET=your-super-secret-key-min-32-chars
JWT_EXPIRY=24h
ENCRYPTION_KEY=your-encryption-key-exactly-32-ch

# Context Window
MCP_MAX_RESPONSE_BYTES=65536        # 64 KB default response budget
MCP_MAX_PAGE_SIZE=100               # Max items per paginated endpoint

# Scalability
MCP_MAX_CONNECTIONS=500             # TCP connection limit
SNAPSHOT_DIR=./snapshots            # State persistence directory
SNAPSHOT_INTERVAL_SECS=60           # Save state every 60 seconds
SNAPSHOT_RESTORE=true               # Restore state on startup

# Cache TTLs (seconds)
CACHE_TTL_TOOL_LIST=30
CACHE_TTL_TOOL_ITEM=60
CACHE_TTL_AGENT_LIST=30
CACHE_TTL_STATS=10
CACHE_TTL_ACTIVITY=300

# Future (not yet wired — provide connection string to enable)
DATABASE_URL=postgresql://user:password@localhost:5432/mcp_tools
REDIS_URL=redis://localhost:6379
```

### MCP Server (`mcp-server-project/credentials/.env`)

```dotenv
# Agent API Keys (convention: AGENT_<AGENTID_UPPERCASE>_KEY)
AGENT_MY_AGENT_KEY=your-agent-secret-key

# Tool credentials (referenced by credential_ref in tool JSON)
OPENAI_API_KEY=sk-...
WEATHER_API_KEY=...
SLACK_BOT_TOKEN=xoxb-...

# Admin
ADMIN_KEY=your-admin-key-for-metrics-endpoint

# Server
MCP_PORT=5001
MCP_MAX_CONNECTIONS=200
MCP_MAX_RESPONSE_BYTES=32768        # 32 KB default per tool response
```

### Tool JSON Fields (MCP Server)

```json
{
  "name": "my_tool",
  "description": "Human-readable description for the LLM",
  "endpoint_url": "https://api.example.com/endpoint",
  "method": "POST",
  "credential_ref": "MY_API_KEY",
  "parameters": {
    "type": "object",
    "properties": {
      "query": { "type": "string", "description": "Search query" }
    },
    "required": ["query"]
  },
  "cache_ttl_seconds": 60,
  "max_response_bytes": 8192,
  "retry_max": 3,
  "timeout_ms": 10000,
  "circuit_failure_threshold": 5,
  "circuit_open_window_ms": 30000
}
```

### Agent JSON Fields (MCP Server)

```json
{
  "agent_id": "my-agent",
  "allowed_tools": ["weather_lookup", "send_email"],
  "expires_at": "2027-01-01T00:00:00Z",
  "disabled": false
}
```

---

## API Reference

### Manager Server (`http://localhost:5000`)

#### Auth
| Method | Path | Auth | Description |
|---|---|:---:|---|
| POST | `/api/auth/login` | — | Get JWT token |
| POST | `/api/auth/register` | — | Create account |
| GET | `/api/auth/me` | ✅ | Current user + API keys |
| POST | `/api/auth/api-keys` | ✅ | Generate new API key |
| DELETE | `/api/auth/api-keys/:key` | ✅ | Revoke API key |

#### Tools
| Method | Path | Auth | Description |
|---|---|:---:|---|
| GET | `/api/tools` | ✅ | List tools (paginated) |
| POST | `/api/tools` | ✅ | Register new tool |
| GET | `/api/tools/:id` | ✅ | Tool details |
| PUT | `/api/tools/:id` | ✅ | Update tool |
| DELETE | `/api/tools/:id` | ✅ | Remove tool |
| POST | `/api/tools/:id/test` | ✅ | Test tool invocation |

#### Agents
| Method | Path | Auth | Description |
|---|---|:---:|---|
| GET | `/api/agents` | ✅ | List agents |
| POST | `/api/agents` | ✅ | Register agent |
| GET | `/api/agents/:id` | ✅ | Agent details |
| PUT | `/api/agents/:id` | ✅ | Update agent |
| DELETE | `/api/agents/:id` | ✅ | Remove agent |
| POST | `/api/agents/:id/tools` | ✅ | Discover tools for agent |

#### Monitoring
| Method | Path | Auth | Description |
|---|---|:---:|---|
| GET | `/api/monitoring/health` | — | Liveness probe |
| GET | `/api/monitoring/stats` | ✅ | Full system stats + cache + snapshot |
| GET | `/api/monitoring/activity` | ✅ | Real hourly call timeline (24h) |
| GET | `/api/monitoring/top-tools` | ✅ | Top N tools by call count |
| GET | `/api/monitoring/audit-log` | ✅ | Audit entries |
| GET | `/api/monitoring/cache` | ✅ | Cache hit-rate + entries |
| GET | `/api/monitoring/snapshot` | ✅ | Last snapshot timestamp + counts |

### MCP Server (`http://localhost:5001`)

| Method | Path | Auth | Description |
|---|---|:---:|---|
| GET | `/health` | — | Liveness probe |
| GET | `/info` | AGENT_KEY | List tools for calling agent |
| GET | `/info/all` | ADMIN_KEY | All tools + all agents |
| GET | `/mcp/tools` | — | Claude Desktop compatible tool list |
| POST | `/mcp/invoke/:tool` | — | MCP protocol invocation |
| POST | `/invoke/:toolName` | AGENT_KEY | Direct tool invocation |
| GET | `/metrics` | ADMIN_KEY | Full monitoring dashboard |
| GET | `/metrics/health` | — | Lightweight liveness probe |
| GET | `/metrics/calls` | ADMIN_KEY | Recent call history |

---

## Implementation Plan

This section documents the complete roadmap — what is built, what is in progress, and what requires infrastructure decisions.

### Phase 1 — Foundation ✅ Complete

- [x] Manager Server REST API (tools, agents, auth, credentials, monitoring)
- [x] In-memory store with full CRUD operations
- [x] JWT + API key dual authentication with RBAC
- [x] AES-256-CBC credential vault
- [x] React dashboard (Tools, Agents, Monitoring, Settings pages)
- [x] WebSocket real-time event broadcasting
- [x] Developer SDK (`src/sdk/index.js`)
- [x] Admin CLI with 17 commands (`src/cli/index.js`)
- [x] MCP Protocol endpoint (Claude Desktop compatible)
- [x] File-based tool/agent registry with hot-reload (`chokidar`)
- [x] Audit log with rolling ring buffer

### Phase 2 — Security Hardening ✅ Complete

- [x] 10-family threat detector (SQL/NoSQL/XSS/SSRF/Shell/Template/Path/CMDi/Null/Header injection)
- [x] Scanner user-agent blocking (sqlmap, nikto, nmap, Burp Suite, 20+ scanners)
- [x] 3-tier rate limiting (global + strict + speed slow-down)
- [x] Auto IP block after brute force (20+ hits)
- [x] Structured security event log with 5 severity levels
- [x] Helmet strict CSP (`defaultSrc: 'none'`)
- [x] Agent key expiry + disabled flag
- [x] Scope enforcement (`requireScope` middleware)
- [x] Auth failure + scope violation security logging
- [x] Security test suite + benchmark (compare hardened vs. unprotected)

### Phase 3 — Operational Capabilities ✅ Complete (This Release)

- [x] **X-Trace-ID** — unique request correlator, propagated through all layers and upstream APIs
- [x] **Circuit Breaker** — CLOSED/OPEN/HALF-OPEN per tool (configurable thresholds)
- [x] **Retry with exponential backoff** — 200ms → 400ms → 800ms, skips 4xx errors
- [x] **Response cache** — TTL per tool/data-type, hit-rate tracking, prefix invalidation
- [x] **Context window limiting** — per-tool byte budget, graceful truncation with signalling
- [x] **Pagination guard** — global `?limit` clamp (default max 100 items)
- [x] **State snapshots** — atomic periodic writes, restored on startup (tools/agents/users survive restart)
- [x] **Connection limit guard** — drops TCP sockets over configurable max
- [x] **Rate limiting activated** (Manager) — 300 global + 15 auth per minute per IP
- [x] **Real monitoring** — activity timeline from actual call data (removed Math.random() mock)
- [x] **`/metrics` endpoint** (MCP) — full admin dashboard (calls, cache, circuit breakers, memory)
- [x] **node-cache activated** (Manager) — TTL presets per data type, hit-rate tracking
- [x] **`/api/monitoring/cache`** and **`/api/monitoring/snapshot`** new endpoints

### Phase 4 — Persistence & Distribution 🔲 Pending Your Input

> These require infrastructure. `pg` and `ioredis` are already installed — only connection strings needed.

- [ ] **PostgreSQL** — migrate `in-memory-store.js` to persistent database
  - `tools`, `agents`, `users`, `api_keys`, `credentials`, `audit_log` tables
  - Connection pool via `pg` (`DATABASE_URL` already in `.env.example`)
- [ ] **Redis** — shared rate limit + session + response cache store
  - Replace node-cache with ioredis for multi-instance safety
  - Shared IP block list across all server instances
  - (`REDIS_URL` already in `.env.example`)
- [ ] **Horizontal scaling** — once Redis + Postgres are wired, deploy N instances behind nginx

### Phase 5 — Developer Experience 🔲 Optional

- [ ] OpenAPI/Swagger spec auto-generation (`swagger-jsdoc`)
- [ ] `zod` env schema validation at startup (fail-fast on missing config)
- [ ] JWT refresh tokens + blacklist
- [ ] Prometheus metrics export (`/metrics/prometheus` endpoint)
- [ ] OpenTelemetry distributed tracing
- [ ] Tool compatibility matrix
- [ ] WebSocket dashboard for real-time circuit breaker state

---

## Security Model

### Manager Server

```
Request
  │
  ├── X-Trace-ID attachment (Layer 0)
  ├── Helmet strict CSP (Layer 1)
  ├── Global rate limit 300/min (Layer 2a)
  ├── Auth rate limit 15/min on /api/auth (Layer 2b)
  ├── Body size limit 2 MB (Layer 3)
  ├── Context window budget (Layer 4)
  ├── Pagination guard max 100 items (Layer 5)
  ├── JWT / API key verification (per-route)
  └── RBAC role check (per-route)
```

### MCP Server

```
Request
  │
  ├── X-Trace-ID attachment (Layer 0)
  ├── Strict Helmet CSP (Layer 1)
  ├── IP block list check (Layer 2)
  ├── Body size guard (Layer 3)
  ├── Context window budget (Layer 4)
  ├── HTTP method whitelist (Layer 5)
  ├── Scanner user-agent block (Layer 6)
  ├── Global rate limit + speed slow-down (Layer 7)
  ├── 10-family threat detection (Layer 8)
  ├── Agent API key auth + expiry + disabled check (per-route)
  ├── Tool scope enforcement (per-route)
  ├── Circuit breaker check (per-tool)
  ├── Response cache lookup (per-tool)
  └── Retry + context limit on upstream call (per-tool)
```

---

## Monitoring & Observability

### Available Data

| Source | What it shows |
|---|---|
| `GET /api/monitoring/stats` | System overview, cache stats, snapshot info, context limit config |
| `GET /api/monitoring/activity` | Real 24h hourly call timeline (success + error counts) |
| `GET /api/monitoring/top-tools` | Top tools by call count + success rate |
| `GET /api/monitoring/audit-log` | All admin actions (tool create/delete, agent add/remove) |
| `GET /api/monitoring/cache` | Cache hit-rate, entry count, evictions |
| `GET /api/monitoring/snapshot` | Last snapshot timestamp + record counts |
| `GET /metrics` (MCP, admin) | Circuit breaker states, call log, cache stats, system memory |
| `GET /metrics/health` (MCP, public) | Uptime + memory (lightweight probe) |
| `logs/calls.ndjson` (MCP) | Full call history: trace ID, agent, tool, latency, success, retries |
| `logs/security.log` (MCP) | All security events: AUTH failures, threats, rate limits, circuit trips |
| `snapshots/meta.json` (Manager) | Last snapshot: timestamp + counts for all data types |

### X-Trace-ID Flow

```
Client → [generates or passes X-Trace-ID]
  → Manager/MCP Server [attaches to req.traceId, echoes in X-Trace-ID response header]
    → Security log entries [include traceId]
      → Call log entries [include traceId]
        → Upstream API call [X-Trace-ID forwarded in headers]
          → Response [traceId in JSON body]
```

---

## Roadmap

### v2.1 (Next)
- Wire PostgreSQL for persistent storage
- Wire Redis for distributed rate limiting + cache
- `zod` env schema validation at startup

### v2.2
- JWT refresh token + blacklist
- Prometheus metrics export
- Per-API-key rate limiting (not IP-based)

### v3.0
- Full OpenAPI spec
- OpenTelemetry distributed tracing
- OAuth2/OIDC federated agent identity

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for branch strategy, PR process, and code style guide.

---

## License

MIT © MCP Tool Manager Team

---

*See [REPORT.md](REPORT.md) for the full technical capability assessment.*