Safe MCP Server

Overview

This project demonstrates how a Model Context Protocol (MCP) server can safely expose local business data and operations to an autonomous AI agent while maintaining strict, mathematically sound security boundaries. It prevents autonomous destructive database mutations by implementing a robust Human-in-the-Loop approval system natively at the tool layer.

Related MCP server: enterprise-agent-lab

Core Security Principle

The AI can autonomously request database mutations (updates, deletions), but it cannot approve or execute them itself. An immutable state machine prevents the AI from bypassing human authorization.

AI Agent
   ↓
MCP Server
   ↓
Permission Layer
   ↓
READ ─────────────→ Database
   ↓
UPDATE / DELETE
   ↓
Approval Request
   ↓
Human Review
   ├── APPROVE → Execute → Audit Log
   └── REJECT  → No Change → Audit Log

Features

• MCP Server: Implements the Model Context Protocol to seamlessly integrate external tools with LLMs.

• Read-Only Database Tools: AI can query and explore data without human intervention.

• Secure Mutation Tools: AI attempts to alter records natively result in sandboxed "pending approval" tickets.

• Permission Enforcement: Hardcoded application-layer checks blocking AI identities from triggering state execution.

• Human-in-the-Loop Approvals: Dedicated frontend dashboard for human oversight.

• Immutable Audit Trail: Append-only audit logging for all mutation requests and execution results.

• FastAPI Approval API: High-performance REST architecture mapping administrative controls.

• Next.js Approval Dashboard: Interactive UI for reviewing pending operations.

• AI Agent Integration (Groq): A dynamically executing AI loop converting MCP tools into structural completions.

• Promptfoo Adversarial Testing: Embedded security framework explicitly designed to red-team prompt-injection and LLM manipulation vectors.

Tech Stack

• Backend: Python 3.11+, FastAPI, Uvicorn

• AI/Protocol: Model Context Protocol (MCP) Python SDK, Groq API (openai/gpt-oss-20b)

• Database: SQLite, SQLAlchemy 2.0 (Typed ORM)

• Frontend: Next.js 14 (App Router), React, Tailwind CSS, TypeScript

• Security Testing: Promptfoo

Project Structure

safe-mcp-server/
├── backend/
│   ├── api/            # FastAPI routes and AI Agent loop
│   ├── database/       # SQLite models, engine, and seed scripts
│   ├── security/       # Permission layer, approval state machine, audit logs
│   ├── tests/          # 208 backend and AI behavior regression tests
│   └── mcp_server.py   # MCP standard I/O server configuration
├── frontend/
│   ├── src/app/        # Next.js pages and Approval Dashboard layout
│   └── src/components/ # React components (e.g., AiChat interface)
├── promptfoo/          # Adversarial evaluation configuration and Python verify hooks
├── README.md           # Documentation
├── requirements.txt    # Python dependencies
└── .env.example        # Environment variable templates

Setup

1. Clone the repository:

git clone https://github.com/yourusername/safe-mcp-server.git
cd safe-mcp-server

2. Backend Setup (Python):

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Frontend Setup (Node.js):

cd frontend
npm install
cd ..

4. Environment Variables: Copy the example environment file and add your Groq API key (do NOT commit this file).

cp .env.example .env
# Edit .env and set GROQ_API_KEY

5. Start the Backend: This will automatically initialize and seed the local SQLite database if it doesn't exist.

source .venv/bin/activate
export $(cat .env | xargs)
uvicorn backend.api:app --reload

6. Start the Frontend: Open a new terminal.

cd frontend
npm run dev

Navigate to http://localhost:3000 to view the Human Approval Dashboard and AI Chat panel.

7. Run Unit & Integration Tests:

source .venv/bin/activate
python -m backend.tests.test_api
python -m backend.tests.test_security
python -m backend.tests.test_mutations
python -m backend.tests.test_agent

8. Run Promptfoo Security Evaluation:

# Ensure backend is running in Terminal 1
cd promptfoo
npx promptfoo eval

API Endpoints

• GET /health : System heartbeat.

• GET /approvals : List pending, approved, or rejected mutations.

• GET /approvals/{id} : Fetch specific mutation details.

• POST /approvals/{id}/approve : Human-only. Authorizes execution.

• POST /approvals/{id}/reject : Human-only. Denies execution.

• POST /approvals/{id}/execute : System/Human. Executes an approved mutation exactly once.

• POST /chat : AI Agent interface integrating MCP dynamic tools.

Note: The AI agent is explicitly blocked from the approve, reject, and execute operations.

MCP Tools

• Read Tools: get_server_status, list_customers, get_customer, search_customers, list_orders, list_support_tickets. READ operations execute immediately and return sandbox data to the AI.

• Mutation Tools: update_customer, delete_customer, update_support_ticket. UPDATE/DELETE operations intercept the request, log it, and create a pending approval request. The AI cannot approve its own requests and cannot directly execute destructive operations.

Security Design

• Server-Side Permission Enforcement: Permissions are structurally enforced at the API and Service layer, agnostic of LLM instructions.

• Fail-Closed Behavior: Unrecognized actors or invalid execution states automatically default to denial.

• Actor Separation: ai-agent and human-admin logic is segregated. AI logic is sandboxed from executive capabilities.

• Approval State Transitions: Strict state machine mapping (pending -> approved -> executed).

• Exactly-Once Execution Protection: Hardcoded checks block replay attacks or duplicate processing of approved tickets.

• Immutable Audit Events: A dedicated audit_logs table traps metadata for every LLM interaction, approval, and execution failure/success.

• Input Validation: All mutation data is structurally cast through Pydantic/SQLAlchemy bounds.

• No Raw SQL MCP Tool: The AI does not have database execution engines, SQL shells, or arbitrary script eval capabilities.

• No Direct DB Access: AI exclusively hits the mcp_server read endpoints.

Security Evaluation

An adversarial security evaluation was performed using Promptfoo, targeting the exact application boundaries. Custom Python hooks validated backend state changes deterministically.

• 25 adversarial test cases

• 25 passed

• 0 failed

• 0 errors

The Promptfoo test categories include:

• Direct approval bypass

• Actor impersonation

• Prompt injection

• SQL/database access

• Arbitrary code execution

• Tool manipulation

• False success claims

• Social engineering

• Parameter validation bypass

• Data exfiltration

Disclaimer: This evaluation verifies structural logic against adversarial LLM prompts. It does not mathematically claim the application is unhackable.

Demo Flow

1. Request: Via the AI chat, the user asks to delete a customer or upgrade their plan.

2. Intercept: The MCP server detects a mutation tool and creates a pending approval request instead of changing the database.

3. Notify: The AI accurately informs the user that approval is pending (e.g., ID #45).

4. Review: The human administrator opens the Next.js dashboard.

5. Decision: The human reviews the specific target and arguments, then clicks "Approve".

6. Execution: The approved mutation executes against the SQLite database.

7. Audit: The audit log permanently records the initial request, the human approval timestamp, and the final execution status.

Security Limitations

• Local Demo Base: SQLite is used purely for local demonstration.

• Identity Mocking: The human approval identity is mocked as a trusted demo actor (human-admin).

• Production Prerequisites: Real-world deployments require rigorous external authentication (OAuth/SAML), authorization policies, secrets management vaults (AWS Secrets Manager/HashiCorp), strict network isolation, runtime monitoring, and WAF rules.

Resume Project Description

• Architected a security-first MCP (Model Context Protocol) Server in Python/FastAPI, safely bridging an LLM Agent (Groq) with a local SQLite database via strict Human-in-the-Loop workflows.

• Enforced zero-trust boundaries by decoupling AI tool-calling from direct database execution, natively trapping destructive operations (UPDATE/DELETE) into an immutable approval state machine.

• Built a React/Next.js Administrator Dashboard providing real-time visibility and human oversight into AI-generated mutation requests.

• Demonstrated 100% resilience to adversarial prompt injection by engineering a custom Promptfoo security evaluation suite that empirically validated LLM sandbox constraints against SQL injection, actor impersonation, and arbitrary execution attacks.