RAG MCP Gateway

A smart proxy server for the Model Context Protocol (MCP) that aggregates multiple downstream MCP servers and provides Natural Language Search capabilities over their tools.

The gateway acts as a single entry point for an MCP client (like Claude Desktop or an Agent), allowing it to discover and use tools from a wide array of connected servers using semantic queries instead of exact naming matching.

Architecture

The system is built on a modular "Gateway" architecture designed for high discoverability and robust connection management.

graph TD
    Client[MCP Client] <-->|Stdio| Gateway[RAG MCP Gateway]
    
    subgraph "Internal Components"
        Gateway --> ConnectionManager
        Gateway --> Indexer
        Gateway --> Retriever
        
        subgraph "Indexing Pipeline"
            Indexer --> Discovery[Tool Discovery]
            Discovery --> Enrichment[LLM Enrichment]
            Enrichment --> Embedding[Vector Embedding]
            Embedding --> Orama[(Orama DB)]
            Enrichment --> Gemini[Google Gemini API]
        end
        
        subgraph "Retrieval Pipeline"
            Retriever --> Search[Parallel Dense/Sparse Search]
            Search --> RRF[RRF Fusion]
            RRF --> Rerank[Cross-Encoder Reranking]
            Rerank --> Model[Transformers.js]
            Search --> Orama
        end
    end
    
    subgraph "Downstream Servers"
        ConnectionManager <-->|Stdio| ServerA[Local Process]
        ConnectionManager <-->|SSE / HTTP| ServerB[Remote Server]
        ConnectionManager <-->|Docker| ServerC[Containerized Tool]
    end

Key Components

1. Connection Manager: Handles persistent connections to multiple downstream MCP servers.

   • Transports: Supports Stdio, SSE, and Streamable-HTTP.

   • Docker Integration: Can manage lifecycle for Docker-based servers, including automatic container cleanup (stop and rm) before startup to avoid name conflicts.

2. Indexer: Synchronizes the local index with downstream servers.

   • Tool Discovery: Polls listTools from all clients.

   • Enrichment: Uses Google Gemini to generate human-readable summaries and potential search questions for tools, significantly increasing search accuracy.

   • Smart Sync: Only re-indexes tools that have changed their name, description, or schema.

3. Vector Store (Orama): A high-performance, in-memory JavaScript vector database that persists to JSON. It handles both vector (dense) and full-text (sparse) indexing.

4. Retriever: Implements a sophisticated search pipeline:

   • Hybrid Search: Simultaneously executes vector search and keyword search.

   • RRF Fusion: Combines results using Reciprocal Rank Fusion to balance semantic and exact matches.

   • Reranking: A second-stage Cross-Encoder (HuggingFace model via Transformers.js) reranks candidates based on the actual technical schema and logic, ensuring the most relevant tool is prioritized.

5. LLM Service: Provides the generative bridge for metadata enrichment, ensuring that even minimally documented tools are discoverable via natural language queries.

Related MCP server: MCP Vector Proxy

Prerequisites

• Node.js: v18 or higher

• NPM: v9 or higher

• Gemini API Key (Optional but Recommended): For generating better tool descriptions and search queries. Get one here.

Installation

1. Clone the repository:

   git clone <repository-url>
   cd rag-mcp

2. Install dependencies:

   npm install

3. Build the project:

   npm run build

Configuration

The gateway is configured using a config.json file in the root directory. You can copy the example file to start:

cp config.example.json config.json

config.json Structure

Define your downstream servers in the mcpServers object:

{
  "mcpServers": {
    "weather": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-weather"]
    },
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "./allowed-dir"]
    },
    "remote-server": {
        "transport": "sse",
        "url": "http://localhost:3000/sse"
    }
  }
}

Environment Variables

You can configure the gateway using the following environment variables. These can be set in your OS or passed via the env object in your Claude Desktop configuration.

Usage

1. Running Locally (Development)

You can run the server directly using ts-node:

# Set your API key first (Windows PowerShell)
$env:GEMINI_API_KEY="your-key-here"
npm run dev

2. Connecting to Claude Desktop

To use this gateway with Claude Desktop, edit your config file:

Windows: %APPDATA%\Claude\claude_desktop_config.json
Mac/Linux: ~/Library/Application Support/Claude/claude_desktop_config.json

Add the gateway to the mcpServers list:

{
  "mcpServers": {
    "rag-gateway": {
      "command": "node",
      "args": ["C:/path/to/rag-mcp/dist/src/server.js"],
      "env": {
        "GEMINI_API_KEY": "your-key-here",
        "RAG_MCP_LOGGING_ENABLED": "true"
      }
    }
  }
}

Note: Always use absolute paths for the command and arguments when configuring Claude Desktop.

How it Works

Once connected, the Gateway exposes two primary tools to the client:

search_tool(query: string, limit?: number)

This is the discovery mechanism. The Agent should call this first when it doesn't know which tool to use.

• Input: query: "I need to check the weather in London", limit: 3

• Process: The gateway embeds this query, searches the vector database, reranks results, and returns up to limit matching tool schemas (default is 10).

execute_tool(tool_name: string, arguments: object)

This is the execution mechanism.

• Input: tool_name: "weather_get_current", arguments: { city: "London" }

• Process: The gateway looks up which downstream server owns "weather_get_current" and proxies the request to it.

Testing & Development

This project includes a suite of verification scripts in the tests/ directory to validate different components without needing a full MCP client.

Running Tests

Use ts-node to run specific test scenarios:

1. Verify Gateway Logic: Simulates a client connecting to the gateway and running searches.

   npx ts-node tests/verify_gateway.ts

2. Verify Index Synchronization: Checks if tools are correctly added, updated, or removed from the vector index when downstream servers change.

   npx ts-node tests/verify_index_sync.ts

3. Verify Transports: Tests the connection managers handling of Stdio and SSE connections.

   npx ts-node tests/verify_transports.ts

Debugging

Since the server communicates over Stdio, standard output (console.log) is reserved for the protocol.

• Logs: Check rag-mcp.log in the project root (must enable RAG_MCP_LOGGING_ENABLED=true).

• Errors: Critical errors are also logged to the file.

Project Structure

rag-mcp/
├── src/
│   ├── server.ts             # Gateway Entry Point (Stdio Server)
│   ├── indexer.ts            # Tool Discovery & Enrichment Logic
│   ├── retriever.ts          # Hybrid Search & Reranking Pipeline
│   ├── connection_manager.ts # Transport Management (Stdio/SSE/Docker)
│   ├── vector_store.ts       # Orama DB Wrapper (Dense/Sparse)
│   ├── models.ts             # Transformer.js Model Management
│   └── llm.ts                # Gemini API Integration
├── data/                     # Local Database & Persistence
├── tests/                    # Verification Scripts
├── config.json               # Downstream Servers Configuration
└── rag-mcp.log               # Debug Logs (if enabled)

Security & Best Practices

• API Keys: Avoid hardcoding GEMINI_API_KEY. Use an environment variable or a secure secret manager.

• Environment Forwarding: When using the Stdio transport, the Gateway forwards process.env plus any specific env defined in config.json to the child process. Be mindful of sensitive variables.

• Local Persistence: Orama data is stored as a plain JSON file in the ./data directory. Ensure this directory is protected.

• Network Access: Transformers.js will attempt to download models from HuggingFace on the first run. Ensure your environment allows this or pre-download the models.

Troubleshooting

"No tools found"

• Verify that downstream servers in config.json are running and accessible.

• Check rag-mcp.log for connection errors (ensure RAG_MCP_LOGGING_ENABLED=true).

• Run refresh_index() tool to force a scan.

"Vector search is inaccurate"

• Enable RAG_MCP_ENABLE_LLM=true and provide a GEMINI_API_KEY. Tools with poor descriptions need LLM enrichment to be discoverable via natural language.

• Adjust RAG_MCP_SEARCH_THRESHOLD. A lower value (e.g., 0.7) returns more candidates but may include irrelevant results.

"Docker errors"

• Ensure the Docker daemon is running.

• The Gateway attempts to stop and rm containers with the same serverId on startup to avoid name conflicts. Ensure the system user has permissions to execute these commands.

"Model download failed"

• If deployment is in an air-gapped environment, you must pre-cache models in the ~/.cache/huggingface (or equivalent) directory.

License & Credits

Project License

The source code for RAG MCP Gateway is licensed under the ISC License

Third-Party Licenses & Terms

This project utilizes several high-quality models and libraries that are subject to their own licenses:

• Inference Engine: Transformers.js is licensed under the Apache License 2.0.

• Vector Database: Orama is licensed under the Apache License 2.0.