Skip to main content
Glama

ToolGate

MCP proxy for dynamic tool gating with semantic search

ToolGate is a Model Context Protocol (MCP) proxy that intelligently filters tools based on semantic relevance, reducing token bloat and improving AI agent performance. Instead of sending all available tools to your AI agent, ToolGate uses semantic search to return only the most relevant tools for each query.

Features

  • šŸŽÆ Semantic Tool Filtering: Uses sentence-transformers and FAISS to find relevant tools

  • šŸ”„ Lazy Two-Phase Loading: Returns truncated descriptions first, full schemas only when called

  • šŸ“Š Session Continuity: Boosts recently-used tools for better context awareness

  • šŸ›”ļø Rule-Based Gating: Always-include/always-exclude patterns for fine-grained control

  • šŸ“ˆ Metrics & Benchmarking: Built-in SQLite metrics tracking and comprehensive benchmark suite

  • šŸš€ Production Ready: Full test coverage, type hints, and robust error handling

Related MCP server: Bifrost-MCP Gateway

Installation

# Clone the repository
git clone https://github.com/yourusername/toolgate.git
cd toolgate

# Install with pip
pip install -e .

# Or install with dev dependencies
pip install -e ".[dev]"

Quick Start

1. Create a configuration file

# config.yaml
upstream_servers:
  - name: filesystem
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/allowed/files"]

  - name: git
    command: npx
    args: ["-y", "@modelcontextprotocol/server-git", "--repository", "/path/to/repo"]

gating:
  top_k: 10
  always_include:
    - "read_*"
    - "git_status"
  always_exclude:
    - "delete_*"
  session_boost: 0.15
  session_window: 5

index:
  model_name: "sentence-transformers/all-MiniLM-L6-v2"
  similarity_metric: "cosine"

metrics:
  enabled: true
  db_path: "~/.toolgate/metrics.db"

2. Run the proxy

toolgate serve --config config.yaml

The proxy will:

  1. Connect to all upstream MCP servers

  2. Fetch and index their tools

  3. Start listening on stdio for MCP requests

  4. Filter tools based on semantic relevance

3. Use with your AI agent

Point your AI agent (Claude Desktop, Continue, etc.) to use ToolGate as an MCP server:

{
  "mcpServers": {
    "toolgate": {
      "command": "toolgate",
      "args": ["serve", "--config", "/path/to/config.yaml"]
    }
  }
}

CLI Commands

Serve

Start the ToolGate proxy server:

toolgate serve --config config.yaml

Index

Display index configuration (dry-run):

toolgate index --config config.yaml

Status

View metrics and statistics:

toolgate status --config config.yaml --limit 20

Benchmark

Run the benchmark suite:

toolgate benchmark --benchmark-dir ./benchmark

How It Works

Architecture

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  AI Agent   │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”˜
       │ MCP (stdio)
       ā–¼
ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│          ToolGate Proxy             │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”  │
│  │   Semantic Search Engine      │  │
│  │  • sentence-transformers      │  │
│  │  • FAISS cosine similarity    │  │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜  │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”  │
│  │      Gating Rules             │  │
│  │  • always_include/exclude     │  │
│  │  • session boost              │  │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜  │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”  │
│  │    Metrics (SQLite)           │  │
│  │  • Token counting             │  │
│  │  • Latency tracking           │  │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜  │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜
       │                  │
       ā–¼                  ā–¼
ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”    ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│ MCP Server  │    │ MCP Server  │
│ (filesystem)│    │    (git)    │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜    ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

Tool Filtering Flow

  1. Query Extraction: ToolGate captures the user's message as a query

  2. Embedding: Query is embedded using sentence-transformers (all-MiniLM-L6-v2)

  3. Search: FAISS finds top-K most similar tools using cosine similarity

  4. Gating: Apply rules (always_include, always_exclude, session boost)

  5. Truncation: Return only tool names + truncated descriptions (phase 1)

  6. JIT Schema Loading: Full schema loaded only when tool is called (phase 2)

Token Reduction

ToolGate achieves significant token savings through:

  • Relevance filtering: Only top-K tools returned (default: 10)

  • Description truncation: Descriptions limited to 200 chars in listTools

  • Lazy schema loading: Full inputSchema loaded only on callTool

Benchmark Results (50 tools, 20 queries):

  • Precision@10: 85%+ (finds relevant tools)

  • Token Reduction: 60%+ average savings

  • Latency: <100ms per query

Configuration Reference

Upstream Servers

upstream_servers:
  - name: server_name        # Unique identifier
    command: executable      # Command to run
    args: [...]             # Command arguments
    env:                    # Optional environment variables
      VAR: value

Gating Rules

gating:
  top_k: 10                 # Max tools to return (default: 10)
  always_include:           # Patterns to always include
    - "git_*"
    - "read_file"
  always_exclude:           # Patterns to always exclude
    - "dangerous_*"
  session_boost: 0.15       # Score boost for recent tools (0-1)
  session_window: 5         # Number of recent turns to track
  description_max_length: 200  # Truncation length

Pattern matching supports wildcards:

  • tool_name - exact match

  • prefix_* - prefix match

  • *_suffix - suffix match

Index Configuration

index:
  model_name: "sentence-transformers/all-MiniLM-L6-v2"
  cache_dir: "~/.cache/toolgate"  # Optional model cache
  similarity_metric: "cosine"     # cosine, euclidean, or dot

Metrics

metrics:
  enabled: true
  db_path: "~/.toolgate/metrics.db"
  token_model: "cl100k_base"  # tiktoken encoding

Development

Running Tests

# Install dev dependencies
pip install -e ".[dev]"

# Run all tests
pytest tests/ -v

# Run with coverage
pytest tests/ --cov=toolgate --cov-report=html

Running Benchmark

# Run benchmark suite
python benchmark/run.py

# Or via CLI
toolgate benchmark

Expected results:

  • āœ“ Precision@10 ≄ 80%

  • āœ“ Token reduction ≄ 60%

  • āœ“ Query latency < 100ms

License

MIT License - see LICENSE file for details.

Contributing

Contributions welcome! Please:

  1. Fork the repository

  2. Create a feature branch

  3. Add tests for new functionality

  4. Ensure all tests pass

  5. Submit a pull request

References

Acknowledgments

Built for the CABAL project - making AI agents smarter through better tool management.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    F
    maintenance
    A universal gateway that aggregates multiple MCP servers into a single interface while providing advanced token optimization, result filtering, and automated summarization. It enables efficient management of large tool catalogs and reduces context usage by up to 95% for major AI clients.
    33 npm
    17
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enterprise-grade dynamic MCP proxy that eliminates token bloat by lazy-loading tool schemas based on semantic intent, enabling efficient orchestration of multiple backend tools from a single endpoint.
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    A single MCP endpoint for AI agents to browse, inspect, and call tools from multiple upstream MCP servers without loading all schemas upfront, reducing context overhead.
    1 npm
    ISC
  • F
    license
    Not graded
    quality
    D
    maintenance
    Enables efficient tool routing and context budgeting for AI agents handling hundreds of MCP tools, with circuit breaker, RBAC, and federation for reliable serving.
    -