🔍 Search Fusion MCP Server

🌏

A High-Availability Multi-Engine Search Aggregation MCP Server providing intelligent failover, unified API, and LLM-optimized content processing. Search Fusion integrates multiple search engines with smart priority-based routing and automatic failover mechanisms.

🆕 What's New in v3.0.0: Major concurrency upgrade! Enhanced multi-threading support with thread-safe operations, intelligent connection pooling, and semaphore-based request limiting. Now supports 50+ concurrent searches without race conditions or data corruption!

✨ Features

🔄 Multi-Engine Integration

• Google Search - Premium performance with API key

• Serper Search - Google search alternative with advanced features

• Jina AI Search - AI-powered search with intelligent content processing

• DuckDuckGo - Free search, no API key required

• Exa Search - AI-powered semantic search

• Bing Search - Microsoft search API

• Baidu Search - Chinese search engine

🚀 Advanced Features

• Intelligent Failover - Automatic engine switching on failures or rate limits

• Priority-Based Routing - Smart engine selection based on availability and performance

• Unified Response Format - Consistent JSON structure across all engines

• Rate Limiting Protection - Built-in cooldown mechanisms

• 🔄 High Concurrency Support - Thread-safe operations with connection pooling

• ⚡ Performance Optimization - Async operations with semaphore-based concurrency control

• LLM-Optimized Content - Advanced web content fetching with pagination support

• Wikipedia Integration - Dedicated Wikipedia search tool

• Wayback Machine - Historical webpage archive search

• Environment Variable Configuration - Pure MCP configuration without config files

• 🌐 Enhanced Proxy Auto-Detection - Intelligent proxy detection with zero configuration

📊 Monitoring & Analytics

• Real-time engine status monitoring

• Success rate tracking

• Error handling and recovery

• Performance metrics

⚡ Concurrency & Performance

• Thread-Safe Operations - All engine statistics and state updates are protected by async locks

• Connection Pooling - Shared HTTP client with configurable connection limits (max 100 connections)

• Semaphore Control - Concurrent request limiting (max 30 simultaneous searches)

• Timeout Protection - 60-second search timeout prevents request accumulation

• Resource Management - Efficient memory usage with automatic connection cleanup

• Race Condition Prevention - Double-checked locking for SearchManager initialization

🏗️ Architecture

🚀 Quick Start

Installation

Option 1: Install from PyPI (Recommended)

Option 2: Install from Source

🌐 Enhanced Proxy Auto-Detection (New in v2.0!)

Search Fusion now features intelligent proxy auto-detection inspired by concurrent-browser-mcp, providing seamless proxy support with zero configuration!

✨ Three-Layer Detection Strategy

1. Environment Variables - Highest priority, checks HTTP_PROXY, HTTPS_PROXY, ALL_PROXY

2. Port Scanning - Scans common proxy ports using socket connection testing

3. System Proxy - Detects OS-level proxy settings (macOS supported)

🔍 Supported Proxy Ports (Priority Order)

• 7890 - Clash default port

• 1087 - V2Ray common port

• 8080 - Generic HTTP proxy port

• 3128 - Squid proxy default port

• 8888 - Other proxy software port

• 10809 - V2Ray SOCKS port

• 20171 - Additional proxy port

🚀 Zero Configuration Usage

Just run directly - proxy will be auto-detected:

Manual override (if needed):

📊 Detection Process

🆚 Comparison with concurrent-browser-mcp

MCP Integration

Environment Variable Configuration

Search Fusion uses pure MCP environment variable configuration without requiring config files.

MCP Client Configuration (PyPI Installation):

MCP Client Configuration (Source Installation):

Supported Environment Variables

Alternative Variable Names:

Engine Priority

Search engines are prioritized automatically:

1. Google Search (Priority 1) - Premium performance with API key

2. Serper Search (Priority 1) - Google alternative with advanced features

3. Jina AI Search (Priority 1.5) - AI-powered search with optional API key for advanced features

4. DuckDuckGo (Priority 2) - Free, no API key required

5. Exa Search (Priority 2) - AI-powered search with API key

6. Bing Search (Priority 3) - Microsoft search API

7. Baidu Search (Priority 3) - Chinese search engine

🛠️ MCP Tools

1. search

Perform web searches with intelligent engine selection and failover.

Parameters:

• query (required): Search query terms

• num_results (default: 10): Number of results to return

• engine (default: "auto"): Engine preference

  • "auto": Automatic engine selection (recommended)

  • "google": Prefer Google Search

  • "serper": Prefer Serper Search

  • "jina": Prefer Jina AI Search

  • "duckduckgo": Prefer DuckDuckGo

  • "exa": Prefer Exa Search

  • "bing": Prefer Bing Search

  • "baidu": Prefer Baidu Search

2. fetch_url

Fetch and process web content with intelligent pagination and multi-method fallback.

Parameters:

• url (required): Web URL to fetch

• use_jina (default: true): Whether to prioritize Jina Reader for LLM-optimized content

• with_image_alt (default: false): Whether to generate alt text for images

• max_length (default: 50000): Maximum content length per page (auto-paginate if exceeded)

• page_number (default: 1): Retrieve specific page from previously fetched content

Features:

• Intelligent Multi-Method Fallback: Tries Jina Reader → Serper Scrape → Direct HTTP

• Automatic Pagination: Splits large content into manageable pages

• Concurrent-Safe Caching: Unique page IDs prevent conflicts in high-concurrency scenarios

• LLM-Optimized Content: Clean markdown format optimized for AI processing

3. get_available_engines

Get current status and availability of all search engines.

4. search_wikipedia

Search Wikipedia articles for entities, people, places, concepts, etc.

Parameters:

• entity (required): Entity to search for

• first_sentences (default: 10): Number of sentences to return (0 for full content)

5. search_archived_webpage

Search archived versions of websites using Wayback Machine.

Parameters:

• url (required): Website URL to search

• year (optional): Target year

• month (optional): Target month

• day (optional): Target day

📖 API Examples

Basic Search

Advanced Web Fetching

Wikipedia Search

🧪 Development

Development Setup

🔧 Configuration Guide

For detailed configuration instructions, see MCP_CONFIG_GUIDE.md.

📊 Performance

• Latency: Sub-second response times with caching

• Availability: 99.9% uptime with intelligent failover

• Throughput: Handles concurrent requests efficiently

• Scalability: Efficient resource utilization and concurrent processing

📈 Concurrency Benchmarks

Tested Performance (v3.0.0+):

• ✅ 50+ concurrent searches - No race conditions or data corruption

• ✅ Thread-safe statistics - Accurate request counting and error tracking

• ⚡ Connection pooling - Efficient HTTP resource management

• 🛡️ Timeout protection - 60s per request prevents system overload

• 📊 Real-time monitoring - Live engine status during high load

Recommended Limits:

• Concurrent searches: 10 (configurable via semaphore)

• Connection pool: 100 max connections, 20 keep-alive

• Request timeout: 60 seconds

• Memory usage: ~50MB baseline + ~2MB per concurrent request

🤝 Contributing

1. Fork the repository

2. Create a feature branch

3. Make your changes

4. Add tests for new functionality

5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🚨 Rate Limiting & Best Practices

• Google Search: 100 queries/day (free tier)

• Serper API: Varies by plan

• Jina AI: Rate limits apply based on subscription

• DuckDuckGo: No official limits, but use responsibly

• Other engines: Check respective API documentation

Always implement appropriate delays and respect rate limits to ensure sustainable usage.

📞 Support

• 📖 Documentation

• 🐛 Issue Tracker

• 💬 Discussions

Made with ❤️ for the MCP community