# MCP Search Server

MCP-compliant server providing web search (Yandex Search API) and URL fetching capabilities over JSON-RPC 2.0. Includes a built-in monitoring dashboard with live log feed and request statistics.

## Quick Start

### Online (with internet)

```bash
pip install git+https://github.com/in40/mcp-search-server.git
```

### Offline (air-gapped, no internet)

**Prerequisites:** Python 3.10+, pip.

#### Download

Two bundles available on the [GitHub Releases page](https://github.com/in40/mcp-search-server/releases):

| Bundle | Size | Playwright | Chromium |
|--------|------|------------|----------|
| `mcp-search-server-offline.tar.gz` | 21 MB | no | no |
| `mcp-search-server-offline-full.tar.gz` | 330 MB | yes | yes |

```bash
# Light (no browser automation):
curl -sL https://github.com/in40/mcp-search-server/releases/download/v2.2.0/mcp-search-server-offline.tar.gz -o mcp-search-server-offline.tar.gz

# Full (with Playwright + Chromium for JS-heavy sites):
curl -sL https://github.com/in40/mcp-search-server/releases/download/v2.2.0/mcp-search-server-offline-full.tar.gz -o mcp-search-server-offline-full.tar.gz
```

To rebuild yourself (e.g. for a different Python version):

```bash
git clone https://github.com/in40/mcp-search-server.git
cd mcp-search-server
bash scripts/build-offline.sh                    # light (21 MB)
bash scripts/build-offline.sh --with-playwright  # full (330 MB)
```

#### Deploy (on the air-gapped machine)

**Light bundle:**
```bash
tar xzf mcp-search-server-offline.tar.gz
cd offline
bash install.sh --venv ./venv
```

**Full bundle (Playwright included):**
```bash
tar xzf mcp-search-server-offline-full.tar.gz
cd offline
bash install.sh --venv ./venv
# install.sh automatically links bundled Chromium
```

#### Manual install (without the script)

```bash
pip install --no-index --find-links=./wheels ./wheels/mcp_search_server-*.whl
```

### Configure & Run

```bash
export YANDEX_SEARCH_API_KEY="your-yandex-api-key"
export YANDEX_FOLDER_ID="your-yandex-folder-id"
export YANDEX_SEARCH_TYPE="SEARCH_TYPE_COM"  # or SEARCH_TYPE_RU
mcp-search-server
```

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                     mcp-search-server                        │
│                                                             │
│  Port 8090              │  Port 8091                        │
│  MCP JSON-RPC 2.0       │  REST API           Dashboard     │
│  ┌───────────────────┐  │  ┌──────────────┐  ┌────────────┐ │
│  │ tools/list         │  │  │ POST /search │  │ GET /      │ │
│  │ tools/call         │  │  │ POST /fetch  │  │  (HTML UI) │ │
│  │  - internet_search │  │  │ GET /health  │  │            │ │
│  │  - fetch_url       │  │  │ GET /stats   │  │            │ │
│  └───────────────────┘  │  │ GET /logs     │  └────────────┘ │
│                         │  └──────────────┘                  │
│  In-memory log ring buffer (24h retention)                   │
└─────────────────────────────────────────────────────────────┘
```

## Features

### Search (Yandex Cloud Search API)

- Primary search provider: Yandex Cloud Search API
- Fallback: Brave Search API (configurable via `SEARCH_PROVIDER`)
- Results include title, URL, description excerpts

### URL Fetch (`fetch_url`)

Multi-layered fallback system:

1. **requests** library with Chrome UA headers (primary)
2. **SSL fallback** - retries without cert verification for broken cert chains
3. **curl subprocess** - bypasses some WAF/bot challenges (QRATOR, Cloudflare)
4. **Playwright/Chromium** - headless browser for JS-heavy sites
5. **Wikipedia API** - direct API for QRATOR-protected Wikipedia mirrors
6. Content extraction via **trafilatura** (HTML-to-Markdown)
7. Russian-language error messages for all failure modes

### Dashboard (Port 8091)

- Real-time stats: total requests, last-hour, errors, search vs fetch breakdown
- Live log feed with auto-refresh (5s interval)
- Per-request detail modal (full JSON)
- Yandex search tab with timing and results
- Themes: blue (default), orange (set `DASHBOARD_THEME=orange`)

## Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `YANDEX_SEARCH_API_KEY` | Yes | *(placeholder)* | Yandex Cloud API key |
| `YANDEX_FOLDER_ID` | Yes | *(placeholder)* | Yandex Cloud folder ID |
| `YANDEX_SEARCH_TYPE` | No | `SEARCH_TYPE_COM` | `SEARCH_TYPE_RU` for Russian results |
| `SEARCH_PROVIDER` | No | `yandex` | `brave` to use Brave Search API |
| `BRAVE_SEARCH_API_KEY` | No | *(placeholder)* | Required if `SEARCH_PROVIDER=brave` |
| `PORT` | No | `8090` | MCP JSON-RPC server port |
| `DASHBOARD_PORT` | No | `8091` | Dashboard + REST API port |
| `LOG_RETENTION_HOURS` | No | `24` | Hours to keep request logs in memory |
| `API_KEYS_PATH` | No | `./api_keys.json` | Path to API keys file |
| `DASHBOARD_THEME` | No | *(none)* | Set to `orange` for orange theme |

## Authentication

Both MCP and Dashboard endpoints require `X-API-Key` header matching keys in `api_keys.json`:

```json
{
    "keys": {
        "your-api-key-here": "read-write"
    }
}
```

## MCP JSON-RPC API

### `initialize`

```json
{"jsonrpc": "2.0", "method": "initialize", "id": 1}
```

### `tools/list`

Returns two tools: `internet_search` and `fetch_url`.

### `tools/call`

#### internet_search

```json
{
    "jsonrpc": "2.0",
    "method": "tools/call",
    "id": 2,
    "params": {
        "name": "internet_search",
        "arguments": { "query": "MCP protocol", "count": 5 }
    }
}
```

#### fetch_url

```json
{
    "jsonrpc": "2.0",
    "method": "tools/call",
    "id": 3,
    "params": {
        "name": "fetch_url",
        "arguments": { "url": "https://example.com" }
    }
}
```

## REST API (Dashboard Port)

| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/health` | No | Health check |
| `GET` | `/stats` | X-API-Key | Aggregated statistics |
| `GET` | `/logs?type=&limit=` | X-API-Key | Request logs |
| `GET` | `/` | X-API-Key | Dashboard HTML UI |
| `POST` | `/api/search` | X-API-Key | JSON: `{"query": "...", "count": 5}` |
| `POST` | `/api/fetch` | X-API-Key | JSON: `{"url": "..."}` |

## Docker

### Online build

```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir \
    "mcp-search-server[playwright] @ git+https://github.com/in40/mcp-search-server.git" \
    && python -m playwright install --with-deps chromium
COPY api_keys.json .
ENV API_KEYS_PATH=/app/api_keys.json
CMD ["mcp-search-server"]
```

### Offline build (air-gapped)

Copy `mcp-search-server-offline.tar.gz` to the build context, then choose your base image:

**Debian/Ubuntu-based (standard):**

```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
COPY mcp-search-server-offline.tar.gz /tmp/
RUN cd /tmp && tar xzf mcp-search-server-offline.tar.gz && \
    pip install --no-index --find-links=/tmp/wheels /tmp/wheels/mcp_search_server-*.whl && \
    rm -rf /tmp/mcp-search-server-offline.tar.gz /tmp/wheels /tmp/install.sh
COPY api_keys.json .
ENV API_KEYS_PATH=/app/api_keys.json
CMD ["mcp-search-server"]
```

**Astra Linux (certified Russian OS):**

```dockerfile
FROM registry.astralinux.ru/library/astra/ubi18:1.8.6
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    git \
    python3 \
    python3-pip \
    python3.11-venv \
    && rm -rf /var/lib/apt/lists/*
COPY mcp-search-server-offline.tar.gz /tmp/
RUN cd /tmp && tar xzf mcp-search-server-offline.tar.gz && \
    python3 -m venv /opt/venv && \
    /opt/venv/bin/pip install --no-index --find-links=/tmp/wheels /tmp/wheels/mcp_search_server-*.whl && \
    rm -rf /tmp/mcp-search-server-offline.tar.gz /tmp/wheels /tmp/install.sh
COPY api_keys.json .
ENV API_KEYS_PATH=/app/api_keys.json
CMD ["mcp-search-server"]
```

## Development

### Online

```bash
git clone https://github.com/in40/mcp-search-server.git
cd mcp-search-server
python -m venv .venv && source .venv/bin/activate
pip install -e ".[all]"
playwright install --with-deps chromium
mcp-search-server
```

### Offline

```bash
tar xzf mcp-search-server-offline.tar.gz
cd offline
bash install.sh --venv ./venv
source ./venv/bin/activate
mcp-search-server
```

## License

Apache 2.0