### MCP Web Utilities Server

Lightweight server exposing web search and page fetching through:
- MCP (`stdio`) for MCP clients
- OpenAPI HTTP (`FastAPI`)

#### Features
- `search_on_web` and `search_on_website` using `ddgs` (DDGS | Dux Distributed Global Search, a multi-source search engine).
- `fetch_webpage` that returns simplified Markdown using `crawl4ai` with stealth settings.

#### Requirements
- Python 3.13+
- `uv` installed



#### Setup

Install `uv` on Windows (PowerShell):
```
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

```
uv sync
```

Then activate the virtual environment and run the crawler setup:
```
.venv\Scripts\activate
crawl4ai-setup
```

#### Run the server

MCP (`stdio`):
```
uv run src\main.py
```

Or when developing:
```
start_mcp_server.cmd
```

OpenAPI HTTP (`127.0.0.1:8000`):
```
uv run uvicorn src.mainhttp:app --host 127.0.0.1 --port 8000
```

Or when developing:
```
start_mcp_server_http.cmd
```

OpenAPI schema:
```
http://127.0.0.1:8000/openapi.json
```

#### LM Studio Configuration
To use this server in LM Studio, add the following to your MCP settings (`mcp.json`):

```json
{
  "mcpServers": {
    "web-utilities": {
      "command": "uv",
      "args": [
        "--directory",
        "D:\\Dev\\McpServer",
        "run",
        "src\\main.py"
      ]
    }
  }
}
```
*Note: Replace `D:\\Dev\\McpServer` with the actual path to your project.*

#### Tools
`search_on_web(query: str, results: int = 10) -> str`
- Returns results formatted as:
```
[title](url)
description
```

`search_on_website(query: str, sites: list[str], results: int = 10) -> str`
- Same format, restricted to the provided `sites`.

`fetch_webpage(target_url: str) -> str`
- Returns simplified Markdown for the target page.

#### OpenAPI Endpoints
- `GET /search/web?query=...&results=10`
- `GET /search/website?query=...&sites=example.com&sites=docs.python.org&results=10`
- `GET /webpage?target_url=https://example.com`
- `GET /date`

#### Tests
```
python -m unittest src.tests
```

#### Notes
- Avoid writing to `STDOUT` (e.g., `print`) when the server is running; it will break JSON RPC communication.
- Network-dependent tests may fail if external services are blocked in the current environment.