# webpage-mcp

A zero-config [Model Context Protocol](https://modelcontextprotocol.io) server that gives any MCP client — Claude Desktop, IDE assistants, or your own agents — the ability to fetch, read, and search the live web.

No API key. No database. No accounts. Point a client at it and it works.

## Tools

| Tool | What it does |
| --- | --- |
| `fetch_page` | Fetch a URL and return its readable text, with scripts, navigation, and page chrome stripped out. |
| `search_page` | Fetch a page and return only the passages most relevant to a query, ranked with BM25 — ideal for long pages or grounded answers. |
| `extract_links` | List a page's hyperlinks as absolute URLs with their anchor text, optionally filtered by a substring. |

## Why

Most "let an AI read the web" setups need an embeddings provider, an API key, and a vector database. This one needs none of that. Retrieval runs in-process with BM25, so it's instant, free, and private — the AI client supplies the reasoning, the server supplies the right passages.

## Install

```bash
git clone https://github.com/Vl4dimirz/webpage-mcp.git
cd webpage-mcp
npm install
npm run build
```

## Use with Claude Desktop

Add this to your `claude_desktop_config.json`:

- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "webpage": {
      "command": "node",
      "args": ["C:/absolute/path/to/webpage-mcp/dist/index.js"]
    }
  }
}
```

Restart Claude Desktop. You can now ask things like:

- "Use the webpage tools to summarize https://example.com/article."
- "Search that page for what it says about pricing."
- "List every link on the homepage that mentions careers."

## Use from any MCP client

The server speaks MCP over stdio. Spawn `node dist/index.js` and connect with the official SDK:

```js
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";

const client = new Client({ name: "my-app", version: "1.0.0" });
await client.connect(new StdioClientTransport({ command: "node", args: ["dist/index.js"] }));

const result = await client.callTool({
  name: "search_page",
  arguments: { url: "https://en.wikipedia.org/wiki/Model_Context_Protocol", query: "who introduced it" },
});
console.log(result.content[0].text);
```

## How it works

- **Fetch + clean** — `cheerio` parses the HTML and removes non-content elements; the body is reduced to plain readable text. Same-URL calls within five minutes are served from a small in-memory cache.
- **Search** — the text is chunked with overlap and scored against the query with BM25 (`k1 = 1.5`, `b = 0.75`), returning the top passages. No embeddings, no external calls.
- **Links** — anchors are resolved to absolute URLs, de-duplicated, and optionally filtered.

## Tech

TypeScript · `@modelcontextprotocol/sdk` · `cheerio` · `zod` · Node 18+ (built-in `fetch`).

## License

MIT