# web-tools-mcp

One [MCP](https://modelcontextprotocol.io) server that gives any AI agent two web tools, `search` and `extract`. The repo runs the same research task through four different clients to show the tools work the same way in each:

- the MCP TypeScript SDK on its own
- the Claude Agent SDK
- the Vercel AI SDK
- LangChain

Each example checks its own result and exits non-zero if the agent skipped a tool or didn't cite a source, so a broken example fails instead of printing something that looks fine. All four last passed on 2026-11-12, with `claude-opus-5-5`.

## The tools

| Tool | What it does | Needs |
| --- | --- | --- |
| `search` | Web search through [Exa](https://exa.ai). Returns ranked results with title, URL, publish date, and excerpts that match the query. | `EXA_API_KEY` |
| `extract` | Fetches one public page and returns its title, description, and main content as Markdown, with navigation, scripts, and footers removed. | Nothing |

Both tools declare an `outputSchema` and return matching `structuredContent`. They also return the same JSON as text, for clients that only read `content`. Both are annotated `readOnlyHint`. Errors such as a bad URL, a 404, or a missing key come back as tool results with `isError: true`, so the model can read them and try something else instead of the run crashing.

## Quickstart

### 1. Install and build

```bash
git clone https://github.com/dice1546/web-tools-mcp && cd web-tools-mcp
pnpm install
pnpm build
```

Requires Node 22.19 or later.

### 2. Call the tools directly (no keys needed)

```bash
pnpm example:client https://example.com
```

This connects with the official MCP client, lists the tools, and calls `extract`. No model is involved:

```
Tools: search, extract

extract → Example Domain
This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.

Skipped search: set EXA_API_KEY to try it.
```

### 3. Add your keys

```bash
cp .env.example .env
```

Fill in `EXA_API_KEY` (from [dashboard.exa.ai](https://dashboard.exa.ai)) and `ANTHROPIC_API_KEY`. The examples load `.env` (or `.env.local`) themselves.

### 4. Give the tools to an agent

```bash
pnpm example:claude
```

Claude gets one task: find the MCP spec page about tools, read it, and explain what `outputSchema` is for. Output from a real run, with the answer shortened:

```
→ mcp__web-tools__search {"query":"Model Context Protocol specification tools outputSchema"}
→ mcp__web-tools__extract {"url":"https://modelcontextprotocol.io/specification/2026-07-28/server/tools","maxChars":20000}

The `outputSchema` field on a tool is for:

- **Declaring what shape the result will have.** It's an optional JSON Schema ...
- **Making servers and clients agree on that shape.** If a tool has an output schema, the server MUST return structured results that conform to it ...
- **Helping clients and models use the data.** ...

URL read: https://modelcontextprotocol.io/specification/2026-07-28/server/tools

✓ verified: 2 tool calls (search, extract), answer cites a URL
```

### What just happened

1. The example started the server as a child process (`node dist/index.js`) and connected to it over stdio.
2. The Agent SDK registered both tools under the server name `web-tools` and turned off Claude Code's built-in web tools, so the only way to reach the web was through this server.
3. Claude searched, picked the official spec page from the results, read it with `extract`, and answered from what it read.
4. `verify()` confirmed that both tools were called and that the answer cites a URL.

## Same task, four clients

| Example | Client | The tool shows up as | Run |
| --- | --- | --- | --- |
| [01](examples/01-mcp-client.ts) | MCP TypeScript SDK `Client` | `extract` | `pnpm example:client` |
| [02](examples/02-claude-agent-sdk.ts) | Claude Agent SDK `query()` | `mcp__web-tools__extract` | `pnpm example:claude` |
| [03](examples/03-vercel-ai-sdk.ts) | Vercel AI SDK `generateText` with `@ai-sdk/mcp` | `extract` | `pnpm example:ai-sdk` |
| [04](examples/04-langchain.ts) | LangChain `createAgent` with `@langchain/mcp-adapters` | `web-tools_extract` | `pnpm example:langchain` |

All four launch the server with the same `{ command, args, env }` object from [`examples/_shared.ts`](examples/_shared.ts). Examples 02 to 04 use `claude-opus-5-5`. Set `WEB_TOOLS_MODEL` to use a different model.

## Use it in Claude Code or Cursor

Claude Code:

```bash
claude mcp add web-tools -e EXA_API_KEY=your-key -- node /absolute/path/to/web-tools-mcp/dist/index.js
```

Cursor, and other clients that read an `mcpServers` JSON config (in Cursor, that's `.cursor/mcp.json`):

```json
{
  "mcpServers": {
    "web-tools": {
      "command": "node",
      "args": ["/absolute/path/to/web-tools-mcp/dist/index.js"],
      "env": { "EXA_API_KEY": "your-key" }
    }
  }
}
```

CI doesn't test these two configs. The examples above are what it tests.

## Things worth knowing

**The server doesn't see your shell's environment variables.** The stdio clients in the MCP TypeScript SDK, the AI SDK, and the LangChain adapter start the server with a short allowlist of variables (`PATH`, `HOME`, `USER` and a few others). If you export `EXA_API_KEY` in your terminal, the server never receives it unless you also pass it in `env`. [`test/clients.test.ts`](test/clients.test.ts) checks this for all three clients, so a change in that behavior will show up in CI.

**Each client names the tools differently.** See the table above. If your code matches tool names, for example in allowlists or logs, match on the end of the name.

**Over stdio, stdout belongs to the protocol.** A stray `console.log` in the server corrupts the JSON-RPC stream. The server logs only to stderr.

**Claude Code has its own web tools.** Example 02 sets `tools: []` to turn off the built-in WebSearch and WebFetch. Without that, you can't tell which tool did the work.

## How `extract` handles untrusted URLs

The agent chooses the URL, so `extract` treats every URL as untrusted:

- It accepts only `http` and `https` URLs.
- It connects only to global unicast addresses. That rules out loopback, private, link-local and carrier-grade NAT ranges, including the cloud metadata address `169.254.169.254`. It unwraps IPv6 forms that hide an IPv4 address, such as `::ffff:7f00:1` (which is `127.0.0.1`), and accepts IPv6 only from `2000::/3`, the block allocated for global unicast.
- It checks the address in the DNS lookup the socket itself uses, so the address it checked is the address it connects to. That covers redirects to `127.0.0.1`, because every connection to a new host goes through the same check, and it covers DNS rebinding, because no second lookup can come back with a different answer.
- It times out after 15 seconds, stops reading after 2 MB, and accepts only HTML and plain-text responses.

Known limits:

- It refuses NAT64 (`64:ff9b::/96`) addresses, so on an IPv6-only network that reaches IPv4 sites through NAT64, those sites won't load.
- It doesn't run JavaScript, so pages that render on the client can come back thin.
- It assumes pages are UTF-8.

## Tests and CI

`pnpm test` runs 45 tests in about three seconds, with no keys and no network. They cover:

- extraction, and the address checks (including a simulated resolver, to show the socket connects to the address that was checked)
- the Exa request and response, against a local fixture server
- the server end to end over stdio
- the AI SDK and LangChain adapters loading the tools and calling them

[CI](.github/workflows/ci.yml) runs in two stages:

- **Every push and pull request:** typecheck, tests, build, and example 01 against the live web.
- **On demand** (Actions → CI → Run workflow): the same checks, plus examples 02 to 04 against a real model, each checking its own result. These need `ANTHROPIC_API_KEY` and `EXA_API_KEY` as repository secrets, and skip without them. They call a paid API, so they don't run on every push.

## Layout

```
src/index.ts          entry point: runs the server over stdio
src/server.ts         registers the tools and shapes every response
src/net.ts            fetch with the URL and address checks
src/tools/search.ts   Exa search
src/tools/extract.ts  HTML to Markdown
examples/             the same task in four clients
test/                 unit tests, end-to-end tests over stdio, client adapter tests
```

## License

MIT