web-research-mcp

A read-only MCP server that fetches public web pages and extracts clean text, metadata, headings, and focus-matched facts.

Demo

Related MCP server: sifthound

Quickstart

Requires Node.js 20 or newer.

git clone <repository-url>
cd web-research-mcp
npm install
npm run build
npm run smoke

Replace <repository-url> with this repository's clone URL. npm run smoke spawns the built server over stdio, lists its tools, and calls both tools against https://example.com/. It requires internet access but no API key.

Run the offline test suite with:

npm test

Connect to Cursor

Create .cursor/mcp.json in your project (or use your global Cursor MCP configuration) and replace the path with the absolute path to this checkout:

{
  "mcpServers": {
    "web-research": {
      "command": "node",
      "args": ["/absolute/path/to/web-research-mcp/dist/index.js"]
    }
  }
}

Restart Cursor after saving the file.

Connect to Claude Desktop

Add the server to claude_desktop_config.json:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

• Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "web-research": {
      "command": "node",
      "args": ["/absolute/path/to/web-research-mcp/dist/index.js"]
    }
  }
}

On Windows, use an absolute path such as C:\\path\\to\\web-research-mcp\\dist\\index.js. Restart Claude Desktop after saving the file.

Test with MCP Inspector

From the repository root, after building:

npx @modelcontextprotocol/inspector node dist/index.js

The Inspector opens a browser UI where you can list the tools and invoke them interactively.

Tool reference

extract_facts includes the facts array only when focus is supplied. On failure, both tools return an MCP error result whose text body is JSON:

{ "code": "INVALID_INPUT", "message": "url: Too small: expected string to have >=1 characters" }

How it works

Placeholder: document the end-to-end flow here (input validation, SSRF checks, HTTP fetch with redirects and limits, Cheerio content extraction, and JSON result formatting).

Known limitations

• No JavaScript-rendered pages. The server fetches and parses the initial HTML response; content rendered client-side by a SPA or infinite scroll is not available.

• No login or authenticated requests. Only public, unauthenticated pages can be fetched.

• DNS rebinding is not fully prevented. Hostnames are resolved and checked before fetch() connects, but the resolve-then-fetch sequence is not atomic, so a malicious DNS server could change the answer in between.

• Main-content extraction is heuristic. The parser prefers article, then main, then body, and removes common chrome; it can still include or omit the wrong content on unusual layouts.

• Responses are limited to HTML, 2 MB, and 10 seconds per fetch. Non-HTML content types and oversized responses are rejected.

Example prompts

1. "Fetch https://example.com and summarize the main points in three bullets."

2. "Extract the title, author, publication date, and JSON-LD types from https://research.example/blog/post."

3. "Find every sentence about pricing on https://example.com/pricing and show the nearest heading for each quote."