Skip to main content
Glama

PulpMiner MCP Server

Source: github.com/pulpminercom/pulpminer-mcp · Issues: github.com/pulpminercom/pulpminer-mcp/issues

An MCP server for PulpMiner. It lets AI assistants (Cursor, Claude Desktop, and any other MCP client):

  • Generate JSON from any web page with AI. PulpMiner scrapes the page and an LLM turns it into structured JSON, optionally in the exact shape you give it.

  • Save a page-to-JSON setup as a reusable REST API (static, or dynamic with {{ variables }} in the URL).

  • Fetch data from saved APIs and list them.

Tools

Tool

What it does

Backend endpoint

Credits

pulpminer_generate_json

Scrape a URL and convert it to JSON with AI (optional jsonFormat sample, CSS selector, extra instructions)

POST /external/ai/json

0.25 (+0.1 renderJs)

pulpminer_save_as_api

Save URL + JSON format (+ optional dynamic URL template, cache) as an API; returns the API id and endpoint

POST /external/ai/json/save

free

pulpminer_fetch_api_data

Call a saved API (GET, or POST with variables for dynamic APIs)

GET/POST /external/{apiId}

0.4 (+0.1 renderJs)

pulpminer_list_saved_apis

List saved APIs (id, url, dynamic URL template and its variables)

GET /external/api/url

free

pulpminer_check_api_key

Validate the API key and show which credentials are configured

GET /external/n8n/auth

free

Option values match the backend: scraper is scraper_1 (default) or scraper_2. llm is llm_2 (OpenAI GPT, default), llm_1 (Google Gemini), or llm_3 (Groq, only if enabled for your account).

Related MCP server: zenrows-mcp

Configuration

Env var

Required

Description

PULPMINER_API_KEY

yes

Your API key from the PulpMiner dashboard → API Keys. Sent as the apikey header. It is the only credential any tool needs.

PULPMINER_BASE_URL

no

Default https://api.pulpminer.com

PULPMINER_TIMEOUT_MS

no

Default 300000 (scraping plus the LLM step can take a while)

Install & build

git clone https://github.com/pulpminercom/pulpminer-mcp.git
cd pulpminer-mcp
npm install
npm run build

Run it:

PULPMINER_API_KEY=... node dist/index.js          # stdio
PULPMINER_API_KEY=... node dist/index.js --http   # Streamable HTTP on http://127.0.0.1:3333/mcp

Once published to npm, this becomes npx -y pulpminer-mcp.

Client configuration

Cursor (~/.cursor/mcp.json or .cursor/mcp.json)

{
  "mcpServers": {
    "pulpminer": {
      "command": "npx",
      "args": ["-y", "pulpminer-mcp"],
      "env": { "PULPMINER_API_KEY": "your-api-key" }
    }
  }
}

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "pulpminer": {
      "command": "npx",
      "args": ["-y", "pulpminer-mcp"],
      "env": { "PULPMINER_API_KEY": "your-api-key" }
    }
  }
}

Before it is published, use a local build: "command": "node", "args": ["/absolute/path/to/pulpminer-mcp/dist/index.js"].

Streamable HTTP clients

Start with --http (PORT, HOST env; it binds to 127.0.0.1:3333 by default) and point the client at http://127.0.0.1:3333/mcp. The server is stateless. Each request can send its own API key in the x-pulpminer-api-key header, which overrides PULPMINER_API_KEY. That makes it usable as a hosted multi-user endpoint (put it behind HTTPS).

{
  "mcpServers": {
    "pulpminer": {
      "url": "http://127.0.0.1:3333/mcp",
      "headers": { "x-pulpminer-api-key": "your-api-key" }
    }
  }
}

Typical flow

  1. pulpminer_generate_json { url, jsonFormat? }: preview the JSON.

  2. pulpminer_save_as_api { url, jsonFormat, cache?, dynamicUrl? } returns apiId and endpoint.

  3. pulpminer_fetch_api_data { apiId, variables? }

Dynamic API example: dynamicUrl: "https://example.com/search?q={{ query }}", then fetch it with variables: { "query": "laptops" }.

Development

npm test        # builds, then runs scripts/smoke-test.mjs against a local mock API (no real calls)
npm run inspect # opens the MCP Inspector UI
npx @modelcontextprotocol/inspector --cli node dist/index.js --method tools/list

Hosted server (remote)

remote/ is a Cloudflare Worker serving the same tools over Streamable HTTP (stateless) at https://mcp.pulpminer.com/mcp. Send your PulpMiner API key as Authorization: Bearer <key> (or x-pulpminer-api-key); it is only forwarded to the PulpMiner API, never stored.

{ "mcpServers": { "pulpminer": { "url": "https://mcp.pulpminer.com/mcp", "headers": { "Authorization": "Bearer YOUR_API_KEY" } } } }

Local dev: cd remote && npm install && npm run dev (Node 22+).

Available Tools

5 tools
pulpminer_check_api_keyCheck PulpMiner credentialsA
Read-only
Inspect

Verify that a PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server) is set and valid (GET /external/n8n/auth). Free.

Inputs: no named parameters.

pulpminer_fetch_api_dataFetch JSON from a saved PulpMiner APIA
Idempotent
Inspect

Call a saved PulpMiner API and return its JSON (GET /external/{apiId}, or POST with variables for dynamic APIs). Uses the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server). Costs 0.4 credits (+0.1 if the API renders JS); cached APIs return fast.

Inputs: apiId (required), variables.

pulpminer_generate_jsonGenerate JSON from a web page with AIAInspect

Scrape a web page and convert it to structured JSON with an LLM (PulpMiner POST /external/ai/json). Pass jsonFormat to force an exact output shape; omit it to let the AI infer a flat JSON. Does not save anything. Costs 0.25 credits (+0.1 with renderJs). Can take 10-60s+. Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).

Inputs: url (required), llm, scraper, renderJs, +4 more.

pulpminer_list_saved_apisList saved PulpMiner APIsA
Read-only
Inspect

List the APIs saved under this API key (GET /external/api/url), oldest first, paginated (pageSize 5-100; smaller values are raised to 5). Returns id, url, dynamic flag and dynamicUrl template for each. Free.

Inputs: id, page, pageSize.

pulpminer_save_as_apiSave a page-to-JSON config as a reusable APIAInspect

Save a URL + JSON format as a PulpMiner API (POST /external/ai/json/save). Afterwards the data is available at {baseUrl}/external/{apiId} with the apikey header (see pulpminer_fetch_api_data). For a dynamic API, pass dynamicUrl with {{ variable }} placeholders, e.g. https://site.com/search?q={{ query }}; callers then supply those variables. Saving itself is free; each later fetch costs 0.4 credits (+0.1 with renderJs). Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).

Inputs: url (required), jsonFormat (required), llm, cache, +6 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

    TDQS

    A4.1/5.0

    Scored across 5 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: one-off scrape-to-JSON, persist a saved API, fetch a saved API's data, list saved APIs, and validate the key. No two tools overlap or could be confused for one another.

    Naming Consistency4/5

    All tools use a consistent snake_case verb_noun pattern with a shared `pulpminer_` prefix, e.g. generate_json, save_as_api, list_saved_apis, check_api_key. The only slight inconsistency is the verb style (generate/save vs fetch/list/check), but it remains predictable and readable.

    Tool Count5/5

    Five tools is well-scoped for a scraping-and-API-builder domain: each one maps to a distinct part of the lifecycle (generate, save, fetch, list, authenticate) and none is redundant. The surface is lean without being thin.

    Completeness4/5

    The lifecycle covers generation, persistence, retrieval, listing, and key validation, so agents can complete the core workflow. Minor gaps exist, such as updating or deleting a saved API and fetching a single saved API's metadata, but these are workarounds rather than dead ends.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers