Skip to main content
Glama

PulpMiner MCP Server

Source: github.com/pulpminercom/pulpminer-mcp · Issues: github.com/pulpminercom/pulpminer-mcp/issues

An MCP server for PulpMiner. It lets AI assistants (Cursor, Claude Desktop, and any other MCP client):

  • Generate JSON from any web page with AI. PulpMiner scrapes the page and an LLM turns it into structured JSON, optionally in the exact shape you give it.

  • Save a page-to-JSON setup as a reusable REST API (static, or dynamic with {{ variables }} in the URL).

  • Fetch data from saved APIs and list them.

Tools

Tool

What it does

Backend endpoint

Credits

pulpminer_generate_json

Scrape a URL and convert it to JSON with AI (optional jsonFormat sample, CSS selector, extra instructions)

POST /external/ai/json

0.25 (+0.1 renderJs)

pulpminer_save_as_api

Save URL + JSON format (+ optional dynamic URL template, cache) as an API; returns the API id and endpoint

POST /external/ai/json/save

free

pulpminer_fetch_api_data

Call a saved API (GET, or POST with variables for dynamic APIs)

GET/POST /external/{apiId}

0.4 (+0.1 renderJs)

pulpminer_list_saved_apis

List saved APIs (id, url, dynamic URL template and its variables)

GET /external/api/url

free

pulpminer_check_api_key

Validate the API key and show which credentials are configured

GET /external/n8n/auth

free

Option values match the backend: scraper is scraper_1 (default) or scraper_2. llm is llm_2 (OpenAI GPT, default), llm_1 (Google Gemini), or llm_3 (Groq, only if enabled for your account).

Related MCP server: zenrows-mcp

Configuration

Env var

Required

Description

PULPMINER_API_KEY

yes

Your API key from the PulpMiner dashboard → API Keys. Sent as the apikey header. It is the only credential any tool needs.

PULPMINER_BASE_URL

no

Default https://api.pulpminer.com

PULPMINER_TIMEOUT_MS

no

Default 300000 (scraping plus the LLM step can take a while)

Install & build

git clone https://github.com/pulpminercom/pulpminer-mcp.git
cd pulpminer-mcp
npm install
npm run build

Run it:

PULPMINER_API_KEY=... node dist/index.js          # stdio
PULPMINER_API_KEY=... node dist/index.js --http   # Streamable HTTP on http://127.0.0.1:3333/mcp

Once published to npm, this becomes npx -y pulpminer-mcp.

Client configuration

Cursor (~/.cursor/mcp.json or .cursor/mcp.json)

{
  "mcpServers": {
    "pulpminer": {
      "command": "npx",
      "args": ["-y", "pulpminer-mcp"],
      "env": { "PULPMINER_API_KEY": "your-api-key" }
    }
  }
}

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "pulpminer": {
      "command": "npx",
      "args": ["-y", "pulpminer-mcp"],
      "env": { "PULPMINER_API_KEY": "your-api-key" }
    }
  }
}

Before it is published, use a local build: "command": "node", "args": ["/absolute/path/to/pulpminer-mcp/dist/index.js"].

Streamable HTTP clients

Start with --http (PORT, HOST env; it binds to 127.0.0.1:3333 by default) and point the client at http://127.0.0.1:3333/mcp. The server is stateless. Each request can send its own API key in the x-pulpminer-api-key header, which overrides PULPMINER_API_KEY. That makes it usable as a hosted multi-user endpoint (put it behind HTTPS).

{
  "mcpServers": {
    "pulpminer": {
      "url": "http://127.0.0.1:3333/mcp",
      "headers": { "x-pulpminer-api-key": "your-api-key" }
    }
  }
}

Typical flow

  1. pulpminer_generate_json { url, jsonFormat? }: preview the JSON.

  2. pulpminer_save_as_api { url, jsonFormat, cache?, dynamicUrl? } returns apiId and endpoint.

  3. pulpminer_fetch_api_data { apiId, variables? }

Dynamic API example: dynamicUrl: "https://example.com/search?q={{ query }}", then fetch it with variables: { "query": "laptops" }.

Development

npm test        # builds, then runs scripts/smoke-test.mjs against a local mock API (no real calls)
npm run inspect # opens the MCP Inspector UI
npx @modelcontextprotocol/inspector --cli node dist/index.js --method tools/list

Hosted server (remote)

remote/ is a Cloudflare Worker serving the same tools over Streamable HTTP (stateless) at https://mcp.pulpminer.com/mcp. Send your PulpMiner API key as Authorization: Bearer <key> (or x-pulpminer-api-key); it is only forwarded to the PulpMiner API, never stored.

{ "mcpServers": { "pulpminer": { "url": "https://mcp.pulpminer.com/mcp", "headers": { "Authorization": "Bearer YOUR_API_KEY" } } } }

Local dev: cd remote && npm install && npm run dev (Node 22+).

Available Tools

5 tools
pulpminer_check_api_keyCheck PulpMiner credentialsA
Read-only
Inspect

Verify that a PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server) is set and valid (GET /external/n8n/auth). Free.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint and openWorldHint, so the description's added detail is genuinely valuable: it explains the two credential mechanisms (PULPMINER_API_KEY env var locally vs Authorization: Bearer on the hosted server), reveals the underlying auth endpoint, and discloses that the call is free. It stops short of saying what a failed or successful check returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with parenthetical detail that is front-loaded on the core action (verify the key), with the endpoint and cost note trailing as supporting facts. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only auth check with no output schema and annotations covering the safety profile, the description supplies the essential context (what is checked, where the key comes from, that it is free). The only omission is any hint about the shape of the result, which is a minor gap given no output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4. The description correctly implies no inputs are required, mentioning only server-side credential sources rather than parameters, so there is nothing further for it to document.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource: verify that a PulpMiner API key is set and valid, and it even names the backing endpoint (GET /external/n8n/auth). Its scope is self-evidently distinct from the generation/save/fetch/list siblings, though it never names or contrasts them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this should be run to confirm credentials before calling the other PulpMiner tools. The 'Free' tag hints there is no cost penalty for calling it, but there is no explicit when-to-use, when-not-to-use, or prerequisite statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pulpminer_fetch_api_dataFetch JSON from a saved PulpMiner APIA
Idempotent
Inspect

Call a saved PulpMiner API and return its JSON (GET /external/{apiId}, or POST with variables for dynamic APIs). Uses the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server). Costs 0.4 credits (+0.1 if the API renders JS); cached APIs return fast.

ParametersJSON Schema
NameRequiredDescriptionDefault
apiIdYesNumeric API id, the value after /external/ in the API URL (e.g. 566).
variablesNoOnly for dynamic APIs: values for every {{ variable }} in its dynamic URL, e.g. {"query": "laptops"}.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare openWorldHint, idempotentHint, non-read-only and non-destructive, but the description adds real behavioral context the annotations lack: cost (0.4 credits, +0.1 for JS-rendered APIs), caching speed, and the auth mechanism (PULPMINER_API_KEY locally or a Bearer token on the hosted server). It does not cover error behavior or rate limits, so not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly packed sentences, front-loaded with purpose, then invocation/auth, then cost — no filler. The middle clause partly restates schema content (variables only for dynamic APIs), which is the only minor redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly states the return (its JSON) and covers the notable complexities an agent must budget for: dual GET/POST modes, authentication, per-call credit cost, and caching. It omits failure/error behavior, which keeps it short of complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so apiId and variables are already fully documented with patterns and examples; the description's restatement ('POST with variables for dynamic APIs') adds little. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a specific verb (call/fetch) and resource (a saved PulpMiner API returning JSON), so the agent knows exactly what the tool produces. It is naturally distinct from list_saved_apis and save_as_api, but it never names or explicitly contrasts a sibling, so it falls short of the 5 bar.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description distinguishes GET (plain call) from POST with variables (dynamic APIs), which is useful invocation context. However, it gives no tool-selection guidance against siblings such as list_saved_apis (to find an apiId) or generate_json, and states no prerequisites beyond implying the API must already be saved.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pulpminer_generate_jsonGenerate JSON from a web page with AIAInspect

Scrape a web page and convert it to structured JSON with an LLM (PulpMiner POST /external/ai/json). Pass jsonFormat to force an exact output shape; omit it to let the AI infer a flat JSON. Does not save anything. Costs 0.25 credits (+0.1 with renderJs). Can take 10-60s+. Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).

ParametersJSON Schema
NameRequiredDescriptionDefault
llmNoLLM used to convert HTML to JSON. llm_2 = OpenAI GPT (default), llm_1 = Google Gemini, llm_3 = Groq (only if enabled for the account; otherwise the API returns 'bad request').llm_2
urlYesFull URL of the web page to extract data from (http/https).
scraperNoScraper backend. scraper_1 (default, supports useSession) or scraper_2. Try the other one if a site blocks scraping.scraper_1
renderJsNoRender JavaScript in a headless browser before extracting (default true; adds 0.1 credits).
selectorNoCSS selector to limit extraction to part of the page, e.g. "main" or "#results". Default: whole body.
jsonFormatNoSample JSON whose keys define the output shape, e.g. {"title": "", "price": "", "items": [{"name": ""}]}. Key names are kept exactly; values are extracted from the page.
useSessionNoUse a persistent scraping session (scraper_1 only; ignored for lifetime-plan accounts).
extraLLMInfoNoExtra natural-language instructions for the LLM, e.g. 'only the first 10 products, prices as numbers'.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (readOnlyHint false, destructiveHint false, idempotentHint false), but the description adds material behavior the annotations cannot convey: exact credit cost (0.25, +0.1 with renderJs), 10-60s+ latency, that nothing is persisted, and the authentication requirements for both local and hosted execution.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense paragraph with the core action front-loaded, followed by cost, latency and auth facts as terse clauses. Every sentence earns its place; nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should carry return-value burden; it explains the shape logic (jsonFormat-driven vs. inferred flat JSON) but not the envelope of the response or error behavior. Cost, timing, auth and persistence are all covered, so it is nearly complete for an 8-parameter nested-schema tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description nonetheless adds meaning beyond the schema by explaining the jsonFormat semantics (force an exact output shape vs. infer a flat JSON) and the renderJs cost relationship and jsonFormat omission default.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Scrape a web page and convert it to structured JSON with an LLM') and even names the underlying endpoint. 'Does not save anything' implicitly separates it from pulpminer_save_as_api, so an agent can pick it apart from siblings without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: pass jsonFormat to force an exact shape, omit it to let the AI infer a flat JSON; renderJs adds cost; only the PulpMiner API key is needed. It stops short of explicitly naming alternative tools or a when-not-to-use condition, so a full 5 isn't warranted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pulpminer_list_saved_apisList saved PulpMiner APIsA
Read-only
Inspect

List the APIs saved under this API key (GET /external/api/url), oldest first, paginated (pageSize 5-100; smaller values are raised to 5). Returns id, url, dynamic flag and dynamicUrl template for each. Free.

ParametersJSON Schema
NameRequiredDescriptionDefault
idNoOnly return the API with this id.
pageNoPage number, starting at 1.
pageSizeNoItems per page, 5-100 (values below 5 are raised to 5).

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnlyHint=true already covering the safety profile, the description still adds real context: oldest-first ordering, all-apis-under-this-key scoping, the pageSize floor of 5, the returned fields (including the dynamicUrl template), and the fact that the call is free. It does not mention rate limits or error modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, all front-loaded with the most important fact (what is listed and its scope) and no filler. The pagination, return shape, and cost details each earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates the returned fields (id, url, dynamic flag, dynamicUrl template), and it covers ordering, pagination floor, and cost. It lacks any guidance on total counts or how to iterate pages, which is the only minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents id, page, and pageSize. The description restates the pageSize range and the raise-to-5 rule but adds no new parameter semantics, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb + resource ("List the APIs saved under this API key") with the underlying endpoint named, plus the ordering (oldest first) and pagination nature. This is clearly distinguishable from siblings like pulpminer_save_as_api and pulpminer_fetch_api_data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the resource (retrieve saved APIs) and the "Free" cost note helps the agent decide, but there is no explicit when-to-use statement or comparison against alternatives such as pulpminer_fetch_api_data.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pulpminer_save_as_apiSave a page-to-JSON config as a reusable APIAInspect

Save a URL + JSON format as a PulpMiner API (POST /external/ai/json/save). Afterwards the data is available at {baseUrl}/external/{apiId} with the apikey header (see pulpminer_fetch_api_data). For a dynamic API, pass dynamicUrl with {{ variable }} placeholders, e.g. https://site.com/search?q={{ query }}; callers then supply those variables. Saving itself is free; each later fetch costs 0.4 credits (+0.1 with renderJs). Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).

ParametersJSON Schema
NameRequiredDescriptionDefault
llmNoLLM used to convert HTML to JSON. llm_2 = OpenAI GPT (default), llm_1 = Google Gemini, llm_3 = Groq (only if enabled for the account; otherwise the API returns 'bad request').llm_2
urlYesPage URL the API scrapes (for dynamic APIs: an example URL with real values).
cacheNoCache results: serve cached JSON (24h TTL) and refresh in the background when older than 15 min.
scraperNoScraper backend. scraper_1 (default, supports useSession) or scraper_2. Try the other one if a site blocks scraping.scraper_1
renderJsNoRender JavaScript in a headless browser before extracting (default true; adds 0.1 credits).
selectorNoCSS selector to limit extraction to part of the page, e.g. "main" or "#results". Default: whole body.
dynamicUrlNoOptional URL template with {{ variable }} placeholders. When set the API becomes dynamic and must be called with variables.
jsonFormatYesRequired. The JSON shape the API should return, usually the output of pulpminer_generate_json (values can be left as examples).
useSessionNoUse a persistent scraping session (scraper_1 only; ignored for lifetime-plan accounts).
extraLLMInfoNoExtra natural-language instructions for the LLM, e.g. 'only the first 10 products, prices as numbers'.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavior well beyond the annotations: the resulting access URL shape and required `apikey` header, the credit model (saving free, 0.4 per fetch, +0.1 with renderJs), and the two auth paths (PULPMINER_API_KEY locally, Authorization Bearer on the hosted server). This is exactly the mutation/cost/auth context an agent needs and annotations do not supply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause and each subsequent sentence covers a distinct concern (dynamic templating, cost, auth) with no filler. It is denser than necessary for a single paragraph and could be split, but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter mutation tool with no output schema, the description covers the returned identifier (apiId used in the fetch URL), authentication, cost implications, and the dynamic-variable workflow, while the schema documents every parameter. An agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3; the description adds a concrete dynamicUrl template example and explains that callers must then supply the placeholders, which slightly exceeds what the schema states. It does not otherwise elaborate on the other eight parameters, so it is an incremental rather than decisive gain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Save a URL + JSON format as a PulpMiner API', and names the concrete endpoint (POST /external/ai/json/save). It also positions itself against siblings by implying the jsonFormat comes from pulpminer_generate_json and the result is read back with pulpminer_fetch_api_data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context for when this tool applies (after producing a JSON format; when a page should be exposed as a reusable/dynamic API) and points to pulpminer_fetch_api_data for consuming the result. It stops short of explicit when-not-to-use or a direct sibling comparison (e.g. vs pulpminer_list_saved_apis), so it is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.1
    • First observedpulpminer_check_api_key
    • First observedpulpminer_fetch_api_data
    • First observedpulpminer_generate_json
    • First observedpulpminer_list_saved_apis
    • First observedpulpminer_save_as_api

TDQS

A4.1/5.0

Scored across 5 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: one-off scrape-to-JSON, persist a saved API, fetch a saved API's data, list saved APIs, and validate the key. No two tools overlap or could be confused for one another.

Naming Consistency4/5

All tools use a consistent snake_case verb_noun pattern with a shared `pulpminer_` prefix, e.g. generate_json, save_as_api, list_saved_apis, check_api_key. The only slight inconsistency is the verb style (generate/save vs fetch/list/check), but it remains predictable and readable.

Tool Count5/5

Five tools is well-scoped for a scraping-and-API-builder domain: each one maps to a distinct part of the lifecycle (generate, save, fetch, list, authenticate) and none is redundant. The surface is lean without being thin.

Completeness4/5

The lifecycle covers generation, persistence, retrieval, listing, and key validation, so agents can complete the core workflow. Minor gaps exist, such as updating or deleting a saved API and fetching a single saved API's metadata, but these are workarounds rather than dead ends.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers