PulpMiner
OfficialPulpMiner MCP Server
Source: github.com/pulpminercom/pulpminer-mcp · Issues: github.com/pulpminercom/pulpminer-mcp/issues
An MCP server for PulpMiner. It lets AI assistants (Cursor, Claude Desktop, and any other MCP client):
Generate JSON from any web page with AI. PulpMiner scrapes the page and an LLM turns it into structured JSON, optionally in the exact shape you give it.
Save a page-to-JSON setup as a reusable REST API (static, or dynamic with
{{ variables }}in the URL).Fetch data from saved APIs and list them.
Tools
Tool | What it does | Backend endpoint | Credits |
| Scrape a URL and convert it to JSON with AI (optional |
| 0.25 (+0.1 renderJs) |
| Save URL + JSON format (+ optional dynamic URL template, cache) as an API; returns the API id and endpoint |
| free |
| Call a saved API (GET, or POST with |
| 0.4 (+0.1 renderJs) |
| List saved APIs (id, url, dynamic URL template and its variables) |
| free |
| Validate the API key and show which credentials are configured |
| free |
Option values match the backend: scraper is scraper_1 (default) or scraper_2. llm is llm_2 (OpenAI GPT, default), llm_1 (Google Gemini), or llm_3 (Groq, only if enabled for your account).
Related MCP server: zenrows-mcp
Configuration
Env var | Required | Description |
| yes | Your API key from the PulpMiner dashboard → API Keys. Sent as the |
| no | Default |
| no | Default |
Install & build
git clone https://github.com/pulpminercom/pulpminer-mcp.git
cd pulpminer-mcp
npm install
npm run buildRun it:
PULPMINER_API_KEY=... node dist/index.js # stdio
PULPMINER_API_KEY=... node dist/index.js --http # Streamable HTTP on http://127.0.0.1:3333/mcpOnce published to npm, this becomes npx -y pulpminer-mcp.
Client configuration
Cursor (~/.cursor/mcp.json or .cursor/mcp.json)
{
"mcpServers": {
"pulpminer": {
"command": "npx",
"args": ["-y", "pulpminer-mcp"],
"env": { "PULPMINER_API_KEY": "your-api-key" }
}
}
}Claude Desktop (claude_desktop_config.json)
{
"mcpServers": {
"pulpminer": {
"command": "npx",
"args": ["-y", "pulpminer-mcp"],
"env": { "PULPMINER_API_KEY": "your-api-key" }
}
}
}Before it is published, use a local build: "command": "node", "args": ["/absolute/path/to/pulpminer-mcp/dist/index.js"].
Streamable HTTP clients
Start with --http (PORT, HOST env; it binds to 127.0.0.1:3333 by default) and point the client at http://127.0.0.1:3333/mcp. The server is stateless. Each request can send its own API key in the x-pulpminer-api-key header, which overrides PULPMINER_API_KEY. That makes it usable as a hosted multi-user endpoint (put it behind HTTPS).
{
"mcpServers": {
"pulpminer": {
"url": "http://127.0.0.1:3333/mcp",
"headers": { "x-pulpminer-api-key": "your-api-key" }
}
}
}Typical flow
pulpminer_generate_json { url, jsonFormat? }: preview the JSON.pulpminer_save_as_api { url, jsonFormat, cache?, dynamicUrl? }returnsapiIdandendpoint.pulpminer_fetch_api_data { apiId, variables? }
Dynamic API example: dynamicUrl: "https://example.com/search?q={{ query }}", then fetch it with variables: { "query": "laptops" }.
Development
npm test # builds, then runs scripts/smoke-test.mjs against a local mock API (no real calls)
npm run inspect # opens the MCP Inspector UI
npx @modelcontextprotocol/inspector --cli node dist/index.js --method tools/listHosted server (remote)
remote/ is a Cloudflare Worker serving the same tools over Streamable HTTP (stateless) at https://mcp.pulpminer.com/mcp. Send your PulpMiner API key as Authorization: Bearer <key> (or x-pulpminer-api-key); it is only forwarded to the PulpMiner API, never stored.
{ "mcpServers": { "pulpminer": { "url": "https://mcp.pulpminer.com/mcp", "headers": { "Authorization": "Bearer YOUR_API_KEY" } } } }Local dev: cd remote && npm install && npm run dev (Node 22+).
Available Tools
5 toolspulpminer_check_api_keyCheck PulpMiner credentialsARead-onlyInspect
Verify that a PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server) is set and valid (GET /external/n8n/auth). Free.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint and openWorldHint, so the description's added detail is genuinely valuable: it explains the two credential mechanisms (PULPMINER_API_KEY env var locally vs Authorization: Bearer on the hosted server), reveals the underlying auth endpoint, and discloses that the call is free. It stops short of saying what a failed or successful check returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with parenthetical detail that is front-loaded on the core action (verify the key), with the endpoint and cost note trailing as supporting facts. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only auth check with no output schema and annotations covering the safety profile, the description supplies the essential context (what is checked, where the key comes from, that it is free). The only omission is any hint about the shape of the result, which is a minor gap given no output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4. The description correctly implies no inputs are required, mentioning only server-side credential sources rather than parameters, so there is nothing further for it to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a precise verb and resource: verify that a PulpMiner API key is set and valid, and it even names the backing endpoint (GET /external/n8n/auth). Its scope is self-evidently distinct from the generation/save/fetch/list siblings, though it never names or contrasts them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: an agent can infer this should be run to confirm credentials before calling the other PulpMiner tools. The 'Free' tag hints there is no cost penalty for calling it, but there is no explicit when-to-use, when-not-to-use, or prerequisite statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pulpminer_fetch_api_dataFetch JSON from a saved PulpMiner APIAIdempotentInspect
Call a saved PulpMiner API and return its JSON (GET /external/{apiId}, or POST with variables for dynamic APIs). Uses the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server). Costs 0.4 credits (+0.1 if the API renders JS); cached APIs return fast.
| Name | Required | Description | Default |
|---|---|---|---|
| apiId | Yes | Numeric API id, the value after /external/ in the API URL (e.g. 566). | |
| variables | No | Only for dynamic APIs: values for every {{ variable }} in its dynamic URL, e.g. {"query": "laptops"}. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare openWorldHint, idempotentHint, non-read-only and non-destructive, but the description adds real behavioral context the annotations lack: cost (0.4 credits, +0.1 for JS-rendered APIs), caching speed, and the auth mechanism (PULPMINER_API_KEY locally or a Bearer token on the hosted server). It does not cover error behavior or rate limits, so not a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tightly packed sentences, front-loaded with purpose, then invocation/auth, then cost — no filler. The middle clause partly restates schema content (variables only for dynamic APIs), which is the only minor redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly states the return (its JSON) and covers the notable complexities an agent must budget for: dual GET/POST modes, authentication, per-call credit cost, and caching. It omits failure/error behavior, which keeps it short of complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so apiId and variables are already fully documented with patterns and examples; the description's restatement ('POST with variables for dynamic APIs') adds little. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence gives a specific verb (call/fetch) and resource (a saved PulpMiner API returning JSON), so the agent knows exactly what the tool produces. It is naturally distinct from list_saved_apis and save_as_api, but it never names or explicitly contrasts a sibling, so it falls short of the 5 bar.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description distinguishes GET (plain call) from POST with variables (dynamic APIs), which is useful invocation context. However, it gives no tool-selection guidance against siblings such as list_saved_apis (to find an apiId) or generate_json, and states no prerequisites beyond implying the API must already be saved.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pulpminer_generate_jsonGenerate JSON from a web page with AIAInspect
Scrape a web page and convert it to structured JSON with an LLM (PulpMiner POST /external/ai/json). Pass jsonFormat to force an exact output shape; omit it to let the AI infer a flat JSON. Does not save anything. Costs 0.25 credits (+0.1 with renderJs). Can take 10-60s+. Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).
| Name | Required | Description | Default |
|---|---|---|---|
| llm | No | LLM used to convert HTML to JSON. llm_2 = OpenAI GPT (default), llm_1 = Google Gemini, llm_3 = Groq (only if enabled for the account; otherwise the API returns 'bad request'). | llm_2 |
| url | Yes | Full URL of the web page to extract data from (http/https). | |
| scraper | No | Scraper backend. scraper_1 (default, supports useSession) or scraper_2. Try the other one if a site blocks scraping. | scraper_1 |
| renderJs | No | Render JavaScript in a headless browser before extracting (default true; adds 0.1 credits). | |
| selector | No | CSS selector to limit extraction to part of the page, e.g. "main" or "#results". Default: whole body. | |
| jsonFormat | No | Sample JSON whose keys define the output shape, e.g. {"title": "", "price": "", "items": [{"name": ""}]}. Key names are kept exactly; values are extracted from the page. | |
| useSession | No | Use a persistent scraping session (scraper_1 only; ignored for lifetime-plan accounts). | |
| extraLLMInfo | No | Extra natural-language instructions for the LLM, e.g. 'only the first 10 products, prices as numbers'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only cover the safety profile (readOnlyHint false, destructiveHint false, idempotentHint false), but the description adds material behavior the annotations cannot convey: exact credit cost (0.25, +0.1 with renderJs), 10-60s+ latency, that nothing is persisted, and the authentication requirements for both local and hosted execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense paragraph with the core action front-loaded, followed by cost, latency and auth facts as terse clauses. Every sentence earns its place; nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should carry return-value burden; it explains the shape logic (jsonFormat-driven vs. inferred flat JSON) but not the envelope of the response or error behavior. Cost, timing, auth and persistence are all covered, so it is nearly complete for an 8-parameter nested-schema tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3; the description nonetheless adds meaning beyond the schema by explaining the jsonFormat semantics (force an exact output shape vs. infer a flat JSON) and the renderJs cost relationship and jsonFormat omission default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Scrape a web page and convert it to structured JSON with an LLM') and even names the underlying endpoint. 'Does not save anything' implicitly separates it from pulpminer_save_as_api, so an agent can pick it apart from siblings without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operating context: pass jsonFormat to force an exact shape, omit it to let the AI infer a flat JSON; renderJs adds cost; only the PulpMiner API key is needed. It stops short of explicitly naming alternative tools or a when-not-to-use condition, so a full 5 isn't warranted.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pulpminer_list_saved_apisList saved PulpMiner APIsARead-onlyInspect
List the APIs saved under this API key (GET /external/api/url), oldest first, paginated (pageSize 5-100; smaller values are raised to 5). Returns id, url, dynamic flag and dynamicUrl template for each. Free.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Only return the API with this id. | |
| page | No | Page number, starting at 1. | |
| pageSize | No | Items per page, 5-100 (values below 5 are raised to 5). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=true already covering the safety profile, the description still adds real context: oldest-first ordering, all-apis-under-this-key scoping, the pageSize floor of 5, the returned fields (including the dynamicUrl template), and the fact that the call is free. It does not mention rate limits or error modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, all front-loaded with the most important fact (what is listed and its scope) and no filler. The pagination, return shape, and cost details each earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully enumerates the returned fields (id, url, dynamic flag, dynamicUrl template), and it covers ordering, pagination floor, and cost. It lacks any guidance on total counts or how to iterate pages, which is the only minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents id, page, and pageSize. The description restates the pageSize range and the raise-to-5 rule but adds no new parameter semantics, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb + resource ("List the APIs saved under this API key") with the underlying endpoint named, plus the ordering (oldest first) and pagination nature. This is clearly distinguishable from siblings like pulpminer_save_as_api and pulpminer_fetch_api_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the resource (retrieve saved APIs) and the "Free" cost note helps the agent decide, but there is no explicit when-to-use statement or comparison against alternatives such as pulpminer_fetch_api_data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pulpminer_save_as_apiSave a page-to-JSON config as a reusable APIAInspect
Save a URL + JSON format as a PulpMiner API (POST /external/ai/json/save). Afterwards the data is available at {baseUrl}/external/{apiId} with the apikey header (see pulpminer_fetch_api_data). For a dynamic API, pass dynamicUrl with {{ variable }} placeholders, e.g. https://site.com/search?q={{ query }}; callers then supply those variables. Saving itself is free; each later fetch costs 0.4 credits (+0.1 with renderJs). Needs only the PulpMiner API key (PULPMINER_API_KEY locally, or Authorization: Bearer <key> on the hosted server).
| Name | Required | Description | Default |
|---|---|---|---|
| llm | No | LLM used to convert HTML to JSON. llm_2 = OpenAI GPT (default), llm_1 = Google Gemini, llm_3 = Groq (only if enabled for the account; otherwise the API returns 'bad request'). | llm_2 |
| url | Yes | Page URL the API scrapes (for dynamic APIs: an example URL with real values). | |
| cache | No | Cache results: serve cached JSON (24h TTL) and refresh in the background when older than 15 min. | |
| scraper | No | Scraper backend. scraper_1 (default, supports useSession) or scraper_2. Try the other one if a site blocks scraping. | scraper_1 |
| renderJs | No | Render JavaScript in a headless browser before extracting (default true; adds 0.1 credits). | |
| selector | No | CSS selector to limit extraction to part of the page, e.g. "main" or "#results". Default: whole body. | |
| dynamicUrl | No | Optional URL template with {{ variable }} placeholders. When set the API becomes dynamic and must be called with variables. | |
| jsonFormat | Yes | Required. The JSON shape the API should return, usually the output of pulpminer_generate_json (values can be left as examples). | |
| useSession | No | Use a persistent scraping session (scraper_1 only; ignored for lifetime-plan accounts). | |
| extraLLMInfo | No | Extra natural-language instructions for the LLM, e.g. 'only the first 10 products, prices as numbers'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavior well beyond the annotations: the resulting access URL shape and required `apikey` header, the credit model (saving free, 0.4 per fetch, +0.1 with renderJs), and the two auth paths (PULPMINER_API_KEY locally, Authorization Bearer on the hosted server). This is exactly the mutation/cost/auth context an agent needs and annotations do not supply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded in the first clause and each subsequent sentence covers a distinct concern (dynamic templating, cost, auth) with no filler. It is denser than necessary for a single paragraph and could be split, but nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter mutation tool with no output schema, the description covers the returned identifier (apiId used in the fetch URL), authentication, cost implications, and the dynamic-variable workflow, while the schema documents every parameter. An agent has everything needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3; the description adds a concrete dynamicUrl template example and explains that callers must then supply the placeholders, which slightly exceeds what the schema states. It does not otherwise elaborate on the other eight parameters, so it is an incremental rather than decisive gain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Save a URL + JSON format as a PulpMiner API', and names the concrete endpoint (POST /external/ai/json/save). It also positions itself against siblings by implying the jsonFormat comes from pulpminer_generate_json and the result is read back with pulpminer_fetch_api_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for when this tool applies (after producing a JSON format; when a page should be exposed as a reusable/dynamic API) and points to pulpminer_fetch_api_data for consuming the result. It stops short of explicit when-not-to-use or a direct sibling comparison (e.g. vs pulpminer_list_saved_apis), so it is strong but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v0.1.1- First observed
pulpminer_check_api_key - First observed
pulpminer_fetch_api_data - First observed
pulpminer_generate_json - First observed
pulpminer_list_saved_apis - First observed
pulpminer_save_as_api
TDQS
Scored across 5 tools
Each tool has a clearly distinct purpose: one-off scrape-to-JSON, persist a saved API, fetch a saved API's data, list saved APIs, and validate the key. No two tools overlap or could be confused for one another.
All tools use a consistent snake_case verb_noun pattern with a shared `pulpminer_` prefix, e.g. generate_json, save_as_api, list_saved_apis, check_api_key. The only slight inconsistency is the verb style (generate/save vs fetch/list/check), but it remains predictable and readable.
Five tools is well-scoped for a scraping-and-API-builder domain: each one maps to a distinct part of the lifecycle (generate, save, fetch, list, authenticate) and none is redundant. The surface is lean without being thin.
The lifecycle covers generation, persistence, retrieval, listing, and key validation, so agents can complete the core workflow. Minor gaps exist, such as updating or deleting a saved API and fetching a single saved API's metadata, but these are workarounds rather than dead ends.
Maintenance
Related MCP Connectors
Web scraping MCP server: any URL to Markdown or JSON, with JavaScript rendering and batch jobs.
One MCP server for 180+ live web-data APIs returning clean JSON from sites that block scrapers.
MCP server for web extraction and rendering via AceDataCloud WebExtrator
Programmatic web-scraping MCP server powered by x402 micro-transactions on Base.
Related MCP Servers
- FlicenseBqualityDmaintenanceAn MCP Server for Web scraping and Crawling, built using Crawl4AI224-

zenrows-mcpofficial
AlicenseAqualityAmaintenanceScrape any webpage and return clean markdown, HTML, or structured JSON. Bypasses anti-bot protection, renders JavaScript (React/Vue/Angular), supports premium residential proxies and CSS extraction. Works with any MCP client — no local install required.521,380 npm20MIT- AlicenseAqualityBmaintenanceWeb extraction MCP server for AI agents. Extract structured data from any URL with built-in Cloudflare bypass, JavaScript rendering, and intelligent parsing. Returns clean markdown or JSON.5794 npm2MIT
- FlicenseNot gradedqualityCmaintenanceTurns any website into a rich set of MCP tools for scraping, crawling, structured data extraction, and automatic API mounting via OpenAPI specs.-