A web extraction server that turns permitted public web pages into structured JSON or Markdown using plain-language prompts, returning explicit error signals (never fabricated data) when pages are blocked, login-required, CAPTCHA-gated, or too sparse.
Key capabilities:
Demo without an API key (try_demo_extract): Verify the server is wired in, explore sample extraction output, and get links to docs, signup, pricing, and free-tier details.
General-purpose structured data extraction (extract_url): Describe in plain English what fields you want (e.g. product names, prices, reviews, contact details) and receive clean JSON, Markdown, or raw HTML. Works with JavaScript-heavy SPAs.
Clean Markdown conversion (extract_markdown): Convert a public page into readable Markdown optimized for RAG pipelines, agent context, or saving as a .md file.
Article content extraction (extract_article): Pull structured fields — title, full body text, author, and publish date — from news articles, blog posts, and editorial pages.
Page metadata retrieval (extract_metadata): Obtain title, meta description, Open Graph tags, Twitter Card tags, canonical URL, and other <head> metadata in a consistent JSON schema.
Usage monitoring (get_usage): Check your current plan, monthly credit limit, used credits, and remaining credits.
Extract canonical URL metadata from web pages as part of the metadata extraction capabilities, identifying the preferred URL for content.
Automatically bypass Cloudflare protection when extracting data from URLs, handling JavaScript rendering and Cloudflare security measures transparently.
Access the MCP server source code and documentation on GitHub, with the server hosted at the hauntapi/mcp-server repository.
Handle JavaScript-rendered content when extracting data from URLs, allowing extraction from modern web applications and dynamic websites.
Extract Open Graph metadata (Facebook/Meta tags) and other social media metadata from web pages as part of the metadata extraction capabilities.
Web extraction for AI agents. Give Claude, Cursor, Windsurf, or any MCP client a clean tool that turns any permitted public web page into structured JSON or Markdown from a plain-language prompt, and returns an honest machine-readable failure instead of fabricated data when a page is blocked.
Website · Docs · Get a free key · Python SDK
Built for agent workflows that need product data, competitor prices, article text, page metadata, small site-wide datasets, lead lists, or research snippets, without brittle CSS selectors. Try it with no API key using the try_demo_extract tool. Free tier: 1,000 credits a month, no card.
This is the block to copy.
{
"mcpServers": {
"haunt": {
"command": "npx",
"args": ["-y", "@hauntapi/mcp-server"],
"env": {
"HAUNT_API_KEY": "your-api-key"
}
}
}
}Get a free API key, no card needed: https://hauntapi.com/?utm_source=github&utm_medium=readme&utm_campaign=mcp_server#signup
Prefer a CLI? npx -y --package @hauntapi/cli@latest haunt-cli init prints the
same block for Claude, Cursor, Windsurf and other MCP clients. It prints config,
it does not edit your files, and it is not itself the server.
Call try_demo_extract first. It returns a fixed sample extraction JSON and trace
plus docs, signup, pricing and free-tier links, without using credits.
Use Haunt's try_demo_extract tool and show me the signup and docs links.Use Haunt to extract the top five story titles from https://news.ycombinator.com/For Markdown output:
Use Haunt extract_markdown to turn https://fastapi.tiangolo.com/tutorial/ into clean Markdown.Reddit, with no Reddit app and no OAuth:
Use Haunt read_reddit to get the top 10 posts from r/webscraping.Map or crawl a small site:
Use Haunt map_site on https://fastapi.tiangolo.com/ and return the first 100 URLs under /tutorial/.Use Haunt crawl_site on https://fastapi.tiangolo.com/tutorial/ for up to 10 pages. Extract each page title and main summary.Related MCP server: Crawl4AI MCP Server
Haunt renders pages in a real browser and reads many Cloudflare-protected and bot-walled pages. It does not go past login walls, paywalls, or CAPTCHA human-verification pages, and it does not promise universal extraction. Those return a typed failure signal (login_required, captcha_required, not_found, upstream_fetch_failed) instead of fabricated data, and a failed read is not charged.
try_demo_extractNo-key activation check. Returns a fixed sample extraction JSON/trace, Haunt's demo endpoint, docs, signup, pricing, MCP info route, and free-tier details. Use this first when a user has installed the MCP server but has not added HAUNT_API_KEY yet.
extract / extract_urlGeneral-purpose extraction from permitted public web pages. extract and extract_url are aliases so hosted MCP docs and local stdio package users can follow the same wording.
Use it for:
Product names, prices, stock status, reviews
Competitor pricing pages
Directories and lead lists
Job boards
Research pages
Supported permitted public pages where you want clean JSON instead of HTML
It accepts optional response_format values: json (default), markdown / md, and raw_html / html.
extract_markdownReturn clean Markdown from a permitted public page. Use this when the agent or workflow wants readable page text for RAG, notes, docs ingestion, or saving as a .md file instead of structured fields.
extract_articleExtract article fields from news, blog, and editorial pages.
Returns title, body text, author, and publish date when available.
extract_metadataExtract page metadata including title, description, Open Graph tags, Twitter Card tags, canonical URL, and related metadata.
map_siteDiscover up to 500 public same-site URLs from robots.txt, sitemaps, and bounded link traversal. Mapping costs no credits and is available on paid plans.
crawl_siteDiscover and apply one prompt and optional JSON Schema across up to 20 public same-site pages. Complete pages use normal extraction credits; partial and failed pages cost nothing. Discovery respects robots rules and fetched crawl content is not stored.
read_redditRead Reddit posts or a post's comment thread as structured JSON. No Reddit app, no OAuth, no developer account. Accepts a subreddit URL, a post URL, or shorthand like r/webdev. Set comments to true to read a thread.
Costs 1 credit, and a blocked read costs nothing.
company_profileTurn a company website URL into a structured profile: what the company does, its product, and the contact and social details published on the site. Read live from the site rather than a stale third-party database.
Costs no credits.
github_repoReturn normalised metadata for a public GitHub repository: description, stars, forks, language, licence, topics, and last push.
Costs no credits.
get_usageCheck current plan, monthly credit limit, used credits, reserved credits, and remaining credits. Use this after a live extraction to see what was charged.
Reddit posts and comments with no Reddit app, OAuth, or developer account
Natural-language prompts instead of fragile CSS selectors
Robots-aware site mapping and bounded multi-page extraction
Supported fetch paths for JavaScript-heavy pages
Challenge-aware extraction with machine-readable verification signals (error_code, captcha_provider, requires_human_verification)
Clean JSON output for agents, databases, and workflows
Clean Markdown output when the job is page text for agents, RAG, notes, or .md files
Free tier for testing
Plan | Credits | Price |
Free | 1,000/mo | £0 |
Starter | 10,000/mo | £19/mo |
Pro | 30,000/mo | £49/mo |
Scale | 80,000/mo | £99/mo |
Credits are not one-to-one requests. Simple public/non-LLM output usually uses 1 credit, normal structured extraction 2, browser-rendered or authenticated extraction 4, and heavy/screenshot extraction 8. Failed, blocked, login/CAPTCHA, provider, and server failures do not burn credits.
Upgrade: https://hauntapi.com/#pricing
Extract the main article content from a news article, blog post, or editorial page. Returns a JSON object with: title (string), body (string, full article text), author (string or null), and published_date (string or null). Use this instead of extract_url when you specifically need article content, it is a focused shortcut with consistent article fields. Read-only, makes no changes to any external system. Requires HAUNT_API_KEY environment variable. Free tier: 1,000 credits/month. Returns an error if rate limit, credit quota, or API key is invalid.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the article or blog post to extract. Must be a valid HTTP or HTTPS URL. Works best on news articles, blog posts, and editorial content. For non-article pages, use extract_url instead. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description declares read-only behavior, no external modifications, and discloses authentication requirements, rate limits, and error triggers. With no annotations, this fully informs the agent of critical behavioral constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, front-loading the core purpose and progressively adding usage guidance, behavioral details, and constraints without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fully informs the agent: clear purpose, usage alternatives, return fields (despite no output schema), prerequisites, rate limits, and error states. No gaps remain given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers the parameter fully (100% coverage), but the description adds value by clarifying the URL must be HTTP/HTTPS and specifying best-use cases, slightly exceeding the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts main article content from news articles, blogs, or editorials, and explicitly distinguishes it from the sibling tool 'extract_url' by specifying when to use each.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance ('Use this instead of extract_url when you specifically need article content'), works-best scenarios, and notes prerequisites (HAUNT_API_KEY), rate limits, and error conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return clean Markdown from a permitted public web page for agents, RAG ingestion, notes, or .md files. This is a low-cost non-LLM output mode when the page can be fetched cleanly. Blocked, login-required, CAPTCHA-gated, paywalled, and too-thin pages return explicit errors instead of fabricated Markdown. Read-only, makes no changes to any external system. Requires HAUNT_API_KEY environment variable.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The full URL of the permitted public page to convert into clean Markdown. Must be a valid HTTP or HTTPS URL. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses read-only behavior, error handling for restricted pages, and the HAUNT_API_KEY requirement. It does not mention rate limits or size constraints, but the core behavioral traits are transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four efficient sentences, front-loaded with the primary purpose, no redundant information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter tool without output schema, the description covers input constraints, behavior, error cases, and environment setup. It is complete enough for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single 'url' parameter, but the description adds meaning beyond the schema by specifying it must be a 'permitted public page' and 'valid HTTP or HTTPS URL', clarifying the input requirements.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns clean Markdown from permitted public web pages, listing specific use cases (agents, RAG ingestion, notes, .md files). It distinguishes itself from siblings by specifying what it does not do (blocked, login-required, etc.) and contrasting with 'non-LLM output mode'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides when-to-use context (low-cost clean page extraction) and when-not-to (blocked, paywalled, etc.), plus an environment variable requirement. It lacks explicit references to sibling tools as alternatives, but the constraints are clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract page metadata from a public or authorised URL: title, meta description, Open Graph tags (og:title, og:description, og:image, og:url), Twitter Card tags, canonical URL, and any other meta information present. Returns a JSON object with all discovered meta tags grouped by type. Use this instead of extract_url when you only need metadata, it is faster and returns a consistent schema. Read-only, makes no changes to any external system. Requires HAUNT_API_KEY environment variable. Free tier: 1,000 credits/month. Returns an error if rate limit, credit quota, or API key is invalid.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to extract metadata from. Must be a valid HTTP or HTTPS URL. Permitted public pages are supported, returns whatever meta tags are present in the HTML head. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description fully discloses behavior: it is read-only, makes no changes, requires HAUNT_API_KEY, has a free tier with 1,000 credits/month, and returns errors on rate limit, quota, or invalid key. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise (4 sentences) and well-structured: purpose first, then return format, usage guidance, behavioral traits, and constraints. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no output schema, and no annotations, the description covers purpose, return format, usage guidance, read-only behavior, authentication, and error conditions. It is fully adequate for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (url) with 100% schema coverage. Description adds meaning beyond schema by specifying it must be HTTP/HTTPS, supports permitted public pages, and returns whatever meta tags are present. Though baseline is 3 due to high coverage, the added nuance justifies a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states what the tool does: extract page metadata (title, meta description, Open Graph tags, etc.) from a URL. It clearly identifies the resource (URL) and the action (extract), and distinguishes itself from sibling extract_url by specifying when to use it. This is a specific verb+resource with differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this instead of extract_url when you only need metadata, it is faster and returns a consistent schema.' Also mentions API key requirement, rate limit, and credit quota, informing the agent when the tool is applicable and what constraints apply.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract structured data from permitted public web pages by providing a URL and describing what you want. Returns clean JSON with exactly the fields you asked for by default. Can also return clean Markdown or raw HTML when response_format is set. Uses supported fetch paths for JavaScript-heavy pages and returns explicit error signals when blocked. It does not solve CAPTCHA, access login/paywall-only pages, or circumvent anti-bot controls. This is the general-purpose extraction tool. Use extract_markdown for LLM/RAG-ready Markdown, extract_article for full article content, or extract_metadata for page meta tags instead, they are optimised shortcuts. Read-only, makes no changes to any external system. Requires HAUNT_API_KEY environment variable. Free tier: 1,000 credits/month. Returns an error if rate limit, credit quota, or API key is invalid.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The full URL of the page to extract data from. Must be a valid HTTP or HTTPS URL. Supports permitted public pages, including some JavaScript-heavy SPAs. Human-verification, login-required, CAPTCHA-gated, paywalled, and blocked pages return explicit errors rather than fabricated data. | |
| prompt | Yes | A plain-English description of what data to extract from the page. Be specific about which fields you want. Examples: 'product name, price, and availability', 'all email addresses and phone numbers', 'the main heading, first paragraph, and all image URLs'. The more specific, the more accurate the extraction. | |
| response_format | No | Optional output mode. Leave blank or use json for structured extraction. Use markdown/md when you want clean page text for an agent, RAG pipeline, or .md file. Use raw_html/html only when you need the fetched HTML. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It fully discloses that the tool reads from external systems (read-only), uses supported fetch paths for JavaScript-heavy pages, returns explicit error signals when blocked, does not solve CAPTCHA, requires HAUNT_API_KEY, and mentions free tier limits and error conditions. This is comprehensive and accurate behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with purpose at the beginning, then behavior, alternatives, and constraints. While slightly lengthy, each sentence adds value. It could be tightened by removing redundancy in the alternatives section, but overall it's efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema or annotations, the description covers all critical aspects: input format, usage, behavior, error handling, authentication (HAUNT_API_KEY), rate limits (free tier), and explicit error signals. No missing information for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% as all parameters have descriptions. The description adds meaningful context beyond the schema: for 'url', it specifies allowed formats and types of pages; for 'prompt', it provides concrete examples; for 'response_format', it explains the modes. Since schema_coverage is high, baseline is 3, but the added details justify a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool extracts structured data from public web pages using a URL and description. It specifies the verb (extract), resource (permitted public web pages), and method. Additionally, it distinguishes from sibling tools by naming them and stating they are optimised shortcuts, making it easy for an agent to select the correct tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explicitly tells when to use this tool (general-purpose extraction) and names alternatives (extract_markdown, extract_article, extract_metadata) with their purposes. It also provides explicit limitations: does not solve CAPTCHA, access login/paywall-only pages, or circumvent anti-bot controls. This gives clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Try Haunt's fixed demo extraction without an API key. Use this first when the user wants to verify the MCP package or see the activation path before signup. Returns demo, docs, signup, pricing, and free-tier links. No API key required. Free tier: 1,000 credits/month.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides key behavioral info: no API key required, free tier limit (1,000 credits/month). While it doesn't explicitly state non-destructive nature, 'demo' strongly implies read-only, making it mostly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each adding value: action, usage, return info, constraints. No redundancy, front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description fully lists return content (demo, docs, signup, pricing, free-tier links) and adds limit info. Complete for a simple demo tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; baseline is 4. Description adds no param info because none needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: a demo extraction without an API key, returning specific links. It distinguishes from sibling extraction tools by emphasizing it's a fixed demo, not a real extraction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: 'Use this first when the user wants to verify the MCP package or see the activation path before signup.' This clearly indicates when to use and implies alternative for actual extraction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.14extract_articleextract_markdownextract_metadataextract_urltry_demo_extractScored across 5 tools
Each tool targets a distinct extraction purpose: article content, markdown, metadata, general extraction, and a demo without API key. Descriptions explicitly guide when to use which, eliminating ambiguity.
All tool names follow a consistent verb_noun snake_case pattern (extract_*, try_demo_extract). No mixed conventions or vague naming.
5 tools is well-scoped for a web extraction server, covering the main use cases without unnecessary redundancy.
Covers the core extraction needs with specialized shortcuts and a general-purpose tool. Minor gap: no tool for extracting specific structured elements like tables, but extract_url can handle it.
Web scraping for AI agents. Converts URLs to clean, LLM-ready Markdown with anti-bot bypass.
MCP server for web extraction and rendering via AceDataCloud WebExtrator
MCP server (stdio): fetch web pages as clean readable markdown via the AgentForge API
Cloud scraping & crawling API for AI agents. Turn any URL into clean, LLM-ready markdown.