A
licenseA
qualityD
maintenanceEnables AI agents to fetch any web page as clean markdown or screenshot it, turning URLs into LLM-ready context.
2
1 npm
MIT
website-content-crawler
Website Content Crawler
爬取一个 URL 列表，并将页面内容作为结构化 JSON 返回。以 Apify Actor 和 MCP server 方式运行，以便 Agent 可以直接调用它。
在线演示：https://website-content-crawler.vercel.app
运行成本
在 Apify 上按事件付费：
事件 | 价格 |
Actor 启动 | $0.00005 |
返回页面记录 | $0.001 |
平台计算费用会在收益结算前扣除；单位经济效益参见 PRICING.md。
Related MCP server: web-reader-mcp
输出
每次运行都会返回以下完全相同的结构。输出 schema 会强制保证该结构。
{
"url": "https://example.com",
"status": 200,
"title": "Example Domain",
"description": null,
"h1": "Example Domain",
"markdown": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\nLearn more\n\nLearn more (https://iana.org/domains/example)",
"links": [
"https://iana.org/domains/example"
],
"wordCount": 23,
"crawledAt": "2026-08-23T09:55:20.140Z",
"error": null
}目录结构
src/— Actor 源代码（Crawlee Cheerio 爬虫）.actor/— Apify Actor 规范：actor.json、INPUT_SCHEMA.json、output_schema.json、dataset_schema.json、Dockerfilemcp-server/— MCP 包装器，将 Actor 暴露为crawl_website工具site/+tools/— 根据真实运行记录构建在线演示页面
本地运行
npm install
npm testMCP 服务器：
cd mcp-server && npm install && node smoke.mjs发布到 Apify Store
当账户激活后：apify push，按事件付费，listing 使用本 README。
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
Web data for agents. Extract web pages, crawl websites, discover URLs, and search the web.
Unblocking and fresh web data for agents: URL to Markdown, YouTube, Maps, Amazon, jobs. Pay per call
Web scraping for agents. Point it at a URL and it returns the page as clean markdown, JavaScript-rendered pages included. Point it at a site and it maps the URLs or crawls the section you need in the background, a few pages at a time so results fit in the conversation. Search the web and read full pages, extract fields with a JSON schema you define (validated, never invented), read a store's catalogue or a blog's posts from the platform's own feed, and check whether a page has changed. Failed requests cost nothing. The free plan includes 1,500 credits a month.
Web pages to Markdown, metadata and small crawls for agents. Free daily calls, then USDC x402/MPP.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.41MIT
JsonLink MCP Serverofficial
AlicenseAqualityCmaintenanceEnables AI agents to read any web page as clean Markdown, pull title/description/preview metadata, capture desktop, mobile, or full-page screenshots, and list every link or image on a URL. JavaScript-heavy pages are rendered in a real browser and blocked requests retried through a proxy, so agents can reach content they'd otherwise miss.5257 npmMIT- AlicenseNot gradedqualityAmaintenanceEnables agents to extract structured data from web pages from a given URL using a prompt that describes the desired data and fields.128 npmMIT