Skip to main content
Glama
deenesjakoruzh
dev55acc-ai

website-content-crawler

by dev55acc-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

Website Content Crawler

URL のリストをクロールし、ページコンテンツを構造化 JSON として返します。Apify Actor として、また MCP サーバー として動作するため、エージェントが直接呼び出せます。

ライブデモ: https://website-content-crawler.vercel.app

実行にかかる費用

Apify ではイベントごとに支払います:

イベント

価格

Actor 開始

$0.00005

ページレコードの返却

$0.001

プラットフォームのコンピューティングは支払い前に差し引かれます。ユニットエコノミクスについては PRICING.md を参照してください。

Related MCP server: cleanfetch

出力

すべての実行はこの正確な形式を返します。出力スキーマがこれを保証します。

{
  "url": "https://example.com",
  "status": 200,
  "title": "Example Domain",
  "description": null,
  "h1": "Example Domain",
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\nLearn more\n\nLearn more (https://iana.org/domains/example)",
  "links": [
    "https://iana.org/domains/example"
  ],
  "wordCount": 23,
  "crawledAt": "2026-08-23T09:55:20.140Z",
  "error": null
}

レイアウト

  • src/ — Actor ソース (Crawlee Cheerio クローラー)

  • .actor/ — Apify Actor 仕様: actor.json, INPUT_SCHEMA.json, output_schema.json, dataset_schema.json, Dockerfile

  • mcp-server/ — Actor を crawl_website ツールとして公開する MCP ラッパー

  • site/ + tools/ — 実際の実行レコードからライブデモページを構築

ローカルでの実行

npm install
npm test

MCP サーバー:

cd mcp-server && npm install && node smoke.mjs

Apify Store への公開

アカウントがアクティブになったら: apify push、価格は Pay per Event、リストにはこの README が使用されます。

This server cannot be deployed

Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

  • agdataA
    dev.shveik

    Unblocking and fresh web data for agents: URL to Markdown, YouTube, Maps, Amazon, jobs. Pay per call

    Browser AutomationWeb Scraping
    311
  • SnoopScanA
    com.snoopscan

    Web scraping for agents. Point it at a URL and it returns the page as clean markdown, JavaScript-rendered pages included. Point it at a site and it maps the URLs or crawls the section you need in the background, a few pages at a time so results fit in the conversation. Search the web and read full pages, extract fields with a JSON schema you define (validated, never invented), read a store's catalogue or a blog's posts from the platform's own feed, and check whether a page has changed. Failed requests cost nothing. The free plan includes 1,500 credits a month.

    Research & DataSearchWeb Scraping
    301
  • PageWireA
    dev.pagewire

    Web pages to Markdown, metadata and small crawls for agents. Free daily calls, then USDC x402/MPP.

    Research & DataSearchWeb Scraping
    31
  • PagewatchA
    dev.pagelens

    Read a URL as clean markdown, screenshot a website, url to PDF. Web access for agents, no signup.

    Browser AutomationWeb Scraping
    52

Related MCP Servers

  • Fetch MCP Server

    aglolz
    A
    license
    B
    quality
    D
    maintenance
    Enables LLMs to retrieve and process web content by fetching URLs and converting HTML to markdown format. Supports chunked reading of large pages and can access both public websites and local networks.
    1
    MIT

  • cleanfetch

    belzorash
    A
    license
    A
    quality
    C
    maintenance
    Enables AI agents to read web pages reliably, returning clean markdown content, hyperlinks, and metadata without navigation or ad noise.
    3
    9 npm
    MIT

  • Crawlberg MCP Server

    xberg-io
    A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI agents to crawl and scrape websites, converting HTML to clean Markdown and structured metadata with support for JavaScript rendering, bot evasion, and SSRF protection.
    185
    MIT