Skip to main content
Glama
Sign In
enesjakozh

mcp-crawler

by orange-fruit01
GitHub
Python
OverviewInspectNewSchemaRelated ServersReviewsScore
Need Help?View Source CodeReport Issue

Integrations

  • Converts crawled web content into markdown format for easier consumption by AI applications

  • Hosts the MCP service as a Docker container, making it accessible via a Render.com URL

MCP Service

This is a Model Control Protocol (MCP) service that provides tools for Cursor and Claude applications.

Connecting to the Service

For Cursor Users

  1. Open Cursor
  2. Go to Settings > AI > Advanced
  3. Under "MCP Endpoint", enter: https://your-render-url.onrender.com
  4. Save settings and restart Cursor

For Claude App Users

  1. Open Claude App
  2. Go to Settings > Advanced
  3. Under "MCP Connection", enter: https://your-render-url.onrender.com
  4. Save settings

Available Tools

  • Web Crawler: Crawls web pages and returns content as markdown

Deployment

This service is deployed on Render.com as a Docker container. The service is accessible at:

https://your-render-url.onrender.com

Replace your-render-url.onrender.com with the actual URL provided by Render after deployment.

Environment Variables

  • MCP_HOST: Host to bind the server (default: 0.0.0.0)
  • MCP_PORT: Port to bind the server (default: 8000)

Monitoring

A health endpoint is available at /health for monitoring the service status.

This server cannot be installed

-
security - not tested
F
license - not found
-
quality - not tested

remote-capable server

The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.

It crawls website

  1. Connecting to the Service
    1. For Cursor Users
    2. For Claude App Users
  2. Available Tools
    1. Deployment
      1. Environment Variables
        1. Monitoring

          Related Resources

          Related MCP Servers

          • MCP Webscan Server

            bsmi021
            A
            security
            A
            license
            A
            quality
            Enables web content scanning and analysis by fetching, analyzing, and extracting information from web pages using tools like page fetching, link extraction, site crawling, and more.
            Last updated -
            6
            7
            TypeScript
            MIT License

          • Scrapbox Cosense MCP Server

            worldnine
            A
            security
            A
            license
            A
            quality
            This server facilitates interaction with cosense/Scrapbox projects, enabling users to retrieve, list, search, and create pages while supporting various query operations and secure access to private projects.
            Last updated -
            4
            12
            TypeScript
            MIT License
            • Apple

          • MCP Server Firecrawl

            Msparihar
            A
            security
            A
            license
            A
            quality
            A server that provides web scraping and intelligent content searching capabilities using the Firecrawl API, enabling AI agents to extract structured data from websites and perform content searches.
            Last updated -
            5
            2
            TypeScript
            MIT License
            • Apple
            • Linux

          • mcp-server-webcrawl

            pragmar
            -
            security
            F
            license
            -
            quality
            Bridge the gap between your web crawl and AI language models. With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously, extracting insights from your web content. Supports WARC, wget, InterroBot, Katana, and SiteOne crawlers.
            Last updated -
            1
            Python
            • Apple

          View all related MCP servers

          Appeared in Searches

          New MCP Servers

          ID: wtt5eqpn0n