Skip to main content
Glama
codewithkate

Robots Compliance Scrape Workflow MCP Server

by codewithkate
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
Python
Remote

Related Servers

Alternatives to Robots Compliance Scrape Workflow MCP Server

No user-submitted related servers found.

    Related Servers

    • magpie-mcp

      MBemera
      A
      license
      Not graded
      quality
      C
      maintenance
      A secure web scraping MCP server for AI agents that fetches pages with token budgeting, robots.txt compliance, and injection warnings, providing parsed content like markdown, metadata, and structured data.
      1
      MIT

    • krwl3r

      garciarsdiego
      A
      license
      Not graded
      quality
      D
      maintenance
      MCP server for web scraping and browser automation, enabling AI agents to extract clean, token-efficient content from web pages.
      1
      MIT

    • mcp-web-tools-server

      ZephyraRR
      A
      license
      Not graded
      quality
      C
      maintenance
      A custom MCP server that provides AI agents with tools to fetch web pages, extract readable article text, extract structured data by CSS selector, and check robots.txt permissions.
      MIT
    • scraperapi-mcp-server by scraperapi

      scraperapi-mcp-serverofficial

      scraperapi
      A
      license
      A
      quality
      B
      maintenance
      A local MCP server that lets AI agents bypass bot detection, geo-restrictions, and JavaScript rendering challenges when scraping the web, backed by ScraperAPI's services
      28
      5
      MIT

    • better-fetch

      dhruvparmar372
      A
      license
      Not graded
      quality
      D
      maintenance
      An MCP server that gives AI tools a reliable way to fetch content from the web, handling anti-bot protection and JavaScript-rendered pages.
      885
      MIT

    Latest Blog Posts

    MCP directory API

    We provide all the information about MCP servers via our MCP API.

    curl -X GET 'https://glama.ai/api/mcp/v1/servers/codewithkate/webscraping-mcp-server'

    If you have feedback or need assistance with the MCP directory API, please join our Discord server