mcp-server-webcrawl

robots.txt•214 B

User-agent: * Allow: /media/static/images Allow: /media/static/scripts/js Allow: /media/static/styles/css Disallow: /media/static Disallow: /appstat/download/ Disallow: /moffitor/download/ Disallow: /qbit/download/

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/pragmar/mcp_server_webcrawl'

If you have feedback or need assistance with the MCP directory API, please join our Discord server