Python Web Search & Scraping MCP
Provides tools for Google Custom Search queries and extraction of public web pages, including combined search-and-scrape workflows.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
GitHub Backend — Python Web Search & Scraping MCP
Independent Python MCP server for Google Custom Search and Python-only public-page extraction. This is the standalone deployment companion for the existing ChatGPT Web Search API plugin.
Routes
Route | Purpose | Authentication |
| Readiness endpoint | Public |
| Python page extraction for existing plugin | Bearer token |
| Streamable HTTP MCP with five tools | Bearer token |
MCP tools: connection_status, google_search, scrape_page, scrape_pages (up to 20), and search_and_scrape.
This scraper does not render JavaScript. It only accesses public HTTPS resources and reports publisher restrictions, including HTTP 403. Publishing the source does not bypass such restrictions.
Official documentation fallback
When https://6sense.com/platform/revvyai returns HTTP 403, extraction makes one independent request to the publicly accessible, publisher-owned https://support.6sense.com/docs/revvyai-in-sales-intelligence. The fallback is limited to this curated RevvyAI URL mapping; arbitrary sites returning 403 remain restricted. The implementation does not attempt to defeat a publisher's access controls.
Successful fallback responses retain the requested URL in url, identify the actual alternative in fallback_source_url, and carry original_http_status: 403, source_type: official_documentation_fallback, and fallback_used: true. The http_status describes the alternative document, not the blocked original. If the alternative fails, the original restricted result is retained with fallback_attempted_url and fallback_status metadata. Consumers should attribute retrieved text to the alternative document, never the original page.
Related MCP server: Google Search MCP Server
Publish public source on GitHub
Use the public repository jatsus22/python-web-search-mcp. This repository is the source for the Python MCP backend.
Upload the contents of this folder, preserving
python_mcp/,tests/,.github/workflows/, and the rootDockerfile. The ZIP is a transfer artifact; do not upload the ZIP itself as the only repository file.Or, from this folder using Git:
git init git add . git commit -m "Initial GitHub Python MCP backend" git branch -M main git remote add origin https://github.com/jatsus22/python-web-search-mcp.git git push -u origin main
Never commit API keys or SCRAPER_SERVICE_TOKEN. Source defaults intentionally contain no secret. Set secrets only in the deployment configuration.
Deploy the Python MCP backend from GitHub (Render)
GitHub stores the source. A Python-capable host runs the server and provides the HTTPS endpoints. Render can build the repository's Dockerfile using the included render.yaml Blueprint; choose a hosting plan suitable for your uptime and cost requirements.
Push this repository to GitHub without including any
.envfile or secret.In the Render dashboard, choose New → Blueprint and connect this GitHub repository. Render reads
render.yamland builds the rootDockerfile.When prompted for deployment environment variables, enter:
Variable
Value
SCRAPER_SERVICE_TOKENA strong random secret, generated locally and stored only in Render and your Web Search API Site's environment
GOOGLE_API_KEYYour Google Custom Search API key; optional for page scraping, required for Google search
MCP_HOST=0.0.0.0andGOOGLE_CX=871edc6cb49394af9are declared inrender.yaml. The Dockerfile defaults to port8080. Never add the token or Google key to the public repository.After deployment, open
https://YOUR_SERVICE.onrender.com/health. A JSON record with"status":"ok"confirms that the Python process is running. Test extraction with a bearer token from a private terminal:curl -fsS https://YOUR_SERVICE.onrender.com/health curl -fsS -X POST https://YOUR_SERVICE.onrender.com/extract \ -H 'Content-Type: application/json' \ -H "Authorization: Bearer $SCRAPER_SERVICE_TOKEN" \ -d '{"url":"https://www.python.org/"}'The direct MCP endpoint is
https://YOUR_SERVICE.onrender.com/mcp(Streamable HTTP). Clients must send the same bearer token. Integrating it into ChatGPT may require compatible OAuth or another supported connection/authentication mechanism.To connect the existing Web Search API Site to this Python service, set its production environment variables:
PYTHON_SCRAPER_URL=https://YOUR_SERVICE.onrender.com/extract SCRAPER_SERVICE_TOKEN=<the same bearer secret>Verify the Site's
connection_statusreportspython_scraper.configured: true, then callscrape_page_pythonand inspect the genuine extraction result. GitHub deployment alone cannot make that status true.
Run locally
python -m pip install -r requirements.txt
export GOOGLE_API_KEY='your-key' # optional, required for Google search
export SCRAPER_SERVICE_TOKEN='test-only-secret'
MCP_HOST=127.0.0.1 PORT=8897 python -m python_mcp.serverIn another terminal:
python -m unittest discover -s tests -v
MCP_URL=http://127.0.0.1:8897/mcp python -m python_mcp.smoke_testsmoke_test.py automatically sends the bearer header when SCRAPER_SERVICE_TOKEN is set. Never disclose a production token in test logs.
Notes
A public GitHub repository is source hosting, not runtime hosting. GitHub Pages cannot run Python server code; Render or another Python-compatible host runs the MCP service.
Running the container on a public interface without
SCRAPER_SERVICE_TOKENis intentionally rejected./mcpand/extractfail closed with HTTP 503 if the bearer token is missing, even when a generic ASGI launcher is used.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
1 Google Search endpoints. Pay per call in USDC via x402.
Pay-per-call web search, keyword trends, and evidence-backed public-page change intelligence.
Web search, page reading, site crawling, data extraction and cited research. OAuth sign-in.
Extract web pages, crawl and map sites, search the web. Extract needs no key; other tools need one.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables web searching and content scraping through Google Custom Search API. Provides tools to search the internet, extract webpage content, and automatically scrape search results for comprehensive information gathering.3MIT
- AlicenseNot gradedqualityDmaintenanceEnables Google search functionality through the Google Custom Search API with enterprise-grade security authentication. Supports streamable HTTP transport protocol for real-time search operations with configurable result limits.MIT
- AlicenseAqualityBmaintenanceEnables JavaScript-rendered Google searches and readable text extraction from public URLs via a real Chrome instance, along with health and CAPTCHA status monitoring.369MIT
- FlicenseNot gradedqualityCmaintenanceEnables aggregated web searches across multiple providers with automatic failover, exposing search and source-listing tools via a streamable HTTP MCP endpoint secured by bearer token authentication.-