VisualWebScraper
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@VisualWebScraperSearch for 'AI' articles on https://example.com/news and summarize them"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
VisualWebScraper
基于 MCP(Model Context Protocol) 的网页抓取服务,使用 Playwright 无头 Chromium 渲染动态网页, 并可调用本地 LM Studio 大模型生成文章简报。
安装
pip install -r requirements.txt
playwright install chromiumRelated MCP server: MCP Playwright Server
启动
python screenshot_scraper_mcp.py
# 或
python -m webscraper.server本地可视化操作台(Web UI)
除 MCP stdio 外,另提供基于标准库 http.server 的网页界面,无需额外依赖,
复用同一套工具层与浏览器池,供非开发者在浏览器中可视化操作。
python run_webui.py # 默认 http://127.0.0.1:8765
python run_webui.py --port 9000 # 自定义端口打开页面后,可在「网页截图 / 文章检索 / 简报生成」三个标签页填写参数并运行; 截图工具会直接渲染返回的长图,检索与简报以文本形式展示(简报生成依赖 LM Studio)。
该 Web UI 与 MCP stdio 入口相互独立,互不影响;退出时浏览器实例同样会被释放。
MCP 客户端配置
{
"mcpServers": {
"VisualWebScraper": {
"command": "python",
"args": ["d:/BaiduSyncdisk/desktop/WebSearchMCP/screenshot_scraper_mcp.py"],
"env": {
"LMSTUDIO_BASE_URL": "http://localhost:1234/v1"
}
}
}
}提供的工具
fetch_webpage_with_screenshot
抓取网页纯文本 + 整页长截图(PNG base64),供多模态模型做视觉分析。
参数 | 类型 | 必填 | 说明 |
| string | 是 | 目标网页链接 |
search_webpage_articles
从栏目/列表页提取文章条目,按标题关键词筛选,按日期倒序输出。
参数 | 类型 | 必填 | 说明 |
| string | 是 | 栏目/列表页链接 |
| string | 是 | 标题筛选关键词 |
| integer | 是 | 输出条数, |
generate_webpage_brief
提取文章 → 抓取详情页正文 → 调用本地大模型逐篇生成摘要 → 输出简报。
参数 | 类型 | 必填 | 说明 |
| string | 是 | 栏目/列表页链接 |
| integer | 是 | 生成条数 |
| string | 否 | 标题筛选关键词,留空为全部 |
| integer | 否 | 每条摘要目标字数,默认 60 |
| string | 否 | LM Studio 模型名,缺省用当前加载的模型 |
三级降级链(保证任何环境下都有输出):
层级 | 触发条件 | 行为 |
L1 | 首次摘要为空 | 换更直接的指令重试 |
L2 | 仍为空 | 取正文首段作为摘要 |
L3 | 全部条目模型调用均失败 | 判定 LM Studio 不可用,落盘 Markdown 到 |
项目结构
webscraper/
├── config.py # 集中配置:超时、限额、环境变量
├── browser.py # Playwright 浏览器池(复用实例)
├── extractors.py # 文章提取、正文抓取、文本清洗
├── llm.py # LM Studio 调用(OpenAI 兼容接口)
├── reporter.py # 简报 Markdown 落盘与文本渲染
├── server.py # MCP 入口:handler 注册与生命周期
└── tools/
├── base.py # 工具注册表、参数校验、结果构造
├── screenshot.py # fetch_webpage_with_screenshot
├── search.py # search_webpage_articles
└── brief.py # generate_webpage_brief站点适配
文章链接识别规则集中在 extractors.py 顶部,换站点时只需调整:
ARTICLE_PATH_MARK = "/article/" # 文章链接特征
ARTICLE_DATE_RE = re.compile(r"/article/(\d{4})/(\d{1,2})/") # URL 日期格式环境变量
变量 | 默认值 | 说明 |
|
| LM Studio OpenAI 兼容接口地址 |
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- AlicenseCqualityDmaintenanceA comprehensive MCP server that provides powerful web automation tools using Playwright, enabling web scraping, testing, and browser interaction through natural language commands.Last updated3815MIT
- Alicense-qualityDmaintenanceAn MCP server that enables AI-powered browser automation, web scraping, and testing using Playwright across Chromium, Firefox, and WebKit. It allows users to perform actions like navigation, clicking, typing, and taking screenshots through natural language interfaces.Last updated14MIT
- Alicense-qualityCmaintenanceMCP server for web search and content extraction using DuckDuckGo or SearXNG, with Playwright-based fetching and LLM-powered data extraction.Last updated140MIT
- Flicense-qualityBmaintenanceAn MCP server for generic browser automation using Playwright. Enables MCP clients to navigate pages, inspect elements, execute JavaScript, capture screenshots, and monitor console logs and network traffic via a headless Chromium instance.Last updated
Related MCP Connectors
Screenshot and HTML render MCP server for AI agents
MCP server exposing the Backtest360 engine API as tools for AI agents.
Firecrawl MCP — wraps the Firecrawl API (firecrawl.dev) for web
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/czhh006564/WebSearchContentToMD_MCP'
If you have feedback or need assistance with the MCP directory API, please join our Discord server