Web Analysis MCP
Uses SearXNG metasearch engine to perform intelligent web searches with domain filtering, result deduplication, and optional LLM-based relevance filtering across multiple search engines.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Web Analysis MCPfind recent developments in quantum computing and summarize key breakthroughs"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Web Analysis MCP
MCP 服务 - 使用 SearXNG 搜索 + Creeper 爬虫,通过 LLM 总结网页内容后返回,避免 token 超限问题。
特性
🔍 SearXNG 智能搜索 + 域名过滤
🧠 LLM 智能过滤 + 主题分类
🕷️ Creeper 高效爬虫 + 并发处理
🧠 DeepSeek LLM 智能总结
⚡ 缓存优化 + 批量处理
Related MCP server: DuckDuckGo MCP Server
快速开始
1. 安装依赖
git clone <repository-url>
cd web-analysis-mcp
npm install2. 部署外部服务
# SearXNG 搜索引擎
docker run -d -p 8086:8080 searxng/searxng
# Creeper 爬虫
git clone https://github.com/ihub-tech/Creeper.git /home/lyf/workspaces/creeper
cd /home/lyf/workspaces/creeper && pip install -r requirements.txt3. 配置环境变量
cp .env.example .env
# 编辑 .env 文件,配置以下必需项:
SEARXNG_BASE_URL=http://127.0.0.1:8086
CREEPER_PATH=/home/lyf/workspaces/creeper
SUMMARY_API_KEY=sk-your-deepseek-key-here
# 可选配置
FILTER_MAX_RESULTS=8
DOMAIN_BLACKLIST=pinterest.com,facebook.com,instagram.com
DOMAIN_WHITELIST=github.com,stackoverflow.com
# LLM 智能过滤(可选)
FILTER_LLM_ENABLED=true
FILTER_LLM_API_KEY=sk-your-filter-api-key
FILTER_LLM_BASE_URL=https://api.deepseek.com
FILTER_LLM_MODEL=glm-4.5-air4. 获取 API Key
DeepSeek: https://platform.deepseek.com/
智谱 AI: https://open.bigmodel.cn/
ModelScope (国内): https://dashscope.aliyuncs.com/
5. 运行
# 开发模式
npm run dev
# 生产模式
npm run build && npm start6. Claude Desktop 配置
{
"mcpServers": {
"web-analysis": {
"command": "node",
"args": ["/path/to/web-analysis-mcp/dist/index.js"],
"env": {
"SEARXNG_BASE_URL": "http://127.0.0.1:8086",
"CREEPER_PATH": "/home/lyf/workspaces/creeper",
"SUMMARY_API_KEY": "sk-your-key"
}
}
}
}可用工具
工具 | 描述 | 主要参数 |
| 智能搜索 + 爬取 + 总结 | query, max_results, include_domains, exclude_domains |
智能过滤机制
规则过滤
黑名单过滤: 排除低质量域名
白名单优先: 白名单结果排在前面
数量限制: 保留指定数量的高质量结果
去重机制: 移除重复链接
LLM 智能过滤(可选)
启用后使用 LLM 进行主题分类和关联性过滤:
识别 10 种主题类型:编程、军事、政治、历史、科学、科技、娱乐、体育、金融、其他
单次调用完成主题分类和结果筛选
大幅降低后续总结成本
# 规则过滤配置
FILTER_MAX_RESULTS=8
DOMAIN_BLACKLIST=pinterest.com,facebook.com
DOMAIN_WHITELIST=github.com,stackoverflow.com
# LLM 过滤配置
FILTER_LLM_ENABLED=true
FILTER_LLM_MODEL=glm-4.5-air测试
# 安装测试工具
npm install -g @modelcontextprotocol/inspector
# 开发模式测试(推荐)
npm run test:dev
# 生产模式测试
npm run test:inspector故障排除
常见问题
SearXNG 连接失败:
curl http://127.0.0.1:8086/search?q=test&format=jsonCreeper 错误:
cd /path/to/creeper && python creeper.py --help调试模式:
LOG_LEVEL=debug npm run dev文档链接
License
MIT
Available Tools
1 toolweb_searchB
使用 SearXNG 搜索网络内容,通过 Creeper 爬取网页,并返回经过 LLM 总结的结果。适用于需要获取最新网络信息的场景。
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | 搜索查询(不能为空或仅包含空格字符) | |
| max_results | No | 最大结果数量 (1-20) | |
| language | No | 搜索语言 | zh |
| time_range | No | 时间范围 | |
| include_domains | No | 只搜索这些域名(白名单) | |
| exclude_domains | No | 排除这些域名(黑名单) | |
| save_to_file | No | 是否将爬取的内容保存到本地文件(需要启用 SAVE_CONTENT_ENABLED 配置) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the multi-step process (search, crawl, LLM summarization) and hints at recency ('最新网络信息'), but fails to disclose critical behavioral traits such as rate limits, authentication needs, potential costs, error handling, or what 'LLM 总结' entails. For a complex tool with no annotations, this leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the core functionality. Every sentence contributes: the first explains what the tool does, and the second provides usage context. There's no wasted text, though it could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multi-step process with 7 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain the return format, error conditions, or behavioral constraints. The agent would lack sufficient context to use this tool effectively without trial and error.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds no parameter-specific information beyond what's in the schema. According to the rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: '使用 SearXNG 搜索网络内容,通过 Creeper 爬取网页,并返回经过 LLM 总结的结果' (search web content using SearXNG, crawl pages via Creeper, and return LLM-summarized results). It specifies the verb (search/crawl/summarize) and resource (web content), but since there are no sibling tools, it cannot demonstrate differentiation from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage context: '适用于需要获取最新网络信息的场景' (suitable for scenarios requiring up-to-date web information). This implies when to use it (for current information), but lacks explicit guidance on when not to use it or comparisons to alternatives. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
TDQS
With only one tool, there is no possibility of ambiguity or overlap between tools. The tool's purpose is clearly distinct as it is the only one available.
A single tool inherently has perfect naming consistency, as there are no other tools to compare against. The name 'web_search' follows a clear verb_noun pattern.
A single tool for a server named 'Web Analysis MCP' feels thin and under-scoped. The tool performs web search and summarization, but the server name suggests broader analysis capabilities (e.g., link analysis, content extraction, performance metrics) that are not covered.
The server's domain appears to be web analysis, but with only a search and summarization tool, there are significant gaps. Missing operations might include analyzing page structure, extracting specific data, checking SEO metrics, or monitoring web changes, making the surface severely incomplete for the implied purpose.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Web search, URL content extraction to Markdown, site mapping, and recursive web crawler.
Web search, fetch, extract, and research for AI agents. Markdown output + AI-synthesized answers.
LLM-ready web search + instant answers + URL-to-clean-text fetch for agents and RAG.
Provides AI assistants with access to Seltz's powerful Web Search capabilities.
Related MCP Servers
- FlicenseBqualityDmaintenanceAllows you to search the web using DuckDuckGo and optionally fetch and summarize content from search results.24
- FlicenseNot gradedqualityCmaintenanceEnables web searching through DuckDuckGo and fetching content from webpages. Provides search capabilities with configurable result limits and webpage content extraction for AI assistants.
- FlicenseNot gradedqualityCmaintenanceEnables web search across multiple search engines (DuckDuckGo, Bing, Startpage) with parallel execution and result deduplication. Also provides web page content extraction capabilities.2
- AlicenseBqualityDmaintenanceEnables web search through DuckDuckGo and webpage content fetching with intelligent text extraction. Features built-in rate limiting and LLM-optimized result formatting for seamless integration with language models.2MIT
Appeared in Searches
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/himly0302/web-analysis-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server