qwen-web-research-mcp
qwen-web-research-mcp
웹 리서치용 MCP 서버: URL(또는 검색할 사이트 + 문구)이 주어지면 전체 페이지 콘텐츠를 가져와서 로컬 Qwen 모델(Ollama를 통해)을 사용해 해당 페이지에 대한 질문을 추출/답변합니다. 긴 페이지는 자동으로 청크로 분할되고 map-reduce 전략으로 분석되므로 건너뛰는 내용이 없습니다.
콘텐츠 추출은 trafilatura를 사용하며, 사이트별 CSS 선택자 대신 콘텐츠 휴리스틱을 기반으로 내비게이션/광고/보일러플레이트를 제거합니다. 따라서 사이트 레이아웃이 변경되어도 계속 작동합니다. 사이트 범위 검색은 DuckDuckGo의 site: 연산자를 사용하므로 사이트별 맞춤 스크래핑 코드 없이 모든 도메인에서 작동합니다.
도구
analyze_page(url, question)— 페이지를 가져와 전체 콘텐츠에 대한question에 답변합니다.search_site_and_analyze(site, phrase, question, max_results)—site에서phrase를 포함하는 페이지를 찾은 다음 각 일치 항목에 대해analyze_page스타일 추출을 실행합니다.list_available_models()— 구성된 Ollama 서버에서 사용 가능한 Qwen/Ollama 모델을 나열합니다.
Related MCP server: searxng-mcp
요구 사항
Python 3.11+
Qwen 모델을 하나 이상 내려받은 Ollama 서버(기본값:
qwen3:14b)
구성(환경 변수)
변수 | 기본값 | 설명 |
|
| 사용할 Ollama 모델 |
|
| Ollama 서버의 포트 |
| 자동 감지 | Ollama 호스트 IP를 재정의합니다. 설정하지 않으면 |
|
|
|
|
| 바인딩 호스트(HTTP 전송 전용) |
|
| 바인딩 포트(HTTP 전송 전용) |
로컬에서 실행
python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python -m qwen_web_research.serverDocker로 실행
docker build -t qwen-web-research-mcp .
docker run -p 8001:8000 \
-e MCP_TRANSPORT=streamable-http \
-e MCP_HOST=0.0.0.0 \
-e OLLAMA_HOST_IP=host.docker.internal \
qwen-web-research-mcp또는 다른 도구(예: Open WebUI)와 함께 docker-compose.yml의 서비스로 실행:
qwen-web-research:
build: ./qwen-web-research-mcp
ports:
- "8001:8000"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- MCP_TRANSPORT=streamable-http
- MCP_HOST=0.0.0.0
- MCP_PORT=8000
- OLLAMA_HOST_IP=host.docker.internal
- QWEN_MODEL=qwen3:14b그런 다음 MCP 클라이언트(예: Open WebUI의 Tools/Connections 설정)를 http://qwen-web-research:8000/mcp(내부 Docker 네트워크) 또는 http://localhost:8001/mcp(호스트에서)로 지정하세요.
Available Tools
3 toolsanalyze_pageA
Fetch a web page, and use a local Qwen model to extract/answer question
about its full content. Long pages are automatically split into chunks and
analyzed piece by piece (map-reduce), so no content is skipped.
Reports progress while working (page fetch retries, per-chunk analysis), so a client that respects MCP progress notifications won't time out waiting on a long page -- this can take 30s-3min or more.
Args: url: The page to fetch and analyze. question: What to extract or answer about the page's content.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| question | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description takes on full burden. It thoroughly discloses behavior: uses local Qwen model, splits long pages into chunks (map-reduce) to avoid skipping content, reports progress (page fetch retries, per-chunk analysis), and warns about long execution time (30s-3min). This is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with each sentence adding value. It front-loads the core function, then explains special behavior (chunking and progress) and a clear Args section. No redundant phrases.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 params, output schema exists, no annotations), the description covers all essential aspects: purpose, parameters, behavior (chunking, progress), and time expectations. The presence of an output schema reduces need for return value details. It is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description provides clear meanings for both parameters: 'url' is the page to fetch/analyze, and 'question' is what to extract or answer. This compensates fully for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies a clear action: fetching a web page and using a Qwen model to extract/answer a question about its content. It distinguishes itself from siblings by focusing on a single URL analysis, whereas search_site_and_analyze implies searching first, and list_available_models is about models.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly states when to use: to fetch a specific page and answer a question about its content. It implicitly contrasts with search_site_and_analyze by mentioning 'a web page' (single URL) vs. site search. However, it does not explicitly mention alternatives or when not to use it, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_available_modelsA
List the Qwen/Ollama models currently available on the local Ollama server.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral aspects such as side effects, permissions, or whether it is read-only, relying solely on the verb 'list' to imply non-mutating behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the purpose without superfluous details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema), the description sufficiently conveys the operation and its scope, meeting all necessary contextual requirements.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters to explain; the description is complete and needs no additional parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and the resource (Qwen/Ollama models) within a specific context (local Ollama server), distinguishing it from sibling tools that focus on page analysis and search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not specify when to use this tool versus alternatives, lacking explicit guidance on conditions or comparisons with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_site_and_analyzeA
Search a specific site for pages/listings containing phrase, then run
analyze_page-style extraction on each match to answer question.
Works on any site without site-specific scraping code: it uses DuckDuckGo's
site: search to find matches, then trafilatura + Qwen to read and filter
each page's content, so it keeps working even if the site's layout changes.
Each match takes roughly 30s-3min to fetch and analyze (more for long pages), so a large max_results will take proportionally long to return. Progress is reported per match and per chunk, for MCP clients that respect it.
Args: site: Domain to search within, e.g. "example.com". phrase: Exact phrase the publication/listing must contain. question: What information to extract from each matching page. max_results: Max number of matching pages to analyze (default 5).
| Name | Required | Description | Default |
|---|---|---|---|
| site | Yes | ||
| phrase | Yes | ||
| question | Yes | ||
| max_results | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full transparency load. It successfully discloses the core behaviors: uses external search engine, parsing through trafilatura and Qwen, handles layout changes, reports progress per match/chunk, and has long latency per match. It does not mention possible rate limits, no-match behavior, failure modes, or network auth, but covers most relevant behavioral traits for an AI agent considering this tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured into a high-level workflow, mechanism, performance note, and Args section. Every sentence contributes useful information such as time estimates, layout-resilience, or parameter semantics. It is concise without sacrificing essential guidance, and the 'Args:' list makes scanability easy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a multi-step search-and-analysis tool, the description covers when to use it, how it works, what each parameter means, time cost, progress reporting, and the fact that output follows analyze-style extraction. Because an output schema exists, avoiding return-value detail is appropriate. The description provides enough for an agent to decide if this tool is appropriate and set expectations about runtime.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description is responsible for parameter meanings. It provides meaningful explanations for all four params: `site` with an example domain, `phrase` as an exact phrase, `question` as what info to extract, and `max_results` as maximum matching pages with a default of 5. This fully bridges the schema gap, though some examples or constraints are minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific, action-oriented phrase: 'Search a specific site for pages/listings containing `phrase`, then run `analyze_page`-style extraction on each match to answer `question`.' This clearly defines the tool's responsibility and differentiates it from the sibling tool `analyze_page`, which focuses on a single page. The scope (site-level search plus extraction) is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly explains when to use it ('Works on any site without site-specific scraping code'), how it works (DuckDuckGo `site:` search, trafilatura + Qwen), and yields a practical performance caveat: 'Each match takes roughly 30s-3min to fetch and analyze.' It does not explicitly name an alternative for single-page analysis, though it references `analyze_page`-style extraction, so it provides clear context but no explicit 'when not to use' statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.1.0- First observed
analyze_page - First observed
list_available_models - First observed
search_site_and_analyze
TDQS
Scored across 3 tools
The two analysis tools are cleanly separated by input: analyze_page targets one explicit URL, while search_site_and_analyze first discovers matching pages on a domain and then analyzes them. list_available_models is clearly a separate utility, so there is no real ambiguity between tools.
All names use snake_case and lead with a verb, making the set predictable and readable. The main deviation is that search_site_and_analyze combines two actions into one name, while the other tools use a simpler verb_noun pattern.
Three tools is small but defensible for a focused web-research helper: two research workflows plus a model discovery utility. It is slightly minimal, but there is no obvious bloat or unnecessary duplication.
The server covers single-page analysis and site-scoped search, which are useful core pieces. However, it lacks a general web-search flow or multi-site research capability, so open-ended web research requests would have no obvious tool to use.
Maintenance
Related MCP Connectors
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Turn any public website into an MCP server for agents to search, read and navigate.
Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceMCP server for local semantic search over web content, enabling AI agents to ingest, index, and query pages with hybrid retrieval and token budget control.4GPL 2.0
- AlicenseAqualityCmaintenanceMCP server for private web search via self-hosted SearXNG with local reranking, full-page content fetching via Firecrawl, and optional Ollama-powered query expansion and summaries.766 npm24MIT
- AlicenseNot gradedqualityCmaintenanceA self-contained web-research MCP server that lets local LLM agents search, fetch, and synthesize web content using tools like web_search, web_fetch, and web_research.2MIT
- AlicenseNot gradedqualityDmaintenanceSemantic web search MCP server that navigates the live web using a cheap LLM and local embeddings, caching results for fast responses to similar queries without relying on general search engines.MIT