research-tools
自主网络研究智能体
一个教学项目:一个 Python 智能体,能够规划、搜索网络、在 RAG 中存储笔记、撰写报告,然后自我批评并循环直到答案足够好。
技术栈:Python 3.11+ · LangChain · LangGraph · MCP · Chroma RAG · LangSmith
你正在构建的内容(课堂地图)
组件 | 文件 | 职责 |
配置 |
| 读取 |
LLM 层 |
| 一个函数,多种提供商 |
工具 |
| 搜索 + 获取页面 |
MCP |
| 通过 MCP 协议提供相同工具 |
RAG |
| 在 Chroma 中记住页面 |
状态 |
| 图的笔记本 |
节点 |
| 每个步骤一个函数 |
图 |
| 连接循环 |
CLI |
| 从终端运行 |
API |
| FastAPI 后端 |
阅读 TEACHING.md 获取完整课程。
Related MCP server: Local-MCP-server
设置
cd ~/Projects/autonomous-web-research-agent
python3.11 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
cp .env.example .env在 .env 中至少放入一个 LLM 密钥(OPENAI_API_KEY 是默认路径)。搜索功能使用免费的 DuckDuckGo。如果之后需要更高质量的搜索,可以添加 TAVILY_API_KEY。添加 LANGCHAIN_API_KEY 可在 LangSmith 中查看追踪。
运行研究任务
python -m research_agent.cli "How does MCP differ from a normal LangChain tool?"运行后端
python -m research_agent.api
# POST http://127.0.0.1:8001/research {"question": "..."}运行 MCP 服务器(用于 Cursor / 其他主机)
python -m research_agent.mcp_server示例 Cursor MCP 配置:
{
"mcpServers": {
"research-tools": {
"command": "/Users/YOU/Projects/autonomous-web-research-agent/.venv/bin/python",
"args": ["-m", "research_agent.mcp_server"]
}
}
}测试(无需 API 密钥)
pytestAvailable Tools
2 toolsfetch_pageB
Download a URL and return cleaned visible text.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states the action is a download (read-only) but does not mention idempotency, error handling, rate limits, or any side effects. The term 'cleaned' is ambiguous regarding filtering and transformations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no extraneous words. It efficiently conveys the core function without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and an output schema, but the description lacks usage guidance and behavioral details. It covers the basic purpose but misses context that would help an agent decide when to invoke it relative to the sibling tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds only that the tool downloads a URL, but does not clarify expected format, protocol (HTTP/HTTPS), or validation rules. The parameter 'url' is left mostly undocumented beyond its name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Download a URL') and the result ('return cleaned visible text'). It distinguishes from sibling 'search_web' by specifying a direct URL fetch rather than a search operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a specific URL to retrieve, but does not provide explicit guidance on when to use this tool versus 'search_web', nor does it mention any exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_webC
Search the public web and return titles, URLs, and snippets.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full behavioral burden. It does not disclose important details such as whether results are limited, if there are any access restrictions or rate limits, or how the search is performed. The description is minimal, missing context about result behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded with the essential action. It avoids unnecessary words, earning its place by being brief yet informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and incomplete schema description, the description fails to fully compensate. An output schema exists, but the description does not explain return values or behavioral details, leaving gaps for a comprehensive understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema describes one required parameter 'query' but has 0% description coverage, meaning the description adds no meaningful context beyond the schema. However, the description implies the query is used for web searching, providing moderate value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches the public web and returns specific fields (titles, URLs, snippets). It effectively conveys the action and resource, though it lacks explicit differentiation from the sibling tool 'fetch_page'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'fetch_page'. The context suggests it's for web searches, but there's no explicit instruction on usage context or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.1.0- First observed
fetch_page - First observed
search_web
TDQS
Scored across 2 tools
search_web is for querying the web and returning snippets, while fetch_page retrieves full content from a specific URL. Their purposes are completely distinct and easily distinguishable.
Both tools follow a consistent verb_noun pattern (search_web, fetch_page), making the naming predictable and clear.
With only two tools, the server feels thin for a general research purpose. While each tool is essential, the minimal count borders on insufficient for complex workflows.
The two tools cover the basic research loop of searching and fetching content. Minor gaps exist (e.g., no ability to refine searches or manage results), but agents can work around them.
Maintenance
Related MCP Connectors
Scrape, crawl and search the web for AI agents via MCP.
Free web search for AI agents. No API key required. Hosted MCP in active development.
Web scraping, extraction and search for AI agents. Hosted MCP with workspace API keys.
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceA Model Context Protocol server that exposes powerful web search and scraping tools to AI agents and MCP-compatible clients.Apache 2.0
- FlicenseNot gradedqualityDmaintenanceEnables tool-calling LLMs to search the internet, capture website images, extract webpage text, and more via a local MCP server.16-
- AlicenseNot gradedqualityCmaintenanceA minimal MCP server for agent-friendly web extraction and search. Offers two tools: fetching real pages with Playwright and Crawl4AI, and searching across 7 engines with automatic fallback.3AGPL 3.0
- AlicenseNot gradedqualityCmaintenanceA self-contained web-research MCP server that lets local LLM agents search, fetch, and synthesize web content using tools like web_search, web_fetch, and web_research.2MIT