这是一个模型上下文协议 (MCP) 服务器实现，集成了 Olostep 的网页抓取、内容提取和搜索功能。 要设置 Olostep MCP 服务器，您需要一个 API 密钥。您可以通过在 Olostep 网站 上注册来获取 API 密钥。
以 HTML、Markdown、JSON 或纯文本格式抓取网站内容（带有可选解析器）
基于解析器的网页搜索，提供结构化结果
AI 回答，包含引用和可选的 JSON 格式输出
批量抓取多达 10,000 个 URL
从起始 URL 开始的自主网站爬取
网站 URL 发现和映射（带有包含/排除过滤器）
针对特定国家的请求路由，用于地理定位内容
可配置的 JavaScript 重型网站等待时间
全面的错误处理和报告
简单的 API 密钥配置
Related MCP server: Parallel Task MCP
有多种方式可以连接到 Olostep MCP 服务器。选择最适合您工作流程的一种。
最简单的方法——无需本地安装。直接连接到我们托管的 MCP 服务器：
https://mcp.olostep.com/mcp身份验证通过 Authorization 标头中的 Bearer 令牌使用您的 Olostep API 密钥完成。有关配置示例，请参阅下方的 客户端设置 部分。
拉取并运行官方 Docker 镜像：
docker pull olostep/mcp-server
docker run -i --rm \
-e OLOSTEP_API_KEY="your-api-key" \
olostep/mcp-server如果您更喜欢从源代码自行构建镜像：
git clone https://github.com/olostep/olostep-mcp-server.git
cd olostep-mcp-server
npm install
npm run build
docker build -t olostep/mcp-server:local .
docker run -i --rm -e OLOSTEP_API_KEY="your-api-key" olostep/mcp-server:local使用 npx 运行，无需任何安装：
env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp在 Windows (PowerShell) 上：
$env:OLOSTEP_API_KEY = "your-api-key"; npx -y olostep-mcp在 Windows (CMD) 上：
set OLOSTEP_API_KEY=your-api-key && npx -y olostep-mcp或者全局安装：
npm install -g olostep-mcp最简单的方法是使用远程端点。在您的项目根目录中创建或编辑 .cursor/mcp.json：
{
"mcpServers": {
"olostep": {
"url": "https://mcp.olostep.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY_HERE"
}
}
}
}替代方案（本地）： 转到 Cursor 设置 > 功能 > MCP 服务器，点击“+ 添加新 MCP 服务器”：
名称： olostep
类型： command
命令： env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp
将其添加到您的 claude_desktop_config.json 中：
{
"mcpServers": {
"mcp-server-olostep": {
"command": "npx",
"args": ["-y", "olostep-mcp"],
"env": {
"OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}替代方案（Docker）：
{
"mcpServers": {
"olostep": {
"command": "docker",
"args": [
"run", "-i", "--rm",
"-e", "OLOSTEP_API_KEY=YOUR_API_KEY_HERE",
"olostep/mcp-server"
]
}
}
}或者通过设备终端中的 Smithery CLI 安装：
npx -y @smithery/cli install @olostep/olostep-mcp-server --client claude将远程端点添加到您的 Claude Code MCP 配置中：
{
"mcpServers": {
"olostep": {
"url": "https://mcp.olostep.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY_HERE"
}
}
}
}替代方案（本地）：
{
"mcpServers": {
"olostep": {
"command": "npx",
"args": ["-y", "olostep-mcp"],
"env": {
"OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}将其添加到您的 ./codeium/windsurf/model_config.json 中：
{
"mcpServers": {
"olostep": {
"serverUrl": "https://mcp.olostep.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY_HERE"
}
}
}
}替代方案（本地）：
{
"mcpServers": {
"mcp-server-olostep": {
"command": "npx",
"args": ["-y", "olostep-mcp"],
"env": {
"OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}将其添加到您的 .vscode/mcp.json 中：
{
"servers": {
"olostep": {
"type": "http",
"url": "https://mcp.olostep.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY_HERE"
}
}
}
}替代方案（本地）：
{
"servers": {
"olostep": {
"type": "stdio",
"command": "npx",
"args": ["-y", "olostep-mcp"],
"env": {
"OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}选项 1：一键安装（推荐）
打开 Metorial 仪表板
导航到 MCP 服务器目录
搜索“Olostep”
点击“安装”并输入您的 API 密钥
选项 2：手动配置
将此添加到您的 Metorial MCP 服务器配置中：
{
"olostep": {
"command": "npx",
"args": ["-y", "olostep-mcp"],
"env": {
"OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
}
}
}之后，Olostep 工具将在您的 Metorial AI 聊天中可用。
OLOSTEP_API_KEY：您的 Olostep API 密钥（必需）
ORBIT_KEY：用于使用 Orbit 路由请求的可选密钥。
scrape_website)从单个 URL 提取内容。支持多种格式和 JavaScript 渲染。
{
"name": "scrape_website",
"arguments": {
"url_to_scrape": "https://example.com",
"output_format": "markdown",
"country": "US",
"wait_before_scraping": 1000,
"parser": "@olostep/amazon-product"
}
}url_to_scrape：您要抓取的网站 URL（必需）
output_format：选择格式（html、markdown、json 或 text） - 默认：markdown
country：可选的国家代码（例如 US、GB、CA），用于特定位置的抓取
wait_before_scraping：抓取前的等待时间（毫秒，0-10000）
parser：用于专门提取的可选解析器 ID
{
"content": [
{
"type": "text",
"text": "{\n \"id\": \"scrp_...\",\n \"url\": \"https://example.com\",\n \"markdown_content\": \"# ...\",\n \"html_content\": null,\n \"json_content\": null,\n \"text_content\": null,\n \"status\": \"succeeded\",\n \"timestamp\": \"2025-11-14T12:34:56Z\",\n \"screenshot_hosted_url\": null,\n \"page_metadata\": { }\n}"
}
]
}search_web)搜索网页以获取给定查询，并获得结构化结果（非 AI，基于解析器）。
{
"name": "search_web",
"arguments": {
"query": "your search query",
"country": "US"
}
}query：搜索查询（必需）
country：用于本地化结果的可选国家代码（默认：US）
代表基于解析器结果的结构化 JSON（作为文本）
answers)搜索网页并以您想要的 JSON 结构返回 AI 驱动的答案，包含来源和引用。
{
"name": "answers",
"arguments": {
"task": "Who are the top 5 competitors to Acme Inc. in the EU?",
"json": "Return a list of the top 5 competitors with name and homepage URL"
}
}task：使用网页数据回答的问题或任务（必需）
json：可选的 JSON 模式/对象或所需输出形状的简短描述
answer_id、object、task、result（如果提供则为 JSON）、sources、created
batch_scrape_urls)同时抓取多达 10,000 个 URL。非常适合大规模数据提取。
{
"name": "batch_scrape_urls",
"arguments": {
"urls_to_scrape": [
{"url": "https://example.com/a", "custom_id": "a"},
{"url": "https://example.com/b", "custom_id": "b"}
],
"output_format": "markdown",
"country": "US",
"wait_before_scraping": 500,
"parser": "@olostep/amazon-product"
}
}batch_id、status、total_urls、created_at、formats、country、parser、urls
create_crawl)启动一个 异步 爬取，通过跟踪链接自主发现并抓取整个网站。返回一个 crawl_id — 爬取在后台运行，此响应中不返回内容。您必须随后使用 crawl_id 调用 get_crawl_results 来轮询状态并检索抓取的页面（与 batch_scrape_urls + get_batch_results 相同的两步模式）。
{
"name": "create_crawl",
"arguments": {
"start_url": "https://example.com/docs",
"max_pages": 25,
"follow_links": true,
"output_format": "markdown",
"country": "US",
"parser": "@olostep/doc-parser"
}
}crawl_id、object、status、start_url、max_pages、follow_links、created、formats、country、parser
将此调用与
get_crawl_results配对 — 不要将crawl_id传递给get_batch_results（爬取和批处理是独立的资源）。
create_map)获取网站上的所有 URL。提取所有 URL 以进行发现和分析。
{
"name": "create_map",
"arguments": {
"website_url": "https://example.com",
"search_query": "blog",
"top_n": 200,
"include_url_patterns": ["/blog/**"],
"exclude_url_patterns": ["/admin/**"]
}
}map_id、object、url、total_urls、urls、search_query、top_n
get_webpage_content)以简洁的 Markdown 格式检索网页内容，支持 JavaScript 渲染。
{
"name": "get_webpage_content",
"arguments": {
"url_to_scrape": "https://example.com",
"wait_before_scraping": 1000,
"country": "US"
}
}url_to_scrape：要抓取的网页 URL（必需）
wait_before_scraping：开始抓取前等待的时间（毫秒，默认：0）
country：加载请求的居住国（例如 US、CA、GB）（可选）
{
"content": [
{
"type": "text",
"text": "# Example Website\n\nThis is the markdown content of the webpage..."
}
]
}get_website_urls)搜索并检索网站的相关 URL，按与您的查询的相关性排序。
{
"name": "get_website_urls",
"arguments": {
"url": "https://example.com",
"search_query": "your search term"
}
}url：要映射的网站 URL（必需）
search_query：用于对 URL 进行排序的搜索查询（必需）
{
"content": [
{
"type": "text",
"text": "Found 42 URLs matching your query:\n\nhttps://example.com/page1\nhttps://example.com/page2\n..."
}
]
}get_batch_results)使用其 batch_id 检索先前提交的批量抓取作业的结果。
{
"name": "get_batch_results",
"arguments": {
"batch_id": "batch_abc123"
}
}batch_id：从 batch_scrape_urls 返回的批处理 ID（必需）
batch_id、status（processing 或 completed）、total_urls、completed_urls、items（每个 URL 的抓取结果数组，包含 url、custom_id、markdown_content、html_content、json_content、text_content、status、page_metadata）
get_crawl_results)检索使用 create_crawl 启动的异步爬取的状态和抓取的页面。这是 create_crawl 的必需配套工具 — create_crawl 仅启动作业并返回 crawl_id；此工具是您实际获取发现的页面及其内容的方式。
{
"name": "get_crawl_results",
"arguments": {
"crawl_id": "crawl_abc123",
"formats": ["markdown"],
"items_limit": 20,
"cursor": 0
}
}crawl_id：从 create_crawl 返回的爬取 ID（必需）
formats：每页要检索的格式数组 — markdown、html、json、text（默认：["markdown"]）
items_limit：要检索内容的最大页面数，1–100（默认：20）
cursor：进入发现页面列表的分页游标（默认：0）
search_query：可选过滤器，用于按与查询的相关性对页面进行排名/选择
进行中时： crawl_id、status (in_progress)、pages_completed、pages_total 以及提示您在大约 10 秒后再次调用的 message。
完成时： crawl_id、status (completed)、pages_returned、next_cursor、has_more 以及一个 pages 数组，其中每个条目都有 url、custom_id 和请求的内容字段（markdown_content、html_content、json_content、text_content）。
服务器提供强大的错误处理：
针对 API 问题的详细错误消息
网络错误报告
身份验证失败处理
速率限制信息
错误响应示例：
{
"isError": true,
"content": [
{
"type": "text",
"text": "Olostep API Error: 401 Unauthorized. Details: {\"error\":\"Invalid API key\"}"
}
]
}MCP 服务器以 Docker 镜像形式提供：
Docker Hub: [olostep/mcp-server](https://hub.docker.com/r/olostep/mcp-server)
官方 Docker MCP 注册表: mcp/olostep（即将推出 - 增强了签名和 SBOM 的安全性）
GitHub 容器注册表: ghcr.io/olostep/olostep-mcp-server
Olostep MCP 服务器正在被添加到 Docker Desktop 的官方 MCP 工具包中，这意味着用户将能够：
在 Docker Desktop 的 MCP 工具包 UI 中发现它
一键安装
可视化配置
将其与任何兼容 MCP 的客户端（Claude Desktop、Cursor 等）一起使用
状态：正在提交至 Docker MCP 注册表
linux/amd64
linux/arm64
# Clone the repository
git clone https://github.com/olostep/olostep-mcp-server.git
cd olostep-mcp-server
# Build the image
npm install
npm run build
docker build -t olostep/mcp-server .
# Run locally
docker run -i --rm -e OLOSTEP_API_KEY="your-key" olostep/mcp-serverISC 许可证
Search the web and return AI-powered answers in the JSON structure you want, with sources and citations.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | Question or task to answer using web data. | |
| json | No | Optional JSON schema/object or a short description of the desired output shape. Example object: { "book_title": "", "author": "", "release_date": "" } |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'AI-powered answers' and 'sources and citations', which hints at synthesis and attribution, but lacks details on behavioral traits like rate limits, authentication needs, response format beyond JSON, or whether it performs web searches in real-time. For a tool with no annotations, this leaves significant gaps in understanding its operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, stating the core functionality in one sentence. Every phrase ('Search the web', 'return AI-powered answers', 'JSON structure you want, with sources and citations') contributes meaning without redundancy. It could be slightly more structured by separating usage hints, but it's efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a tool that performs web searches and AI synthesis, the description is incomplete. It doesn't cover critical aspects like response format details, error handling, limitations (e.g., search depth), or how 'sources and citations' are structured in the output. For a complex tool with 2 parameters, this leaves too much undefined for effective agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('task' and 'json') well. The description adds minimal value beyond the schema, mentioning 'JSON structure you want' which aligns with the 'json' parameter but doesn't provide additional semantics like examples or constraints. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Search the web and return AI-powered answers' with specific outputs ('JSON structure you want, with sources and citations'). It distinguishes from siblings like 'google_search' or 'scrape_website' by emphasizing AI-powered answer generation rather than raw search results or content extraction. However, it doesn't explicitly contrast with 'search_web' which might be similar.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'google_search', 'search_web', or 'get_webpage_content'. The description implies usage for AI-powered answers with structured JSON output, but doesn't specify scenarios where this is preferred over simpler search tools or when not to use it (e.g., for raw data vs. synthesized answers).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scrape up to 10k URLs at the same time. Perfect for large-scale data extraction.
| Name | Required | Description | Default |
|---|---|---|---|
| urls_to_scrape | Yes | JSON array of objects with "url" and optional "custom_id". | |
| output_format | No | Choose format for all URLs. Default: "markdown". | markdown |
| country | No | Optional country code for location-specific scraping. | |
| wait_before_scraping | No | Wait time in milliseconds before scraping each URL. | |
| parser | No | Optional parser ID for specialized extraction. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral disclosure. It mentions scale ('up to 10k URLs') but doesn't cover critical aspects like rate limits, authentication needs, error handling, or what 'scrape' entails (e.g., does it extract text, metadata, full HTML?). The phrase 'at the same time' hints at concurrency but lacks specifics. For a batch operation tool with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two sentences) and front-loaded with the core functionality. Every word earns its place: first sentence defines the tool, second provides usage context. No wasted words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (batch scraping with 5 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what 'scrape' returns (e.g., content, status codes), how errors are handled for partial failures, or performance considerations. For a tool that could involve significant processing and network usage, more context is needed to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no parameter-specific information beyond implying the 'urls_to_scrape' parameter supports batch operations. No additional semantics, constraints, or usage examples are provided. Baseline 3 is appropriate when schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Scrape up to 10k URLs at the same time' specifies the verb (scrape) and resource (URLs) with a quantitative limit. It distinguishes from siblings like 'scrape_website' (singular) and 'get_webpage_content' (single page) by emphasizing batch capability. However, it doesn't explicitly differentiate from 'create_crawl' which might also handle multiple URLs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage context: 'Perfect for large-scale data extraction' suggests when to use this tool (for bulk operations). However, it lacks explicit guidance on when NOT to use it or alternatives (e.g., use 'scrape_website' for single URLs, 'get_webpage_content' for simpler extraction). No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Autonomously discover and scrape entire websites by following links from a start URL.
| Name | Required | Description | Default |
|---|---|---|---|
| start_url | Yes | Starting URL for the crawl. | |
| max_pages | No | Maximum number of pages to crawl. | |
| follow_links | No | Whether to follow links found on pages. | |
| output_format | No | Format for scraped content. Default: "markdown". | markdown |
| country | No | Optional country code for location-specific crawling. | |
| parser | No | Optional parser ID for specialized content extraction. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions autonomous discovery and link-following, it lacks critical behavioral details like rate limits, authentication requirements, potential for being blocked by websites, or what happens when max_pages is reached. The description doesn't explain what 'scrape' entails beyond content extraction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's front-loaded with the main action and resource, making it immediately understandable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex crawling tool with 6 parameters and no annotations or output schema, the description is insufficient. It doesn't address important contextual aspects like what the tool returns (scraped content format, error handling), performance characteristics, or limitations of autonomous website discovery.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema. The baseline score of 3 reflects adequate parameter documentation through the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('discover and scrape entire websites') and resource ('websites'), distinguishing it from siblings like 'scrape_website' or 'get_webpage_content' by emphasizing autonomous link-following behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'scrape_website' or 'batch_scrape_urls'. It mentions following links but doesn't specify scenarios where this comprehensive crawling approach is preferred over targeted scraping.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get all URLs on a website. Extract URLs for discovery and site analysis.
| Name | Required | Description | Default |
|---|---|---|---|
| website_url | Yes | Website URL to extract links from. | |
| search_query | No | Optional search query to filter URLs (e.g., "blog"). | |
| top_n | No | Optional limit for number of URLs returned. | |
| include_url_patterns | No | Optional glob patterns to include (e.g., "/blog/**"). | |
| exclude_url_patterns | No | Optional glob patterns to exclude (e.g., "/admin/**"). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool extracts URLs but doesn't describe how it works (e.g., crawling depth, handling of dynamic content, rate limits, authentication needs, or error conditions). The phrase 'Get all URLs' might imply comprehensive extraction, but without behavioral details, the agent lacks transparency about what to expect from the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise and front-loaded: the first sentence 'Get all URLs on a website' directly states the core purpose. The second sentence adds context about use cases without redundancy. Every word earns its place, making it efficient and easy to parse for an AI agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no annotations, no output schema), the description is insufficient. It doesn't explain what the tool returns (e.g., list structure, error formats), behavioral constraints, or how it differs from similar siblings. For a URL extraction tool with multiple filtering options, more context is needed to guide effective use, especially without annotations or output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, providing clear documentation for all 5 parameters. The description adds minimal value beyond this, only implying URL extraction without detailing parameter interactions or usage examples. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, though the description could have enhanced understanding with practical context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get all URLs on a website' specifies the verb (get/extract) and resource (URLs from a website). It distinguishes from some siblings like 'get_webpage_content' (which gets content rather than URLs) and 'google_search' (which searches the web rather than extracting from a specific site). However, it doesn't explicitly differentiate from 'get_website_urls' (which might have similar functionality), keeping it from a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions 'discovery and site analysis' as general use cases, but doesn't specify when to choose this over siblings like 'get_website_urls', 'scrape_website', or 'create_crawl'. There's no mention of prerequisites, limitations, or comparative advantages, leaving the agent with minimal usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve content of a webpage in markdown
| Name | Required | Description | Default |
|---|---|---|---|
| url_to_scrape | Yes | The URL of the webpage to scrape. | |
| wait_before_scraping | No | Time to wait in milliseconds before starting the scrape. | |
| country | No | Residential country to load the request from (e.g., US, CA, GB). Optional. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'retrieve' and 'scrape', implying a read-only operation, but doesn't address potential issues like rate limits, authentication needs, error handling, or what happens with dynamic content. This is inadequate for a scraping tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes the output format, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a scraping tool with no annotations and no output schema, the description is incomplete. It doesn't explain return values (e.g., structure of markdown content), error conditions, or behavioral traits like handling of JavaScript-rendered pages. This leaves significant gaps for an AI agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the schema fully documents all three parameters. The description adds no additional meaning beyond what's in the schema, such as explaining why 'wait_before_scraping' might be needed or how 'country' affects the scrape. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'retrieve' and resource 'content of a webpage', specifying the output format 'in markdown'. However, it doesn't differentiate from sibling tools like 'scrape_website' or 'batch_scrape_urls', which likely have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'scrape_website' or 'batch_scrape_urls'. The description lacks context about use cases, prerequisites, or exclusions, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search and retrieve relevant URLs from a website
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL of the website to map. | |
| search_query | Yes | The search query to sort URLs by. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'search and retrieve' but doesn't explain how the search works (e.g., depth, scope, or limitations), what 'relevant' means, potential rate limits, or authentication needs. This leaves significant gaps for a tool that interacts with external websites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded and appropriately sized for its purpose, earning full marks for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of web interaction tools and the lack of annotations and output schema, the description is incomplete. It doesn't cover behavioral aspects like error handling, return format, or limitations, which are crucial for an agent to use this tool effectively in real-world scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('url' and 'search_query') adequately. The description implies these parameters are used for searching and retrieving, but doesn't add meaningful semantic context beyond what the schema provides, such as examples or constraints on the search query.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('search and retrieve') and resource ('relevant URLs from a website'). However, it doesn't explicitly distinguish this tool from sibling tools like 'scrape_website' or 'search_web', which might have overlapping functionality, preventing a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple sibling tools like 'scrape_website', 'google_search', and 'search_web', there's no indication of context, prerequisites, or exclusions, leaving the agent to guess based on tool names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve structured data from Google search results
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | The search query to perform | |
| country | No | Country code for localized results (e.g., US, GB) | US |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves structured data, implying a read-only operation, but doesn't cover critical aspects like rate limits, authentication needs, error handling, or what 'structured data' entails (e.g., format, fields). This leaves significant gaps for a tool interacting with an external service.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place by specifying the action, data type, and source. No fluff or redundancy is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a search tool with no annotations and no output schema, the description is incomplete. It doesn't explain the return values (what 'structured data' includes), error conditions, or operational constraints like rate limits. For a tool that likely involves external API calls, more context is needed to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents both parameters ('query' and 'country'). The description adds no additional meaning beyond what's in the schema, such as query formatting examples or country code implications. Baseline 3 is appropriate when the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Retrieve') and resource ('structured data from Google search results'), making the purpose immediately understandable. It distinguishes this from siblings like 'search_web' by specifying Google as the source and structured data as the output. However, it doesn't explicitly contrast with all siblings (e.g., 'answers' might also retrieve information).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'search_web' or 'answers'. It doesn't mention prerequisites, constraints, or typical use cases. The agent must infer usage from the name and description alone without explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract content from a single URL. Supports multiple formats and JavaScript rendering.
| Name | Required | Description | Default |
|---|---|---|---|
| url_to_scrape | Yes | The URL of the website you want to scrape. | |
| output_format | No | Choose format ("html", "markdown", "json", or "text"). Default: "markdown" | markdown |
| country | No | Optional country code (e.g., US, GB, CA) for location-specific scraping. | |
| wait_before_scraping | No | Wait time in milliseconds before scraping (0-10000). Useful for dynamic content. | |
| parser | No | Optional parser ID for specialized extraction (e.g., "@olostep/amazon-product"). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'JavaScript rendering' (useful context) and 'Supports multiple formats' (output behavior), but lacks critical details: whether scraping respects robots.txt, rate limits, authentication needs, error handling, or what 'extract content' specifically means. For a scraping tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two short sentences) and front-loaded with the core purpose. Every sentence adds value: first states the main action, second adds key capabilities. No wasted words, though it could be slightly more structured for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no annotations, and no output schema, the description is moderately complete. It covers the basic purpose and key features (formats, JavaScript), but lacks details on scraping behavior, error cases, or output structure. For a tool with this complexity and no structured safety/behavior annotations, it should provide more context about limitations or typical use cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are well-documented in the schema. The description adds minimal value beyond the schema: it implies format support ('Supports multiple formats') and JavaScript capability, but doesn't explain parameter interactions or provide additional semantic context. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Extract content from a single URL' specifies the verb and resource. It distinguishes from sibling 'batch_scrape_urls' by emphasizing 'single URL' and from 'get_webpage_content' by mentioning format support and JavaScript rendering. However, it doesn't explicitly contrast with all siblings like 'google_search' or 'search_web'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through 'single URL' (vs. batch) and 'JavaScript rendering' (for dynamic content), but doesn't provide explicit when-to-use guidance or alternatives. It mentions 'Supports multiple formats' which suggests format flexibility, but no clear exclusions or comparisons to siblings like 'get_webpage_content' are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search the web for a given query and return structured results (non-AI, parser-based).
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Search query | |
| country | No | Optional country code for localized results (e.g., US, GB). | US |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'parser-based' and 'non-AI' which adds some behavioral context, but fails to disclose critical traits like rate limits, authentication needs, result format, pagination, or error handling for a web search tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words, clearly front-loading the core functionality. Every part earns its place by specifying the action, resource, and method.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of web search and lack of annotations or output schema, the description is incomplete. It omits details on result structure, limitations, error cases, and how it differs from siblings, leaving significant gaps for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('query' and 'country'). The description adds no additional meaning beyond what the schema provides, such as query formatting tips or country code examples, meeting the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Search') and resource ('the web'), specifying it returns structured results via parser-based (non-AI) methods. It distinguishes from AI-based search tools but doesn't explicitly differentiate from sibling tools like 'google_search' or 'answers'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'google_search' or 'answers', nor does it mention prerequisites or exclusions. The description implies a general web search context but lacks explicit usage instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0answersbatch_scrape_urlscreate_crawlcreate_mapget_webpage_content2 fields changedInput schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#"Input schema / additionalPropertiesAdded value: +falseget_website_urls2 fields changedInput schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#"Input schema / additionalPropertiesAdded value: +falsegoogle_search2 fields changedInput schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#"Input schema / additionalPropertiesAdded value: +falsescrape_websitesearch_webget_webpage_contentget_website_urlsgoogle_searchScored across 9 tools
There is significant overlap between tools like 'answers', 'google_search', and 'search_web' for web search functionality, and between 'scrape_website', 'get_webpage_content', and 'batch_scrape_urls' for content extraction. However, the descriptions help clarify some distinctions, such as AI-powered vs. parser-based search or single vs. batch scraping.
Most tools follow a consistent verb_noun pattern (e.g., 'create_crawl', 'get_webpage_content', 'scrape_website'), with only minor deviations like 'answers' (a noun alone) and 'google_search' (noun_verb). The naming is generally readable and predictable across the set.
With 9 tools, the count is reasonable for a web data extraction and search server. It covers multiple aspects of the domain without being overwhelming, though some overlap suggests potential consolidation could refine the scope slightly.
The toolset provides comprehensive coverage for web search, content scraping, and URL discovery, including batch operations and autonomous crawling. Minor gaps might include more advanced filtering or data processing tools, but core workflows are well-supported without dead ends.
Domain/IP, web capture, Google search/maps, YouTube, Instagram, TikTok, X and Amazon data jobs
Stealth scraping & search. Bypasses Cloudflare, DataDome & LinkedIn via Cyborg HITL approach.
Emails, phones and social links from any list of websites. Pay only for sites with a contact.
Web search, read any page, verify or find emails, screenshots, keyword data. One key, pay per call.
