网页抓取服务器
一个功能强大的MCP服务器，提供不受robots.txt限制的网页抓取功能，支持多种HTTP方法和自定义请求设置。 A powerful MCP server that offers web scraping capabilities unrestricted by robots.txt, supports multiple HTTP methods and custom request Settings.## 工具列表 Tool List
本MCP服务封装下列工具，可让模型通过标准化接口调用以下功能。 本MCP服务封装下列工具，可让模型通过标准化接口调用以下功能。
工具 Tool | 描述 Description |
fetch | Fetch content from a URL without following robots.txt restrictions. Returns the response body as text. |
检查服务 ## Inspector
工具在线测试： https://mcp.xiaobenyang.com/inspector/1777316659329027
Online Tool test https://mcp.xiaobenyang.com/inspector/1777316659329027
服务配置 MCP Server Config
如何获取 XBY-APIKEY ？ How to get XBY-APIKEY ?
访问小笨羊科技网站 https://xiaobenyang.com，注册用户即可获得APIKEY Visit XiaoBenYang website https://xiaobenyang.com, register and get the APIKEY.
SSE
{
"mcpServers": {
"网页抓取服务器": {
"headers": {
"XBY-APIKEY": "<YOUR_XBY_APIKEY>"
},
"type": "sse",
"url": "https://mcp.xiaobenyang.com/1777316659329027/sse"
}
}
}
STREAMABLE HTTP
{
"mcpServers": {
"网页抓取服务器": {
"headers": {
"XBY-APIKEY": "<YOUR_XBY_APIKEY>"
},
"type": "streamable_http",
"url": "https://mcp.xiaobenyang.com/1777316659329027/mcp"
}
}
}
STDIO
{
"mcpServers": {
"网页抓取服务器": {
"command": "npx",
"args": [
"-y",
"xiaobenyang-mcp"
],
"env": {
"XBY_APIKEY": "<YOUR_XBY_APIKEY>",
"mcpId": "1777316659329027",
},
"transport": "stdio"
}
}
}