Skip to main content
Glama

斥候(veddata)

一个模型上下文协议(MCP)服务器:在网页上做数据源发现 —— 捕获网络请求、提取内嵌数据、分析脚本,并通过交互与监测点跟踪数据流向。

English

功能

  • 网络数据源:页面发出的全部请求与响应,按接口归并并给出字段结构

  • 内嵌数据:<script> 标签里的 JSON 与 window 全局变量

  • 脚本分析:列出与搜索已加载脚本、读取源码、追踪一个值出现在哪些位置

  • 交互:点击、输入、选择、滚动,按可见文本或 css=、//(XPath)选择器定位

  • 监测点:请求监测(匹配到指定模式即记录)与 JS 断点监测(命中时抓取作用域变量快照)

  • 文档结构:DOM 树快照、关键词定位、按路径定位,全部在服务端内存中完成

  • 工具链:一次调用顺序执行多个工具(支持等待与重复),并为每个动作给出因果差分

  • 页面事实:地址、标题、正文长度、关键词命中的原文片段、可见与隐藏的组件

  • 下载:发生下载时在消息中给出文件名、保存路径、状态与大小

  • 服务端留存:结果留在服务端,返回压缩摘要,明细随时按需获取

Related MCP server: Ruishu MCP

环境要求

  • Python 3.10 及以上

  • Chrome 或 Edge(默认以有窗口模式启动,并使用持久化配置目录保留登录状态)

安装与使用

一次性运行(无需安装,推荐):

{
  "mcpServers": {
    "veddata": {
      "command": "uvx",
      "args": ["veddata", "--response-dir", "E:\\path\\to\\out"]
    }
  }
}

或先安装再使用:uv tool install veddata(或 pip install veddata),然后

{
  "mcpServers": {
    "veddata": {
      "command": "veddata",
      "args": ["--response-dir", "E:\\path\\to\\out"]
    }
  }
}

从源码运行(开发用):

uv venv
uv pip install -e ".[dev]"

客户端配置(标准输入输出):

{
  "mcpServers": {
    "veddata": {
      "command": "E:\\path\\to\\veddata\\.venv\\Scripts\\python.exe",
      "args": ["-m", "veddata.server",
               "--profile-dir", "E:\\path\\to\\profile",
               "--response-dir", "E:\\path\\to\\out"]
    }
  }
}

配置项

参数

说明

--profile-dir <目录>

浏览器配置目录(持久化,保留登录状态)。默认 <用户数据根>/veddata/profile

--response-dir <目录>

ved_export、ved_screenshot 的落盘目录。未配置时这两个工具在写盘时报错

环境变量

说明

HEADLESS=true

无窗口模式(默认为有窗口)

BROWSER_PATH <路径>

指定浏览器可执行文件(填 edge 表示使用 Edge)

BROWSER_ADDRESS <地址>

接管已在运行的浏览器(调试协议地址),而不是自行启动

工具

会话与标签页

  • ved_open — 启动会话(自行启动一个普通 Chrome 并通过调试协议接管)

  • ved_status — 当前页面事实、标签页清单、已捕获条数、状态账本

  • ved_tabs、ved_tab_switch、ved_tab_close — 标签页清单、切换、关闭;关闭支持 tab="A,B" 或 tabs=["A","B"],最后一个标签不会被关掉

  • ved_close — 关闭浏览器并清空捕获数据

导航与读取

  • ved_goto — 导航,返回文档结构、数据源预览与页面事实

  • ved_fetch — 取当前页正文(分页)

  • ved_dom_tree、ved_dom_search、ved_dom_locate — 树快照、关键词、路径;快照默认跳过 svg sprite / 纯图标子树(<symbol>、<defs>、<path> 等)

  • ved_console — 执行 JS 或读取控制台消息

  • ved_cookies、ved_screenshot — cookie 与截图落盘;filter="sessionid,sid_tt" 只取指定的几个(命中时值不裁剪,用来取完整票据)

交互

  • ved_act — input、click、select、scroll;可单步或链式;target 支持可见文本、css= 或 //;定位只查询不点击(探测不产生副作用),未命中会给出最接近的可见文本;页面自跳后用 force=true 免去手动 ved_status

数据源

  • ved_apis、ved_inspect — 捕获清单与单条详情(请求头、响应体、字段结构)

  • ved_export、ved_export_all — 导出到磁盘(原始或精简)

  • ved_peek — 打开网址并直接返回数据源清单

  • ved_request — 重放已捕获的接口或发送自定义请求。params / headers 里传 null 表示删除该键;默认从浏览器同步 cookie(你显式给的 cookie 头优先);raw=true 是「原样发出」——不继承捕获记录的头/参数、不注入 cookie 与 XSRF,只发你给的东西。返回里回显实际发出的请求(最终 URL、每个请求头的来源、被删除 / 未发送的键),据此判断「哪些东西是必需的」

  • ved_search、ved_context、ved_trace_value — 跨网络、内嵌、脚本、全局变量的搜索与追踪

脚本

  • ved_list_scripts、ved_search_scripts、ved_script_source — 清单、搜索、源码;搜索零命中时按路径片段反查脚本,并列出已捕获请求里含这些片段的相关接口

监测点

  • ved_watch — 注册、列表、收集、删除;分为请求与 JS 两类;命中快照留在服务端

扫描

  • ved_scan — 一次性汇总网络、文档结构与内嵌数据

工具链

  • ved_chain — 顺序执行多个工具;等待支持毫秒、网络命中、网络静默、元素可见性;重复块支持一层

行为说明

  • 只陈述事实:输出说明有什么,不含判断与下一步提示;页面状态以地址、标题、正文长度、关键词命中片段、可见组件等事实呈现。

  • 结果留在服务端:返回压缩摘要与分页页脚,明细用 ved_apis、ved_inspect、ved_watch(collect) 等按需获取。

  • 状态观测:发生在工具之外的变化(页面自行跳转、标签页增删)之后,依赖活状态的工具会返回一条事实说明而不执行;ved_status、ved_tabs 会重新建立观测。

  • 人机协作:界面可见,登录或验证由用户在窗口中完成;工具照实报告当前页面事实。

  • 浏览器生命周期:自行启动的浏览器由 ved_close 关闭;通过 BROWSER_ADDRESS 接管的仅断开连接。

示例:没有接口文档的站点,怎么拿到下载直链

以 svgrepo 为例(该站点没有公开接口文档,AI 只能自己侦察)。一条工具链跑完:

[
  {"tool": "ved_open"},
  {"tool": "ved_goto", "args": {"url": "https://www.svgrepo.com/vectors/arrow/"}},
  {"tool": "ved_goto", "args": {"url": "https://www.svgrepo.com/svg/535197/arrow-u-up-left"}},
  {"tool": "ved_act",  "args": {"action": "click", "target": "css=a[href*='/download/']"}}
]

消息里直接给出事实(动作、因果、落盘位置):

[click] Clicked 'css=a[href*='/download/']' → +1 new APIs
    GET    https://www.svgrepo.com/_next/data/XJiZPe18H5paekHV…/tools.json
download: arrow-u-up-left-svgrepo-com.svg → E:\Downloads\arrow-u-up-left-svgrepo-com.svg (completed, 399 B)

开发

pytest tests -q
python scripts/smoke.py

许可

MIT License,详见 LICENSE。

免责声明

本项目仅用于学习、研究、技术交流与合法合规的自动化测试,不得用于任何违法、侵权或违背公序良俗的用途。使用前请完整阅读本声明;下载、安装、运行或以任何方式使用本项目,即视为已阅读、理解并同意本声明的全部内容。如不同意,请立即停止使用并删除本项目。

  1. 合规责任由使用者自负。使用者应自行确认并遵守:目标网站的服务条款、robots.txt 与使用规则;所在国家或地区的法律法规(包括但不限于网络安全、数据安全、个人信息保护、著作权、反不正当竞争、计算机信息系统相关法律);以及任何适用的行业规范与合同约定。

  2. 禁止用途。不得用于:未经授权访问、干扰或破坏他人系统与数据;绕过技术保护措施或访问控制;大规模抓取、囤积或转售他人数据;侵犯他人知识产权、商业秘密或个人隐私;任何形式的欺诈、骚扰、垃圾信息或其他恶意行为。

  3. 风险自担。使用本项目可能导致账号被封禁、访问受限、服务中断、数据丢失或其他损失,上述风险与后果由使用者自行承担。

  4. 无担保。本项目按「原样」(AS IS)提供,作者不对其适用性、可靠性、准确性、安全性或不侵权作出任何明示或默示的担保,也不承诺任何功能可用、持续可用或不会被目标站点检测。

  5. 责任限制。在适用法律允许的最大范围内,作者与贡献者不对因使用或无法使用本项目而产生的任何直接、间接、附带、特殊、惩罚性或后果性损害(包括但不限于利润损失、数据丢失、业务中断、第三方索赔、行政处罚或法律责任)承担任何责任。

  6. 不构成法律意见。本项目不提供任何法律意见或合规承诺;涉及合规判断请咨询专业律师。

  7. 对第三方的主张由使用者承担。因使用者的行为引发的任何第三方主张、争议、投诉或索赔,由使用者自行处理并承担全部责任,与作者及贡献者无关。

  8. 权利保留。作者保留随时修改、暂停或终止本项目的权利,且无需事先通知;本声明亦可随时更新,更新后的版本自发布之日起生效。

Available Tools

21 tools
scout_actA

Execute actions on the page: input, scroll, click, or select.

Two modes:

  1. Single step: scout_act("scroll", "bottom") scout_act("input", "python教程", target="搜索") scout_act("click", target="下一页") scout_act("select", "最多播放", target="综合排序")

  2. Chain: scout_act(actions=[ {"action": "input", "value": "python教程", "target": "搜索"}, {"action": "scroll", "value": "bottom"}, {"action": "click", "target": "最多播放"}, ])

Each step reports new API method + path inline so AI sees them immediately.

Inputs: value, action, target, actions. Returns: result

scout_apisA

List all captured API endpoints, optionally filtered by keyword.

Inputs: tab, keyword. Returns: result

scout_closeA

Close the browser and clear all captured data.

Kills the Chromium process on port 9222 and resets all state.

Returns: Status message.

Inputs: no named parameters. Returns: result

scout_contextC

Search all data sources for keyword, returning field paths and values.

Inputs: keyword (required), tab. Returns: result

scout_cookiesC

View cookies for the current page.

Inputs: tab, all_info, all_domains. Returns: result

scout_elementsB

List interactive page elements, repeated DOM containers, and common actions.

Scans for clickable buttons, links, inputs and detects repeated container structures (card layouts, list items, etc.).

Returns: Numbered list of interactive elements + DOM containers + Common Actions.

Inputs: no named parameters. Returns: result

scout_exportC

Export one or more captured API data sources.

Inputs: tab, index, format, indices, +1 more. Returns: result

scout_export_allB

Export all captured API data sources at once.

Inputs: tab, format, output_dir. Returns: result

scout_fetchA

Fetch full page content — scrolls to bottom, dumps innerText + AXTree links, writes to cache file, returns chunked segments.

First call scrolls to bottom and writes a JSON cache file. Subsequent calls with start_index read from cache (no re-scroll).

Inputs: tab, max_length, start_index. Returns: result

scout_gotoA

Navigate to a URL and capture API requests.

Starts network monitoring BEFORE navigating (required: listen.start() must precede any action that triggers requests). Returns page text and interactive elements.

Use new_tab=True to open in a new tab while keeping the current page.

Inputs: url (required), new_tab. Returns: result

scout_inspectC

Show full request and response details for one or more APIs.

Inputs: tab, index, detail, indices. Returns: result

scout_loginA

Wait for the user to manually log in via the browser window.

Detects login by polling cookies: if cookie names change or ≥2 values change simultaneously, login is detected. Works across all sites without site-specific logic.

Inputs: timeout. Returns: result

scout_openA

Open / manage the browser session.

Finds or launches a Chromium on port 9222. Clears old tabs and resets the API pool. Does NOT navigate — use scout_goto() for that.

Returns: Browser session status.

Inputs: no named parameters. Returns: result

scout_peekC

One-shot API discovery: open a URL, capture and inspect matching API.

Inputs: url (required), method, path_contains. Returns: result

scout_requestA

Send a test HTTP request — replay from captured API or fully custom.

Two modes:

  1. Replay: index=3, params='{"page":2}' — uses API #3's URL/method/headers.

  2. Manual: url="...", method="POST", body="..." — fully custom.

Cookies are auto-synced from the current browser tab.

Inputs: tab, url, body, index, +3 more. Returns: result

scout_scanA

Comprehensive page data source scanner with two modes.

MODE "all" — full page scan (default):

  1. Network APIs — XHR/Fetch requests already captured by the listener

  2. SSR Embedded JSON — window.INITIAL_STATE, NEXT_DATA, etc.

  3. DOM Containers — repeated HTML structures (card layouts, list items)

MODE "dom" — keyword-targeted DOM scan: Searches DOM for containers matching the keyword.

Inputs: url, mode, keyword. Returns: result

scout_screenshotC

Take a screenshot of the current page.

Inputs: name, full_page. Returns: result

scout_tab_closeA

Close browser tab(s) by CDP short ID and prune their API records.

Supports comma-separated IDs for batch close (e.g. "C724404D,5FD84E84"). Empty = close current tab.

Inputs: tab. Returns: result

scout_tabsA

List all open browser tabs with CDP short IDs.

Returns: Tab list with short IDs and current marker.

Inputs: no named parameters. Returns: result

scout_tab_switchA

Switch the active tab by CDP short ID (from scout_tabs output).

After switching, scout_goto() targets the new tab.

Inputs: tab (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 21 tools

    Disambiguation4/5

    Most tools have distinct purposes, but there is some overlap between scout_scan, scout_search, and scout_context, which all involve searching different data sources. Descriptions help differentiate but could still confuse an agent.

    Naming Consistency3/5

    All tools share the 'scout_' prefix, but the naming pattern mixes verbs (e.g., scout_act, scout_fetch) and nouns (e.g., scout_apis, scout_cookies). This inconsistency, while readable, could be more uniform.

    Tool Count4/5

    21 tools is slightly above the typical well-scoped range (3-15), but the domain of browser automation and API discovery justifies many of them. The count feels reasonable, not excessive.

    Completeness4/5

    The tool set covers navigation, interaction, API capture, data export, tab management, and login. Minor gaps like cookie management or local storage access exist, but core workflows are well-supported.

    Maintenance

    ActivitySlowing
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      A
      quality
      A
      maintenance
      An MCP server that bridges LLMs with dynamic real-world data by leveraging Chrome DevTools Protocol to intercept and reconstruct network traffic, enabling AI agents to extract high-quality structured data from complex web environments.
      3
      89
      -
    • A
      license
      Not graded
      quality
      D
      maintenance
      Secure, agent-driven web data extraction MCP server that extracts structured data from websites using APIs, RSS, and HTML without requiring a browser.
      Apache 2.0
    • A
      license
      A
      quality
      B
      maintenance
      An MCP server that provides passive and low-impact active reconnaissance tools for authorized bug bounty and security assessments, enabling LLMs to perform structured recon and generate reports.
      11
      Apache 2.0