Skip to main content
Glama
guanyuyan
by guanyuyan

web_batch

Fetches a batch of URLs concurrently through the complete tiered scraping pipeline, returning one result per URL. Handles anti-bot, login, and JS rendering with per-URL token budgets.

Instructions

并发抓取一批 URL,各自走完整分级链路,返回逐个结果。

Args: urls: 目标 URL 列表(最多 batch_max_urls 个,默认 20)。 max_tokens: 每个 URL 的正文 token 预算上限。 link_policy: 同 web_fetch。 format: 同 web_fetch(markdown / text / json)。 headers: 所有 URL 共用的自定义请求头;传入时不走共享缓存。 cookies: 所有 URL 共用的自定义 cookie;传入时不走共享缓存。 timeout: 单次请求超时秒数。 deadline_ms: 整批共用的总预算(毫秒),不传表示不限。预算耗尽后还没开始的 子任务直接返回 timeout(diagnostics 为 deadline:exceeded),不再占用网络与浏览器。 bypass_cache: 跳过并绕过响应缓存。 debug: 每条结果附带 diagnostics(决策轨迹与计数,不含页面内容)。 session: 登录态会话名(默认 default);对该批所有 URL 生效。 proxy: 该批所有 URL 共用的代理;不传则按代理池分配,未配池时沿用全局配置。 显式传入代理时该批不读也不写共享缓存。 ctx: MCP 上下文,由框架注入,用于逐条上报进度(客户端请求进度时才发通知)。

Input Schema

NameRequiredDescriptionDefault
urlsYes
debugNo
proxyNo
formatNomarkdown
cookiesNo
headersNo
sessionNodefault
timeoutNo
max_tokensNo
deadline_msNo
link_policyNointernal
bypass_cacheNo

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed9 schema fields changedv0.10.1
    • addedInput schema / properties / bypass_cache
      Added value: +{
      +  "default": false,
      +  "title": "Bypass Cache",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / cookies
      Added value: +{
      +  "anyOf": [
      +    {
      +      "additionalProperties": {
      +        "type": "string"
      +      },
      +      "type": "object"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Cookies"
      +}
    • addedInput schema / properties / deadline_ms
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "integer"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Deadline Ms"
      +}
    • addedInput schema / properties / debug
      Added value: +{
      +  "default": false,
      +  "title": "Debug",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / format
      Added value: +{
      +  "default": "markdown",
      +  "title": "Format",
      +  "type": "string"
      +}
    • addedInput schema / properties / headers
      Added value: +{
      +  "anyOf": [
      +    {
      +      "additionalProperties": {
      +        "type": "string"
      +      },
      +      "type": "object"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Headers"
      +}
    • addedInput schema / properties / proxy
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Proxy"
      +}
    • addedInput schema / properties / session
      Added value: +{
      +  "default": "default",
      +  "title": "Session",
      +  "type": "string"
      +}
    • addedInput schema / properties / timeout
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Timeout"
      +}
  2. First observedv0.1.0

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: it discloses concurrency, that deadline exhaustion returns per-subtask timeout with diagnostics 'deadline:exceeded' without consuming network/browser resources, that custom headers/cookies/proxy disable shared cache, and that progress is only reported when the client requests it. These are real behavioral traits beyond the schema. It omits rate-limit/concurrency-cap behavior and partial-failure semantics for the batch as a whole.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded summary sentence followed by a per-argument list, so the reader gets purpose before detail. The argument block is long but each entry is one line and earns its place by documenting otherwise-undocumented params. Minor verbosity in the ctx entry, which is framework-injected and of little use to an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value detail is rightly omitted, and the description still states 'returns per-URL results'. For a 12-param concurrent tool with no annotations, the description covers scoping, deadline behavior, cache effects, and session/proxy propagation adequately. It could say more about failure isolation across the batch and concurrency limits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% across 12 params, so the description must compensate and largely does: every parameter is explained, including non-obvious semantics like deadline_ms being a batch-wide budget, session applying to all URLs, and the cache side-effects of headers/cookies/proxy. The 'same as web_fetch' references for link_policy and format defer meaning to another tool's docs rather than stating it here, which is the main gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line gives a specific verb+resource: concurrent fetch of a batch of URLs, each through the full tiered pipeline, returning per-URL results. It's clearly distinct from web_fetch (batch vs single) though it never explicitly names the boundary. Sibling differentiation is implied via 'link_policy/format 同 web_fetch' rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'batch' vs the singular web_fetch sibling, but there is no explicit 'use this when you have multiple URLs, use web_fetch for one'. No when-not guidance, no prerequisites. The cache-implication notes (headers/cookies/proxy disable shared cache) provide operational context but not selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.