Skip to main content
Glama

scrape_incremental

Refresh a documentation index by fetching the sitemap, scraping new or failed pages, and re-indexing only pages with changed content.

Instructions

Incremental scrape: fetch the sitemap, scrape any new pages not previously indexed, retry previously failed pages, and re-check existing pages for content changes. Only re-indexes pages whose content has actually changed.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
concurrencyNoNumber of parallel browser tabs (1-5). Default: 3.
language_filterNoLanguage to scrape. Same codes as scrape_full. Default: 'en'.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.0

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and does well: it discloses that already-indexed pages are only re-indexed when content actually changed, and that previously failed pages are retried. It omits write/mutation scope (it updates an index), permission needs, and runtime/long-running behavior, so it is not fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence, front-loaded with the tool's identity and then the four pipeline stages in order. Every clause conveys distinct behavior; nothing is redundant or padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a stateful scrape tool with no output schema, the description explains the operational lifecycle well enough to call it correctly. It could go further by indicating the returned result or how progress is observed (e.g., via get_scrape_status), but the core behavior is covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with only two optional params, so the schema already documents concurrency and language_filter fully. The description adds no syntax, default, or interaction detail beyond that, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb+resource ('Incremental scrape') and enumerates the exact pipeline: fetch sitemap, scrape new pages, retry failed pages, re-check existing pages. The 'incremental' framing, combined with 'not previously indexed', clearly separates it from the scrape_full sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives strong context for when this tool applies — for updating an existing index rather than building one — and implies the alternative is a full scrape. However, it never explicitly says 'use this instead of scrape_full when X', leaving the routing decision partly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.