Skip to main content
Glama

Scrape a page

pagesift_scrape
Read-only

Fetch a single known URL and return clean Markdown or structured content—HTML, links, images, or metadata—for AI and RAG workflows.

Instructions

Fetch one URL and return clean content: markdown by default, or html, rawHtml, links, images and metadata. Use it when you know the page you need. For several known URLs use pagesift_batch_scrape; to find URLs on a site use pagesift_map; to collect a whole site use pagesift_crawl. pagesift respects robots.txt and reports blocked pages as errors instead of returning their content.

Input Schema

NameRequiredDescriptionDefault
urlYesThe page to scrape.
proxyNoOnly basic and auto are supported; pagesift does not bypass bot protection.
engineNopagesift extension: fetch with plain HTTP, a headless browser, or decide automatically (default).
maxAgeNoAccept a cached copy at most this many milliseconds old; 0 forces a live fetch.
mobileNo
actionsNoBrowser actions to run before capturing the page.
formatsNoOutput formats. Defaults to ["markdown"].
parsersNo
profileNo
timeoutNoTimeout in milliseconds.
waitForNoMilliseconds to wait before capturing the page.
locationNo
lockdownNo
redactPIINo
pdfOptionsNo
excludeTagsNo
ifNoneMatchNopagesift extension: ETag for a conditional request.
includeTagsNo
jsonOptionsNoOptions for the json format.
queryOptionsNoOptions for the query format.
storeInCacheNo
ifModifiedSinceNopagesift extension: HTTP date for a conditional request.
onlyMainContentNoKeep only the main content. Defaults to true.
screenshotOptionsNo
zeroDataRetentionNo
removeBase64ImagesNo
skipTlsVerificationNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.81.2

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, open-world behavior, so the description is not obligated to restate safety. It adds genuinely new context: robots.txt is respected and blocked pages surface as errors rather than content, which prevents an agent from misreading an empty result as a successful scrape. It does not cover caching behavior or rate/latency characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: capability first, sibling routing second, a behavioral caveat last. No filler, and the most important information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex, 27-parameter tool with no output schema, the description covers purpose, alternatives, default output, and the robots.txt failure mode, which is enough to call it correctly in the common case. Gaps remain around caching/conditional-request semantics and browser actions, but they are secondary for the typical single-page scrape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 27 parameters and only ~48% schema description coverage, the description needs to compensate and largely does not. It only clarifies the formats parameter (markdown is the default, plus html/rawHtml/links/images/metadata); high-impact options like proxy restrictions, engine, actions, redactPII, lockdown, zeroDataRetention, storeInCache, and conditional-request fields are left to the schema or undocumented entirely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (fetch one URL, return clean content) and enumerates the default and alternative output formats. It also names each sibling it is not, so an agent can distinguish single-page scraping from batch, map, and crawl without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit selection rule: use this when you already know the page, use pagesift_batch_scrape for several known URLs, pagesift_map to discover URLs, pagesift_crawl to collect a whole site. Every alternative is named with the condition that selects it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.