Skip to main content
Glama
CaptureKit MCP Server

CaptureKit MCP Server

Official

Website Crawler

website_crawler
Read-only

Fetches raw HTML from any webpage in request mode without rendering, enabling scraping, audits, monitoring, and link or metadata extraction.

Instructions

Website Crawler. Website Crawler API fetches raw HTML from any webpage in request mode (no rendering) for fast scraping, audits, monitoring, and link/metadata extraction. Credit cost: 1 credit per call.

Input Schema

NameRequiredDescriptionDefault
queryYesA website URL with either the http or https protocol.
fieldsNoComma-separated list of fields to keep in the response, with dot notation for nested fields (e.g. name,location.city). Does not change the credit cost.
allow_redirectsNoIf set to false, do not follow redirects. Default: true.
return_page_sourceNoIf set to true, the response will be the HTML source of the page. Default: false.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, openWorld, and non-destructive, so the safety profile is covered. The description adds two meaningful facts beyond that: the request-mode/no-rendering behavior (so dynamically rendered content will not be fetched) and the cost of 1 credit per call. It does not describe pagination, rate limits, or error behavior on unreachable URLs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core operation, then modes, then cost. The only inefficiency is the redundant opening 'Website Crawler. Website Crawler API...' which restates the title twice before saying anything new.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter read-only fetch tool with full schema coverage and no output schema, the description covers operation mode, typical uses, and cost. The main gap is the default response shape, since it is unclear what comes back when 'return_page_source' is false, but that is partly addressed by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters including 'query', 'fields', 'allow_redirects', and 'return_page_source' are already documented in the schema. The description adds no extra parameter detail such as URL format constraints or how 'fields' interacts with the HTML-source mode. Baseline 3 applies when the schema carries the semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it fetches raw HTML from any webpage in request mode. The parenthetical '(no rendering)' and the listed use cases (scraping, audits, monitoring, link/metadata extraction) make the operation concrete. It does not name any sibling tool that would do a rendering-based capture instead, so differentiation is only implied.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use cases are listed (fast scraping, audits, monitoring, link/metadata extraction) and 'no rendering' hints that JavaScript-heavy pages need a different tool. However, it never names an alternative sibling such as capture or content, nor does it state when this tool is the wrong choice. The guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools