Skip to main content
Glama
Sign In
webscraping-ai

WebScraping-AI MCP Server

Official
by webscraping-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

Get Page Text

webscraping_ai_text
Read-only

Extract a web page’s main content as structured Markdown, stripping boilerplate while preserving headings, lists, tables, and links for reading or LLM input.

Instructions

Get a web page's main content as Markdown, with boilerplate stripped and structure (headings, lists, tables, links) kept. Best for reading a page or passing it to an LLM. API docs: https://webscraping.ai/docs#text

Input Schema

NameRequiredDescriptionDefault
jsNoExecute on-page JavaScript using a headless browser (false by default).
urlYesURL of the target page.
proxyNoType of proxy: datacenter, residential, or stealth (datacenter by default). Residential and stealth requests cost more than datacenter — see the pricing page.datacenter
deviceNoType of device emulation.
countryNoCountry of the proxy to use (US by default).
timeoutNoMaximum web page retrieval time in ms (20000 by default, maximum is 30000).
wait_forNoCSS selector to wait for before returning the page content.
js_scriptNoCustom JavaScript code to execute on the target page.
js_timeoutNoMaximum JavaScript rendering time in ms (3000 by default).
text_formatNoFormat of the text response.json
custom_proxyNoYour own proxy URL in "http://user:password@host:port" format.
error_on_404NoReturn error on 404 HTTP status on the target page (false by default).
return_linksNoReturn links from the page body text.
error_on_redirectNoReturn error on redirect on the target page (false by default).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv1.4.0
    • removedInput schema / additionalProperties
      Removed value: -false
    • changedInput schema / properties / proxy / description
      Previous value: -"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Residential and stealth requests cost more than datacenter — see the pricing page."
  2. Changed1 schema field changedv1.2.2
    • addedInput schema / additionalProperties
      Added value: +false
  3. Changed1 schema field changedv1.0.5
    • changedInput schema / properties / country / enum
      Previous value: -[
-  "us",
-  "gb",
-  "de",
-  "it",
-  "fr",
-  "ca",
-  "es",
-  "ru",
-  "jp",
-  "kr",
-  "in"
-]New value: +[
+  "us",
+  "gb",
+  "de",
+  "it",
+  "fr",
+  "ca",
+  "es",
+  "ru",
+  "jp",
+  "kr",
+  "in",
+  "hk",
+  "tr"
+]
  4. Changed2 schema fields changedv1.0.4
    • changedInput schema / properties / proxy / description
      Previous value: -"Type of proxy, datacenter or residential (datacenter by default)."New value: +"Type of proxy: datacenter, residential, or stealth (datacenter by default). Use residential if the site restricts datacenter traffic, or stealth for the most heavily protected sites with advanced anti-bot detection. Residential and stealth requests cost more than datacenter — see the pricing page."
    • changedInput schema / properties / proxy / enum
      Previous value: -[
-  "datacenter",
-  "residential"
-]New value: +[
+  "datacenter",
+  "residential",
+  "stealth"
+]
  5. Addedv1.0.2

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true and destructiveHint=false, so the safety profile is covered. The description contributes useful output-shape context (Markdown, boilerplate removal, preserved structure) but says nothing about cost implications of proxies, failure modes, or what happens with the 14 behavioral knobs. It also asserts Markdown output while the schema's text_format defaults to 'json', an ambiguity left unresolved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences plus a docs link; the output shape is front-loaded before the usage hint. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 14-parameter, no-output-schema tool the description covers the primary return shape but omits how text_format (plain/xml/json, default json) changes the response and never mentions the cost-bearing proxy options or the JS/headless-browser path. Adequate to call the tool with just a URL, but thin for anything beyond the default case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and parameter docs are detailed (proxy cost note, timeout max, CSS selector semantics), so the schema carries the burden. The description adds no parameter-level guidance of its own, which is the expected baseline here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get a web page's main content as Markdown') plus the transformation performed ('boilerplate stripped, structure... kept'), which contrasts implicitly with the raw-HTML sibling. It does not name the sibling (webscraping_ai_html) or any other alternative, so differentiation is inferable rather than explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Best for reading a page or passing it to an LLM' gives a positive use case, but there is no exclusion or routing guidance versus the nine siblings (html, fields, selected, question, etc.). The agent must infer that this is the reading/LLM-context tool rather than the extraction tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools