Skip to main content
Glama
TadMSTR

searxng-mcp

by TadMSTR
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

fetch_url

Fetch any URL and extract clean, readable content as markdown. Supports token limits, CSS selectors, and wait conditions for JS-rendered pages, while blocking private addresses.

Instructions

Fetch and extract readable content from any URL. GitHub URLs are fetched via the GitHub API; all others go through a fetch cascade: Firecrawl → Crawl4AI → raw HTTP. Returns clean markdown where possible. Content is trimmed to a token budget (default ~2000 tokens / 8000 chars; raise with max_tokens). Results cached for 24 hours. Blocked domains and private/internal addresses are refused.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesURL to fetch and extract content from
max_tokensNoApproximate token budget for the returned content (chars ≈ tokens × 4). Omit for the ~2000-token / 8000-char default; max 10000 tokens.
domain_profileNoNamed domain profile to apply: 'homelab', 'dev', or omit for default filters
target_selectorNoCSS selector to scope extraction to a specific element (e.g. 'article', 'main .content'). Honored by Firecrawl/Crawl4AI and applied client-side on the raw-HTTP tier; ignored by fast paths and if it matches nothing.
wait_for_selectorNoCSS selector to wait for before extracting, for JS-rendered pages. Honored by the rendering tiers (Firecrawl/Crawl4AI); ignored on raw HTTP (no JS).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changedv3.15.1
    • addedInput schema / properties / max_tokens
      Added value: +{
+  "description": "Approximate token budget for the returned content (chars ≈ tokens × 4). Omit for the ~2000-token / 8000-char default; max 10000 tokens.",
+  "maximum": 10000,
+  "minimum": 100,
+  "type": "integer"
+}
    • addedInput schema / properties / target_selector
      Added value: +{
+  "description": "CSS selector to scope extraction to a specific element (e.g. 'article', 'main .content'). Honored by Firecrawl/Crawl4AI and applied client-side on the raw-HTTP tier; ignored by fast paths and if it matches nothing.",
+  "maxLength": 500,
+  "type": "string"
+}
    • addedInput schema / properties / wait_for_selector
      Added value: +{
+  "description": "CSS selector to wait for before extracting, for JS-rendered pages. Honored by the rendering tiers (Firecrawl/Crawl4AI); ignored on raw HTTP (no JS).",
+  "maxLength": 500,
+  "type": "string"
+}
  2. First observedv3.12.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It details the fetch cascade, token budget, caching (24 hours), and refusal of blocked/internal domains. It does not mention rate limits or error handling, but the main behaviors are well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the primary action. The second sentence provides essential details without unnecessary words. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explains return is clean markdown and trimmed to token budget. It covers caching and refusal behavior. It could mention return format more explicitly but is adequate for a fetch tool with moderate complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining the default token budget (2000 tokens / 8000 chars) and how max_tokens relates to characters. It also clarifies behavior for domain_profile, target_selector, and wait_for_selector, which goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches and extracts readable content from any URL, with specific handling for GitHub URLs and a cascade of backends. It distinguishes itself from siblings like crawl_site (multi-page) and search (search) by focusing on single URL fetch and extraction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states what the tool does and provides context for when to use it (fetching any URL). It mentions blocked domains and private addresses are refused, giving a clear exclusion criterion. However, it does not explicitly contrast with siblings like crawl_site or search_and_fetch, which could be used for similar tasks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools