Agent Margin Router

Web Crawl

web_crawl

Read-only

Mini crawler: fetches a start page and follows same-domain links breadth-first (max 10 pages), returning clean LLM-ready Markdown, title, word count and discovered links per page; honours robots.txt by default. SSRF-guarded. Pay per call with USDC or USDT on Base via x402. 3 free calls per wallet.

Input Schema

TableJSON Schema

Name	Required	Description
`wallet`	No	Your EVM wallet address (0x...). Unlocks the free tier (3 free calls per wallet).
`api_key`	No	Optional AMR enterprise API key.
`max_pages`	No	1-10 (default 5).
`start_url`	Yes	Public page to start from.
`x_payment`	No	Optional signed x402 payment payload (base64 JSON) for USDC/USDT on Base.
`follow_links`	No	Follow same-domain links (default true).
`include_links`	No	Keep hyperlinks in Markdown (default true).
`respect_robots`	No	Honour robots.txt Disallow for '*' (default true).

Output Schema

TableJSON Schema

Name	Required	Description	Default
`pages`	Yes
`token`	No	Token actually used for payment (USDC, USDT or FREE).	USDC
`domain`	Yes
`routing`	Yes	How the request was routed (provider, fallback, cache, latency).
`cost_usdc`	Yes	Price charged in stablecoin units (USDC/USDT 1:1); 0 on the free tier.
`max_pages`	Yes
`start_url`	Yes
`fetched_at`	No	UTC timestamp when the data was fetched.
`page_count`	Yes
`total_words`	Yes
`follow_links`	Yes
`pages_failed`	Yes
`robots_blocked`	Yes
`freshness_seconds`	Yes	Age of the underlying data in seconds (0 = fetched live).
`queued_not_fetched`	Yes
`x_payment_response`	No	Base64-encoded x402 settlement receipt (mirrors the X-PAYMENT-RESPONSE header). Only present when the call was paid with `x_payment`.

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedSep 15, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "description": "Response for ``POST /web-crawl``.",
+  "properties": {
+    "cost_usdc": {
+      "description": "Price charged in stablecoin units (USDC/USDT 1:1); 0 on the free tier.",
+      "title": "Cost Usdc",
+      "type": "number"
+    },
+    "domain": {
+      "title": "Domain",
+      "type": "string"
+    },
+    "fetched_at": {
+      "description": "UTC timestamp when the data was fetched.",
+      "format": "date-time",
+      "title": "Fetched At",
+      "type": "string"
+    },
+    "follow_links": {
+      "title": "Follow Links",
+      "type": "boolean"
+    },
+    "freshness_seconds": {
+      "description": "Age of the underlying data in seconds (0 = fetched live).",
+      "title": "Freshness Seconds",
+      "type": "integer"
+    },
+    "max_pages": {
+      "title": "Max Pages",
+      "type": "integer"
+    },
+    "page_count": {
+      "title": "Page Count",
+      "type": "integer"
+    },
+    "pages": {
+      "items": {
+        "properties": {
+          "depth": {
+            "title": "Depth",
+            "type": "integer"
+          },
+          "error": {
+            "anyOf": [
+              {
+                "type": "string"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Error"
+          },
+          "final_url": {
+            "title": "Final Url",
+            "type": "string"
+          },
+          "internal_links": {
+            "items": {
+              "type": "string"
+            },
+            "title": "Internal Links",
+            "type": "array"
+          },
+          "links_found": {
+            "title": "Links Found",
+            "type": "integer"
+          },
+          "markdown": {
+            "title": "Markdown",
+            "type": "string"
+          },
+          "status_code": {
+            "anyOf": [
+              {
+                "type": "integer"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Status Code"
+          },
+          "title": {
+            "anyOf": [
+              {
+                "type": "string"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Title"
+          },
+          "url": {
+            "title": "Url",
+            "type": "string"
+          },
+          "word_count": {
+            "title": "Word Count",
+            "type": "integer"
+          }
+        },
+        "required": [
+          "url",
+          "final_url",
+          "markdown",
+          "word_count",
+          "links_found",
+          "internal_links",
+          "depth"
+        ],
+        "title": "CrawledPage",
+        "type": "object"
+      },
+      "title": "Pages",
+      "type": "array"
+    },
+    "pages_failed": {
+      "title": "Pages Failed",
+      "type": "integer"
+    },
+    "queued_not_fetched": {
+      "title": "Queued Not Fetched",
+      "type": "integer"
+    },
+    "robots_blocked": {
+      "title": "Robots Blocked",
+      "type": "integer"
+    },
+    "routing": {
+      "description": "How the request was routed (provider, fallback, cache, latency).",
+      "properties": {
+        "cache_hit": {
+          "default": false,
+          "description": "True if the result was served from cache.",
+          "title": "Cache Hit",
+          "type": "boolean"
+        },
+        "fallback_used": {
+          "default": false,
+          "description": "True if the primary provider failed and a fallback served the result.",
+          "title": "Fallback Used",
+          "type": "boolean"
+        },
+        "latency_ms": {
+          "default": 0,
+          "description": "End-to-end routing latency in milliseconds.",
+          "title": "Latency Ms",
+          "type": "integer"
+        },
+        "provider": {
+          "description": "Name of the upstream provider that served the result.",
+          "title": "Provider",
+          "type": "string"
+        },
+        "providers_tried": {
+          "description": "Providers attempted, in order.",
+          "items": {
+            "type": "string"
+          },
+          "title": "Providers Tried",
+          "type": "array"
+        }
+      },
+      "required": [
+        "provider"
+      ],
+      "title": "RoutingMeta",
+      "type": "object"
+    },
+    "start_url": {
+      "title": "Start Url",
+      "type": "string"
+    },
+    "token": {
+      "default": "USDC",
+      "description": "Token actually used for payment (USDC, USDT or FREE).",
+      "title": "Token",
+      "type": "string"
+    },
+    "total_words": {
+      "title": "Total Words",
+      "type": "integer"
+    },
+    "x_payment_response": {
+      "description": "Base64-encoded x402 settlement receipt (mirrors the X-PAYMENT-RESPONSE header). Only present when the call was paid with `x_payment`.",
+      "type": "string"
+    }
+  },
+  "required": [
+    "freshness_seconds",
+    "cost_usdc",
+    "routing",
+    "start_url",
+    "domain",
+    "pages",
+    "page_count",
+    "pages_failed",
+    "max_pages",
+    "follow_links",
+    "robots_blocked",
+    "queued_not_fetched",
+    "total_words"
+  ],
+  "type": "object"
+}

Changed7 schema fields changedSep 13, 2026
- addedInput schema / properties / follow_links / default
```
Added value: +true
```
- addedInput schema / properties / include_links / default
```
Added value: +true
```
- addedInput schema / properties / max_pages / default
```
Added value: +5
```
- addedInput schema / properties / max_pages / maximum
```
Added value: +10
```
- addedInput schema / properties / max_pages / minimum
```
Added value: +1
```
- addedInput schema / properties / respect_robots / default
```
Added value: +true
```
- changedInput schema / properties / wallet / description
```
Previous value: -"Your EVM wallet address (0x...). Unlocks the free tier (3 calls per wallet)."New value: +"Your EVM wallet address (0x...). Unlocks the free tier (3 free calls per wallet)."
```
AddedSep 13, 2026

TDQS

A4.4/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/openWorld annotations, the description discloses breadth-first crawling, the 10-page limit, default robots.txt handling, SSRF guarding, and the payment model (USDC/USDT on Base via x402, 3 free calls per wallet). This is substantial behavioral context and does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three dense, purposeful sentences: core behavior and output first, then security behavior, then cost/payment context. No filler or redundant repetition of schema fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a rich output schema and 100% schema coverage, the description fills important gaps: page limit, same-domain behavior, robots default, SSRF safety, and payment terms. It does not fully explain how to obtain an x402 payload, but the schema field and free-tier note give an agent enough to proceed effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all 8 parameters with descriptions, so the baseline is 3. The description mostly restates crawl behavior already reflected in max_pages, follow_links, and respect_robots, and adds little parameter-specific meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly names the resource ('start page'), the operation ('fetches and follows same-domain links breadth-first'), and the output ('LLM-ready Markdown, title, word count and discovered links'). This makes it easy to distinguish from single-page tools like html_to_markdown or extract_clean and from web_search.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear context for use: crawling multiple same-domain pages up to 10 pages and returning Markdown. However, it does not explicitly name sibling alternatives or state when not to use it, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Agent Margin Router tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.