Skip to main content
Glama

AstraVerify Domain Trust Check

Fetch the root page as a named crawler

fetch_as_crawler
Read-onlyIdempotent

Show exactly what one named crawler receives from the domain's root page. Use when the person asks "what does Google see?" or "can ChatGPT read my site?", and whenever check_discoverability reports a crawl-access problem - this is the tool that explains such a result instead of restating it. robots.txt can read perfectly while a CDN or firewall returns 403 to one crawler and 200 to another, and only a real fetch shows that. Inputs: domain (bare, like example.com) and agent, exactly one of googlebot, bingbot, oai-searchbot, perplexitybot, claudebot, gptbot. Call it once per crawler to compare them. Returns the HTTP status, the full redirect chain, response headers, content type, the text that crawler would index, and flags for an outright block or a bot-challenge page - which is how cloaking becomes visible. One live fetch of the root page, nothing else, and it changes nothing.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
agentYes
domainYesThe domain to check, as the person gave it (no scheme, no path). Reserved names such as example.com are refused.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
msNo
agentYes
chainNoRedirect hops
errorNo
labelNoThe crawler as its operator names it
titleNo
domainYes
statusNoHTTP status the crawler received
blockedNoThe response refused the crawler (403, 429 or similar)
extractNoThe text this crawler would index; null when the fetch failed
headersNo
challengeNoA bot-challenge page rather than the site
final_urlNo
request_idNoreq_... for this call. Quote it when reporting a problem with the result.
content_typeNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedOutput schema / properties / request_id
      Added value: +{
      +  "description": "req_... for this call. Quote it when reporting a problem with the result.",
      +  "type": "string"
      +}
  2. Changed5 schema fields changed
    • changedOutput schema / properties / content_type / type
      Previous value: -"string"New value: +[
      +  "string",
      +  "null"
      +]
    • changedOutput schema / properties / extract / description
      Previous value: -"The text this crawler would index"New value: +"The text this crawler would index; null when the fetch failed"
    • changedOutput schema / properties / extract / type
      Previous value: -"string"New value: +[
      +  "string",
      +  "null"
      +]
    • changedOutput schema / properties / final_url / type
      Previous value: -"string"New value: +[
      +  "string",
      +  "null"
      +]
    • changedOutput schema / properties / title / type
      Previous value: -"string"New value: +[
      +  "string",
      +  "null"
      +]
  3. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "properties": {
      +    "agent": {
      +      "type": "string"
      +    },
      +    "blocked": {
      +      "description": "The response refused the crawler (403, 429 or similar)",
      +      "type": "boolean"
      +    },
      +    "chain": {
      +      "description": "Redirect hops",
      +      "items": {
      +        "properties": {
      +          "status": {
      +            "type": "integer"
      +          },
      +          "url": {
      +            "type": "string"
      +          }
      +        },
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    "challenge": {
      +      "description": "A bot-challenge page rather than the site",
      +      "type": "boolean"
      +    },
      +    "content_type": {
      +      "type": "string"
      +    },
      +    "domain": {
      +      "type": "string"
      +    },
      +    "error": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "extract": {
      +      "description": "The text this crawler would index",
      +      "type": "string"
      +    },
      +    "final_url": {
      +      "type": "string"
      +    },
      +    "headers": {
      +      "additionalProperties": {
      +        "type": "string"
      +      },
      +      "type": "object"
      +    },
      +    "label": {
      +      "description": "The crawler as its operator names it",
      +      "type": "string"
      +    },
      +    "ms": {
      +      "type": "integer"
      +    },
      +    "status": {
      +      "description": "HTTP status the crawler received",
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "title": {
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "domain",
      +    "agent"
      +  ],
      +  "type": "object"
      +}
  4. First observed

TDQS

Score is being calculated.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources