dossier_web_surface
Fetch a domain's robots.txt, sitemap.xml, and homepage metadata (title, description, OpenGraph, Twitter cards) concurrently. Use for SEO audits, content discovery, or verifying metadata before sharing.
Instructions
Snapshot a domain's public web surface: robots.txt, sitemap.xml, and the home-page metadata (title, description, OpenGraph, Twitter cards). Use for SEO audits, content discovery, or verifying metadata before sharing; for HTTP headers use dossier_headers, for redirect behavior use dossier_redirects. Fetches /, /robots.txt, and /sitemap.xml concurrently via HTTPS, 5 s each; parses with a lightweight HTML parser. Returns a composite CheckResult: {status:"ok", meta:{title, description, og, twitter}, robots, sitemapPresent} or {status:"error", reason}.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Public FQDN, e.g. example.com. Must be resolvable on the public internet; IPs, ports, paths, and protocol prefixes are rejected. |