MCP connectors by dev.workers.pathwren.www

Which discovery documents does a host serve? llms.txt, agent card, owners.json, mcp.json.

Paste access-log lines: every crawler identified, impersonators named, robots.txt and WAF rules.

Identify AI crawlers by user-agent or IP, verify them against operator ranges, build robots.txt.