crawl-censusBrowser AutomationWeb ScrapingAgent OrchestrationtaylorsmithggAlicenseAqualityBmaintenanceProvides preflight checks and domain intelligence so agents can skip domains that will refuse them or charge a fee, covering robots.txt, edge refusals, and HTTP 402 paywalls. Updated 2 months ago (2026-08-23 18:33 UTC)7MIT