Crawl websites to generate structured LLM-ready markdown files using Firecrawl, extracting data for AI inference and storing results in S3.
-
Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.
Firecrawl MCP — wraps the Firecrawl API (firecrawl.dev) for web scraping / crawling / search…