scrape
Extract structured content from web pages by providing a URL. This tool parses HTML to retrieve text, metadata, and other data for analysis or integration.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
Implementation Reference
- main.py:80-86 (handler)Handler function for the 'scrape' tool, registered via @mcp.tool() decorator. It calls the scrape_urls helper on the WebTools instance to perform the scraping.@mcp.tool() async def scrape(url: str) -> str: try: url_scraped = webtools.scrape_urls(url) return url_scraped except Exception as e: return f"Error scraping url {url}: {str(e)}"
- tools/webtools.py:58-71 (helper)Helper method in WebTools class that implements the core scraping logic using FirecrawlApp.scrape_url, supporting markdown and HTML formats with screenshot action.def scrape_urls(self, url: list[str]): try: urls_scraped = self.firecrawl.scrape_url( url, params={ "formats": ["markdown", "html"], "actions": [ {"type": "screenshot"}, ], }, ) return urls_scraped except Exception as e: return f"Error scrapping ulr {url}: {str(e)}"