Verify that Screaming Frog SEO Spider is installed and the CLI is accessible. Returns version info and license status.

Start a background Screaming Frog crawl that saves to SF's internal database.

Args: url: The URL to crawl (e.g. https://example.com) config_file: Optional path to a .seospiderconfig file for crawl settings. To limit the number of URLs crawled, set the limit in a config file (Configuration > Spider > Limits in the SF GUI) and pass it here. label: Optional label for identifying this crawl (e.g. 'freshgovjobs')

Returns: A crawl_id to use with crawl_status to check progress. The crawl runs in the background - use crawl_status to poll.

Check the status of a running or completed crawl.

Args: crawl_id: The crawl_id returned by crawl_site

List all crawls saved in Screaming Frog's internal database. Returns crawl names, Database IDs, and sizes. Use the Database ID with export_crawl or delete_crawl.

Load a saved crawl from SF's database and export data as CSV files.

Args: db_id: The Database ID from list_crawls (e.g. '1234' or a crawl identifier) export_tabs: Comma-separated export tabs (default: Internal:All,Response Codes:All,Page Titles:All,Meta Description:All,H1:All,H2:All,Images:All,Canonicals:All,Directives:All). See the export-reference resource for all options. bulk_export: Optional bulk export types (e.g. 'All Inlinks,All Outlinks') save_report: Optional reports to save (e.g. 'Crawl Overview')

Returns: An export_id and list of generated CSV files. Use read_crawl_data to read them.

Read CSV data from an export. Use after export_crawl.

Args: export_id: The export_id from export_crawl file: CSV filename to read (from the file list in export_crawl output) limit: Max rows to return (default 100, max 1000) offset: Number of rows to skip (for pagination) filter_column: Optional column name to filter by filter_value: Optional value to match in the filter column filter_mode: How to match filter_value: "contains" (default, case-insensitive substring), "exact" (case-insensitive exact match), or "regex" (Python regex) columns: Optional comma-separated column names to return. Wide exports (Internal:All has dozens of columns) flood the context; request just the ones you need, e.g. "Address,Status Code".

Returns: CSV data as formatted text with column headers.

Aggregate CSV data from an export: row counts and group-by breakdowns. Use instead of read_crawl_data when the question needs counts or a distribution ("how many 404s", "status code breakdown") rather than the rows themselves.

Args: export_id: The export_id from export_crawl file: CSV filename to aggregate (from the file list in export_crawl output) group_by: Optional column name; counts rows per distinct value of it top: Max distinct values to show, most common first (default 20, max 100) filter_column: Optional column name to filter by before aggregating filter_value: Optional value to match in the filter column filter_mode: How to match filter_value: "contains" (default), "exact", or "regex"

Returns: Total matching row count, plus per-value counts with percentages when group_by is set.

Delete a crawl from Screaming Frog's internal database to free disk space.

Args: db_id: The Database ID from list_crawls

WARNING: This permanently deletes the crawl data. It cannot be undone.

Show disk usage of Screaming Frog's internal crawl storage. Returns total size and per-crawl breakdown of ProjectInstanceData.