Screaming Frog SEO Spider MCP Server
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| SF_CLI_PATH | No | Path to the Screaming Frog CLI executable | /Applications/Screaming Frog SEO Spider.app/Contents/MacOS/ScreamingFrogSEOSpiderLauncher |
| SF_CONFIG_DIR | No | Directory containing `.seospiderconfig` files that `crawl_site` can load. | ~/.config/sf-mcp/configs/ |
| SF_ALLOWED_DOMAINS | No | Comma-separated list of allowed crawl target domains. When set, `crawl_site` only accepts URLs matching these domains. | |
| SF_EXPORT_TTL_SECONDS | No | How long exported CSV files are kept before auto-cleanup. | 3600 |
| SF_EXPORT_TIMEOUT_SECONDS | No | Max time to wait for an `export_crawl` operation to complete. | 300 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| sf_checkA | Verify that Screaming Frog SEO Spider is installed and the CLI is accessible. Returns version info and license status. |
| crawl_siteA | Start a background Screaming Frog crawl that saves to SF's internal database. Args: url: The URL to crawl (e.g. https://example.com) config_file: Optional path to a .seospiderconfig file for crawl settings. To limit the number of URLs crawled, set the limit in a config file (Configuration > Spider > Limits in the SF GUI) and pass it here. label: Optional label for identifying this crawl (e.g. 'freshgovjobs') Returns: A crawl_id to use with crawl_status to check progress. The crawl runs in the background - use crawl_status to poll. |
| crawl_statusA | Check the status of a running or completed crawl. Args: crawl_id: The crawl_id returned by crawl_site |
| list_crawlsA | List all crawls saved in Screaming Frog's internal database. Returns crawl names, Database IDs, and sizes. Use the Database ID with export_crawl or delete_crawl. |
| export_crawlA | Load a saved crawl from SF's database and export data as CSV files. Args: db_id: The Database ID from list_crawls (e.g. '1234' or a crawl identifier) export_tabs: Comma-separated export tabs (default: Internal:All,Response Codes:All,Page Titles:All,Meta Description:All,H1:All,H2:All,Images:All,Canonicals:All,Directives:All). See the export-reference resource for all options. bulk_export: Optional bulk export types (e.g. 'Links:All Inlinks,Content:Soft 404 Inlinks') save_report: Optional reports to save (e.g. 'Crawl Overview') Returns: An export_id and list of generated CSV files. Use read_crawl_data to read them. |
| read_crawl_dataA | Read CSV data from an export. Use after export_crawl. Args: export_id: The export_id from export_crawl file: CSV filename to read (from the file list in export_crawl output) limit: Max rows to return (default 100, max 1000) offset: Number of rows to skip (for pagination) filter_column: Optional column name to filter by filter_value: Optional value to match in the filter column filter_mode: How to match filter_value: "contains" (default, case-insensitive substring), "exact" (case-insensitive exact match), or "regex" (Python regex) columns: Optional comma-separated column names to return. Wide exports (Internal:All has dozens of columns) flood the context; request just the ones you need, e.g. "Address,Status Code". Returns: CSV data as formatted text with column headers. |
| aggregate_crawl_dataA | Aggregate CSV data from an export: row counts and group-by breakdowns. Use instead of read_crawl_data when the question needs counts or a distribution ("how many 404s", "status code breakdown") rather than the rows themselves. Args: export_id: The export_id from export_crawl file: CSV filename to aggregate (from the file list in export_crawl output) group_by: Optional column name; counts rows per distinct value of it top: Max distinct values to show, most common first (default 20, max 100) filter_column: Optional column name to filter by before aggregating filter_value: Optional value to match in the filter column filter_mode: How to match filter_value: "contains" (default), "exact", or "regex" Returns: Total matching row count, plus per-value counts with percentages when group_by is set. |
| delete_crawlA | Delete a crawl from Screaming Frog's internal database to free disk space. Args: db_id: The Database ID from list_crawls WARNING: This permanently deletes the crawl data. It cannot be undone. |
| storage_summaryA | Show disk usage of Screaming Frog's internal crawl storage. Returns total size and per-crawl breakdown of ProjectInstanceData. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| get_export_reference | Complete reference of all Screaming Frog export options. |
TDQS
Scored across 9 tools
Each tool maps to a distinct step in the crawl lifecycle: install check, start, status, list, storage, export, read, aggregate, and delete. Even the most similar pair, read_crawl_data and aggregate_crawl_data, is clearly separated by row-level versus count/group-by output.
Most tools follow a clear verb_noun snake_case pattern (crawl_site, export_crawl, delete_crawl), but sf_check is a noun_verb exception and storage_summary is noun-only. The naming is still predictable and readable overall.
Nine tools cover crawl initiation, monitoring, storage management, export, data access, and cleanup without redundancy. The count is appropriate for the server's focused SEO crawling purpose.
The full crawl workflow is covered: start, check status, list crawls, export results, read/aggregate data, and delete to free space. The main gap is no way to stop or cancel an in-progress background crawl, which could leave agents with no abort path for long-running crawls.