Skip to main content
Glama
Sign In
deenesjakoruzh
screaming-frog-mcp

screaming-frog-mcp

by marykovziridze
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

screaming-frog-mcp

MCP server that lets Claude run Screaming Frog SEO Spider headless crawls, export data, and manage crawl storage — without anyone opening the GUI.

Type a URL into Claude. Screaming Frog runs in the background. You get the data back. That's it.

Forked from bzsasson/screaming-frog-mcp v0.1.0 with bug fixes. The original had issues that made it unusable in practice — pipe deadlocks that hung crawls, false GUI detection that blocked everything after the first run, a delete command that could wipe your entire crawl database. All fixed.

What's fixed

Bug

Fix

Pipe deadlock

stdout/stderr redirected to log files instead of PIPE. Crawls no longer hang when SF produces large output.

GUI detection

Uses psutil instead of ps aux. Works on Mac and Windows. Headless CLI processes no longer get mistaken for the GUI.

Stale crawl cleanup

SF leaves a temp crawl.seospider file inside its own app bundle when a crawl gets interrupted. Every crawl after that fails. Now auto-cleaned before each run.

Delete safety

delete_crawl(".") used to resolve to the root data directory and wipe everything. Fixed.

Export dir leak

Failed exports left temp directories on disk. Now cleaned up.

Input validation

Stricter character allowlists for CLI arguments and db_id.

Related MCP server: Screaming Frog SEO Spider MCP Server

Requirements

Installation

Mac

uvx --from git+https://github.com/marykovziridze/screaming-frog-mcp screaming-frog-mcp

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "screaming-frog": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"]
    }
  }
}

Windows

Install uv first:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Add to C:\Users\[name]\AppData\Roaming\Claude\claude_desktop_config.json:

{
  "mcpServers": {
    "screaming-frog": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"],
      "env": {
        "SF_CLI_PATH": "C:\\Program Files (x86)\\Screaming Frog SEO Spider\\ScreamingFrogSEOSpiderCli.exe"
      }
    }
  }
}

Restart Claude Desktop after editing the config.

Tools

Tool

What it does

sf_check

Verify SF is installed and licensed

crawl_site

Start a headless crawl

crawl_status

Check crawl progress

list_crawls

List saved crawls in SF's database

export_crawl

Export crawl data as CSV

read_crawl_data

Read and filter exported CSV data

delete_crawl

Delete a saved crawl

storage_summary

Show disk usage of crawl storage

Configuration

Variable

Default

Notes

SF_CLI_PATH

Mac: auto-detected

Set manually on Windows or custom installs

Known limitations

  • Windows stale crawl path — auto-cleanup works on Mac. On Windows, if crawls fail after an interruption, check for a crawl.seospider file in your SF install directory and delete it manually.

  • No crawl progress percentage — SF's headless CLI doesn't report progress mid-crawl. You know when it starts and when it finishes.

  • Large sites — tested on sites up to ~160 pages. Not stress-tested on 10k+ page sites.

License

MIT — see LICENSE

Credits

Original MCP server by Boaz Sasson.

Available Tools

8 tools
crawl_siteA

Start a background Screaming Frog crawl that saves to SF's internal database.

Args: url: The URL to crawl (e.g. https://example.com) config_file: Optional path to a .seospiderconfig file for crawl settings (including crawl limits) label: Optional label for identifying this crawl (e.g. 'freshgovjobs')

Returns: A crawl_id to use with crawl_status to check progress. The crawl runs in the background - use crawl_status to poll.

Note: To limit the number of URLs crawled, export a .seospiderconfig from the SF GUI with the desired crawl limit, then pass it via config_file.

Inputs: url (required), config_file, label. Returns: result

crawl_statusC

Check the status of a running or completed crawl.

Args: crawl_id: The crawl_id returned by crawl_site

Inputs: crawl_id (required). Returns: result

delete_crawlA

Delete a crawl from Screaming Frog's internal database to free disk space.

Args: db_id: The Database ID from list_crawls

WARNING: This permanently deletes the crawl data. It cannot be undone.

Inputs: db_id (required). Returns: result

export_crawlA

Load a saved crawl from SF's database and export data as CSV files.

Args: db_id: The Database ID from list_crawls (e.g. '1234' or a crawl identifier) export_tabs: Comma-separated export tabs (default: Internal:All,Response Codes:All,Page Titles:All,Meta Description:All,H1:All,H2:All,Images:All,Canonicals:All,Directives:All). See the export-reference resource for all options. bulk_export: Optional bulk export types (e.g. 'All Inlinks,All Outlinks') save_report: Optional reports to save (e.g. 'Crawl Overview')

Returns: An export_id and list of generated CSV files. Use read_crawl_data to read them.

Inputs: db_id (required), export_tabs, bulk_export, save_report. Returns: result

list_crawlsA

List all crawls saved in Screaming Frog's internal database. Returns crawl names, Database IDs, and sizes. Use the Database ID with export_crawl or delete_crawl.

Inputs: no named parameters. Returns: result

read_crawl_dataA

Read CSV data from an export. Use after export_crawl.

Args: export_id: The export_id from export_crawl file: CSV filename to read (from the file list in export_crawl output) limit: Max rows to return (default 100) offset: Number of rows to skip (for pagination) filter_column: Optional column name to filter by filter_value: Optional value to match in the filter column (case-insensitive substring)

Returns: CSV data as formatted text with column headers.

Inputs: export_id (required), file (required), limit, offset, +2 more. Returns: result

sf_checkA

Verify that Screaming Frog SEO Spider is installed and the CLI is accessible. Returns version info and license status.

Inputs: no named parameters. Returns: result

storage_summaryA

Show disk usage of Screaming Frog's internal crawl storage. Returns total size and per-crawl breakdown of ProjectInstanceData.

Inputs: no named parameters. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Deploy Server
    Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

    TDQS

    A4.2/5.0

    Scored across 8 tools

    Disambiguation5/5

    Each tool has a distinct purpose with no overlap: crawl_site initiates crawls, crawl_status checks progress, delete_crawl removes data, export_crawl exports data, list_crawls lists saved crawls, read_crawl_data reads exported data, sf_check verifies installation, and storage_summary shows disk usage. The descriptions clearly differentiate their functions, preventing agent misselection.

    Naming Consistency5/5

    All tools follow a consistent snake_case naming pattern with clear verb_noun structures (e.g., crawl_site, list_crawls, read_crawl_data). The naming is predictable and readable throughout the set, with no deviations in style or convention.

    Tool Count5/5

    With 8 tools, this server is well-scoped for managing Screaming Frog crawls. Each tool earns its place by covering essential operations like starting, monitoring, exporting, and managing crawls, without being overly sparse or bloated. The count aligns perfectly with the domain's workflow needs.

    Completeness5/5

    The tool set provides complete CRUD/lifecycle coverage for Screaming Frog crawls: create (crawl_site), read (export_crawl, read_crawl_data), update (implied via crawl_status monitoring), delete (delete_crawl), and management (list_crawls, storage_summary, sf_check). There are no obvious gaps, and agents can handle all core tasks without dead ends.

    Resources

    Related MCP Connectors

    Related MCP Servers

    Appeared in Searches