screaming-frog-mcp
Provides integration with GitHub repositories for installation and configuration of the MCP server through git-based package management.
screaming-frog-mcp
MCP server that lets Claude run Screaming Frog SEO Spider headless crawls, export data, and manage crawl storage — without anyone opening the GUI.
Type a URL into Claude. Screaming Frog runs in the background. You get the data back. That's it.
Forked from bzsasson/screaming-frog-mcp v0.1.0 with bug fixes. The original had issues that made it unusable in practice — pipe deadlocks that hung crawls, false GUI detection that blocked everything after the first run, a delete command that could wipe your entire crawl database. All fixed.
What's fixed
Bug | Fix |
Pipe deadlock | stdout/stderr redirected to log files instead of PIPE. Crawls no longer hang when SF produces large output. |
GUI detection | Uses |
Stale crawl cleanup | SF leaves a temp |
Delete safety |
|
Export dir leak | Failed exports left temp directories on disk. Now cleaned up. |
Input validation | Stricter character allowlists for CLI arguments and db_id. |
Related MCP server: Screaming Frog SEO Spider MCP Server
Requirements
Screaming Frog SEO Spider with a paid license — headless crawls require a license
Python 3.10+
uv (recommended) or pip
Installation
Mac
uvx --from git+https://github.com/marykovziridze/screaming-frog-mcp screaming-frog-mcpAdd to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"screaming-frog": {
"command": "uvx",
"args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"]
}
}
}Windows
Install uv first:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"Add to C:\Users\[name]\AppData\Roaming\Claude\claude_desktop_config.json:
{
"mcpServers": {
"screaming-frog": {
"command": "uvx",
"args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"],
"env": {
"SF_CLI_PATH": "C:\\Program Files (x86)\\Screaming Frog SEO Spider\\ScreamingFrogSEOSpiderCli.exe"
}
}
}
}Restart Claude Desktop after editing the config.
Tools
Tool | What it does |
| Verify SF is installed and licensed |
| Start a headless crawl |
| Check crawl progress |
| List saved crawls in SF's database |
| Export crawl data as CSV |
| Read and filter exported CSV data |
| Delete a saved crawl |
| Show disk usage of crawl storage |
Configuration
Variable | Default | Notes |
| Mac: auto-detected | Set manually on Windows or custom installs |
Known limitations
Windows stale crawl path — auto-cleanup works on Mac. On Windows, if crawls fail after an interruption, check for a
crawl.seospiderfile in your SF install directory and delete it manually.No crawl progress percentage — SF's headless CLI doesn't report progress mid-crawl. You know when it starts and when it finishes.
Large sites — tested on sites up to ~160 pages. Not stress-tested on 10k+ page sites.
License
MIT — see LICENSE
Credits
Original MCP server by Boaz Sasson.
Available Tools
8 toolscrawl_siteA
Start a background Screaming Frog crawl that saves to SF's internal database.
Args: url: The URL to crawl (e.g. https://example.com) config_file: Optional path to a .seospiderconfig file for crawl settings (including crawl limits) label: Optional label for identifying this crawl (e.g. 'freshgovjobs')
Returns: A crawl_id to use with crawl_status to check progress. The crawl runs in the background - use crawl_status to poll.
Note: To limit the number of URLs crawled, export a .seospiderconfig from the SF GUI with the desired crawl limit, then pass it via config_file.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| config_file | No | ||
| label | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses key behavioral traits: the crawl runs in the background, returns a crawl_id for tracking, and requires polling via crawl_status. However, it doesn't mention potential side effects (resource consumption), authentication needs, rate limits, or error conditions that would be important for a background job tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Perfectly structured with clear sections (description, Args, Returns, Note), zero wasted sentences, and front-loaded critical information. Every sentence earns its place by providing essential operational details or practical guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (background job with 3 parameters), no annotations, but with output schema (implied by 'Returns' section), the description is mostly complete. It covers purpose, parameters, return value, and operational flow, but lacks information about error handling, performance characteristics, or integration with other tools beyond crawl_status.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by explaining all three parameters: url (URL to crawl with example), config_file (optional .seospiderconfig for settings/limits with practical guidance), and label (optional identifier with example). It adds meaningful context beyond basic schema types, especially for config_file usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Start a background Screaming Frog crawl') and resource ('saves to SF's internal database'), distinguishing it from siblings like crawl_status (check progress) or export_crawl (export data). It uses precise technical terminology that identifies the exact operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context about when to use this tool (to initiate a crawl) and references crawl_status for checking progress, but doesn't explicitly contrast when to use this versus alternatives like list_crawls or delete_crawl. The note about config_file usage offers practical guidance but not sibling differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
crawl_statusC
Check the status of a running or completed crawl.
Args: crawl_id: The crawl_id returned by crawl_site
| Name | Required | Description | Default |
|---|---|---|---|
| crawl_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It only states 'Check the status' without revealing whether the operation is purely read-only, if it has side effects, or what errors may occur. This is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two lines) and front-loaded with the purpose. The 'Args:' section is somewhat redundant with the schema but does not waste many words. It could be further compressed by removing the 'Args' line since it restates the parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity (1 parameter, no nested objects), the description omits crucial details. With an output schema indicated but not described, the tool's return format is unknown. An agent would not know whether to expect a status string, progress percentage, or error details. No error handling or edge cases are mentioned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds minimal context: 'The crawl_id returned by crawl_site', which clarifies the parameter's origin and format. However, it does not explain required constraints (e.g., must be a valid UUID) or provide examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the verb 'Check' and resource 'status of a running or completed crawl', clearly indicating the tool's function. It distinguishes from siblings like 'crawl_site' (starts a crawl) and 'delete_crawl' (deletes). However, it does not elaborate on what status entails (e.g., progress, completion, errors).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after calling 'crawl_site' by referencing the crawl_id returned by that tool. It provides no explicit guidance on when to use this tool vs alternatives (e.g., 'list_crawls' or 'read_crawl_data'), nor does it mention when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_crawlA
Delete a crawl from Screaming Frog's internal database to free disk space.
Args: db_id: The Database ID from list_crawls
WARNING: This permanently deletes the crawl data. It cannot be undone.
| Name | Required | Description | Default |
|---|---|---|---|
| db_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Explicitly warns 'This permanently deletes the crawl data. It cannot be undone.' which covers the destructive nature. No annotations provided, so description carries full burden. Could mention prerequisites like ensuring crawl is not in use, but still strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences plus a warning. Front-loaded with action and purpose. No fluff. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Completeness is high for a simple delete tool. Output schema exists (not shown), so return value explanation is not needed. Could mention success/failure behavior or side effects, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but description fully explains the sole parameter 'db_id' as 'The Database ID from list_crawls', adding clear provenance beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete a crawl' with specific resource and verb. Distinguishes from sibling tools like list_crawls or crawl_site by noting it deletes from the internal database to free disk space.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context (free disk space) and source for db_id (list_crawls), but lacks explicit guidance on when not to use or alternatives like export_crawl before deletion. Adequate but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export_crawlA
Load a saved crawl from SF's database and export data as CSV files.
Args: db_id: The Database ID from list_crawls (e.g. '1234' or a crawl identifier) export_tabs: Comma-separated export tabs (default: Internal:All,Response Codes:All,Page Titles:All,Meta Description:All,H1:All,H2:All,Images:All,Canonicals:All,Directives:All). See the export-reference resource for all options. bulk_export: Optional bulk export types (e.g. 'All Inlinks,All Outlinks') save_report: Optional reports to save (e.g. 'Crawl Overview')
Returns: An export_id and list of generated CSV files. Use read_crawl_data to read them.
| Name | Required | Description | Default |
|---|---|---|---|
| db_id | Yes | ||
| export_tabs | No | ||
| bulk_export | No | ||
| save_report | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It mentions loading a saved crawl and exporting as CSV, but does not disclose potential side effects, authentication needs, failure modes (e.g., missing crawl), or whether it is idempotent. The behavior is implied but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args and Returns sections. The default value for export_tabs is lengthy but necessary. Overall, it is concise without being terse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 params, 1 required, no enums) and the presence of an output schema, the description covers all aspects: parameter explanations, default values, cross-references to other resources, and return value. It is complete for an export tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds meaning: db_id is explained as coming from list_crawls, export_tabs includes default and reference to external resource, bulk_export and save_report have examples. This compensates well for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'export', the resource 'crawl data', and the output format 'CSV files'. It distinguishes from sibling tools like crawl_site (creation) and read_crawl_data (reading).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that db_id comes from list_crawls and suggests using read_crawl_data to read the results. However, it does not explicitly state when to use this tool versus alternatives like read_crawl_data or how to decide between exporting vs direct reading.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_crawlsA
List all crawls saved in Screaming Frog's internal database. Returns crawl names, Database IDs, and sizes. Use the Database ID with export_crawl or delete_crawl.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states the tool returns specific fields and is a read operation. While adequate for a simple list, it does not mention any potential scale or performance considerations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: purpose, return info, usage hint. No waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity (no params) and presence of output schema, the description is sufficient. It covers what the tool does, what it returns, and how to use the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, schema coverage 100% trivially. Description adds meaning by explaining the tool returns crawl names, DB IDs, and sizes, and how to use the IDs with sibling tools, which compensates for the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List all crawls' with specific verb and resource. It distinguishes from sibling tools like crawl_site (initiate) and delete_crawl (delete).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides context that list_crawls is used to obtain Database IDs for export_crawl and delete_crawl. However, it does not explicitly contrast with other siblings like crawl_status or read_crawl_data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
read_crawl_dataA
Read CSV data from an export. Use after export_crawl.
Args: export_id: The export_id from export_crawl file: CSV filename to read (from the file list in export_crawl output) limit: Max rows to return (default 100) offset: Number of rows to skip (for pagination) filter_column: Optional column name to filter by filter_value: Optional value to match in the filter column (case-insensitive substring)
Returns: CSV data as formatted text with column headers.
| Name | Required | Description | Default |
|---|---|---|---|
| export_id | Yes | ||
| file | Yes | ||
| limit | No | ||
| offset | No | ||
| filter_column | No | ||
| filter_value | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool reads data (non-destructive) and returns formatted text, but lacks details on permissions, rate limits, error handling, or data format specifics. It adds basic context but misses key behavioral traits for a read operation with filtering capabilities.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded: purpose first, then Args and Returns sections. Every sentence earns its place—no fluff. The bullet-point style for parameters is efficient, and the text is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, 0% schema coverage, no annotations, but an output schema exists, the description does well. It explains the tool's purpose, usage context, and parameter semantics thoroughly. The output schema handles return values, so the description doesn't need to detail them. It could improve by addressing error cases or authentication needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides clear semantics for all 6 parameters: export_id links to export_crawl, file specifies the CSV filename, limit/offset handle pagination, and filter_column/filter_value enable case-insensitive substring filtering. This adds substantial meaning beyond the bare schema, though it doesn't cover all edge cases.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Read CSV data from an export.' It specifies the verb ('Read'), resource ('CSV data'), and source ('from an export'), distinguishing it from siblings like crawl_site or export_crawl. The mention of 'Use after export_crawl' further clarifies its role in the workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides usage guidance: 'Use after export_crawl.' This indicates a prerequisite and timing context, distinguishing it from alternatives like list_crawls or crawl_status. It effectively tells the agent when to invoke this tool in relation to other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sf_checkA
Verify that Screaming Frog SEO Spider is installed and the CLI is accessible. Returns version info and license status.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It discloses return values (version info, license status) but does not mention any side effects, auth requirements, or network access needs. The behavior is implied as a read-only check, which is adequate for a simple verification tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that front-load the core purpose. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has zero parameters and an output schema exists, the description covers the essential purpose and return information. It is complete enough for a simple verification tool, though additional behavioral detail (e.g., error states) could elevate it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist in the input schema, so baseline is 4. The description adds meaning by specifying what is being verified and what is returned, fulfilling the purpose of parameter documentation by explaining the tool's context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Verify') and clearly identifies the resource (Screaming Frog SEO Spider installation/CLI). It distinguishes the tool from siblings like crawl_site and crawl_status by focusing on installation and accessibility checks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. The usage context is implied as a prerequisite check, but no alternatives or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
storage_summaryA
Show disk usage of Screaming Frog's internal crawl storage. Returns total size and per-crawl breakdown of ProjectInstanceData.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read-only operation but does not explicitly state it is non-destructive or require any permissions. With no annotations, it provides minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences that front-load the core purpose and add relevant detail without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, the description covers the essential behavior (returns total size and per-crawl breakdown) and does not require additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description adds no parameter-level details. However, schema coverage is 100% and the description is sufficient for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Show disk usage') and resource ('internal crawl storage'), and distinguishes it from sibling tools which focus on crawling, status, or data export.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (to check storage usage) but does not explicitly mention when not to use it or how it compares to alternatives like list_crawls or sf_check.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
8 tool updates
v0.2.0- First observed
crawl_site - First observed
crawl_status - First observed
delete_crawl - First observed
export_crawl - First observed
list_crawls - First observed
read_crawl_data - First observed
sf_check - First observed
storage_summary
TDQS
Scored across 8 tools
Each tool has a distinct purpose with no overlap: crawl_site initiates crawls, crawl_status checks progress, delete_crawl removes data, export_crawl exports data, list_crawls lists saved crawls, read_crawl_data reads exported data, sf_check verifies installation, and storage_summary shows disk usage. The descriptions clearly differentiate their functions, preventing agent misselection.
All tools follow a consistent snake_case naming pattern with clear verb_noun structures (e.g., crawl_site, list_crawls, read_crawl_data). The naming is predictable and readable throughout the set, with no deviations in style or convention.
With 8 tools, this server is well-scoped for managing Screaming Frog crawls. Each tool earns its place by covering essential operations like starting, monitoring, exporting, and managing crawls, without being overly sparse or bloated. The count aligns perfectly with the domain's workflow needs.
The tool set provides complete CRUD/lifecycle coverage for Screaming Frog crawls: create (crawl_site), read (export_crawl, read_crawl_data), update (implied via crawl_status monitoring), delete (delete_crawl), and management (list_crawls, storage_summary, sf_check). There are no obvious gaps, and agents can handle all core tasks without dead ends.
Maintenance
Related MCP Connectors
Crawlbase MCP — wraps the Crawlbase Crawling API (crawlbase.com, formerly
Official SerpApi MCP server for Google, Bing, and other search engines.
One MCP server for 180+ live web-data APIs returning clean JSON from sites that block scrapers.
- CalmSEOOAuthcom.calmseo
SEO MCP server for keyword research, SERP analysis, audits, and Search Console workflows.
Related MCP Servers
- FlicenseBqualityDmaintenanceAn MCP Server for Web scraping and Crawling, built using Crawl4AI224-
- AlicenseAqualityAmaintenanceEnables AI assistants to crawl websites, export crawl data, and manage crawl storage using Screaming Frog SEO Spider through the MCP protocol.983MIT
- AlicenseAqualityDmaintenanceA headless web-scraping MCP server built on Scrapy, providing tools for polite fetching, CSS/XPath extraction, link/table extraction, sitemap and robots.txt reading, and bounded asynchronous crawls.10MIT
- AlicenseNot gradedqualityCmaintenanceMCP server for controlling Screaming Frog SEO Spider through AI assistants, enabling crawling, SEO issue analysis, and quick page checks via natural language commands.1MIT