This server is an MCP integration for the CrawlGraph backlink-intelligence API, giving any MCP client backlink lookups and competitor analysis built on Common Crawl data.
backlinks – Look up every referring domain for a target domain, with host counts and CrawlGraph authority/rank; supports limits, sorting, and querying a specific Common Crawl release.
gap_analysis – Find domains that link to your competitors but not to you, including which competitors each domain links to (found_on).
gap_outreach_targets – The warm-outreach play: prioritize domains linking to all (or 2+) competitors, filter platform/CDN noise, and optionally authority-score the top targets.
backlink_changes – Compare two Common Crawl snapshots to see added, removed, and authority-moved referring domains over time.
releases – List the Common Crawl snapshots the API can query, free of charge.
Five MCP tools for backlink research using CrawlGraph
and Common Crawl snapshots: backlinks, backlink_changes, gap_analysis,
gap_outreach_targets, and releases. Outreach tools identify research
candidates; they do not send messages or create contacts.
Surface | Credentials | Tool profile |
npm / local stdio | Your | Legacy |
| Your API key in | Legacy |
| Account-linked OAuth access token in the Authorization header | Bounded directory profile, when enabled |
Legacy /mcp also accepts Smithery's ?apiKey=… and base64 JSON ?config=…
forms. Header credentials take precedence, then apiKey, then packed config.
Prefer the Authorization header. The connector endpoint rejects API keys,
query credentials and packed configuration; OAuth tokens never go to the
public /api/v1 API.
Directory approval and real Claude/OpenAI compatibility verification remain pending. No Verified listing or logo is claimed. This checkout's source version is 0.4.0; it does not establish the current npm or hosted version. Verify each distribution separately through the operator runbooks linked in DEPLOY.md.
Related MCP server: oncrawl-mcp-server
Obtain a key through CrawlGraph's API onboarding. The free-key flow emails a key; it does not sign you into an account.
For Claude Desktop / Claude Code, put this in your client's MCP configuration:
{
"mcpServers": {
"crawlgraph": {
"command": "npx",
"args": ["-y", "crawlgraph-mcp"],
"env": {
"CRAWLGRAPH_API_KEY": "cg_live_your_key_here"
}
}
}
}Cursor, Windsurf, Cline and Zed use the same command/environment arrangement.
Streamable HTTP clients can use https://crawlgraph.com/mcp with their own
bearer API key. Confirm the hosted tools/list response before relying on a
specific source feature.
Local stdio setting | Required | Default |
| Yes for tool execution; resolved lazily | — |
| No; legacy REST adapter only |
|
When the operator enables the reviewed service, connect to
https://crawlgraph.com/mcp/connectors using an existing CrawlGraph account,
sign in and approve consent in the original connection browser. Account
creation is outside OAuth. If you need an account, follow the disclosed
onboarding flow, then return and request sign-in; an emailed API key alone is
not a login session.
The issuer is https://crawlgraph.com, the exact resource is
https://crawlgraph.com/mcp/connectors, and the only scope is
crawlgraph:read. Public clients use authorization code plus S256 PKCE and
token_endpoint_auth_method=none. Omitted registration auth method is
normalized to none and any SDK-generated secret is discarded. Confidential
client methods and metadata-document fetching are unsupported. Callbacks
must be exact operator-approved HTTPS URLs. OpenAI's callback must come from
the actual portal; this repository does not supply a guessed callback.
Authorization and code exchange require the exact raw resource value. At code
exchange a supplied redirect_uri must exactly match the stored callback;
omission retains that callback. Refresh may omit resource and scope to retain
the original grant. PKCE is validated by the backend during the atomic code
exchange, alongside client/resource/redirect/scope checks.
Use account connection management to disconnect and revoke the entire grant
and refresh family. An expired access token can be refreshed while its grant
and refresh family remain valid; revoked or expired grants require
reconnecting. Discovery, OAuth and introspection outages return 503 without
a relink challenge. Execution outages return a safe tool error inside the
HTTP 200 MCP result, without relink metadata. Tool-time revocation is reported
through mcp/www_authenticate result metadata. Refresh rotation
has no retry grace: concurrent reuse revokes the family, and a lost token
response may require reconnecting. A 503 after a possible commit does not
promise an unused code or refresh token.
Both auth profiles share the account's UTC calendar-month counters. Free
accounts have 15 backlinks calls and 0 gap jobs; accounts with current paid
access have 1,000 backlinks calls and 50 gap jobs. Paid eligibility follows
the backend's current entitlement rules. Release inventory consumes no
research quota. backlink_changes shares the backlinks counter.
Directory inputs and costs:
Tool | Arguments | Cost |
|
| 1 backlinks call |
|
| 1 backlinks call |
|
| New submission: 1 gap job; polling/resume: 0 |
| Gap inputs plus | New submission: 1 gap job; optional N extra backlinks calls |
|
| 0 |
Directory lists default to 20, maximum 100. The complete serialized
UTF-8 CallToolResult, including structured data, text and metadata, is
limited to 65,536 bytes. Additional output clipping is disclosed in
crawlgraph/output metadata, retaining original observed totals, source caps,
quota accounting, provenance and job handles. Large exports belong on the
account/API surface.
Gap tools poll for at most 75 seconds within a 90-second tool budget. A
pending result includes job_id and the normalized request. Resume the same
tool with that handle and query; a fresh valid grant for the same owner can
resume after reconnecting. Polling costs zero gap jobs. Never automatically
submit a replacement after timeout, cancellation, network loss or a stale
worker: the accepted job may still complete. Cancellation stops further
polling. An accepted submission remains charged; a lost/malformed response
reports unknown consumption rather than claiming zero.
Node writes bounded JSON telemetry to stderr. Each directory tool call records
an internally generated correlation ID, fixed tool name (or unknown), final
envelope status, isError, allowlisted primary and partial error codes, latency,
and the exact final serialized UTF-8 result size, excluding transport framing.
Known charged-call counts and unknown-consumption categories come from the
final envelope. Pending cancellations and completed results with partial
revocation retain their distinct status and error fields. HTTP/OAuth errors
record only a fixed route category, status, allowlisted code and a separate
response-local correlation ID. Logging failures cannot change responses.
Logs omit arguments, domains, results, job/account/grant/client/source IDs,
caller request IDs, exception text, URLs, headers, configuration and credentials.
OAuth source admission groups IPv6 addresses by /64 only in the local bucket key; mapped IPv4 addresses use the same bucket as their IPv4 form. Private RPCs still receive the full normalized verified ingress source. The bounded bucket map fails closed at capacity without evicting live windows; the backend continues enforcing its own policies.
Outreach preserves the backend ranking by competitor overlap, authority and
domain. Platform-filter counts describe only the returned backend sample.
Enrichment defaults to 0; explicitly request enrich_authority_top=N
(0–5) for up to N extra backlinks calls on retained targets. Partial
failures retain completed results, per-item provenance, actual known charges,
the latest observed quota and any unknown consumption. Verified jobs use
their attested release for enrichment; legacy jobs keep null provenance and
record separate lookup releases.
Legacy tools retain their original fields and defaults: backlinks defaults to
1,000 rows at the public API, maximum 10,000; outreach uses enrich_top, default
10, maximum 25, with one backlinks call per enrichment. Legacy gap tools
submit and poll within their existing 90-second window and have no directory
job_id input. Check the selected profile before estimating quota use.
Common Crawl is a periodic observation, not live link monitoring. Named
release provenance does not prove that a page currently links to a target.
A removed referring domain means it was not observed in the newer snapshot,
not proven live deletion. Unavailable comparison differs from zero changes;
unknown totals and source-capped lower bounds differ from complete empty
results. Gap provenance is saved at execution and retained on later polling.
Older unattested artifacts report legacy_unverified and null release
identity. A referring-domain overlap suggests a research candidate, not that
a publisher has never heard of you.
Example research prompt:
Find backlink gaps for mine.example against a.example and b.example. If pending, resume the returned job without resubmitting. Report provenance, caps and quota use before proposing any optional authority enrichment.
npm install
npm run build
node --test tests/*.test.mjsThe build is also the TypeScript check; no separate lint script is configured. Tests use the installed SDK, local HTTP fixtures and fake private/public backend replies. They require permission to bind loopback sockets and never need production credentials or live provider access. Node fixtures do not prove the backend's SQLite transactions, quota isolation or query pinning; separate crawlback backend HTTP tests and the full release gates cover those.
Implementation references: HTTP surfaces, OAuth router, provider, private backend client, directory tools, and legacy server.
MIT
Look up referring domains (backlinks) for a single target domain from the Common Crawl webgraph. Returns each linking domain with host count and CrawlGraph authority score, plus the target's own authority/rank. Costs one backlinks call against the monthly quota (1,000/mo on lifetime).
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | Target domain, e.g. 'stripe.com'. | |
| limit | No | Max rows (1..10000, default 1000). | |
| sort | No | 'authority' (default) or 'hosts'. | |
| release_id | No | Common Crawl release id (defaults to latest; see the releases tool). |
| Name | Required | Description |
|---|---|---|
| domain | Yes | |
| release_id | Yes | |
| release_label | Yes | |
| total_linking_domains | Yes | |
| returned | Yes | |
| cg_authority | Yes | |
| cg_rank | Yes | |
| results | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. The description adds the critical quota usage detail (1,000/mo) and data source (Common Crawl webgraph), providing extra behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences. First sentence front-loads purpose and output, second adds quota info. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With good annotations, full schema, and output schema, the description covers purpose, source, output summary, and quota. Missing a note on default sort or pagination, but otherwise very complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3. The description does not add any parameter-specific meaning beyond what the schema already provides; it mainly describes output behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool looks up referring domains for a single target domain from Common Crawl, specifying the output (linking domain, host count, authority score, target authority/rank). This distinguishes it from siblings like gap_analysis or releases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions quota cost but does not explicitly state when to use this versus alternatives like gap_analysis or releases. It implies use for backlink data but lacks exclusion criteria or comparative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a competitor backlink gap analysis: find domains that link to one or more of your competitors but NOT to you. Submits an async job and polls until done (usually 5-30s). Returns every gap with found_on listing which competitors each domain links to. Costs one gap job against the monthly quota (50/mo on lifetime).
| Name | Required | Description | Default |
|---|---|---|---|
| my_domain | Yes | Your domain. | |
| competitor_domains | Yes | 1 to 5 competitor domains. |
| Name | Required | Description |
|---|---|---|
| my_domain | Yes | |
| competitor_domains | Yes | |
| total_gaps | Yes | |
| gaps | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond annotations: async job with polling (5-30s), return format with 'found_on' listing, and quota limits (50/month). Annotations already indicate readOnly, openWorld, idempotent, and non-destructive, and the description aligns without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences. The first sentence front-loads the purpose, followed by operational details and return info. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (context signals indicate true), the description adequately covers return format and execution behavior. Quota and async details are included. Slight improvement could mention output schema existence or pagination, but overall complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters (my_domain, competitor_domains) well-described in the schema. The description does not add extra parameter-specific semantics, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a competitor backlink gap analysis, specifically finding domains linking to competitors but not to the user's domain. It distinguishes itself from siblings like 'backlinks' and 'gap_outreach_targets' by specifying the unique gap analysis functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to find linking domains) and provides context on async job execution and quota costs. However, it does not explicitly state when not to use it or contrast with alternatives, though the sibling list implies distinct use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The warm-outreach play. Runs a gap analysis, then ranks results: PRIORITY = domains linking to ALL your competitors but not you (publishers who cover your whole space and have never heard of you), SECONDARY = domains linking to 2+ competitors. Platform/CDN noise is filtered, top N priority targets are scored by authority. Use 2-3 competitors. Costs one gap job + one backlinks call per enriched target.
| Name | Required | Description | Default |
|---|---|---|---|
| my_domain | Yes | Your domain. | |
| competitor_domains | Yes | 2 to 5 competitor domains (2-3 recommended). | |
| include_platforms | No | Keep platform/CDN/social domains in the list. Default false. | |
| enrich_top | No | Authority-score the top N priority targets. Default 10; each costs one backlinks call. 0 disables. |
| Name | Required | Description |
|---|---|---|
| my_domain | Yes | |
| competitor_domains | Yes | |
| priority_targets | Yes | |
| secondary_targets | Yes | |
| total_gaps | Yes | |
| platforms_filtered | Yes | |
| authority_enriched | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses costs ('one gap job + one backlinks call per enriched target'), noise filtering, and ranking behavior. Adds value beyond annotations (readOnlyHint, etc.) by detailing operational impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise 5 sentences, front-loaded with key purpose, each sentence adds unique value. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, ranking logic, costs, filtering, and parameter specifics. With output schema existing, no need to detail return values. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage). Description adds minor extra context (e.g., cost per enrich_top, default for include_platforms), but not substantial beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it finds outreach targets based on gap analysis, ranking priority and secondary domains. Distinct from sibling tools (backlinks, gap_analysis, releases) by combining both analyses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use 2-3 competitors' and describes the ranking logic, giving clear context. Does not include explicit when-not-to-use, but the purpose is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the Common Crawl releases the API can query. Does not count against any quota. Use a release id with the backlinks tool to query a specific snapshot.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| releases | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, destructiveHint. The description adds that it does not count against quota, which is valuable behavioral context beyond annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with purpose, no superfluous words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with no parameters and an output schema. The description fully explains purpose, quota impact, and how to use the output with a sibling tool. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline is 4. The description adds no parameter info, but none is needed since there are no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists Common Crawl releases the API can query, with a specific verb and resource. It distinguishes from siblings by mentioning using a release id with the backlinks tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it does not count against quota, and advises to use a release id with the backlinks tool to query a specific snapshot, providing clear context for when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.2.2backlinksgap_analysisgap_outreach_targetsreleasesScored across 4 tools
Each tool has a clearly distinct purpose: backlinks for single domain lookup, gap_analysis for competitor gap detection, gap_outreach_targets for ranked outreach targets, and releases for listing data snapshots. No overlap.
All tool names follow a consistent pattern of lowercase with underscores, using descriptive noun phrases (e.g., gap_analysis, gap_outreach_targets). No mixing of conventions.
With 4 tools, the set is well-scoped for a specialized backlink analysis server. Each tool is necessary and the count is appropriate for the domain.
Covers essential workflows: single domain lookup, competitor gap analysis, and enriched outreach targeting. Missing bulk queries or historical comparisons, but the releases tool enables snapshot selection, mitigating gaps.
CrawlGraph MCP — backlink intelligence on the public Common Crawl webgraph
Remote MCP server that gives Claude, ChatGPT, Cursor or any MCP client Google search volume, CPC, competition and similar keywords for 50+ countries. Works with no signup at 20 lookups a day; sign in with OAuth or an API key for higher limits, or $4.99 a month for unlimited.
Your agent needs the link graph — who points at a competitor, what anchor text they use, what was won or lost last month, and which sites link to all of your rivals but not to you. **What you can ask for** • "Who links to my competitor and not to me?" • "What anchors point at this domain, and how spammy are the sources?" • "Which links did this site win and lose in the last 30 days?" • "Compare the referring domains of these five competitors." • "Summarise the backlink profile of these 100 domains in bulk." **How to use it** Point any MCP client at https://mcp.aisa.one/seo-backlinks/mcp and sign in with OAuth — there is no key to create or paste. 27 tools: backlinks and referring domains, anchors, new and lost links over time, domain and page intersections, referring networks, spam scores, bulk summaries for many domains at once, plus Semrush's own backlink and indexed-page sets. **Why this rather than the source** Two independent link indexes behind one account — coverage differs, and the gap is usually the interesting part. **It is also a door to the rest** The same login reaches 26 sources and 580+ operations. Find the link gap here, then ask the same agent who runs those sites and how to reach them — without adding a second server. **What it costs** Finding and inspecting an operation is free. Running one is billed per call at API prices, with no seat and no monthly minimum, and every call takes max_price_usd so an agent cannot overspend by accident. **Where else it reaches** https://mcp.aisa.one/seo/mcp for all of it at once — rankings, keywords, backlinks, site health and AI-answer visibility across DataForSEO, Semrush and Ahrefs.
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
