Skip to main content
Glama
Site Crawler for Links and Tables MCP Server

Site Crawler for Links and Tables MCP Server

Site Crawler for Links and Tables MCP Server

npm version license

An MCP server for the Mamba Labs Site Crawler for Links and Tables actor on Apify. Crawl a website and get its links, download links, tables, and lists as rows.

What's Inside

Related MCP server: Crawl4AI MCP Server

What it does

Give a start address. The actor crawls the site over plain HTTP, following links and the sitemap, and returns one row per page with internal links, external links, file download links, every HTML table as rows, and every list. The output is deterministic, with no AI. robots.txt is obeyed by default. Pages built by JavaScript return little, because no browser runs.

All of the work runs on Apify. This package is a thin client that starts the actor run, waits for it, and hands back the dataset rows.

Quick start

You need Node.js 18 or newer and an Apify account with an API token.

Add this to your Claude Desktop config (claude_desktop_config.json), Cursor, or any MCP client:

{
  "mcpServers": {
    "mamba-site-crawler-links-tables": {
      "command": "npx",
      "args": [
        "-y",
        "@mambalabsdev/mcp-site-crawler-links-tables"
      ],
      "env": {
        "APIFY_TOKEN": "your-apify-token"
      }
    }
  }
}

Restart the client. The crawl_site_links_tables tool appears.

Prerequisites

Example prompts

  • "List every external download link on this website."

  • "Pull all the tables from these pages into rows."

  • "Crawl this site and show me which pages link out to other domains."

Tool

crawl_site_links_tables. One tool. Example arguments:

{
  "start_urls": [
    "https://webscraper.io/test-sites/tables"
  ],
  "extract": [
    "tables",
    "links"
  ],
  "max_pages": 10,
  "max_depth": 1
}

Inputs

Every buyer facing input of the actor. Omitted inputs take the actor's own defaults.

Input

Type

Required

Description

start_urls

array

no

Web addresses to start from, up to 100. A bare domain such as example.com is read as https.

max_pages

number

no

Pages to read across the whole run, up to 10,000. Default 100.

max_depth

number

no

How many links to follow from a start address. 0 reads only the start address. Default 10.

same_domain_only

string

no

yes (default): follow links only on the start address's host and its subdomains. no: also follow links to other sites, up to Maximum pages. Links to other sites are listed either way.

extract

array

no

Any of links, tables, lists, download_links. Default links, tables, lists. download_links adds one separate row per file link, charged as download-link-row; the download_links field on page rows is always filled at no extra charge.

download_extensions

array

no

File extensions that count as download links. Default pdf, zip, xlsx, xls, docx, doc, pptx, csv, mp3, mp4, exe, dmg, msi, pkg, gz, tar, tgz, 7z, rar, iso, apk. Files are listed, never downloaded.

use_sitemap

string

no

yes (default): also queue the addresses in /sitemap.xml, which finds pages no link reaches. no: follow links only.

obey_robots

string

no

yes (default): skip pages robots.txt disallows and honor its crawl delay up to 5 seconds. no: read them, only if you have the right to.

max_charge_usd

string

no

Optional. The run stops before a charge that would pass this amount. Summary rows are always returned.

proxy_mode

string

no

pool (default): the Apify proxy pool only, charged as page-row; a page that refuses it gets a free blocked row. auto: a refused page is retried on residential addresses, and a page read that way is charged as residential-page-row. residential: every page uses residential addresses and is charged as residential-page-row.

Pricing

Pay per event on Apify, billed to your Apify account. Prices in USD.

Event

Free plan

Bronze

Silver

Gold

page-row, one page read on the proxy pool

$0.006

$0.0057

$0.0054

$0.0051

residential-page-row, one page read on residential addresses, only with proxy_mode auto or residential

$0.02

$0.019

$0.018

$0.017

download-link-row, one separate file link row, only when extract includes download_links

$0.0001

$0.000095

$0.00009

$0.000085

apify-actor-start, per run start

$0.0055

$0.0055

$0.0055

$0.0055

Failed or refused pages, summary rows, pages skipped by robots.txt, and refused input are free. Mamba Labs pays the platform costs of the run, Apify proxy traffic included. Mamba Labs pays the platform costs of the run, Apify proxy traffic included.

Full actor documentation

apify.com/mambalabs/site-crawler-links-tables

License

MIT. See LICENSE.

Available Tools

1 tool

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev1.0.0
    • First observedcrawl_site_links_tables

TDQS

A4.1/5.0

Scored across 1 tool

Disambiguation5/5

There is only one tool, so there is no possible overlap or misselection risk. Its purpose (crawl a site and return links, tables, and lists) is stated explicitly and leaves no ambiguity about when to use it.

Naming Consistency4/5

The single name crawl_site_links_tables follows snake_case with a leading verb, which is a predictable convention. It is descriptive but concatenates multiple output types into one long name, which is slightly less clean than a tighter verb_noun form.

Tool Count3/5

One tool is thin for a server, though this tool is substantial and bundles several extraction outputs rather than being trivial. A single entry point is workable for a focused crawler, but splitting configuration or output modes would make the surface easier to target.

Completeness4/5

The tool covers the stated domain well: internal/external/file links, tables, and lists, with robots.txt compliance noted. Gaps are minor-to-moderate: no depth or max-page control surfaced, no JS rendering fallback, and no way to isolate a single page versus a full crawl.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    Enables web content scanning and analysis by fetching, analyzing, and extracting information from web pages using tools like page fetching, link extraction, site crawling, and more.
    6
    13
    MIT
  • F
    license
    Not graded
    quality
    D
    maintenance
    Enables advanced web crawling and content extraction with JavaScript support, AI-powered analysis, PDF/Office document processing, YouTube transcript extraction, Google search integration, and multi-format data export capabilities.
    2
    -
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables web crawling and content extraction from web pages, supporting multiple output formats like text, markdown, XML, and JSON, with robots.txt compliance and rate limiting.
    14 npm
    1
    MIT