KeepLink-MCP

A local daemon that lets AI agents archive web pages to the Wayback Machine in the background, so they don't block on Internet Archive's slow/rate-limited API.

Works with any MCP-compatible client (Kiro, Cursor, Claude Desktop, etc).

Version: 1.1.0

Why?

When AI agents do deep research, they read a lot of web pages. Those pages disappear all the time (link rot). Archiving them to the Internet Archive is the obvious fix — but IA's API is slow and rate-limited. If your agent calls it synchronously, it blocks for seconds or gets 429'd.

This tool solves that. The agent calls archive_url, gets a task ID back in milliseconds, and moves on. A background worker handles the actual archiving with proper retry logic.

Related MCP server: MIDAS

What it does

• Exposes archive_url and get_archive_status as MCP tools

• Queues requests in local SQLite (survives restarts, no Redis needed)

• Background worker retries with exponential backoff on 429/5xx

• Deduplicates — same URL within 24h won't be re-archived

• Binds to localhost only, no telemetry, nothing phones home

Architecture

AI Client (Kiro/Cursor)
    │ stdio (MCP JSON-RPC)
    ▼
┌─────────────────────┐
│   MCP Server        │  ← Separate process
│   (archive_url,     │
│    get_archive_status)│
└────────┬────────────┘
         │ httpx (localhost:19210)
┌────────▼────────────────────────┐
│   FastAPI Service + Worker      │  ← Main process
│   ┌─────────┐ ┌──────────────┐ │
│   │  Routes  │ │ Background   │ │
│   │  /api/*  │ │ Worker       │ │
│   └────┬─────┘ └──────┬───────┘ │
│        │               │         │
│   ┌────▼───────────────▼───────┐ │
│   │    SQLite (WAL mode)       │ │
│   └────────────────────────────┘ │
└──────────────────────────────────┘
         │
         │ waybackpy
         ▼
   Internet Archive SPN2

Two processes: the MCP server talks stdio with your AI client, and forwards requests over HTTP to the FastAPI service. The FastAPI service manages the queue and runs the background worker.

Getting started

You need Python 3.10+.

git clone https://github.com/keeplink/keeplink-mcp.git
cd keeplink-mcp
pip install -e ".[dev]"

Start the backend service:

python -m keeplink_mcp.main

Runs on 127.0.0.1:19210 by default.

Run the tests:

pytest test/ -v   # 84 tests, takes ~12s

Docker

Run with Docker (v1.1.0+):

# Build the image
docker build -t keeplink-mcp .

# Run the container
docker run -d \
  --name keeplink \
  -p 19210:19210 \
  -v keeplink-data:/app/data \
  -e KEEPLINK_DB_PATH=/app/data/task.db \
  -e KEEPLINK_LOG_FILE=/app/data/archiver.log \
  keeplink-mcp

Or use docker-compose:

# Start the service
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the service
docker-compose down

The docker-compose.yml maps port 19210 and creates a persistent volume for the SQLite database.

MCP tools

archive_url

Queue a URL for archiving.

Returns: { "task_id": "...", "status": "pending", "url": "..." }

get_archive_status

Check on a task. Pass either task_id or url (at least one).

Returns the task status, archive URL (if done), error info (if failed).

get_archive_status_batch

Query status of multiple tasks in a single request (v1.1.0+).

Returns: { "results": [...], "total_requested": N, "total_found": M }

Each result contains the same fields as get_archive_status. Non-matching identifiers are silently omitted.

HTTP API Endpoints

Health Check

GET /api/health — Check service liveness, readiness, and operational stats (v1.1.0+).

Response (200 when healthy, 503 when degraded):

{
  "liveness": true,
  "readiness": true,
  "readiness_error": null,
  "stats": {
    "queue_depth": 5,
    "success_count": 42,
    "failure_count": 3
  }
}

• liveness: Always true if the server responds

• readiness: true only when database is reachable and worker has completed at least one poll cycle

• stats.queue_depth: Number of pending tasks

• stats.success_count: Successfully archived tasks (last 24h)

• stats.failure_count: Failed tasks (last 24h)

Batch Status Query

GET /api/status/batch — Query multiple tasks at once (v1.1.0+).

Query Parameters:

Constraints:

• At least one parameter required

• Combined total of identifiers ≤ 50

• Returns HTTP 422 if constraints violated

Response:

{
  "results": [
    {
      "task_id": "abc123",
      "url": "https://example.com",
      "status": "success",
      "result_url": "https://web.archive.org/...",
      "error_message": null,
      "retry_count": 0,
      "created_at": "2024-01-15T10:30:00Z",
      "updated_at": "2024-01-15T10:35:00Z"
    }
  ],
  "total_requested": 3,
  "total_found": 1
}

Hooking it up to your editor

You need both: the backend service running, AND the MCP server configured in your client.

Kiro — .kiro/settings/mcp.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Cursor — .cursor/mcp.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Claude Desktop — claude_desktop_config.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Don't forget to start the backend first: python -m keeplink_mcp.main

Configuration

Everything's controlled via env vars (prefix KEEPLINK_):

Core Settings

Rate Limiting (v1.1.0+)

Log Rotation (v1.1.0+)

How it works under the hood

1. Agent calls archive_url → MCP server validates the URL → POSTs to FastAPI

2. FastAPI checks if this URL was already submitted in the last 24h (dedup) → writes to SQLite → returns task ID

3. Worker picks up pending tasks every 5s → calls Internet Archive via waybackpy

4. Success? Stores the archive URL. Got 429/5xx? Backs off and retries. Got 403? Gives up.

License

MIT. See LICENSE.

Contributing

See CONTRIBUTING.md.