Skip to main content
Glama
libering

OmniArchive-MCP

by libering

KeepLink-MCP

A local daemon that lets AI agents archive web pages to the Wayback Machine in the background, so they don't block on Internet Archive's slow/rate-limited API.

Works with any MCP-compatible client (Kiro, Cursor, Claude Desktop, etc).

Version: 1.2.0

Why?

When AI agents do deep research, they read a lot of web pages. Those pages disappear all the time (link rot). Archiving them to the Internet Archive is the obvious fix — but IA's API is slow and rate-limited. If your agent calls it synchronously, it blocks for seconds or gets 429'd.

This tool solves that. The agent calls archive_url, gets a task ID back in milliseconds, and moves on. A background worker handles the actual archiving with proper retry logic.

Related MCP server: Clone Space MCP

What it does

  • Exposes archive_url and get_archive_status as MCP tools

  • Queues requests in local SQLite (survives restarts, no Redis needed)

  • Background worker retries with exponential backoff on 429/5xx

  • Deduplicates — same URL within 24h won't be re-archived

  • Binds to localhost only, no telemetry, nothing phones home

Architecture

AI Client (Kiro/Cursor)
    │ stdio (MCP JSON-RPC)
    ▼
┌─────────────────────┐
│   MCP Server        │  ← Separate process
│   (archive_url,     │
│    get_archive_status)│
└────────┬────────────┘
         │ httpx (localhost:19210)
┌────────▼────────────────────────┐
│   FastAPI Service + Worker      │  ← Main process
│   ┌─────────┐ ┌──────────────┐ │
│   │  Routes  │ │ Background   │ │
│   │  /api/*  │ │ Worker       │ │
│   └────┬─────┘ └──────┬───────┘ │
│        │               │         │
│   ┌────▼───────────────▼───────┐ │
│   │    SQLite (WAL mode)       │ │
│   └────────────────────────────┘ │
└──────────────────────────────────┘
         │
         │ waybackpy
         ▼
   Internet Archive SPN2

Two processes: the MCP server talks stdio with your AI client, and forwards requests over HTTP to the FastAPI service. The FastAPI service manages the queue and runs the background worker.

Getting started

You need Python 3.10+.

git clone https://github.com/libering/keeplink-mcp.git
cd keeplink-mcp
pip install -e ".[dev]"

Start the backend service:

python -m keeplink_mcp.main

Runs on 127.0.0.1:19210 by default.

Run the tests:

pytest test/ -v   # 84 tests, takes ~12s

Docker

Run with Docker (v1.1.0+):

# Build the image
docker build -t keeplink-mcp .

# Run the container
docker run -d \
  --name keeplink \
  -p 19210:19210 \
  -v keeplink-data:/app/data \
  -e KEEPLINK_DB_PATH=/app/data/task.db \
  -e KEEPLINK_LOG_FILE=/app/data/archiver.log \
  keeplink-mcp

Or use docker-compose:

# Start the service
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the service
docker-compose down

The docker-compose.yml maps port 19210 and creates a persistent volume for the SQLite database.

MCP tools

archive_url

Queue a URL for archiving.

Param

Type

Required

url

string

yes

Must be http or https

Returns: { "task_id": "...", "status": "pending", "url": "..." }

get_archive_status

Check on a task. Pass either task_id or url (at least one).

Param

Type

Required

task_id

string

no

The ID from archive_url

url

string

no

Looks up the most recent task for this URL

Returns the task status, archive URL (if done), error info (if failed).

get_archive_status_batch

Query status of multiple tasks in a single request (v1.1.0+).

Param

Type

Required

task_ids

string

no

Comma-separated task IDs (max 50 combined with urls)

urls

string

no

Comma-separated URLs (max 50 combined with task_ids)

Returns: { "results": [...], "total_requested": N, "total_found": M }

Each result contains the same fields as get_archive_status. Non-matching identifiers are silently omitted.

archive_and_cite

Archive a cited web source and get back a paste-ready structured citation in one call. Non-blocking, same as archive_url — it returns immediately without waiting for archiving to finish.

Param

Type

Required

url

string

yes

Must be http or https

title

string

no

The page title you already read; used as the citation link text

Returns a Citation object: { "title": ..., "original_url": ..., "archived_url": ..., "archived_at": ..., "task_id": ..., "formatted": ... }.

The response comes in one of two shapes depending on archive state:

  • Cache_Hit — if the URL was already archived within the 24h dedup window, you get a complete citation immediately: archived_url points at the permanent Wayback Machine snapshot, archived_at is set, and formatted is a ready-to-paste markdown reference like [title](archived_url) (original: original_url, archived YYYY-MM-DD). When you don't supply a title, the archived_url is used as the link text instead.

  • Pending_Citation — otherwise archiving is queued/in progress: archived_url and archived_at are null, and formatted states that archiving is in progress and includes the original_url and task_id. Call get_archive_status(task_id) later to obtain the permanent archived_url and complete the citation.

KeepLink stays fully deterministic and LLM-free when producing a citation: it does not fetch the page content (supply the title yourself) and does not invoke any large language model.

HTTP API Endpoints

Health Check

GET /api/health — Check service liveness, readiness, and operational stats (v1.1.0+).

Response (200 when healthy, 503 when degraded):

{
  "liveness": true,
  "readiness": true,
  "readiness_error": null,
  "stats": {
    "queue_depth": 5,
    "success_count": 42,
    "failure_count": 3
  }
}
  • liveness: Always true if the server responds

  • readiness: true only when database is reachable and worker has completed at least one poll cycle

  • stats.queue_depth: Number of pending tasks

  • stats.success_count: Successfully archived tasks (last 24h)

  • stats.failure_count: Failed tasks (last 24h)

Batch Status Query

GET /api/status/batch — Query multiple tasks at once (v1.1.0+).

Query Parameters:

Param

Description

task_ids

Comma-separated task IDs

urls

Comma-separated URLs

Constraints:

  • At least one parameter required

  • Combined total of identifiers ≤ 50

  • Returns HTTP 422 if constraints violated

Response:

{
  "results": [
    {
      "task_id": "abc123",
      "url": "https://example.com",
      "status": "success",
      "result_url": "https://web.archive.org/...",
      "error_message": null,
      "retry_count": 0,
      "created_at": "2024-01-15T10:30:00Z",
      "updated_at": "2024-01-15T10:35:00Z"
    }
  ],
  "total_requested": 3,
  "total_found": 1
}

Hooking it up to your editor

You need both: the backend service running, AND the MCP server configured in your client.

Kiro — .kiro/settings/mcp.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Cursor — .cursor/mcp.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Claude Desktop — claude_desktop_config.json:

{
  "mcpServers": {
    "keeplink": {
      "command": "python",
      "args": ["-m", "keeplink_mcp.mcp_server.main"]
    }
  }
}

Don't forget to start the backend first: python -m keeplink_mcp.main

Configuration

Everything's controlled via env vars (prefix KEEPLINK_):

Core Settings

Variable

Default

What it does

KEEPLINK_API_HOST

127.0.0.1

Bind address

KEEPLINK_API_PORT

19210

Port

KEEPLINK_DB_PATH

./data/task.db

Where the SQLite file lives

KEEPLINK_MAX_RETRIES

5

How many times to retry a failed archive

KEEPLINK_BASE_BACKOFF

300.0

Base retry delay in seconds (doubles each time, 5min aligns with IA cooldown)

KEEPLINK_WORKER_CONCURRENCY

1

How many tasks to process per poll cycle

KEEPLINK_POLL_INTERVAL

5.0

Seconds between queue polls

KEEPLINK_IA_ACCESS_KEY

—

Your IA S3 key (optional, for higher rate limits)

KEEPLINK_IA_SECRET_KEY

—

Your IA S3 secret

KEEPLINK_LOG_LEVEL

INFO

Log verbosity

KEEPLINK_LOG_FILE

./data/archiver.log

Log file location

Rate Limiting (v1.1.0+)

Variable

Default

What it does

KEEPLINK_RATE_LIMIT_TOKENS

7

Max tokens in bucket (burst capacity). Set to 0 to disable rate limiting.

KEEPLINK_RATE_LIMIT_INTERVAL_SEC

60.0

Seconds between token refills

KEEPLINK_RATE_LIMIT_TIMEOUT

30.0

Max seconds to wait for a token before retrying later

Log Rotation (v1.1.0+)

Variable

Default

What it does

KEEPLINK_LOG_MAX_BYTES

10485760

Max log file size before rotation (10 MB default)

KEEPLINK_LOG_BACKUP_COUNT

5

Number of rotated log files to keep

How it works under the hood

  1. Agent calls archive_url → MCP server validates the URL → POSTs to FastAPI

  2. FastAPI checks if this URL was already submitted in the last 24h (dedup) → writes to SQLite → returns task ID

  3. Worker picks up pending tasks every 5s → calls Internet Archive via waybackpy

  4. Success? Stores the archive URL. Got 429/5xx? Backs off and retries. Got 403? Gives up.

License

MIT. See LICENSE.

Contributing

See CONTRIBUTING.md.

Available Tools

4 tools
archive_and_citeA

Archive a cited web source AND get back a paste-ready citation in one call. Non-blocking (<50ms): archiving runs in the background. If the URL was already archived within 24h, you get a COMPLETE citation immediately (formatted markdown link to the permanent Wayback Machine URL). Otherwise you get a PENDING citation with a task_id — call get_archive_status with that task_id later to obtain the permanent archived_url and complete the citation. KeepLink does NOT fetch page content or call any LLM; supply the page title yourself if you want it in the citation. Response fields: title, original_url, archived_url, archived_at, task_id, formatted.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe source URL to archive and cite (http or https).
titleNoOptional page title you already read; used as the citation link text. Whitespace-only titles are treated as absent.
formatNoCitation output format for the `formatted` field. One of markdown (default), bibtex, apa, plain. Only affects `formatted`; structured fields are unchanged.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses non-blocking latency (<50ms), background execution, the 24h dedupe cache that yields an immediate complete citation, and explicit non-behaviors (no page-content fetch, no LLM calls). It also enumerates the returned fields, which matters because there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core compound action and latency, then flows into the branching behavior and caveats. It is fairly dense but every sentence contributes; the trailing 'Response fields:' list is slightly list-like but justified given the absence of an output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-param async tool with no output schema and no annotations, the description covers the call semantics, the async follow-up path via get_archive_status, the caching shortcut, and the return payload. An agent has everything needed to invoke it and handle both response shapes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds rationale beyond the schema: it explains that the title must be supplied by the caller precisely because the tool does not fetch content, and that format only affects the `formatted` field. That connects the parameters to the tool's behavior rather than restating field docs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific compound action (archive + return a paste-ready citation) and distinguishes itself from the sibling archive_url by the 'AND get back a paste-ready citation' clause. It also clarifies the two possible outcomes (complete vs pending) so an agent knows exactly what the call produces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear operational routing: if a task_id is returned, call get_archive_status later to finish the citation. That is a genuine when-to-use-next instruction. It stops short of explicitly saying when to prefer this over archive_url, leaving that comparison to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

archive_urlA

Submit a URL for archiving to the Internet Archive. Returns immediately with a task_id (non-blocking, <50ms). The actual archiving happens asynchronously in the background. Use this to preserve web page evidence during research — pages may change or disappear, so archive early and often. Duplicate submissions within 24 hours are deduplicated automatically. Response fields: task_id (tracking ID), status (always 'pending' for new submissions), url (submitted URL), is_deduplicated (true if an existing task was reused).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to archive (must be http or https). The URL is validated and normalized before submission.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses non-blocking behavior (<50ms), background async processing, 24-hour dedup semantics, and the exact response fields including is_deduplicated. These are precisely the behavioral traits an agent needs and none are in structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and latency characteristics, then usage rationale, then response fields. Every sentence carries information, though the trailing enumeration of response fields is slightly list-like.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description's enumeration of return fields (task_id, status, url, is_deduplicated) is essential and present. Combined with async and dedup semantics, an agent has everything needed to call and interpret this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and there is a single well-documented 'url' parameter, so the schema already carries the validation/normalization detail. The description adds no further parameter-level meaning beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Submit a URL for archiving to the Internet Archive') and immediately distinguishes the operation's character as non-blocking and asynchronous. This separates it cleanly from siblings like get_archive_status and retry_task without needing to name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to-use guidance ('preserve web page evidence during research... archive early and often') and notes automatic deduplication within 24 hours. It does not explicitly route to alternatives such as archive_and_cite, which appears to be a superset tool, so sibling selection is left partly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_archive_statusA

Check the status of an archive task. Provide either a task_id or a url (at least one required). Status values: 'PENDING' (queued, waiting to be processed), 'PROCESSING' (worker is archiving now), 'SUCCESS' (archived — result_url contains the Wayback Machine link), 'FAILED' (permanent failure after retries exhausted). When status is SUCCESS, use result_url as a permanent citation link. If status is PENDING/PROCESSING, check again after a few seconds.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNoThe URL to look up the most recent task for. Returns the latest task regardless of status.
task_idNoThe task ID returned by archive_url. Use this for precise lookup of a specific submission.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it enumerates all four status values with meaning, notes FAILED means retries are exhausted, and that a url lookup returns the latest task regardless of status. It omits auth/rate-limit expectations, but the state semantics are genuinely informative.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then the input rule, then status semantics, then next-action advice. Slightly long because of the status enumeration, but every clause carries actionable information and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter polling tool with no output schema and no annotations, the description covers inputs, status meanings, the result_url return value, and polling cadence. Only the FAILED/retry_task handoff is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so baseline is 3, but the description adds a real constraint the schema does not encode: the schema shows zero required parameters, yet the description states 'at least one required'. That extra rule materially affects correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Check the status of an archive task') and explicitly ties its inputs to the sibling archive_url ('task_id returned by archive_url'), so an agent can separate it from archive_url/retry_task/archive_and_cite without reading schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear invocation rules (supply task_id OR url, at least one) and follow-up guidance (use result_url as citation when SUCCESS; re-poll after a few seconds when PENDING/PROCESSING). It does not, however, route the agent anywhere for FAILED even though retry_task is a sibling, which is the one obvious missing branch.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

retry_taskA

Re-queue a permanently FAILED archive task so the background worker processes it again (e.g. after a transient 403/429). Only tasks whose status is 'failed' can be retried; the task is reset to 'pending' with its retry budget refreshed. Returns the task_id and new status ('pending'). If the task is not failed (pending/processing/success) the call is rejected; if the task_id does not exist a not-found error is returned.

ParametersJSON Schema
NameRequiredDescriptionDefault
task_idYesThe task_id of the failed task to re-queue.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses the mutation (reset to 'pending', retry budget refreshed), the precondition (status must be 'failed'), rejection on non-failed states, and a not-found error for a missing task_id.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four dense sentences, all carrying real information (precondition, effect, return, error cases) with the mutation front-loaded. Slightly longer than strictly necessary but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema and no annotations, the description specifies the return value (task_id and new status 'pending') and all failure modes, so an agent has everything needed to call it correctly and interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is fully documented in the schema, so the baseline is 3. The description reinforces that task_id must refer to a failed task but adds no format or syntax details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Re-queue') and resource ('permanently FAILED archive task') with an explicit example motivation (transient 403/429). This clearly distinguishes it from the sibling list/read tools (archive_url, get_archive_status, archive_and_cite).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives strong state-based gating: 'Only tasks whose status is failed can be retried' and enumerates the rejected states (pending/processing/success). It does not name a sibling alternative to use instead, but the when/when-not guidance is otherwise explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedarchive_and_cite
    • First observedarchive_url
    • First observedget_archive_status
    • First observedretry_task

TDQS

A4.1/5.0

Scored across 4 tools

Disambiguation3/5

archive_url and archive_and_cite both submit a URL for archiving, so an agent must decide between them; the citation convenience in archive_and_cite is the only real differentiator and descriptions acknowledge the overlap. retry_task and get_archive_status are clearly distinct.

Naming Consistency4/5

All four names are verb-first snake_case (archive_url, retry_task, get_archive_status, archive_and_cite), which is predictable overall. The compound 'archive_and_cite' deviates slightly from the verb_noun pattern but remains readable.

Tool Count4/5

Four tools fit the narrow URL-archiving domain well: submit, check status, retry, and cite. It is a touch lean but each tool covers a distinct stage of the workflow.

Completeness3/5

The set covers submission, status polling, retry, and citation, but there is no way to list existing tasks, cancel a pending archive, or batch-submit URLs—notable gaps for managing multiple tasks during research.

Maintenance

ActivityActive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Bridges AI agents to a real browser using a persistent daemon and Chrome extension for driving actual login sessions, cookies, and tabs.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables LLM agents to perform web search, scraping, and summarization outside their context window, receiving compact cited briefs via MCP while full pages are cached and viewable in a local web UI.
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    A local-first MCP server that lets AI agents read any webpage as clean Markdown, crawl whole sites within configured limits, search without API keys, and solve supported captchas locally — all without cloud services or third-party keys.
    30
    19
    AGPL 3.0