OmniArchive-MCP
Allows archiving web pages to the Internet Archive's Wayback Machine, with queue management and retry logic.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@OmniArchive-MCParchive https://example.com/article"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
KeepLink-MCP
A local daemon that lets AI agents archive web pages to the Wayback Machine in the background, so they don't block on Internet Archive's slow/rate-limited API.
Works with any MCP-compatible client (Kiro, Cursor, Claude Desktop, etc).
Version: 1.2.0
Why?
When AI agents do deep research, they read a lot of web pages. Those pages disappear all the time (link rot). Archiving them to the Internet Archive is the obvious fix — but IA's API is slow and rate-limited. If your agent calls it synchronously, it blocks for seconds or gets 429'd.
This tool solves that. The agent calls archive_url, gets a task ID back in milliseconds, and moves on. A background worker handles the actual archiving with proper retry logic.
Related MCP server: Clone Space MCP
What it does
Exposes
archive_urlandget_archive_statusas MCP toolsQueues requests in local SQLite (survives restarts, no Redis needed)
Background worker retries with exponential backoff on 429/5xx
Deduplicates — same URL within 24h won't be re-archived
Binds to localhost only, no telemetry, nothing phones home
Architecture
AI Client (Kiro/Cursor)
│ stdio (MCP JSON-RPC)
▼
┌─────────────────────┐
│ MCP Server │ ← Separate process
│ (archive_url, │
│ get_archive_status)│
└────────┬────────────┘
│ httpx (localhost:19210)
┌────────▼────────────────────────┐
│ FastAPI Service + Worker │ ← Main process
│ ┌─────────┐ ┌──────────────┐ │
│ │ Routes │ │ Background │ │
│ │ /api/* │ │ Worker │ │
│ └────┬─────┘ └──────┬───────┘ │
│ │ │ │
│ ┌────▼───────────────▼───────┐ │
│ │ SQLite (WAL mode) │ │
│ └────────────────────────────┘ │
└──────────────────────────────────┘
│
│ waybackpy
▼
Internet Archive SPN2Two processes: the MCP server talks stdio with your AI client, and forwards requests over HTTP to the FastAPI service. The FastAPI service manages the queue and runs the background worker.
Getting started
You need Python 3.10+.
git clone https://github.com/libering/keeplink-mcp.git
cd keeplink-mcp
pip install -e ".[dev]"Start the backend service:
python -m keeplink_mcp.mainRuns on 127.0.0.1:19210 by default.
Run the tests:
pytest test/ -v # 84 tests, takes ~12sDocker
Run with Docker (v1.1.0+):
# Build the image
docker build -t keeplink-mcp .
# Run the container
docker run -d \
--name keeplink \
-p 19210:19210 \
-v keeplink-data:/app/data \
-e KEEPLINK_DB_PATH=/app/data/task.db \
-e KEEPLINK_LOG_FILE=/app/data/archiver.log \
keeplink-mcpOr use docker-compose:
# Start the service
docker-compose up -d
# View logs
docker-compose logs -f
# Stop the service
docker-compose downThe docker-compose.yml maps port 19210 and creates a persistent volume for the SQLite database.
MCP tools
archive_url
Queue a URL for archiving.
Param | Type | Required | |
url | string | yes | Must be http or https |
Returns: { "task_id": "...", "status": "pending", "url": "..." }
get_archive_status
Check on a task. Pass either task_id or url (at least one).
Param | Type | Required | |
task_id | string | no | The ID from archive_url |
url | string | no | Looks up the most recent task for this URL |
Returns the task status, archive URL (if done), error info (if failed).
get_archive_status_batch
Query status of multiple tasks in a single request (v1.1.0+).
Param | Type | Required | |
task_ids | string | no | Comma-separated task IDs (max 50 combined with urls) |
urls | string | no | Comma-separated URLs (max 50 combined with task_ids) |
Returns: { "results": [...], "total_requested": N, "total_found": M }
Each result contains the same fields as get_archive_status. Non-matching identifiers are silently omitted.
archive_and_cite
Archive a cited web source and get back a paste-ready structured citation in one call. Non-blocking, same as archive_url — it returns immediately without waiting for archiving to finish.
Param | Type | Required | |
url | string | yes | Must be http or https |
title | string | no | The page title you already read; used as the citation link text |
Returns a Citation object: { "title": ..., "original_url": ..., "archived_url": ..., "archived_at": ..., "task_id": ..., "formatted": ... }.
The response comes in one of two shapes depending on archive state:
Cache_Hit — if the URL was already archived within the 24h dedup window, you get a complete citation immediately:
archived_urlpoints at the permanent Wayback Machine snapshot,archived_atis set, andformattedis a ready-to-paste markdown reference like[title](archived_url) (original: original_url, archived YYYY-MM-DD). When you don't supply atitle, thearchived_urlis used as the link text instead.Pending_Citation — otherwise archiving is queued/in progress:
archived_urlandarchived_atarenull, andformattedstates that archiving is in progress and includes theoriginal_urlandtask_id. Callget_archive_status(task_id)later to obtain the permanentarchived_urland complete the citation.
KeepLink stays fully deterministic and LLM-free when producing a citation: it does not fetch the page content (supply the title yourself) and does not invoke any large language model.
HTTP API Endpoints
Health Check
GET /api/health — Check service liveness, readiness, and operational stats (v1.1.0+).
Response (200 when healthy, 503 when degraded):
{
"liveness": true,
"readiness": true,
"readiness_error": null,
"stats": {
"queue_depth": 5,
"success_count": 42,
"failure_count": 3
}
}liveness: Alwaystrueif the server respondsreadiness:trueonly when database is reachable and worker has completed at least one poll cyclestats.queue_depth: Number of pending tasksstats.success_count: Successfully archived tasks (last 24h)stats.failure_count: Failed tasks (last 24h)
Batch Status Query
GET /api/status/batch — Query multiple tasks at once (v1.1.0+).
Query Parameters:
Param | Description |
| Comma-separated task IDs |
| Comma-separated URLs |
Constraints:
At least one parameter required
Combined total of identifiers ≤ 50
Returns HTTP 422 if constraints violated
Response:
{
"results": [
{
"task_id": "abc123",
"url": "https://example.com",
"status": "success",
"result_url": "https://web.archive.org/...",
"error_message": null,
"retry_count": 0,
"created_at": "2024-01-15T10:30:00Z",
"updated_at": "2024-01-15T10:35:00Z"
}
],
"total_requested": 3,
"total_found": 1
}Hooking it up to your editor
You need both: the backend service running, AND the MCP server configured in your client.
Kiro — .kiro/settings/mcp.json:
{
"mcpServers": {
"keeplink": {
"command": "python",
"args": ["-m", "keeplink_mcp.mcp_server.main"]
}
}
}Cursor — .cursor/mcp.json:
{
"mcpServers": {
"keeplink": {
"command": "python",
"args": ["-m", "keeplink_mcp.mcp_server.main"]
}
}
}Claude Desktop — claude_desktop_config.json:
{
"mcpServers": {
"keeplink": {
"command": "python",
"args": ["-m", "keeplink_mcp.mcp_server.main"]
}
}
}Don't forget to start the backend first: python -m keeplink_mcp.main
Configuration
Everything's controlled via env vars (prefix KEEPLINK_):
Core Settings
Variable | Default | What it does |
|
| Bind address |
|
| Port |
|
| Where the SQLite file lives |
|
| How many times to retry a failed archive |
|
| Base retry delay in seconds (doubles each time, 5min aligns with IA cooldown) |
|
| How many tasks to process per poll cycle |
|
| Seconds between queue polls |
| — | Your IA S3 key (optional, for higher rate limits) |
| — | Your IA S3 secret |
|
| Log verbosity |
|
| Log file location |
Rate Limiting (v1.1.0+)
Variable | Default | What it does |
|
| Max tokens in bucket (burst capacity). Set to |
|
| Seconds between token refills |
|
| Max seconds to wait for a token before retrying later |
Log Rotation (v1.1.0+)
Variable | Default | What it does |
|
| Max log file size before rotation (10 MB default) |
|
| Number of rotated log files to keep |
How it works under the hood
Agent calls
archive_url→ MCP server validates the URL → POSTs to FastAPIFastAPI checks if this URL was already submitted in the last 24h (dedup) → writes to SQLite → returns task ID
Worker picks up pending tasks every 5s → calls Internet Archive via waybackpy
Success? Stores the archive URL. Got 429/5xx? Backs off and retries. Got 403? Gives up.
License
MIT. See LICENSE.
Contributing
See CONTRIBUTING.md.
Available Tools
4 toolsarchive_and_citeA
Archive a cited web source AND get back a paste-ready citation in one call. Non-blocking (<50ms): archiving runs in the background. If the URL was already archived within 24h, you get a COMPLETE citation immediately (formatted markdown link to the permanent Wayback Machine URL). Otherwise you get a PENDING citation with a task_id — call get_archive_status with that task_id later to obtain the permanent archived_url and complete the citation. KeepLink does NOT fetch page content or call any LLM; supply the page title yourself if you want it in the citation. Response fields: title, original_url, archived_url, archived_at, task_id, formatted.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The source URL to archive and cite (http or https). | |
| title | No | Optional page title you already read; used as the citation link text. Whitespace-only titles are treated as absent. | |
| format | No | Citation output format for the `formatted` field. One of markdown (default), bibtex, apa, plain. Only affects `formatted`; structured fields are unchanged. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so well: it discloses non-blocking latency (<50ms), background execution, the 24h dedupe cache that yields an immediate complete citation, and explicit non-behaviors (no page-content fetch, no LLM calls). It also enumerates the returned fields, which matters because there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core compound action and latency, then flows into the branching behavior and caveats. It is fairly dense but every sentence contributes; the trailing 'Response fields:' list is slightly list-like but justified given the absence of an output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-param async tool with no output schema and no annotations, the description covers the call semantics, the async follow-up path via get_archive_status, the caching shortcut, and the return payload. An agent has everything needed to invoke it and handle both response shapes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds rationale beyond the schema: it explains that the title must be supplied by the caller precisely because the tool does not fetch content, and that format only affects the `formatted` field. That connects the parameters to the tool's behavior rather than restating field docs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific compound action (archive + return a paste-ready citation) and distinguishes itself from the sibling archive_url by the 'AND get back a paste-ready citation' clause. It also clarifies the two possible outcomes (complete vs pending) so an agent knows exactly what the call produces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear operational routing: if a task_id is returned, call get_archive_status later to finish the citation. That is a genuine when-to-use-next instruction. It stops short of explicitly saying when to prefer this over archive_url, leaving that comparison to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
archive_urlA
Submit a URL for archiving to the Internet Archive. Returns immediately with a task_id (non-blocking, <50ms). The actual archiving happens asynchronously in the background. Use this to preserve web page evidence during research — pages may change or disappear, so archive early and often. Duplicate submissions within 24 hours are deduplicated automatically. Response fields: task_id (tracking ID), status (always 'pending' for new submissions), url (submitted URL), is_deduplicated (true if an existing task was reused).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to archive (must be http or https). The URL is validated and normalized before submission. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so well: it discloses non-blocking behavior (<50ms), background async processing, 24-hour dedup semantics, and the exact response fields including is_deduplicated. These are precisely the behavioral traits an agent needs and none are in structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and latency characteristics, then usage rationale, then response fields. Every sentence carries information, though the trailing enumeration of response fields is slightly list-like.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description's enumeration of return fields (task_id, status, url, is_deduplicated) is essential and present. Combined with async and dedup semantics, an agent has everything needed to call and interpret this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and there is a single well-documented 'url' parameter, so the schema already carries the validation/normalization detail. The description adds no further parameter-level meaning beyond what the schema states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Submit a URL for archiving to the Internet Archive') and immediately distinguishes the operation's character as non-blocking and asynchronous. This separates it cleanly from siblings like get_archive_status and retry_task without needing to name them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear when-to-use guidance ('preserve web page evidence during research... archive early and often') and notes automatic deduplication within 24 hours. It does not explicitly route to alternatives such as archive_and_cite, which appears to be a superset tool, so sibling selection is left partly to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_archive_statusA
Check the status of an archive task. Provide either a task_id or a url (at least one required). Status values: 'PENDING' (queued, waiting to be processed), 'PROCESSING' (worker is archiving now), 'SUCCESS' (archived — result_url contains the Wayback Machine link), 'FAILED' (permanent failure after retries exhausted). When status is SUCCESS, use result_url as a permanent citation link. If status is PENDING/PROCESSING, check again after a few seconds.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | The URL to look up the most recent task for. Returns the latest task regardless of status. | |
| task_id | No | The task ID returned by archive_url. Use this for precise lookup of a specific submission. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does well: it enumerates all four status values with meaning, notes FAILED means retries are exhausted, and that a url lookup returns the latest task regardless of status. It omits auth/rate-limit expectations, but the state semantics are genuinely informative.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose, then the input rule, then status semantics, then next-action advice. Slightly long because of the status enumeration, but every clause carries actionable information and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter polling tool with no output schema and no annotations, the description covers inputs, status meanings, the result_url return value, and polling cadence. Only the FAILED/retry_task handoff is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3, but the description adds a real constraint the schema does not encode: the schema shows zero required parameters, yet the description states 'at least one required'. That extra rule materially affects correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Check the status of an archive task') and explicitly ties its inputs to the sibling archive_url ('task_id returned by archive_url'), so an agent can separate it from archive_url/retry_task/archive_and_cite without reading schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear invocation rules (supply task_id OR url, at least one) and follow-up guidance (use result_url as citation when SUCCESS; re-poll after a few seconds when PENDING/PROCESSING). It does not, however, route the agent anywhere for FAILED even though retry_task is a sibling, which is the one obvious missing branch.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
retry_taskA
Re-queue a permanently FAILED archive task so the background worker processes it again (e.g. after a transient 403/429). Only tasks whose status is 'failed' can be retried; the task is reset to 'pending' with its retry budget refreshed. Returns the task_id and new status ('pending'). If the task is not failed (pending/processing/success) the call is rejected; if the task_id does not exist a not-found error is returned.
| Name | Required | Description | Default |
|---|---|---|---|
| task_id | Yes | The task_id of the failed task to re-queue. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so well: it discloses the mutation (reset to 'pending', retry budget refreshed), the precondition (status must be 'failed'), rejection on non-failed states, and a not-found error for a missing task_id.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four dense sentences, all carrying real information (precondition, effect, return, error cases) with the mutation front-loaded. Slightly longer than strictly necessary but no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema and no annotations, the description specifies the return value (task_id and new status 'pending') and all failure modes, so an agent has everything needed to call it correctly and interpret the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single parameter is fully documented in the schema, so the baseline is 3. The description reinforces that task_id must refer to a failed task but adds no format or syntax details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Re-queue') and resource ('permanently FAILED archive task') with an explicit example motivation (transient 403/429). This clearly distinguishes it from the sibling list/read tools (archive_url, get_archive_status, archive_and_cite).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives strong state-based gating: 'Only tasks whose status is failed can be retried' and enumerates the rejected states (pending/processing/success). It does not name a sibling alternative to use instead, but the when/when-not guidance is otherwise explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
archive_and_cite - First observed
archive_url - First observed
get_archive_status - First observed
retry_task
TDQS
Scored across 4 tools
archive_url and archive_and_cite both submit a URL for archiving, so an agent must decide between them; the citation convenience in archive_and_cite is the only real differentiator and descriptions acknowledge the overlap. retry_task and get_archive_status are clearly distinct.
All four names are verb-first snake_case (archive_url, retry_task, get_archive_status, archive_and_cite), which is predictable overall. The compound 'archive_and_cite' deviates slightly from the verb_noun pattern but remains readable.
Four tools fit the narrow URL-archiving domain well: submit, check status, retry, and cite. It is a touch lean but each tool covers a distinct stage of the workflow.
The set covers submission, status polling, retry, and citation, but there is no way to list existing tasks, cancel a pending archive, or batch-submit URLs—notable gaps for managing multiple tasks during research.
Maintenance
Related MCP Connectors
Shared copies of public web pages for AI agents. Search stored pages or fetch a URL.
Deterministic web intake and data utilities for autonomous agents.
Build agents to automate any background task. Works with your ChatGPT/Claude subscription.
- fastCRWOAuthio.github.us
Scrape, crawl, map & search the web. Open-source, self-hostable Rust crawler & search for AI agents.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceBridges AI agents to a real browser using a persistent daemon and Chrome extension for driving actual login sessions, cookies, and tabs.MIT
- FlicenseNot gradedqualityBmaintenanceArchives live web pages for offline replay with full interactivity, enabling AI agents to analyze page structure and behavior.1-
- AlicenseNot gradedqualityCmaintenanceEnables LLM agents to perform web search, scraping, and summarization outside their context window, receiving compact cited briefs via MCP while full pages are cached and viewable in a local web UI.MIT
- AlicenseAqualityAmaintenanceA local-first MCP server that lets AI agents read any webpage as clean Markdown, crawl whole sites within configured limits, search without API keys, and solve supported captchas locally — all without cloud services or third-party keys.3019AGPL 3.0