The ArchiveBox API server provides programmatic access to web archiving functionality through a Model Context Protocol (MCP) interface, enabling AI agents and applications to manage snapshots, archive results, and tags.
Core Capabilities:
Authentication Management: Generate and validate API tokens for secure access to ArchiveBox instances
Snapshot Operations: Retrieve, filter, and manage web page snapshots with extensive filtering options (by ID, URL, tag, title, timestamp, creator, date ranges) and search functionality
ArchiveResult Access: List, filter, and search archive extraction results by extractor type, status, command, output, and metadata
Tag Management: Retrieve specific tags by ID or ABID with optional snapshot inclusion
Universal Lookup: Retrieve any resource (Snapshot, ArchiveResult, or Tag) by its unique ABID in a unified interface
CLI Command Execution: Programmatically execute ArchiveBox commands:
Add URLs: Archive new URLs with configurable tags, crawl depth, parser selection, extractors, and update/overwrite behavior
Update Archives: Update existing snapshots with filtering by timestamp ranges, status, and pattern matching
Schedule Operations: Set up recurring archive jobs with configurable frequency, depth, and import paths
List Archives: Generate formatted outputs (JSON/CSV/HTML) with sorting and filtering
Remove Archives: Delete snapshots matching specific criteria
Advanced Features:
Pagination Support: All list operations support page/limit/offset parameters for efficient data retrieval
Pattern Matching: Multiple filter types (exact, substring) and date range filtering with comparison operators
Batch Operations: Execute operations on multiple URLs or snapshots simultaneously
Extractor Control: Specify which archiving extractors to use for each operation
Security: Supports multiple authentication types (none, static, JWT, OAuth Proxy, OIDC Proxy, Remote OAuth) and optional policy-based authorization using Eunomia
Middleware: Built-in error handling, rate limiting, timing, and comprehensive logging
Flexible Deployment: Runs as an MCP server via stdio or HTTP transport, easily containerized with Docker
Supports deployment of the ArchiveBox MCP server as a containerized service using Docker with configurable authentication and authorization options
Hosted on GitHub with repository management, issue tracking, and collaborative development features
Supports OpenID Connect (OIDC) proxy authentication for secure access control with configurable client credentials and provider integration
Distributed as a Python package through PyPI, enabling easy installation and dependency management
Built as a Python wrapper providing programmatic access to ArchiveBox API functionality for web archiving operations
Uses Shields.io badges to display project metrics including version, downloads, stars, license, and build status
Uses Vercel's GitHub README stats service to display contributor statistics and project metrics
Supports Docker Compose configuration using YAML format for service deployment and environment management
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@ArchiveBox APIadd https://example.com to my archive"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
ArchiveBox API - A2A | AG-UI | MCP
Version: 0.0.13
Overview
ArchiveBox API Python Wrapper & Fast MCP Server!
This repository provides a Python wrapper for interacting with the ArchiveBox API, enabling programmatic access to web archiving functionality. It includes a Model Context Protocol (MCP) server for Agentic AI, enhanced with various authentication mechanisms, middleware for observability and control, and optional Eunomia authorization for policy-based access control.
Contributions are welcome!
All API Response objects are customized for the response call. You can access return values in a parent.value.nested_value format, or use parent.json() to get the response as a dictionary.
Features:
Authentication: Supports multiple authentication types including none (disabled), static (internal tokens), JWT, OAuth Proxy, OIDC Proxy, and Remote OAuth for external identity providers.
Middleware: Includes logging, timing, rate limiting, and error handling for robust server operation.
Eunomia Authorization: Optional policy-based authorization with embedded or remote Eunomia server integration.
Resources: Provides
instance_configfor ArchiveBox configuration.Prompts: Includes
cli_add_promptfor AI-driven interactions.
API
API Calls:
Authentication
Core Model (Snapshots, ArchiveResults, Tags)
CLI Commands (add, update, schedule, list, remove)
If your API call isn't supported, you can extend the functionality by adding custom endpoints or modifying the existing wrapper.
These are the API endpoints currently supported
MCP
All the available API Calls above are wrapped in MCP Tools. You can find those below with their tool descriptions and associated tag.
MCP Tools
Function Name | Description | Tag(s) |
| Generate an API token for a given username & password. |
|
| Validate an API token to make sure it's valid and non-expired. |
|
| Retrieve list of snapshots. |
|
| Get a specific Snapshot by abid or id. |
|
| List all ArchiveResult entries matching these filters. |
|
| Get a specific Tag by id or abid. |
|
| Get a specific Snapshot, ArchiveResult, or Tag by abid. |
|
| Execute archivebox add command. |
|
| Execute archivebox update command. |
|
| Execute archivebox schedule command. |
|
| Execute archivebox list command. |
|
| Execute archivebox remove command. |
|
A2A Agent
Architecture:
Component Interaction Diagram
Usage
MCP
MCP CLI
Short Flag | Long Flag | Description |
-h | --help | Display help information |
-t | --transport | Transport method: 'stdio', 'http', or 'sse' [legacy] (default: stdio) |
-s | --host | Host address for HTTP transport (default: 0.0.0.0) |
-p | --port | Port number for HTTP transport (default: 8000) |
--auth-type | Authentication type: 'none', 'static', 'jwt', 'oauth-proxy', 'oidc-proxy', 'remote-oauth' (default: none) | |
--token-jwks-uri | JWKS URI for JWT verification | |
--token-issuer | Issuer for JWT verification | |
--token-audience | Audience for JWT verification | |
--oauth-upstream-auth-endpoint | Upstream authorization endpoint for OAuth Proxy | |
--oauth-upstream-token-endpoint | Upstream token endpoint for OAuth Proxy | |
--oauth-upstream-client-id | Upstream client ID for OAuth Proxy | |
--oauth-upstream-client-secret | Upstream client secret for OAuth Proxy | |
--oauth-base-url | Base URL for OAuth Proxy | |
--oidc-config-url | OIDC configuration URL | |
--oidc-client-id | OIDC client ID | |
--oidc-client-secret | OIDC client secret | |
--oidc-base-url | Base URL for OIDC Proxy | |
--remote-auth-servers | Comma-separated list of authorization servers for Remote OAuth | |
--remote-base-url | Base URL for Remote OAuth | |
--allowed-client-redirect-uris | Comma-separated list of allowed client redirect URIs | |
--eunomia-type | Eunomia authorization type: 'none', 'embedded', 'remote' (default: none) | |
--eunomia-policy-file | Policy file for embedded Eunomia (default: mcp_policies.json) | |
--eunomia-remote-url | URL for remote Eunomia server |
Using as an MCP Server
The MCP Server can be run in two modes: stdio (for local testing) or http (for networked access). To start the server, use the following commands:
Run in stdio mode (default):
Run in HTTP mode:
Basic API Usage
Token Authentication
Basic Authentication
API Key Authentication
SSL Verify
Deploy MCP Server as a Service
The ArchiveBox MCP server can be deployed using Docker, with configurable authentication, middleware, and Eunomia authorization.
Using Docker Run
For advanced authentication (e.g., JWT, OAuth Proxy, OIDC Proxy, Remote OAuth) or Eunomia, add the relevant environment variables:
Using Docker Compose
Create a docker-compose.yml file:
For advanced setups with authentication and Eunomia:
Run the service:
Configure mcp.json for AI Integration
Recommended: Store secrets in environment variables with lookup in the JSON file.
For Testing Only: Plain text storage will also work, although not recommended.
CLI Parameters
The archivebox-mcp command supports the following CLI options for configuration:
--transport: Transport method (stdio,http,sse) [default:http]--host: Host address for HTTP transport [default:0.0.0.0]--port: Port number for HTTP transport [default:8000]--auth-type: Authentication type (none,static,jwt,oauth-proxy,oidc-proxy,remote-oauth) [default:none]--token-jwks-uri: JWKS URI for JWT verification--token-issuer: Issuer for JWT verification--token-audience: Audience for JWT verification--oauth-upstream-auth-endpoint: Upstream authorization endpoint for OAuth Proxy--oauth-upstream-token-endpoint: Upstream token endpoint for OAuth Proxy--oauth-upstream-client-id: Upstream client ID for OAuth Proxy--oauth-upstream-client-secret: Upstream client secret for OAuth Proxy--oauth-base-url: Base URL for OAuth Proxy--oidc-config-url: OIDC configuration URL--oidc-client-id: OIDC client ID--oidc-client-secret: OIDC client secret--oidc-base-url: Base URL for OIDC Proxy--remote-auth-servers: Comma-separated list of authorization servers for Remote OAuth--remote-base-url: Base URL for Remote OAuth--allowed-client-redirect-uris: Comma-separated list of allowed client redirect URIs--eunomia-type: Eunomia authorization type (none,embedded,remote) [default:none]--eunomia-policy-file: Policy file for embedded Eunomia [default:mcp_policies.json]--eunomia-remote-url: URL for remote Eunomia server
Middleware
The MCP server includes the following built-in middleware for enhanced functionality:
ErrorHandlingMiddleware: Provides comprehensive error logging and transformation.
RateLimitingMiddleware: Limits request frequency with a token bucket algorithm (10 requests/second, burst capacity of 20).
TimingMiddleware: Tracks execution time of requests.
LoggingMiddleware: Logs all requests and responses for observability.
Eunomia Authorization
The server supports optional Eunomia authorization for policy-based access control:
Disabled (: No authorization checks.
Embedded (: Runs an embedded Eunomia server with a local policy file (
mcp_policies.jsonby default).Remote (: Connects to an external Eunomia server for centralized policy decisions.
To configure Eunomia policies:
A2A CLI
Endpoints
Web UI:
http://localhost:8000/(if enabled)A2A:
http://localhost:8000/a2a(Discovery:/a2a/.well-known/agent.json)AG-UI:
http://localhost:8000/ag-ui(POST)
Short Flag | Long Flag | Description |
-h | --help | Display help information |
--host | Host to bind the server to (default: 0.0.0.0) | |
--port | Port to bind the server to (default: 9000) | |
--reload | Enable auto-reload | |
--provider | LLM Provider: 'openai', 'anthropic', 'google', 'huggingface' | |
--model-id | LLM Model ID (default: qwen3:4b) | |
--base-url | LLM Base URL (for OpenAI compatible providers) | |
--api-key | LLM API Key |
| | --mcp-url | MCP Server URL (default: http://localhost:8000/mcp) | | | --web | Enable Pydantic AI Web UI | False (Env: ENABLE_WEB_UI) |