What can you do with this server?

This server provides a persistent web archiving service that automatically saves fetched pages and web search results for later retrieval and full-text search. * Fetch & Archive Web Pages: Retrieve any public URL, convert HTML to Markdown, and archive it. Supports optional authentication and timeout configuration; private/internal IPs are blocked for security. * Search & Archive Web Results: Perform DuckDuckGo searches and archive the results automatically. * List Archived Entries: Browse previously saved entries with metadata, optional date filtering and result limits. * Read Archive Files: Access the contents of a specific archive file by its ID, showing newest entries first. * Rebuild Search Index: Trigger a full-text search index rebuild, making archived content searchable through the unified-history-mcp integration.

Which integrations are available for this server?

Allows performing web searches via DuckDuckGo, with results persisted and indexed for later retrieval.

How do I use web-archive-mcp?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@web-archive-mcp fetch and archive https://example.com/article" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

web-archive-mcp

by palimpsest-labs

Overview Schema Source Related Servers Discussions Admin

Python

Local

web-archive-mcp

MCP server for persistent web fetch and search archiving. Every web_fetch and web_search result is saved as timestamped JSONL, indexed by fst-indexer, and searchable via unified-history-mcp.

Part of the Palimpsest investigative toolkit.

Why

web_fetch and web_search results normally evaporate when a session ends. Pages change, get deleted, or get memory-holed. This closes that gap — every result is persisted, content-addressed for dedup, and fed into the same search pipeline as your session logs and transcripts. Three months later, a search(domain="all", query="target name") still finds the page that's been 404'd since July.

Related MCP server: grok-mcp

Architecture

web_fetch / web_search          playwright-archive-mcp (browser capture)
        │                                │
        ▼                                ▼
  web-archive-store  (shared JSONL write-path + SSRF URL validation)
        │
        ▼
  ~/.local/share/web-archive/*.jsonl
        │                                │
        │                        fst-indexer (Jsonl extractor)
        │                                │
        ▼                                ▼
  returns content              index.fst + manifest.json
                                       │
                                       ▼
                              unified-history-mcp
                              domain: "web-archive"

Tools

Tool	Description
`web_fetch(url, timeout, token, preview, redact_html, method, body, content_type)`	Fetch a URL, convert to markdown, persist, return

web_fetch supports the standard HTTP verbs via method (GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS; default GET). Pass body for the request body and content_type for the Content-Type header (e.g. a JSON POST). The method is stamped on each archived entry and included in the response. | web_search(query) | Search the web (DuckDuckGo), persist results | | archive_list(date_from, date_to, max) | List archived entries with metadata | | archive_read(id, max_entries) | Read entries from an archive file | | rebuild | Rebuild FST index for the web-archive domain |

Installation

web-archive-mcp depends on the shared web-archive-store package (archive write-path + SSRF URL validation). Install it first:

git clone https://github.com/palimpsest-labs/web-archive-store
cd web-archive-store
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

cd ..
git clone https://github.com/palimpsest-labs/web-archive-mcp
cd web-archive-mcp
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Once web-archive-store is published to PyPI this becomes a single pip install -e ..

Browser-driven traffic capture (the playwright_* tools) moved to the separate playwright-archive-mcp server, which records HTTP traffic into the same store.

Integration with unified-history-mcp

Add to your unified-history TOML config:

[domains.web-archive]
dir = "~/.local/share/web-archive"
pattern = "*.jsonl"
extractor = "jsonl"
label = "web-archive entry"
filters = []

Then rebuild: search(domain="web-archive", query="rebuild") or call rebuild directly.

Once indexed, a search(domain="all", query="your search") scans your sessions, transcripts, notifications, and every web page you've ever fetched — in a single query.

Entry format

{
  "type": "fetch",
  "source": "https://example.com/page",
  "title": "Example Page",
  "content": "# Example\n\nMarkdown content...",
  "timestamp": "2026-07-30T21:15:00Z",
  "content_hash": "abc123..."
}

For searches, source holds the query string and type is "search".

Content-addressed dedup prevents storing identical entries. Same source + same content hash = skipped.

License

MIT

Available Tools

5 tools

archive_listA

List archived web fetch/search entries with metadata.

Args: date_from: Optional start of date range (YYYY-MM-DD) date_to: Optional end of date range (YYYY-MM-DD, inclusive) max_results: Maximum entries to show (default 50)

ParametersJSON Schema

Name	Required	Description	Default
`date_to`	No
`date_from`	No
`max_results`	No

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A3.8/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral disclosure burden. It states 'List' which implies read-only, and it documents date range filtering, but it does not explicitly state side effects, sort order, or behavior with no matching entries. The output schema covers return structure, but not behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely compact and well-structured: one sentence states the purpose, followed by a short Args list. Every sentence is necessary, and the key information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple list tool with three optional parameters and an output schema. The description covers purpose and all parameters adequately. The only gap is the lack of usage guidance or explicit differentiation from sibling tools, but given the low complexity, the description is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has no per-property descriptions (0% coverage), so the description's Args section adds meaningful semantic value: it specifies date format (YYYY-MM-DD), inclusivity for date_to, and the default for max_results. All three parameters are explained, exceeding the schema's bare types and defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'List archived web fetch/search entries with metadata,' naming both the verb ('List') and the resource ('archived web fetch/search entries'). This clearly distinguishes the tool from sibling 'archive_read', which presumably reads a single entry.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'archive_read' or the web_fetch/web_search tools. It simply lists parameters without addressing use cases, exclusions, or preferred contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

archive_readA

Read entries from an archive file.

Args: file_id: Archive file name (e.g., '2026-07-30-fetch-example.jsonl') max_entries: Maximum entries to return, newest first (default 50)

ParametersJSON Schema

Name	Required	Description	Default
`file_id`	Yes
`max_entries`	No

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A3.7/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the transparency burden. It discloses the read-only nature via 'Read' and adds ordering information ('newest first'), but it does not mention failure modes, side effects, or the format of returned entries.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise, front-loaded with the core purpose in the first sentence, followed by a compact args block. Every sentence adds value and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read tool with an output schema, the description covers the core operation and parameter semantics well. It lacks explicit guidance on obtaining file_id or handling error cases, which prevents a perfect score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, but the description fully compensates by explaining file_id with an example filename and max_entries with 'Maximum entries to return, newest first'. This adds meaningful semantics beyond the raw schema types and defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Read entries from an archive file.' It clearly states the action and object, but it does not explicitly differentiate this tool from the sibling archive_list tool, which limits it to a 4 rather than a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives. The description simply states what the tool does, leaving the agent to infer that archive_read is appropriate for reading entries, and it does not mention exclusions or sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rebuildA

Rebuild the FST index for the web-archive domain.

Calls fst-indexer to rebuild the full-text index so archived content is searchable via unified-history-mcp.

ParametersJSON Schema

Name	Required	Description	Default
No parameters

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A3.7/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It mentions that it 'Calls fst-indexer to rebuild the full-text index', which is helpful, but it does not disclose side effects such as overwriting the existing index, potential downtime, or whether the operation is long-running or safe to invoke repeatedly. For a rebuild operation, this is a notable gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action, and contains no filler or repetition. Every sentence contributes to explaining what the tool does and why.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-param tool with an output schema, the description covers the core action and outcome. However, it omits operational context such as whether this replaces the current index, how long it might take, or any prerequisites (e.g., fst-indexer availability). Given that this is a rebuild operation, slightly more context would be useful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description does not need to explain parameter semantics. It gains a baseline 4 because there is nothing to compensate for; the schema is empty and no parameter documentation is required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Rebuild'), the resource ('the FST index for the web-archive domain'), and the purpose ('so archived content is searchable'). This distinguishes it from sibling tools like archive_list/archive_read (which read) and web_search (which searches).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'so archived content is searchable' implies the tool is used to make archived content searchable, but there is no explicit guidance on when to use it vs. alternatives, no exclusions, and no mention of prerequisites like needing to run this after content updates. Usage context is implied but not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

web_fetchA

Fetch a URL and archive the result.

Fetches the URL, converts HTML to markdown, and persists the result as a timestamped JSONL entry in the web-archive. Only http and https URLs are allowed; private/internal IPs are blocked.

Args: url: The URL to fetch (http/https only) timeout: Request timeout in seconds (default 30, max 120) token: Optional Bearer token for authenticated requests

ParametersJSON Schema

Name	Required	Description	Default
`url`	Yes
`token`	No
`timeout`	No

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses side effects (persisting to archive), format conversion (HTML to markdown), URL restrictions, and optional auth token. It doesn't detail error handling or redirect behavior, but the core behaviors are well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the primary purpose, and uses an Args list for parameter details. No redundant fluff; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool's complexity is moderate (3 params, 1 required) and an output schema exists. The description covers behavior, side effects, constraints, and all parameters, making it sufficient for correct selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains all three parameters: url (http/https only), timeout (seconds, default 30, max 120), and token (optional Bearer). This adds essential meaning beyond the schema's bare types and defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it fetches a URL and archives the result, converting HTML to markdown and persisting as a timestamped JSONL entry. This specific verb+resource+scope distinguishes it from sibling tools like web_search (search) and playwright_* (browser automation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for fetching and archiving web content, but it does not explicitly state when to use it over alternatives like web_search or playwright_navigate. It provides constraints (http/https only, private IPs blocked) but no direct comparison or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

web_searchA

Search the web and archive the results.

Performs a web search via DuckDuckGo, persists the results, and returns them formatted. The results are archived for later search.

Args: query: The search query (max 500 chars)

ParametersJSON Schema

Name	Required	Description	Default
`query`	Yes

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A4.5/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden of behavioral disclosure. It explicitly mentions that results are persisted and archived for later search, which is a significant side effect beyond the basic search operation. It does not mention rate limits or error behavior, but the disclosed archiving behavior adds valuable transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately short and front-loaded with the main purpose. The 'Args' section clearly attaches semantics to the parameter. Every sentence earns its place, with no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter schema, the presence of an output schema, and the clear description of the tool's behavior and side effects, this description is complete. The agent has everything needed to invoke the tool correctly and understand its archiving side effect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides only the parameter type and name with no description, so the description must compensate. It adds meaning by defining 'query' as the search query and specifying a 500-character limit, which is actionable. The explanation is sufficient for a single required parameter, though it could elaborate on query formatting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Search the web and archive the results.' It clearly indicates the tool performs a web search via DuckDuckGo, which distinguishes it from sibling tools like web_fetch (fetching a specific URL) and playwright_* (browser automation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies use for web search, not for direct fetching or browser interaction. It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to select this tool for search queries. A named alternative would push it to 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

5 tool updatesv0.1.0Aug 1, 2026
- First observedarchive_list
- First observedarchive_read
- First observedrebuild
- First observedweb_fetch
- First observedweb_search

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.2/5.0

Scored 2026-08-01 03:00 across 5 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: fetching a URL, searching the web, listing archives, reading archive entries, and rebuilding the index. No overlap or ambiguity exists between them.

Naming Consistency4/5

Most tools follow a verb_noun pattern (web_fetch, web_search, archive_list, archive_read), but 'rebuild' is a single verb that breaks the pattern and mixes with the web_/archive_ prefixes. The inconsistency is minor and does not hinder comprehension.

Tool Count5/5

Five tools is well-scoped for a web-archive server, covering core operations (fetch, search, list, read, maintain) without unnecessary bloat.

Completeness5/5

The tool surface covers the full lifecycle of archiving web content: creating archive entries via web_fetch and web_search, reading them via archive_list and archive_read, and maintaining the search index with rebuild. No obvious gaps exist for the stated purpose.

Maintenance

ActivitySlowing

ResponsivenessNo issues

Resources

GitHub Repository

Need Help?

Related Servers

Related MCP Connectors

mcp A
ai.llmse
Public MCP server for the LLM Search Engine
Search RAG Systems Developer Tools
102
MysocialOAuth
io.mysocial
Social media MCP server: your Instagram, TikTok, YouTube, LinkedIn and Threads history for your AI.
Social Media
1
Probe V0 A
io.github.lexbritten
Experimental MCP server for current empirical verification of explicit public HTTPS endpoint claims.
API Testing Penetration Testing
21
Synap Memory A
ai.maximem
Persistent memory for AI agents — log and recall conversation context over MCP.
Knowledge & Memory AI & Machine Learning
4248

Related MCP Servers

scout
Browser Automation Web Scraping
tools-for-agents
A
license
Not graded
quality
B
maintenance
MCP server that fetches web pages, extracts clean markdown (reducing token count), caches results, and provides searchable reading history.
Updated a month ago (2026-08-05 17:26 UTC)
MIT
grok-mcp
Search Web Scraping Social Media
dq52099
F
license
Not graded
quality
C
maintenance
MCP server providing web search, news search, and X/Twitter search capabilities via HTTP or stdio.
Updated 2 months ago (2026-07-26 22:44 UTC)
-
url-context-mcp
Browser Automation Web Scraping
SkillfulElectro
A
license
A
quality
B
maintenance
An MCP server that fetches web pages and extracts clean, AI-usable context from them, enabling tools for link discovery, content search, and integrated fetch-and-search operations.
Updated 2 months ago (2026-07-14 05:50 UTC)
5
8 npm
1
MIT
worklog-mcp
Developer Tools Project Management Observability
aadityakushwaha
A
license
B
quality
C
maintenance
An MCP server that enables coding agents to log their work, decisions, test runs, PRs, and progress as structured events, either to a local JSONL file or an HTTP endpoint.
Updated a month ago (2026-08-16 19:35 UTC)
6
0
MIT