Searches the Wayback Machine via the Internet Archive to retrieve archived copies of paywalled articles and extract clean text.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@remove-paywall-mcp remove paywall from https://www.wsj.com/articles/abc123"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP server that removes article paywalls by searching internet archives. Give it a URL, get back the article text.
You give it a paywalled article URL
Tracking params are stripped and the URL is normalized
It tries multiple approaches in parallel:
Direct fetch with Googlebot user-agent (FT, WSJ, and many others serve full content to crawlers)
12ft.io proxy for hard paywalls
iitty textise for plain-text rendering
Wayback Machine CDX API, archive.ph/is mirrors, and Wayback Availability API
It extracts the article body with readability-lxml, stripping navigation, ads, and sidebar cruft
Post-extraction check: if the result still contains paywall text (e.g., the snapshot captured the paywall itself), it retries with the next archive
It returns clean text with the title and snapshot URL
It also learns from every attempt — success rates per domain per archive source are tracked in a local SQLite database, and archive search order is re-ranked automatically using Laplace smoothing.
Related MCP server: mcp-server-wayback
# zero-install (recommended — works everywhere uvx is available)
uvx remove-paywall-mcp
# from PyPI
pip install remove-paywall-mcp
# from source
pip install git+https://github.com/jasval/remove-paywall-mcp.git
# Docker
docker run -i --rm remove-paywall-mcp
docker compose up -d # HTTP mode on port 8000Once installed, add this to your MCP client config:
{
"mcp": {
"remove-paywall": {
"type": "local",
"command": ["uvx", "remove-paywall-mcp"],
"enabled": true
}
}
}{
"mcpServers": {
"remove-paywall": {
"command": "uvx",
"args": ["remove-paywall-mcp"]
}
}
}mcp_tools:
remove_paywall:
type: "stdio"
command: "uvx"
args: ["remove-paywall-mcp"]{"command": "docker", "args": ["run", "-i", "--rm", "remove-paywall-mcp"]}remove_paywallMain tool. Removes a paywall from an article URL and returns clean article text.
Parameter | Type | Description |
| string | The paywalled article URL |
search_archivesSearch all archive sources for snapshots without extracting content. Useful to see what's available.
Parameter | Type | Description |
| string | The article URL to search for |
get_from_archiveFetch from a specific archive source.
Parameter | Type | Description |
| string | The article URL |
| string |
|
domain_infoLook up a domain in the knowledge base — paywall status, notes, and per-archive success rates.
Parameter | Type | Description |
| string | Domain name (e.g. |
add_domainRegister a domain in the knowledge base. Mark paywalled domains so archives are searched first, or non-paywalled domains so the live page is fetched directly.
Parameter | Type | Description |
| string | Domain name |
| boolean |
|
| string? | Optional description |
The server provides 3 prompt templates for LLMs to use the tools effectively.
remove_paywall_promptFull instruct for bypassing a specific URL. Tells the assistant to use remove_paywall, fall back to search_archives, and check domain_info.
Parameter | Type | Description |
| string | The paywalled article URL |
bypass_paywallShort alias — just tells the assistant to call remove_paywall on the URL.
Parameter | Type | Description |
| string | The paywalled article URL |
handle_paywallsSystem prompt fragment. No arguments — returns instructions for the assistant to automatically call remove_paywall whenever it encounters a paywall, login wall, or metered content. Paste this into your system prompt or load it as a prompt at session start.
Seeded with 33 well-known paywalled domains (NYT, WSJ, Bloomberg, Medium, etc.), stored in SQLite at ~/.remove-paywall-mcp/domains.db. Tracks every archive success/failure per domain and re-ranks archive search order automatically — domains where archive.is consistently fails won't waste time on it.
Variable | Default | Description |
|
|
|
|
| Bind address (HTTP mode) |
|
| Port (HTTP mode) |
|
| Database directory |
Source | Priority | Notes |
Googlebot direct | 1 | Fetches directly with |
12ft.io | 2 | Proxy at |
iitty | 3 |
|
Wayback Machine | 4 | CDX API, newest-first ( |
archive.is/ph mirrors | 5 | Tries newest/oldest across archive.ph, archive.is, archive.today, archive.md |
Wayback Availability | 6 |
|
Priority is dynamically re-ranked per domain based on historical success rates recorded in the knowledge base.
MCP client (Claude/OpenCode/LiteLLM)
│ stdio or HTTP
▼
┌─────────────────┐
│ server.py │ MCPServer with 5 tools
│ +3 prompts │
└────────┬────────┘
│
┌────┴────┐
▼ ▼
┌────────┐ ┌──────────┐
│archives│ │ extractors│
│ .py │ │ .py │
│ │ │ │
│googlebot│ │readability│
│ 12ft │ │Beautiful │
│ iitty │ │Soup │
│ wayback│ │ │
│ archive│ │ │
│ .is/ph │ │ │
│ wayback│ │ │
│ avail │ │ │
└───┬────┘ └──────────┘
│
▼
┌──────────────┐
│domain_store │
│ .py │
│ │
│ SQLite knows │
│ which domains│
│ have paywalls│
│ and which │
│ archives work│
│ best (Laplace│
│ smoothed) │
└──────────────┘MIT
Register a domain in the paywall knowledge base.
Set has_paywall=true for sites that normally have paywalls (so archives are tried first). Set has_paywall=false for sites that don't (so archive search is skipped and the live page is fetched directly).
| Name | Required | Description | Default |
|---|---|---|---|
| notes | No | ||
| domain | Yes | ||
| has_paywall | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It discloses a meaningful behavioral consequence: the has_paywall flag determines whether archive search is attempted first or skipped. Still, it does not mention outcomes like duplicate-domain handling, idempotency, or permissions, leaving notable gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: one for purpose, two for parameter behavior. No filler or repetition; front-loaded and easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose and the key behavioral switch, and an output schema exists to document returns. It could be more complete regarding duplicate/format expectations, but it is adequate for this relatively simple CRUD tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It explains has_paywall semantics in detail and implies domain through the tool name/purpose, but notes is left undefined. This compensates for the two required parameters but not the optional one.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Register a domain in the paywall knowledge base,' using a specific verb and resource. This clearly distinguishes it from sibling tools like remove_paywall and search_archives. No ambiguity in intent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear conditional guidance: set has_paywall=true for paywalled sites (archives tried first) and false for non-paywalled sites (archive search skipped). However, it doesn't explicitly state when to choose add_domain over sibling tools like remove_paywall, so it lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Look up stored knowledge about a paywall domain.
Returns paywall status, user notes, historical archive success rates, and the best archive order for this domain. Use add_domain to register new domains.
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly implies a read-only lookup ('Look up stored knowledge') and details the information returned. Doesn't cover missing-domain behavior, but the operation's nature is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences front-load the purpose, list the return values, and point to an alternative. Every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter lookup tool with an output schema, the description sufficiently covers functionality and returns. The cross-reference to add_domain adds context. Minor edge cases are not essential here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It implies the 'domain' parameter is the target domain, but gives no format or example (e.g., bare domain vs full URL), leaving ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description starts with a specific verb+resource: 'Look up stored knowledge about a paywall domain.' It enumerates the returned data (status, notes, success rates, best archive order), which clearly distinguishes it from sibling tools like search_archives or add_domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: it's for looking up stored metadata about a domain. Explicitly names add_domain as the alternative for registering new domains. Doesn't exclude search_archives or get_from_archive, but the scope is self-evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch an archived version of a URL from a specific source.
source must be one of: googlebot, 12ft, iitty, wayback, archive_is, wayback_available. Returns the extracted article text.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| source | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool returns extracted article text, which is a useful behavioral detail. However, it omits other contextual traits such as rate limits, failure modes, or what happens when a source is unavailable, so it only partially meets the transparency bar.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded, with the main action in the first sentence and a compact list of allowed sources. It avoids fluff, though the formatting of the list with line breaks could be cleaner.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose, parameter constraints, and return value, which is acceptable for a simple 2-parameter tool with an output schema. However, it lacks usage context relative to sibling tools and any error/edge-case behavior, making it only minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds the list of valid values for 'source', which is essential given the schema has no enums. It also implies 'url' is the original URL to fetch. However, it provides no further context about URL format or what each source actually represents, so the added meaning is limited.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Fetch'), resource ('an archived version of a URL'), and constraint ('from a specific source'). It distinguishes from siblings like 'search_archives' by implying a direct fetch rather than a search, though it does not explicitly compare to alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives like 'search_archives' or 'remove_paywall'. The description lists allowed sources but does not explain the selection context or any exclusions, leaving the choice of tool ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Remove a paywall from an article URL by searching internet archives.
First tries a direct fetch with Googlebot user-agent (many sites serve full content to crawlers), then 12ft.io proxy, iitty textise, Wayback Machine (CDX API with dedup + newest-first), archive.is/ph mirrors, and Wayback Availability API. Archives are tried in parallel using historical success rates to prioritize the best one for each domain.
Returns extracted article text with title and snapshot info.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It transparently lists the sequence of attempts (Googlebot fetch, 12ft.io, archive services) and the prioritization logic, but it doesn't disclose potential failure modes, rate limits, or that some sources may be unreliable. It describes what it does but not edge-case behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact yet rich: first sentence states purpose, middle details the process in an ordered list-like manner, last states output. No redundancy, every sentence adds information, and the most important info (purpose) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is complex (multiple fallbacks, parallel requests), and the description covers the methodology well while the output schema handles return details. Missing a note on failure behavior or expected latency, but given the output schema and clear process, it's nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It does: the first sentence specifies 'article URL', giving the single `url` parameter clear semantic meaning. It doesn't dictate format, but for a single-string parameter this is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Remove') and resource ('paywall') and immediately distinguishes itself from sibling tools like search_archives or get_from_archive by stating the goal is paywall removal. It further clarifies it works through internet archives, making the tool's scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use this tool: when a paywalled article URL needs full content extraction. It details the multi-step fallback strategy, but does not explicitly compare with sibling tools or state exclusions (e.g., when not to use), so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search all archive sources for snapshots of a URL.
Returns a list of available snapshot URLs from each archive source: Googlebot direct fetch, 12ft.io proxy, iitty textise, Wayback Machine, archive.is/ph mirrors, and Wayback Availability API. Does not extract content — use remove_paywall for full article retrieval.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the output (list of snapshot URLs), enumerates six archive sources, and explicitly states what the tool does not do (extract content). This gives a clear behavioral picture for a simple search tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the main action, then output description and a pointer to an alternative. Every sentence earns its place with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter tool and an existing output schema, the description covers the tool's purpose, return type, and boundaries. The list of archive sources adds useful context beyond what structured fields would provide.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description implies the URL parameter is the target of the search ('Search all archive sources for snapshots of a URL'). However, it does not add any format or constraint details (e.g., encoding, absolute URL), leaving some ambiguity for a single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search' and the resource 'all archive sources for snapshots of a URL'. It lists the specific archive sources and explicitly differentiates from sibling tools like remove_paywall by stating it does not extract content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides an explicit alternative: 'use remove_paywall for full article retrieval'. It implies the tool is for discovering snapshots, not fetching content. However, it does not differentiate from other siblings like get_from_archive, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Each tool has a distinct purpose: remove_paywall is the high-level bypass, search_archives lists snapshots, get_from_archive fetches from a named source, and domain_info/add_domain manage domain knowledge. No two tools appear to do the same thing.
Most tool names follow a verb_noun pattern (remove_paywall, search_archives, add_domain), but get_from_archive uses verb-preposition-noun and domain_info is a noun phrase. This is a minor deviation from an otherwise consistent snake_case style.
With 5 tools, the server is well-scoped for its purpose: one main operation, two low-level retrieval ops, and two knowledge-base ops. No redundancy and no missing essential functionality.
The tool surface covers the full workflow: automatic bypass, manual archive search/fetch, and persistent domain knowledge. The ability to consult and add domain info ensures agents can adapt to known paywall patterns. No major gaps evident.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
One MCP server for 180+ live web-data APIs returning clean JSON from sites that block scrapers.
Free remote MCP server for fetching public web pages through a rotating proxy pool.
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/jasval/remove-paywall-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
