Newsclip

Search, read and privately archive news through a command-line interface, Python API, or MCP server. Newsclip uses your existing subscriptions and public publisher endpoints. It returns an article only when its identity and full content pass the provider's verification checks.

Subscription sign-in uses signin-window: open a short-lived link and sign in on the publisher's page yourself. Newsclip keeps the verified session afterwards. It does not need your password in a configuration file or an assistant conversation.

• Ten built-in providers: Financial Times, WSJ, Bloomberg, CNBC, Reuters, The Economist, The New York Times, The Guardian, AP News and Axios.

• Fair federated search with source status, date filters and coverage limits.

• Separate user tokens, sessions, retained browser profiles and SQLite archives.

• Transient reading with get; durable storage and Markdown export with save.

• Local CLI/Python access and authenticated HTTP/MCP access to the same core.

• MIT-licensed original code, with synthetic tests and no bundled account data.

Quick start

The reference server runs on Linux AMD64 with Docker Compose. Python 3.12+ and uv are used for setup and the CLI. The sign-in viewer works in a browser or phone; the browser service itself runs on Linux.

git clone https://github.com/benjamin-di/newsclip-mcp.git
cd newsclip-mcp
uv sync --locked --extra local --extra mcp --extra signin
uv run python scripts/setup.py --user alice --user bob
# Configure OIDC in .env as described in docs/DEPLOYMENT.md before startup.
export NEWSCLIP_VERSION="$(python3 -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')"
export VCS_REF="$(git rev-parse HEAD)" # required if Compose builds source images
docker compose up -d

Setup writes private token files and configuration without printing secrets. Remote sign-in requires an OIDC provider such as Pocket ID and a verified identity-to-user binding. Open /connections/ to manage your own subscriptions or explicitly use one another user shared with you; assistant API access remains separate from browser management. Sharing is selected per publisher, and saved articles always stay in your private archive. See shared subscriptions. This identity check is required for every remote management attempt, including the viewer and its WebSocket. A copied link or MCP bearer grants no management authority. See the permission boundaries. Each person uses their own URL and token file:

export NEWSCLIP_URL=http://127.0.0.1:8000/users/alice
export NEWSCLIP_TOKEN_FILE="$PWD/local/tokens/alice.token"

uv run newsclip search "energy markets" --source ap --json
uv run newsclip auth connect ft
# Open the returned link, complete sign-in, then check the session.
uv run newsclip auth status ft --json
uv run newsclip get 'https://www.ft.com/content/<article-id>'
uv run newsclip save 'https://www.ft.com/content/<article-id>' --out article.md
uv run newsclip archive-search "energy" --json

get does not archive. save commits the verified article and can additionally write a Markdown file on the CLI client's machine. Failed, partial and unauthorized pages never become archived articles. Use archive-get ARTICLE_ID to read a saved article without contacting its publisher.

The default deployment is reachable only on the server itself. For phone or remote-client access, configure an HTTPS public origin and reverse proxy as explained in deployment.

Related MCP server: feedkit

Install the CLI or Python package

For an existing Newsclip server, install the client from PyPI with Python 3.12 or later:

pip install newsclip
newsclip --help
newsclip --server http://127.0.0.1:8000/users/alice \
  --token-file /path/to/alice.token search "energy markets" --source ap --json

The base package is a remote client and installs no publisher browser or sign-in runtime. For local Python/CLI use, install newsclip[local]; for the complete server use newsclip[local,mcp,signin] or the Compose setup above. Local use of a base-only installation reports the required extra.

The same package provides newsclip.client.NewsclipClient for Python programs. Use the Docker Compose quick start above for the complete server and sign-in browser setup.

Interfaces

Retained diagnostics identify the installed application version, full source revision and an opaque process instance. Native MCP reports that same product version. Timing and browser traces retain their source fingerprints and read older records without assigning them the current build. Article, result and archive output keep their existing contracts.

Release images and distributions embed identity before installation; runtime environment variables cannot relabel them. Noneditable installations reject a missing or conflicting stamp. Editable development reports an explicit unknown revision. Direct source packages must first run uv run python scripts/embed-build-identity.py <product-version> <full-revision>. Direct image builds require VERSION and full VCS_REF build arguments.

For MCP, connect your client to http://127.0.0.1:8000/users/alice/mcp with Alice's bearer token. The server exposes search, get, save, archive_search, archive_get, reextract, session_status, and connect. connect returns the owner's Connections page without starting or replacing sign-in (expires_at is null for that permanent page). Only the authenticated owner can press Connect. Upgrade clients with the server for this nullable response field. Read operations never submit credentials.

Pocket ID establishes human browser identity. Agentgateway or another admission gateway separately checks an agent's audience, scopes, explicit grant and group ceiling. Newsclip then enforces the selected registered user's application credential and storage boundary. Saved publisher sessions establish provider access only for that user; signing in does not delegate the user's API access to an agent. Backend tokens do not encode OAuth scope policy, so protect direct backend routes from bypassing the gateway's admission decisions.

For Python:

from pathlib import Path
from newsclip.client import NewsclipClient

with NewsclipClient(
    "http://127.0.0.1:8000/users/alice",
    Path("local/tokens/alice.token").read_text(),
) as client:
    response = client.search_many("energy markets", provider_keys=["ap"])
    for result in response.results:
        print(result.headline, result.url)

The local CLI and synchronous NewsclipService are also available without running an HTTP server. newsclip --user alice ... selects isolated local storage; it is an administrator/local-process option, not remote authorization. newsclip mcp serves local stdio MCP with the same local user's access.

Providers and limits

See the provider matrix for authentication, search coverage and retrieval behavior. A supported adapter cannot guarantee that a publisher will accept every network, account or browser. Articles outside a subscription are reported as not_entitled. Provider failures remain visible beside healthy search results.

Newsclip is for content you are authorized to access. You supply the necessary subscriptions and API permissions. It does not purchase access, redistribute publisher content, or grant rights to that content through its software license.

Documentation

• CLI reference and Python/HTTP/MCP API

• Sign-in and user isolation

• Deployment, configuration, backups and upgrades

• Provider support and extension

• Architecture, development and roadmap

• Contributing, security reporting and third-party notices

Development

uv sync --locked --extra local --extra mcp --extra signin
uv run patchright install chromium
make check
make check-mcp

Offline tests use synthetic content and require no publisher credentials. Live provider checks are explicit private operator actions and are never run on a contributor's pull request. Public images and release archives are built from the public release source.

License

MIT for original code. Dependencies retain their own licenses; see NOTICE. Newsclip is not affiliated with the supported publishers.

Optional operator maintenance, per-account credential configuration and trusted reverse proxy deployment are described in Maintenance. Human sign-in remains the default; maintenance uses the same public package with separate runtime authority.

See Upgrading for account, maintenance and health-contract changes.

Required controls for hosted adoption

Set CONNECTIONS_REQUIRE_CONTROLS=1 for hosted readers and credential writers. Missing control records then refuse access, including after a state directory is replaced. Existing standalone behavior remains available for unadopted state; saved initialization evidence still prevents recreating a lost adopted record.

Before starting matching control-aware roles, explicitly initialize each reviewed account in its existing private state, using the same runtime user and paths:

python -m newsclip.connection_controls --initialize --root <user-state-root> --provider <configured-publisher>

This local operation makes no provider request. It preserves existing valid controls and refuses a missing record beside retained initialization evidence. It must not run automatically to repair a damaged or missing adopted volume. Keep markers with current controls, bindings, credentials and accounting; check actual mounted state before startup and recover on disposable copies. An older backup or an image that ignores controls is not a safe way to clear a disconnect.