tg-archive
Provides search and retrieval over an archive of public Telegram channel posts, including keyword and semantic search, fetching individual posts, listing recent posts and topics, and viewing archive statistics.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@tg-archive search for posts about AI regulation"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
tg-ml-archive
tg-ml-archive copies the posts of public Telegram channels into a SQLite database. An OpenAI model tags each post by topic and writes a one-line English summary. An MCP server lets Claude search the archive by keywords and by meaning. A digest page shows the news of the last 3 days.
systemd timer (hourly) ── ingest.py ──► archive.db (SQLite: posts + FTS5 + sqlite-vec)
├─ enrich.py ─┘ (OpenAI: tags + summary + embeddings)
└─ media.py ──► /var/www/tg-digest/media (pictures and videos for the digest)
systemd timer (daily) ──── digest.py ──► /var/www/tg-digest/index.html + feed.xml
Claude ──HTTPS + token──► Caddy ──► mcp_server.py --httpFile | What it does |
| Gets new posts of each channel in |
| Tags each post with OpenAI (topics, kind, availability, entities, summary) and stores an embedding. |
| Downloads the pictures and videos of new posts for the digest page. |
| Builds the digest page for the latest finished 3-day window. |
| MCP tools: |
| The channel list, the channel descriptions, the topic list, and the digest settings. |
| Scripts and systemd units for the VPS. |
Setup guide
This guide is written so that a person or an AI agent can do it from start to end. Do the parts in this order:
Collect the accounts and values (Part A).
Install the project on your computer and fill the archive (Part B).
Deploy to the VPS (Part C).
Make the MCP server and the digest public over HTTPS (Part D).
Connect Claude (Part E).
If you only want local search on your computer, do Part A, Part B, and step E1.
Rules for AI agents
Do not print, copy into chat, or commit
.env,tg.session, or/etc/tg-archive-mcp.env.tg.sessiongives full access to the Telegram account. Treat it as a password.Only the owner of the Telegram account can type the phone number and the login code. Ask the owner to run step B4 in a terminal.
After the VPS has
tg.session, do not runingest.pyon the local computer. Telegram can revoke a session that runs in two places at the same time.
Part A. Collect the accounts and values
You need these items before you start:
Item | Where to get it | Goes into |
Telegram API ID and API hash | https://my.telegram.org → API development tools |
|
A Telegram account with a phone number | Your own account. The code reads channels as this user. | Login in step B4 |
OpenAI API key |
| |
A VPS with Ubuntu and root SSH key login | Any provider. Tested on Ubuntu 24.04 with 2 GB of memory. |
|
A domain or subdomain that you control | Your DNS provider |
|
The public channels to archive | The |
|
The local computer needs git, ssh, rsync, sqlite3, and uv. macOS and Linux both work.
Part B. Install locally and fill the archive
B1. Clone the repo and install the dependencies:
git clone <repo-url> tg-ml-archive
cd tg-ml-archive
uv syncB2. Make the .env file:
cp .env.example .envThen open .env and set these keys: TG_API_ID, TG_API_HASH, OPENAI_API_KEY, DEPLOY_HOST, DOMAIN.
Leave MCP_TOKEN empty. Step D1 makes the token on the server.
B3. Edit channels.toml:
Put your channels in the
channelslist (public usernames, with or without@).In
[about], write one description for each channel. Use the username in lowercase as the key. The tagger reads this text, so describe what the channel posts and how its posts look.In
[strip], remove the entries of channels that you do not archive. Add a regular expression only if a channel repeats an ad footer.Change
[topics],[kinds], and[availability]only if you need a different taxonomy.
B4. Get the posts (the account owner must do this step):
uv run ingest.pyOn the first run, Telethon asks for the phone number and the login code. Then it saves tg.session and downloads the full history of each channel. A large channel can take some minutes.
Expected result: ingest.py prints the number of new posts for each channel.
B5. Tag a small sample, then examine the tags:
uv run enrich.py --sample 20
sqlite3 archive.db "select summary, topics, kind from posts where enriched_at is not null order by random() limit 20"If the tags are wrong, improve [about] and [topics] in channels.toml. Then run the sample again.
B6. Tag and embed all posts:
uv run enrich.pyThis step sends every post to OpenAI once. The cost depends on the number of posts and on TAG_MODEL (default gpt-5-mini).
B7. Do a test of the local search:
claude mcp add tg-archive -- uv run --directory "$PWD" mcp_server.pyThen ask Claude a question about the channels. Claude must call the tg-archive tools.
Part C. Deploy to the VPS
C1. Make sure that SSH key login to the VPS works without a password prompt:
ssh -o BatchMode=yes "$(grep '^DEPLOY_HOST=' .env | cut -d= -f2)" true && echo okC2. Copy the code, the secrets, and the database to the VPS:
deploy/push.sh --with-dataThe script does these steps:
It makes the system users
tgarchiveandtgmcp, and the grouptgarchive-db.It installs
ffmpeganduvon the VPS if they are not there.It copies the files that git tracks to
/opt/tg-ml-archive.It copies
.envandtg.sessionto/opt/tg-ml-archive(mode 600), andarchive.dbto/var/lib/tg-archive/.It installs and starts the hourly sync timer and the daily digest timer.
Run --with-data only one time. Later runs of --with-data overwrite the server database with your old local copy.
C3. Make sure that the sync works on the VPS:
ssh <DEPLOY_HOST> 'systemctl start tg-archive-sync && systemctl show -p Result --value tg-archive-sync'Expected result: success. To read the log, run journalctl -u tg-archive-sync -n 50 on the VPS.
From now on, the VPS is the only place that runs ingest.py. Your local archive.db does not get new posts.
Part D. Public HTTPS endpoint and digest page
D1. Run the setup script one time:
deploy/go-public.shThe script does these steps on the VPS:
It stops the sync for a short time. If the database is still in
/opt/tg-ml-archive, it makes a backup in/rootand moves the database to/var/lib/tg-archive/.It makes
/etc/tg-archive-mcp.env(root only) with a new randomMCP_TOKENand the OpenAI key. The script does not print the token.It deploys the code again and starts
tg-archive-mcp.serviceas usertgmcp. This user cannot readtg.sessionor.env.It installs Caddy and writes
/etc/caddy/sites/tg-ml-archive.caddyforDOMAIN. The main Caddyfile only imports/etc/caddy/sites/*.caddy.It does a test of the MCP server: 401 without the token, 200 with the token.
You can run the script again. Each step checks the state before it changes something.
D2. At your DNS provider, add an A record: DOMAIN → the IP address of the VPS.
D3. If the VPS has a firewall, open TCP ports 80 and 443. Caddy needs them to get the TLS certificate.
D4. When the DNS record works, do a test of the endpoint:
curl -s -o /dev/null -w '%{http_code}\n' "https://<DOMAIN>/mcp"Expected result: 401. This shows that HTTPS works and that the server refuses requests without the token.
D5. The first digest page comes after the first 3-day window is finished. The windows start on DIGEST_EPOCH (default 2026-10-17) at 00:00 UTC. To build a page now for a past window, run this on the VPS:
cd /opt/tg-ml-archive && runuser -u tgarchive -- env ARCHIVE_DB=/var/lib/tg-archive/archive.db .venv/bin/python digest.py --window 2026-10-17Change the date to the first day of a finished window. Then open https://<DOMAIN>/.
Part E. Connect Claude
E1. Claude Code on the local computer, local database: see step B7. This uses your local archive.db, which stops getting new posts after Part C.
E2. Claude Code, VPS database (recommended): get the token from the VPS into a shell variable, then add the server:
TOKEN=$(ssh <DEPLOY_HOST> "grep '^MCP_TOKEN=' /etc/tg-archive-mcp.env | cut -d= -f2")
claude mcp add --transport http -s user tg-archive "https://<DOMAIN>/mcp" --header "Authorization: Bearer $TOKEN"
unset TOKENE3. claude.ai (web, desktop, and phone): go to Settings → Connectors → Add custom connector. Use this URL:
https://<DOMAIN>/mcp?token=<MCP_TOKEN>claude.ai connectors cannot send headers, so the token goes in the URL. Caddy keeps no access log, so the token is not written to disk on the server.
E4. Do a test: ask Claude "Show the archive stats". Claude must call the stats tool and show the post count of each channel.
Related MCP server: Telegram Bot MCP Server
Daily operation
Task | Command (run on the VPS unless noted) |
Deploy code changes (on the local computer) |
|
Show the timers |
|
Run a sync now |
|
Read the sync log |
|
Read the MCP server log |
|
Add a channel | Edit |
Change the topics | Edit |
Rotate the MCP token | Edit |
Run Python commands on the VPS as the user tgarchive, with the database path:
cd /opt/tg-ml-archive && runuser -u tgarchive -- env ARCHIVE_DB=/var/lib/tg-archive/archive.db .venv/bin/python <script>.py <flags>Configuration
All settings are in .env. .env.example lists them with their defaults.
Key | Default | Use |
| (required) | Telegram API credentials |
|
| Session file path without the |
| (required) | Tagging, embeddings, and the digest |
|
| The tagger |
|
| Embeddings. Set |
|
| Database path. The systemd units set |
| (required for | The HTTP MCP server |
| (required for deploy) | SSH target of the VPS, for example |
| (required for Part D) | Public domain of the MCP endpoint and the digest page |
|
| Base URL in the Atom feed |
|
| First day of the first digest window |
|
| The model that writes the digest text |
|
|
|
Digest page
https://<DOMAIN>/ shows the latest finished 3-day window. The server keeps no old digests, and /digest/... links go to /. /feed.xml is an Atom feed with one entry for each new digest.
media.pyruns in the hourly sync. It saves the pictures, videos, and GIFs of new posts (full albums too) to/var/www/tg-digest/media/. ffmpeg converts videos to H.264 at 720p or less, with a poster frame. Posts that have only topics from[digest] no_mediaget no downloads.digest.pyruns daily at 01:30 UTC (timertg-archive-digest). When a new window is finished, it does these steps:It removes humor, off-topic posts, and empty posts (less than 60 characters and no useful links).
It puts posts about the same thing into one item and gives each item an importance (gpt-5-mini). Then it merges items whose posts link the same GitHub repo.
It writes each item in the language of its sources, and writes again each item that comes back in the wrong language.
It writes the overview as short lists by theme, in Russian. To change the language, edit
OVERVIEW_SYSTEMindigest.py.It saves the result to
/var/www/tg-digest/data.json, renderstemplates/toindex.htmlandfeed.xml, and deletes the media of posts from before the new window.
[digest]inchannels.tomlsets the section order and the sections without media.After a template change, run
digest.py --render-only. This makes no AI calls. To write only a new overview, rundigest.py --redo-overview.Cost: all calls use the OpenAI Flex tier at half price. If OpenAI refuses a Flex call, the code sends it again on the standard tier. One digest costs about $0.12 with
gpt-6-sol.data.jsonstores the real cost underusage.Media on disk: the current window and the next window, 250 MB at most.
Security model
tg.sessionand.envare readable only bytgarchive(mode 600).The public MCP server runs as
tgmcp. systemd blocks its access totg.session,.env,/root, and/home. Its secrets are in/etc/tg-archive-mcp.env, which only root can read.The database is in
/var/lib/tg-archive/. The grouptgarchive-dbgives both users access.Clients send
Authorization: Bearer <MCP_TOKEN>, or?token=when they cannot send headers.Caddy keeps no access log, so tokens in URLs are not written to disk.
Notes
Keyword search uses prefix matching, not stemming. Semantic search (embeddings) finds paraphrases and works across Russian and English.
The sync uses about 4 s of CPU per hour. The database is about 73 MB for 17,000 posts.
License
MIT. See LICENSE.
Available Tools
5 toolsget_postA
Full text and metadata of one post (use after search_posts returns a snippet).
Inputs: msg_id (required), channel (required).
list_topicsB
Topics, post kinds and availability values (with descriptions and post counts), plus the author's most used hashtags. Call this to pick filters for search_posts.
Inputs: top_hashtags.
recentB
Posts from the last N days, newest first, optionally filtered by topics/channels. Good for "what happened this week in agents".
Inputs: days, limit, topics, channels. Returns: result
search_postsA
Hybrid keyword + semantic search over all archived posts.
query: natural-language or keyword query (any language; exact names like "Qwen3" or arXiv ids work too).
channels: restrict to these channel usernames.
since / until: ISO dates, e.g. "2026-09-01" (until is exclusive).
topics: any-of filter, values from list_topics().
kind: one post type from list_topics() (e.g. model-release, tool-release, paper, tutorial, humor).
hashtags: any-of filter on the author's own hashtags, e.g. ["text2video", "gaussian"] (very precise).
availability: any-of filter, e.g. ["open-weights", "code-only"] for things you can run locally.
Returns posts with date, channel, summary, topics, links, text snippet and permalink.
Inputs: query (required), kind, limit, since, +5 more. Returns: result
statsB
Per-channel post counts, date range and enrichment progress.
Inputs: no named parameters. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
get_post, list_topics, and stats are clearly distinct, and the descriptions even prescribe a workflow (list_topics → search_posts → get_post). The only mild overlap is between recent (time-windowed browsing) and search_posts (query-based retrieval), both returning filtered post lists, but the time-vs-query distinction is documented well enough to separate them.
Two tools use the verb_noun pattern (get_post, list_topics) while the other three are bare nouns or adjectives (recent, stats, search_posts), and 'recent' in particular reads more like a parameter than a tool name. The set is still readable and unambiguous, but the convention is mixed.
Five tools is a tight, well-scoped surface for a read-only archive server: discovery (list_topics, stats), retrieval (search_posts, recent), and expansion (get_post). Each tool has a clear job and none feels redundant.
The read-only lifecycle is covered end to end: find filters, search, browse recent, expand full text, and inspect corpus stats. Minor gaps remain — no explicit channel-enumeration tool (channels must be guessed or inferred from stats) and no pagination/offset control on search or recent results for large result sets.
Related MCP Connectors
Search your newsletter and YouTube archive, drafted actions and working context from any MCP client.
Search your AI chat history (ChatGPT, Claude, Codex) from any MCP client. Remote, private, read-only
Personal knowledge MCP: capture bookmarks, notes & todos by chat; archive pages; search memory.
Hosted Telegram MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables Claude to interact with Telegram channels and groups through API or web scraping for reading posts and searching.1535 npmMIT
- AlicenseAqualityDmaintenanceEnables AI assistants to publish, edit, search, and manage messages in Telegram channels via a set of MCP tools.8MIT
- AlicenseBqualityDmaintenanceEnables users to read, search, and manage Telegram messages in channels, groups, and private chats through MCP tools.8MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI assistants like Claude to read, search, and send messages through your personal Telegram account via MCP tools.MIT