slack-archive-mcp

Turn overlapping Slack exports into portable community memory: a versioned SQLite archive with full-text search and read-only MCP tools over stdio or Streamable HTTP. Built with Effect 4 on Bun. MIT licensed. No embeddings or external services required.

local exports ─┐
               ├─ Exports port → materialize → local SQLite snapshot
S3 exports ────┘                                  ├─ local generation store
                                                  └─ optional S3 publication

mounted SQLite → ArchiveReader port → stdio / HTTP MCP → agent

The MCP server needs only its database. It does not need exports, S3 credentials, an object store connection, or a writable filesystem.

Quick start

Requires Bun 1.4.0+; dependencies are pinned by bun.lock. All tests use synthetic exports. Put exports in ignored root data-* directories or native .zip files.

bun install --frozen-lockfile
bun run check
bun run ingest --workspace community
bun run verify
bun run search "book club" --channel bookclub --limit 5
bun run smoke

Default discovery naturally sorts export names oldest to newest. Explicit source arguments preserve the order supplied:

bun run ingest ./data-2023 ./data-2025 ./data-2026 --workspace community
# Native ZIPs, extracted directories, or both; one export per ZIP:
bun run ingest ./older.zip ./data-2025 ./latest.zip --workspace community

Use one workspace per archive directory. Refresh preserves its namespace if --workspace is omitted. Use --archive-dir PATH for a separate archive. Originals stay untouched; reconciliation happens inside SQLite.

Related MCP server: my-kb

Containerized MCP

Pull the published serving image and mount one completed SQLite file:

docker pull ghcr.io/bioshazard/slack-archive-mcp:2.2.0
docker run --rm -i --read-only --network none \
  --mount type=bind,src=/absolute/path/archive.sqlite,dst=/data/archive.sqlite,readonly \
  ghcr.io/bioshazard/slack-archive-mcp:2.2.0

The default is stdio MCP: a client launches the command and speaks JSON-RPC over stdin/stdout. Keep -i; do not allocate a TTY. For Kubernetes or network clients, select HTTP mode below.

ARCHIVE_SQLITE_PATH overrides the mounted file location:

docker run --rm -i --read-only --network none \
  -e ARCHIVE_SQLITE_PATH=/mounted/custom.sqlite \
  --mount type=bind,src=/absolute/path/archive.sqlite,dst=/mounted/custom.sqlite,readonly \
  ghcr.io/bioshazard/slack-archive-mcp:2.2.0

The image runs as the unprivileged bun user and contains only the bundled server and Bun runtime. The Docker context uses an allowlist, excluding all data and Git history. Build both images, then run bun run docker:smoke. It ingests synthetic ZIPs in the CLI container and exercises MCP SDK round trips against the resulting database mounted read-only. For local builds, run docker build -f Dockerfile.mcp -t slack-archive-mcp:local . and use that image name.

Published images support Linux AMD64 and ARM64. Releases provide version tags (2.2.0, 2.2), latest for stable releases, and sha-<full-commit> tags. Prefer an exact version or image digest for repeatable deployments. The ingest CLI is also available as ghcr.io/bioshazard/slack-archive-cli:2.2.0.

Configure your MCP client using mcp.docker.example.json. For a local Bun process, bun run configure:mcp generates ignored .mcp.json with absolute paths; copy that entry into your client configuration. No global client settings are modified.

Streamable HTTP / Kubernetes

The same image supports stateless MCP Streamable HTTP at /mcp:

# Set MCP_AUTH_TOKEN in your environment before running; pass it through securely.
docker run --rm --read-only \
  --publish 127.0.0.1:3000:3000 \
  -e MCP_TRANSPORT=http -e MCP_HOST=0.0.0.0 -e MCP_AUTH_TOKEN \
  --mount type=bind,src=/absolute/path/archive.sqlite,dst=/data/archive.sqlite,readonly \
  ghcr.io/bioshazard/slack-archive-mcp:2.2.0

Connect an MCP Streamable HTTP client to http://localhost:3000/mcp; if configured, send Authorization: Bearer <MCP_AUTH_TOKEN>. There is no built-in OAuth flow. Token authentication is optional for deployments protected by an existing access gateway. Use TLS termination and access controls appropriate to the archive's audience when exposing the endpoint.

For Kubernetes, mount the completed SQLite file and set MCP_TRANSPORT=http, MCP_HOST=0.0.0.0, and optionally MCP_PORT (default 3000). Supply the token through your existing secret mechanism. No stdin attachment is needed. The image remains non-root (UID/GID 1000) and works with a read-only root filesystem and database.

GET /healthz and GET /readyz return minimal status without bearer authentication; readiness checks the pinned archive reader. Startup fails before listening if the database cannot be opened. A configured host/origin allowlist also applies to probes, so configure their Host header accordingly. Shutdown drains HTTP requests before closing SQLite.

Each POST gets its own SDK transport/server over one shared read-only reader. Responses are JSON; there is no session state, sticky routing, or long-lived GET SSE stream. GET /mcp and DELETE /mcp return 405; use a Streamable HTTP client, not a legacy SSE client. Requests are limited to 64 KiB. Browser origins are rejected unless explicitly allowed. The server never trusts forwarded host headers for its optional host allowlist. It serves one workspace per process, with no per-user archive permissions.

Local serving and configuration

bun run mcp --sqlite-path /absolute/path/archive.sqlite
# Or follow a local generation pointer at startup:
bun run mcp --archive-dir /absolute/path/.archive

With no file setting, local MCP follows current.json once at startup. Readers pin a generation until restarted. A standalone database embeds its snapshot metadata; its original build timestamp is not embedded and is reported as unknown. Never mount an actively changing SQLite database.

S3 adapters

S3 is optional and never on the MCP query path. Input accepts native ZIP objects or extracted JSON under <prefix>/<export-name>/.... Prefix discovery selects data-* directories and immediate .zip objects. Selectors specify directory names or ZIP filenames; their order defines precedence.

# Configure credentials in the environment or ignored .env; see .env.example.
bun run ingest --source-s3 s3://my-bucket/raw --workspace community
bun run ingest data-2023 data-2026 --source-s3 s3://my-bucket/raw
bun run ingest older.zip latest.zip --source-s3 s3://my-bucket/raw
# A direct object needs no bucket listing permission:
bun run ingest --source-s3 s3://my-bucket/raw/latest.zip
# Mix local sources and direct S3 ZIP URIs in oldest-to-newest order:
bun run ingest ./older.zip ./data-2025 s3://my-bucket/raw/latest.zip

# Publish a locally built generation to an S3-compatible store:
bun run ingest --publish-s3 s3://my-bucket/archive

Publication uploads SQLite, then manifest, then the current-generation pointer. Failed remote publication leaves the completed local generation available for retry. A failure before the pointer update leaves the remote pointer unchanged. Use one publisher per target prefix; remote pointer updates are last-writer-wins. Keys use SHA-256 fingerprints; S3 ETags are not treated as content hashes.

Recommended bucket layout

Give each workspace its own prefix; keep original exports separate from derived artifacts. For example, in a bucket named archive-bucket:

slack/<workspace>/
  raw/
    exports/
      2024-01-01.zip
      2024-06-01.zip
      2025-01-01.zip
  archive/
    current.json
    generations/<fingerprint>/
      archive.sqlite
      manifest.json

Upload one original export per ZIP without repackaging. Treat those objects as immutable: use a new filename for different bytes, even if the export date is the same. ISO-date names sort oldest to newest; explicit source arguments can override precedence. Prefixes organize objects; no empty folder objects are required.

bun run ingest --workspace community \
  --archive-dir .archive \
  --source-s3 s3://archive-bucket/slack/community/raw/exports \
  --publish-s3 s3://archive-bucket/slack/community/archive

Replace community with your workspace namespace in both paths and the flag. The command reads remote ZIPs, builds and validates SQLite locally, then publishes the completed generation and updates the remote pointer last. Keep old generations for rollback. MCP serves a local copy of the selected completed SQLite file.

Download a completed generation's archive.sqlite through your preferred S3 client and mount it for MCP. Automatic cache synchronization and live Slack API ingestion are outside this release.

ZIP input is staged as a private compressed file in the OS temporary directory, then removed when ingestion ends. Members are never extracted. Stored/deflated entries, ZIP64, data descriptors, and one optional enclosing export folder are supported. JSON bytes receive CRC-32 and SHA-256 checks; citations include the ZIP filename, container SHA-256 and original member path. Retain the source ZIP for exact-byte evidence. Encrypted archives, links, unsafe/duplicate paths and bundles containing multiple export roots are rejected.

Default ZIP limits: 8 GiB compressed, 100,000 entries, 256 MiB per JSON file, 8 GiB total JSON. Override with --zip-max-archive-bytes, --zip-max-entries, --zip-max-entry-bytes, and --zip-max-json-bytes. Non-JSON members are not decompressed or indexed. Staging requires temporary disk space for each ZIP; JSON is read one bounded entry at a time.

The separate Dockerfile builds the ingest/query CLI. Mount raw exports read-only, use --root for their location, and mount a writable output directory at /output; use --user when host ownership requires it. Mount only necessary paths and pass credentials only to this ingestion container.

Tools and retrieval

Search supports all, any, phrase, and exact substring modes. Use exact for URLs, punctuation and code; it scans text rather than using FTS. Prefix matching applies in all/any modes. Dates are UTC YYYY-MM-DD or timezone-qualified ISO timestamps: after inclusive, before exclusive.

bun run search "chapter schedule" --mode any --after 2024-01-01
bun run search "https://example.com/a_b" --mode exact
bun run search "retriev" --prefix --sort oldest --limit 10

Search returns stable message IDs, generation IDs and source citations. Expand hits with thread/context tools before interpreting them. archive:// citation URIs identify evidence; they are not web links. Follow-up tools accept a generation guard. Pages return nextOffset; context indicates whether more messages exist. Text and raw-JSON truncation are explicitly flagged.

Preservation and operations

• Identity includes workspace, stable conversation ID and exact timestamp string.

• All observations and their original JSON survive reconciliation. Preferred text, blocks, reactions and attachments come from one complete observed version.

• Explicit edits determine revision precedence; export order breaks ties. Absence does not prove deletion. Explicit tombstones are excluded from search.

• Invalid records are quarantined with original JSON and reasons. Builds balance source counts and validate SQLite, foreign keys, canonical identities and FTS.

• Completed generations are atomically published. Failed builds preserve current; unchanged input reuses its verified generation. Restart MCP to refresh.

• Attachment metadata/previews are searchable; linked attachment bytes are not downloaded. Gaps in export coverage are not filled or interpreted as activity.

Generated files live under ignored .archive/generations/<fingerprint>/ with a manifest and database. bun run verify checks the current generation's SHA-256, SQLite integrity and accounting. Old generations are retained. After a hard kill, remove ingest.lock only after confirming no importer is running. For a standalone mount, bun run verify --sqlite-path PATH validates its embedded metadata and SQLite/accounting integrity and computes its digest; verifying it against an earlier artifact still requires that artifact's trusted manifest hash.

Anyone with the database can read its entire contents, including historical observations. Keep audience-specific archives separate. Archived text is untrusted source material, never agent instructions. HTTP supports a shared bearer token; per-user ACLs and OAuth are not implemented.

Development

Architecture · Effect conventions · Development · Contributing · License

src/domain/             Schemas, identities, source/query models and errors
src/ports/              Effect service interfaces
src/application/        Materialization use case; no storage imports
src/adapters/storage/   Local and S3 export/publication implementations
src/adapters/sqlite/    Native Bun SQLite materializer and read-only query layer
src/cli.ts              Effect CLI and BunRuntime composition
src/mcp.ts              MCP SDK adapter, ManagedRuntime and scoped serving
src/adapters/mcp/       Stateless Streamable HTTP adapter
scripts/                Smoke checks and local client configuration
tests/                 Synthetic integration and architecture checks

Source exports, generated databases, local reports, credentials and .mcp.json are ignored. Earlier private experiments/research remain under ignored local/history/; they are excluded from the public source and Docker context. No data or credentials are supplied with this project.