super-log

# super-log

One hub for every log stream you have — devices, servers, containers, browsers, chains and apps.

This project is a consolidation of a patchwork of tools I have used, in one form or another, over the last fifteen years — the log mergers, port watchers, build wrappers, ad-hoc proxies and one-off scripts every long-running bench accumulates — rebuilt here as one coherent thing, on one wire protocol, with one screen.

Free and self-hosted, forever. It collects and consolidates; analysis is a separate, cleaner concern — hand the consolidated stream to super-log.com for real-time LLM analysis and team features, or to your own store. See Collection is not analysis.

Twelve producers on one screen, interleaved by arrival: C++ through both SN_LOG and spdlog, Rust, Go, Python, Swift, Fortran, a POSIX shell script, two React Native devices, Metal GPU work reporting real bandwidth, and a live Binance WebSocket. The last of those is running at hundreds of frames a second — so it is rate-capped, and says so rather than silently dropping them.

If you build across devices, you know the ritual: a Metro console for the iOS simulator, another for the Android emulator, adb logcat for the phone on your desk, a terminal for the C++ engine, another for the Rust service, browser devtools for the web build, and an ssh session to the box in the cloud. Six places to look, none agreeing on timestamps, and the bug is always in the interleaving.

super-log converges all of it on one process and one screen:

 apps, 14+ languages ┐
 GPU and graphics    │
 devices and boards  │                                     ┌─▶ native viewer (ImGui)
 machines, services  ├── POST NDJSON ──▶  superlogd :7333 ─┼─▶ web viewer (React)
 network and DNS     │                    (fan-out+replay) ├─▶ journal → search / replay
 builds and repos    │                                     ├─▶ GET /recent  (scripts)
 blockchain          │                                     ├─▶ MCP tools    (agents)
 anything that prints┘                                     └─▶ alerts → webhook

Every producer speaks one small wire protocol (docs/PROTOCOL.md: one JSON event per line, batched over plain HTTP POST), the hub fans out to any number of readers with replay-on-connect, and everything is interleaved by hub sequence — not by device clocks, which drift.

Quick start

macOS — Homebrew:

brew install saxonnicholls/tap/super-log && brew services start super-log

Ubuntu (incl. Ubuntu Server on a Pi) — the PPA, every architecture, auto-updating:

sudo add-apt-repository ppa:super-log/stable && sudo apt update && sudo apt install super-log

Debian / Raspberry Pi OS — the .deb directly (PPAs are Ubuntu-only; don't use add-apt-repository here). Pick your arch:

curl -fsSL -O https://github.com/saxonnicholls/super-log/releases/download/v0.4.0/super-log_0.4.0_arm64.deb   # or _amd64.deb
sudo apt install -y ./super-log_0.4.0_arm64.deb

Then see it work in two lines — the hub is already running on :7333:

echo "the bench is live" | superlog tee --topic hello
curl -s "http://127.0.0.1:7333/recent?topic=hello"

Fedora/RHEL, npm, vcpkg, building from source, and the web + native viewers are in Install below.

Related MCP server: Local Lens

The superlog CLI

One command reads the bench, manages the tailers, and pipes streams onto the hub:

superlog status            what's running, and the hub's health
superlog alarms            firing alarms   (every read command is NDJSON when piped — | jq)
superlog versions          the version inventory, per host
superlog start vitals      start a tailer in the background
superlog stop vitals       stop it
superlog viewer            open the native viewer  (builds it once if needed)
superlog tee               a stream onto the hub   (make 2>&1 | superlog tee --topic build)
superlog git install-hooks stamp every commit onto the bench
superlog git recall <sha>  replay the logs the bench saw behind a commit
superlog login             open super-log Cloud in your browser

Full reference: docs/CLI.md. Or superlog help.

What it does for you

One screen, everything on it. Streams colour-coded by source and level, filtered by stream, minimum level or substring. Pause freezes the display while collection continues; copy a row or the whole filtered view; export JSON, CSV or plain text.

The minimum level set to ERROR: 29 rows out of 1131. The same pricing failure surfaces from Swift, Python, Go and a shell script side by side — each in its own language's idiom, a PricingError, a KeyError, a returned error, a shell test — plus the GPU refusing an allocation four times the size of the card. Python's row carries the local variables from the failing frame, which is the part you would otherwise be adding a print statement to find.

Your apps need almost nothing. Fourteen dependency-free SDKs: header-only C++ (both a spdlog sink and a native snicholls::log one), plain C in one stb-style header — zero-alloc, and a production build provably contains no logging at all — a Rust crate with an optional tracing layer, Python plugging into stdlib logging, Go with a log/slog handler, Java with a java.util.logging bridge (Kotlin and Scala ride it, one import, zero glue), Swift, Fortran over raw POSIX sockets, OCaml over the same raw sockets, Haskell needing only GHC's boot libraries and curl, Lean 4 for the proof jobs that run all night (core IO plus curl — Lean grew a kernel before it grew sockets), Perl from core modules alone (HTTP::Tiny has shipped with Perl since 5.14), Lua making the same curl bargain as the shell SDK (Lua never grew sockets at all), C# from the BCL alone — everywhere .NET goes, including Unity profiles and Xbox Dev Mode — Zig through @cImport of the C header itself, COBOL through a 20-line C shim onto the header-only C SDK, Ruby from the stdlib — with a drop-in ::Logger adapter, which makes the whole Rails story one config.logger assignment — a sh one-liner for scripts, and one JS client for React Native, the browser and Node — patchConsole: true and every console.log is on the bench.

Six of them hook the logging framework the language already has — the spdlog sink, logging.Handler, slog.Handler, java.util.logging.Handler, Ruby's ::Logger adapter and patchConsole — so everything a program already logs reaches the bench without a single call site changing.

Why the interleaving is the point. In the middle, one Swift tick unfolds in order — the tick at INFO, a DEBUG pricing pass, the ERROR its exception raised, and the WARN that followed — while eleven other producers keep writing around it. Reconstructing that sequence from separate terminals is the ritual this replaces.

Follow one action across every tier. A tap becomes a request, a database write and a chain call on four streams. withTrace() mints a correlation id, carries it across awaits, and puts it on outbound HTTP automatically; a server adopts it and logs under the same id. Then one query — a ⇢ in the viewer, GET /recent?trace=…, or an agent tool — returns the whole story in order.

Every error, including the ones nobody logged. Uncaught exceptions and unhandled rejections are captured by default in every SDK, with whole stacks — not clipped to the throw site, because a deep React Native render error runs 100+ frames through the bridge and that path is the thing you actually need to paste. Capture chains to whatever was already installed, so React Native still shows its red box, Node still exits 1, and C++ still aborts. C++ traces are demangled (pricer::Engine::quote(int)) with no boost dependency. For the hardest class — an exception a library throws and a component boundary catches and displays, so it never reaches the global handler — wrap the tree in SuperLogErrorBoundary (or forward your own boundary's componentDidCatch to log.exception): that lands the component stack too, the one thing a JS stack never contains — it names the component that threw. Plus an opt-in breadcrumb on every Error construction.

Zero-app-change fallbacks. Host-side tailers scrape what already exists: adb logcat (scoped to one app, because an OEM handset emits ~600 lines a second), the iOS simulator's log stream, the macOS unified log, journald, Docker containers, and any log file. A catalog knows where ~20 common services log on macOS and Linux — postgres, mysql, mongodb, redis, nginx, apache, kafka, elasticsearch, rocksdb — including both Homebrew prefixes. The catalog also knows the engines and content tools: Unity's Editor.log (the C# compiler's error CS1234 becomes ERROR, a thrown exception too, while a folder named "Exceptions" stays INFO) and Unreal Engine's per-project editor logs (LogNet: Warning: maps by Unreal's own verbosity words, category kept). Blender logs to stdout — that is what the tee is for — and AutoCAD's LOGFILEMODE file tails like any other.

Whole fleets, pulled over ssh. One config file brings up every stream on every server: OS logs, service logs, container logs. Nothing is installed remotely, no port is opened, and the servers never need to reach the hub — so the hub can stay loopback-bound while still watching production.

HTTP calls, request and response. Front a service with the logging proxy and every call is one event (method, path, status, latency, size), or turn on patchNetwork in the app and see the calls it makes. HTTPS targets need no certificate work. Bodies are opt-in; credentials are always redacted.

Blockchain addresses, beside the code that touched them. Watch any EVM address and its transfers, contract events and native balance moves land on the same screen, in hub order, next to the app code that sent them.

Operational keys that never silently run dry. A keeper or oracle out of gas stops a production system as surely as a crashed server, and it fails politely — it just stops, and nothing says why. superlog-gas polls the balances you name (native coin or ERC-20, per chain, per key, from a gitignored config — a labelled key list is a map for an attacker) and applies the bench's discipline: readings are metric events for the chart, crossing your fund-now line is CRITICAL said once, your low line WARN once, and refunding is announced. One JSON-RPC batch per chain per poll, because a public RPC's rate limit is part of the design surface.

Infrastructure that only speaks when something changes. DNS records and TLS expiry, listening ports and the processes that own them — all watched by diffing snapshots, so the stream is silent until it matters: an NS record you did not change, a certificate three weeks out, a new public listener on a production box, a service that restarted without saying so.

Builds as events, not walls of text. Wrap any build — cmake, clang, gcc, cargo, npm, xcodebuild, local or over ssh — and compiler diagnostics become WARN/ERROR rows with file:line, with one summary event carrying exit status, duration and counts. Linking included: GNU ld's undefined reference and multiple definition carry no severity word and would otherwise scroll past as INFO; they, Apple's Undefined symbols and duplicate symbol, lld, collect2, and the driver's own clang: error: linker command failed all land as ERROR. Debuggers ride the same rails: pipe a batch session through the tee — gdb --batch -ex run -ex bt ./app 2>&1 | superlog tee --topic dbg.app --classify (lldb likewise) — or tail gdb's set logging file like any other.

Massive downloads, watched. A 70B model from Hugging Face is fifteen shards and half a day of \r-rewritten progress bars that exist only on the terminal that started them — and tqdm, curl and wget all mute or reshape those bars the moment their output is a pipe, so tee sees nothing. Wrap the fetch in superlog-dl and percent, bytes and rate become metric events; or point --watch at the destination directory and progress is measured at the filesystem, which no tool can mute and which is the only honest aggregate when every shard resets its own bar. The event that matters most is the stall: no bytes for 30 seconds is a WARN on the bench — hours before the fetch's own patience runs out at 97% of 140GB.

Watts, thermals, and who is drawing them. On macOS every bench run samples CPU package power, die temperatures, fan RPM, aggregate CPU as one number, and the top energy consumers — because a runaway process announces itself through the fans long after a chart would have caught it (see the power row below for the incident that earned this).

The machine's own life events. Crash reports and kernel panics parsed out of DiagnosticReports — process, exception, signal — the moment they land (and from the recent past at startup, because a crash writes its report before the reboot that restarts the watcher); the previous shutdown cause once per boot, translated (cause -128: uncontrolled power loss) and ERROR when unclean; volume mounts, unmounts and renames — a rename moves every path on the volume, which is how long writes die with nothing recording why; and sleep/wake, which explains every gap in every other stream. This bench crashed four times with all of that evidence sitting unread; the first live run surfaced an unclean shutdown and 38 crashes from the preceding three days. The demo starts it unconditionally on macOS, beside power.

Files, and the lines that changed. A generated header, a lock file a tool rewrote, a config a script overwrote — superlog-watch puts the change in the same ordered stream as the build that then failed, and with --diff it says which lines: one event per hunk, removed and added together, every hunk of a save sharing a trace with its modified event so one query returns the whole edit. Idempotent by content hash — a rewrite that changes no bytes publishes nothing, which an mtime watcher cannot promise.

History, not just the last few minutes. The journal writes every frame verbatim to disk; search reads it back with the same filters as the live feed (including --trace), and replay re-publishes it at original pace. A 1 GB / 4.8M-event journal searches in ~2.3 s.

Readable by scripts and agents. GET /recent?since=<cursor>&level=ERROR answers "what happened since I last looked", with a cursor that never misses or repeats an event.

An MCP server, so a coding agent can read the bench. Seven tools, and the shape of them matters: an agent's context is small and a firehose is not, so every tool filters first, caps its output, and returns one compact line per event. The documentation is queryable too: stream_guide explains any capability in detail — what power.*'s metrics mean, why a dl.* stall escalates, why a diff can be silent — from guide.json, and the same file's playbooks (triage, follow-a-trace, silent-stream…) are served as native MCP prompts. Every logging capability ships with an entry there, so an agent never has to guess what a topic means.

npm run demo:mcp        # drives all six over stdio and prints what an agent sees
# from a clone:
claude mcp add super-log --scope user -- node $PWD/sdk/js/packages/mcp/bin/superlog-mcp.mjs
# or, once published, with no clone at all:
claude mcp add super-log --scope user -- npx -y @super-log/mcp

Registered once per machine, not per project: one hub serves every project and agents narrow by topic prefix. Read-only by construction — with one deliberate, narrow exception: agent telemetry, which can land only on agent.* status topics (see the agents blotter below) — and dependency-free — MCP over stdio is newline-delimited JSON-RPC 2.0.

Not just Claude — any MCP client. The same server drops into every MCP-capable tool; the config is always npx -y @super-log/mcp:

// the universal server entry — Cursor, VS Code, Cline, Windsurf, Zed, Gemini CLI, …
{ "command": "npx", "args": ["-y", "@super-log/mcp"], "env": { "SUPER_LOG_URL": "http://127.0.0.1:7333" } }

Three ways to add it:

• MCP Registry — listed as com.super-log/super-log, discoverable in-app and installable by name (claude mcp add com.super-log/super-log); it propagates to the Cursor / VS Code / Glama / PulseMCP catalogs on its own.

• Claude Code plugin — /plugin marketplace add saxonnicholls/super-log then /plugin install super-log@super-log (adds the server and a /super-log:stream-guide skill).

• Claude Desktop — double-click the .mcpb from the release, no CLI.

Per-client setup for Cursor, VS Code, Cline, Windsurf, Zed, Continue, LibreChat and Gemini CLI — plus the OpenAI Agents SDK / Google ADK recipes — is in docs/CONNECT.md.

Already on OpenTelemetry? Join with one exporter stanza, no new SDK. superlog-otlp is an OTLP/HTTP server on the standard 127.0.0.1:4318, so pointing an existing exporter at it needs zero endpoint config: OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318. It speaks both OTLP/JSON and OTLP/protobuf (a hand-rolled bounded decoder, because most SDKs default to protobuf), maps severityNumber to bench levels, and turns each resource's service.name into an otlp.<service> topic. The detail that makes it more than a gateway: an OTel span's trace_id becomes the event's trace, so it lines up with withTrace()'s own ids — one /recent?trace= returns the OTel spans and the bench's lines as a single story. Metrics arrive as metric readings (histograms summarized to count and sum, never invented percentiles); spans as trace-carrying DEBUG events, because this is an inlet, not a span store.

And the reverse: feed your existing OpenTelemetry backend from the bench. superlog-otlp-export reads the hub and POSTs OTLP/JSON to any collector or vendor — superlog-otlp-export --endpoint https://otlp.example.com --header 'authorization: Bearer …'. Every event becomes an OTLP LogRecord (the severity band inverted, service.name from origin.app, fields as attributes, a 32-hex trace as traceId); a bench metric event becomes an OTLP gauge point. JSON only, on purpose — it is what every collector accepts; a backend that demands gRPC/protobuf is one OTel Collector hop away (see docs/OTLP.md). The bench has no spans of its own, so traces are not exported. Together the inlet and outlet make super-log a two-way OTLP member of a bench, not a walled garden.

Something reaches you when nobody is watching. Rules over the live feed fire through one channel registry — console, desktop, webhook, command today; Telegram, Twilio SMS / WhatsApp and email are config-gated entries in the same registry (superlog-alert --channels prints the roster and exactly what each missing channel needs) — and back onto the bench as alert.*. Four rule shapes, because production breaks in four ways: something bad was logged, too much of it was logged, something stopped being logged (a quiet server and a dead server look identical until you check), and several things happened together — a combo rule fires when all its conditions land inside one window, the correlation ("a deploy happened AND errors spiked") no single filter can say.

Production can raise the alarm, without logging a thing. Every SDK's PRODUCTION mode ships nothing by design — but an ALARM is not logging, it is a rare deliberate act, and superlog-alarm is its door: a token-guarded public webhook through a Cloudflare tunnel — a quick tunnel with zero config, or a stable named tunnel auto-provisioned via the API given a token. Cloudflare is the verified, daily-driven path: quick tunnels, the named tunnel, DNS provisioning and the endpoint factory all run against it on a real bench. --tunnel ngrok and --tunnel zrok exist as code paths but have not been driven the same way yet — see Future directions. One curl from any language fires alert.inbound.<name>; the same key re-firing is one alarm with a repeat count, not 113 pages; recovery closes the loop; and every checker can POST a heartbeat so the gateway itself raises monitor_dead:<name> when a watcher goes silent — the alarm the dead watcher cannot send. Both viewers carry an alarm blotter — a sparse panel, deliberately unlike the firehose, one row per key — with a test-alarm button that proves the whole path step by step: hub, tunnel, a real round-trip from the internet back through the public URL, the notification channels (including the diagnosis when your own router's DNS filters the tunnel's name while production's resolves it fine), and then every route on the books, each with its own verdict — a capture endpoint passes only when a probe posted through its public URL lands back on the hub as a wh.<name> event, a forwarded port that answers 502 is reported as "tunnel up, your service is not", and a watch-only URL passes on any HTTP answer, because any answer proves the wire.

And straight from code: SN_ALARM. The gateway is the door for a production system across a network; for an alarm the running program itself wants to raise — an assertion that should wake someone, a dependency gone dark — every SDK has a first-class primitive that skips the log and lands in the blotter directly:

SN_ALARM("settlement engine unreachable");        // C++: P0, fires in the panel
SN_ALARM_CLEAR("settlement engine unreachable");  // clears it

The same call is in every language super-log speaks — sn_alarm(), sn_alarm!, .alarm() and their kin — because an alarm you can only raise from one language is a gap an incident finds. It emits on alert.native.<key>, which the rules engine ignores by design (so it never trips a rule), keyed and edge-triggered so a hot loop is one alarm and not a flood; CRITICAL fires it, an INFO RECOVERED closes it. It posts straight to the hub — immediate and local; the gateway's cross-process dedup and repeat-counting are for alarms that cross a network. And it is deliberately not gated by PRODUCTION mode: an alarm you asked for is the one thing that must not go quiet in production (SUPER_LOG_ALARMS=0 mutes it if you must). Every demo client fires one so you can watch it land.

The gateway is also an endpoint factory, and the viewers split its two audiences: alarms (production) — the sparse blotter plus the alarm path's own routes — and webhooks (development) — the endpoint grid and a live feed of every captured delivery with its payload, signature verdict and relay status. Both are toggled from a menu bar that the ImGui and React viewers render from the same declarative viewer/menu.json — alongside an agents blotter (View → Agents): every agent working the bench, one row each — name, which LLM it runs on, latest status, last seen. MCP consumers appear the moment they connect (the server announces its own initialize handshakes) and their tool calls show as requesting events; anything can report via the agent_report MCP tool or one POST to agent.<name>, declaring its LLM, task, percent, and the cadence it promises — every 15 minutes by default, plus on events, which is how an 8-hour job stays visible. The freshness light is held to each agent's own promise: green inside 2×, late, then silent — an agent that broke its own word looks like it. And a servers board: every event carries origin.device, so the hub's traffic is the server list — one row per machine with a recency light (up / quiet / silent), last seen, and the loudest level of the last minute, whatever the mechanism that produced the events (a vitals reading, a ping metric, an app's own SDK). "Is the build box fine" becomes one glance; when the answer is grey, the detailed logs are one topic filter away in the firehose — and a silence rule in alerts.json is what turns grey into an alarm. Every route is a grid row — status : route : url : seen : ping — expandable into full diagnostics, with ping (measure this route now, on the same watchdog books), copy (the full public URL) and delete buttons, each route pinged on its own configurable clock (two consecutive failures raises tunnel_down:<name>, recovery clears it). + endpoint provisions in one click — a name alone captures deliveries as wh.<name> events (paste the URL into a Stripe or GitHub webhook form and watch them arrive), a name plus port forwards a local service. Many endpoints live better in a file: npm run alarm -- --provision endpoints.json applies a declarative manifest ({"name":"stripe"} capture, {"name":"webapp","port":5173} forward, {"name":"partner","url":"https://…"} watch-only, each with optional interval_s) and keeps it applied as the file changes — new names appear, removed names are torn down, though only names the file created. endpoints.json.example shows every shape; the real file is gitignored because names, ports and signing secrets describe the bench. Every URL the gateway currently owns is rewritten to endpoints.env (gitignored) on every change, so scripts and agents can simply source it — and agents can also just ask: the MCP server's list_webhooks returns the roster with public URLs and health.

Webhook testing, done properly — Stripe as the worked example. A capture endpoint alone is already a webhook inspector: paste its URL into the Stripe dashboard (or stripe trigger payment_intent.succeeded at it), and every delivery lands as a wh.stripe event with method, headers (stripe-signature kept) and up to 32KB of body — a real invoice event runs 5–15KB and arrives whole. Two additions make it a development tool rather than a peephole. Give the endpoint your signing secret ("secret":"whsec_..." in the manifest, or STRIPE_WEBHOOK_SECRET) and the gateway verifies the Stripe signature scheme on arrival: each event carries sig: verified or sig: FAILED (WARN), with stale timestamps flagged as possible replays — and no secret means the field stays absent, never a fake verdict. Give it a relay ({"name":"stripe","relay":5000} or a full URL like http://127.0.0.1:5000/webhook) and each delivery is also handed to your local handler, whose real response goes back to Stripe — stripe listen --forward-to, except every delivery, signature verdict and your handler's status code are on the bench, in order, beside your app's own logs. A handler that is down shows up as relay unreachable at WARN — the exact finding a webhook test exists to produce. (local: true skips the tunnel entirely when the bench only needs to test against itself.) GitHub's scheme (x-hub-signature-256) verifies beside Stripe's against the same secret; other providers are one clause each.

The webhooks panel's deliveries feed is built for reading the results: filter by endpoint and by level exactly like the main log (WARN isolates the failed signatures and unreachable relays), expand any delivery's payload in place, and copy — one button per delivery, one for the whole filtered set — hands you paste-able evidence: the feed line, the signature and relay verdicts, then the payload verbatim. The whole loop is exercised end-to-end with the Stripe CLI:

stripe listen --print-secret            # the whsec_... for endpoints.json
stripe listen --forward-to http://127.0.0.1:7336/hook/stripe-test
stripe trigger payment_intent.succeeded # real events, signatures verified

Producers never block. Every SDK uses a bounded queue that drops oldest under burst — counted, never hidden. A logger that can stall the app it observes is worse than no logger.

Off in production, by construction. Every SDK requires you to declare DEVELOPMENT or PRODUCTION (neither or both refuses to build), and each mode ships only what its policy allows. Production defaults to nothing.

What this is, and what it is not

It is a development tool. The bench you sit at: everything your machine and your devices are saying, on one screen, in the order it happened, while you are working. It is optimised for the ten seconds after something breaks — one hub, no schema to declare, no agent to install, a stream added by typing one command, and nothing to configure before the first line appears.

It is not an observability suite, and should not be used as one. The distinction is not modesty, it is design: several things that make it good at the first job make it unfit for the second.

Concretely, do not point this at production and walk away. The hub has no authentication: anyone who can reach the port can read every stream and publish to any topic. It keeps 2000 events per topic in memory and forgets the rest unless the journal is running. Throughput, though, is not the limit that stops you: on one developer machine a single hub ingested a 250-machine fleet's burst — 500,000 events in 2.3 seconds (~217,000 events/second, 67 MB/s), none dropped — and a level-filtered /recent still answered in 16 ms while it arrived (bench/loadtest.mjs — reproduce it yourself against a throwaway hub). What makes it unfit for production is the two sentences above, no auth and an in-memory ring, not the ingest path.

What it is fair to do in production is pull: the ssh tailer and the fleet runner read remote logs onto your bench over ssh, so production never needs to reach the hub and the hub never needs to be exposed. That is how the fleet support is meant to be used.

When you outgrow it, you have not wasted anything — the wire format is NDJSON on plain HTTP (see docs/PROTOCOL.md), so a forwarder into Loki, Elasticsearch or an OTLP collector is a small script that subscribes to the firehose and re-posts. The two tools answer different questions and it is reasonable to run both: this one for "what is happening right now while I am looking", that one for "what happened last Tuesday at three in the morning".

Collection is not analysis — and that is the whole design

super-log does one thing: it collects and consolidates. Every stream a bench produces, on one wire, in one order, on one screen. It deliberately does not analyse — no query language, no dashboards to build, no rules engine you must program before the first insight. That separation is not a missing feature; it is the point. Collection should be dumb, fast, and trustworthy; analysis is a different job with different tools, and coupling the two is how logging stacks become the thing you have to operate instead of the thing that helps you operate.

This is the opposite of the log4j lesson. The Java logging world put a large, powerful, evaluating framework inside every application — and in 2021 the world learned what that costs, when a string in a log message became remote code execution (log4shell) across half the internet. The framework that was supposed to observe the app could compromise it. super-log inverts every part of that:

• Logging lives outside the app. A super-log producer is a thin, zero-dependency SDK — often just one file, or no code at all: point an existing OpenTelemetry exporter at it, or tail a file. There is no heavy framework in your process to be a liability.

• Content is data, never code. Every parser on the collection side is bounded and treats input as bytes to be stored, never expressions to be evaluated — the log4shell class of bug is structurally impossible here.

• Production ships nothing by default. Every SDK is DEVELOPMENT-xor- PRODUCTION, and PRODUCTION is an inert shell that sends nothing until you opt a level in — the compiled languages compile the wire code out entirely, provably (strings finds no endpoint in the binary).

So the app stays simple and safe, the bench stays a bench, and what you do with the consolidated stream is a clean, separate choice — eyeball it, forward it to your long-term store, or hand it to something that reasons about it.

super-log.com — analysis and AI, for teams

The open-source bench is the collection and consolidation layer, free and self-hosted, forever. super-log.com is the analysis layer built on top of it — the commercial service for when a consolidated stream is more than one person can watch:

• Real-time LLM analysis of the consolidated log. An AI reads the whole interleaved firehose as it happens — every service, device and chain at once — and tells you what changed, what correlates, and what is about to break, in the plain-language a teammate would use. The thing a human does in the ten seconds after something breaks, done continuously across streams no human can watch all of.

• Built for teams. One consolidated view of everyone's benches and services, shared context, history that outlives a laptop, and alerting that reaches the person who can fix it. The servers, agents and PR boards in the open-source viewers hint at the shape; the hosted service makes them a shared, durable, analysed surface.

• Same clean boundary. super-log.com consumes the exact same NDJSON wire the open-source hub speaks — nothing new to instrument. Your collection stays yours and self-hosted; you choose to send a stream up for analysis, and you can stop at any time. Collection and analysis stay separate all the way through.

The open-source project is complete and useful on its own — the commercial service is analysis you would otherwise build, not a paywall around the bench.

Install

The fastest way onto the bench — Homebrew — and a runnable example in the same breath:

brew install saxonnicholls/tap/super-log     # the hub + all the tailers

# See it work in three lines:
brew services start super-log                        # hub on :7333
echo "the bench is live" | superlog tee --topic hello    # a line of your own
curl -s "http://127.0.0.1:7333/recent?topic=hello"   # read it back

# Then put your machine on the bench — any of these, immediately:
superlog-netstate      # gateway, Wi-Fi, VPN, resolvers — changes only
superlog-otlp          # OpenTelemetry inlet on :4318
superlog-otlp-export   # OpenTelemetry outlet: bench -> OTLP/HTTP JSON backend
your-build 2>&1 | superlog tee --topic build    # anything that prints

Other channels, each landing the hub and the tailers:

# npm — the tailers and the MCP server as commands, anywhere Node ≥18 runs
npm install -g @super-log/tailers @super-log/mcp

# Ubuntu (incl. Ubuntu Server on a Pi) — the PPA: apt install super-log, every
# architecture Launchpad builds (arm64 included), with apt upgrade after.
sudo add-apt-repository ppa:super-log/stable
sudo apt update && sudo apt install super-log

# Debian / Raspberry Pi OS — PPAs are Ubuntu-only (no add-apt-repository here),
# so take the .deb from the release (apt, not dpkg -i, so nodejs and libatomic1
# come with it). Pick your arch — arm64 for a 64-bit Pi, amd64 for x86:
curl -fsSL -O https://github.com/saxonnicholls/super-log/releases/download/v0.4.0/super-log_0.4.0_arm64.deb   # or _amd64.deb
sudo apt install -y ./super-log_0.4.0_arm64.deb

# Fedora / RHEL / Rocky / Alma — dnf takes the URL directly:
sudo dnf install -y https://github.com/saxonnicholls/super-log/releases/download/v0.4.0/super-log-0.4.0-1.fc41.x86_64.rpm

# vcpkg — the SDK for C/C++ (incl. Windows/MSVC). C header by default;
# [cpp] adds the header-only C++ SDK (pulls the ts-moveables vcpkg port).
vcpkg install super-log          # the zero-dependency C header
vcpkg install super-log[cpp]     # + the C++ SDK (event model, SN_LOG)

Pick the .deb for your architecture (dpkg --print-architecture). On a systemd host the hub starts on install; in a container without systemd, run superlogd yourself.

On install the hub binds loopback (127.0.0.1) — right for a dev box, but if this machine is the collector other devices ship logs to over the LAN (a common Raspberry Pi role), they cannot reach it until you bind the network: set SUPER_LOG_LAN=1 (or SUPER_LOG_BIND=0.0.0.0) in the service environment and restart superlogd. The failure is silent at both ends — the device shows "never reported" and the hub logs nothing — so if you are standing one up as a LAN sink, see docs/DEVICES.md for the "loopback trap" and how to make the LAN binding persist across restarts.

The MCP server also runs straight from npx, no install: claude mcp add super-log -- npx -y @super-log/mcp. The viewers (native and web) come with a clone — see Quick start below. Packaging sources and the maintainer publish steps live in packaging/.

Quick start (from a clone)

git clone --recurse-submodules --shallow-submodules <this repo>
cd super-log
cp .env.example .env             # optional: chain endpoints, hub defaults

# The one-command install: preflight the toolchain, build the hub and
# viewer, and verify every SDK actually delivers to a real hub before
# calling itself done. Add --persist to start the hub and the default-on
# tailers at login and keep them running across reboots.
./scripts/install.sh             # build + verify
./scripts/install.sh --persist   # ...and run at login, forever

# The whole demo: hub, C++/Rust/iOS/Android/browser/container clocks,
# OS-log streams, both viewers - one command
npm run demo                     # see demo/README.md for the tour

# ...and the other languages on the same screen, if their toolchains are here
SUPER_LOG_LANGS="go python java swift fortran shell" npm run demo

# Or piece by piece:
./scripts/build_all.sh           # rebuild everything (hub, native + web viewers, SDK tests)
./scripts/dev.sh                 # build + run the hub
npm install && npm run viewer    # web viewer on http://localhost:7334
npm run tail:android             # first stream: the Android emulator

The demo binds to loopback. Real phones need the hub on the LAN: SUPER_LOG_LAN=1 ./demo/run.sh — read the security section first. The viewer finds the hub from the host that served the page, so opening it from another machine needs no configuration.

Configuration files

Most streams need no config — you just run them. A handful that carry secrets (API keys, RPC URLs with keys, wallet addresses, tokens) read a file that is gitignored, so your credentials never reach a commit. Every one ships a .example twin beside it: copy, then edit. That's the whole ritual — cp <name>.example <name>, fill in your values, run.

Config-free by design — nothing to create: the servers, devices (USB), agents and PRs boards derive from the hub's own traffic or a gh/env setting, and the OS/app/network tailers take command-line flags. If a tailer needs a file it can't find, it tells you the exact cp line and stops — it never runs half-configured.

Installing it in a project

Clone it once, use it from every project. super-log is not a dependency you add to a repo — it is a tool you install on a machine, like a debugger. The hub is machine-wide and shared: one per bench, every project on it. Two hubs would be two ports competing and two viewers each showing a third of the picture.

That also keeps your repo clean, and it avoids a real cost — a Cargo git dependency on this repo clones all five submodules and 36 MB of C++ that the Rust crate never touches.

git clone --recurse-submodules --shallow-submodules \
  https://github.com/saxonnicholls/super-log ~/dev/super-log
cd ~/dev/super-log && npm install        # once, for the tailers and the web viewer

Then, per project:

~/dev/super-log/scripts/setup.sh ~/code/my-app

That writes exactly two files into your project and touches nothing else:

.superlog/ (pids and logs) is added to your .gitignore. The project type is detected, so the config arrives pre-filled rather than blank — a package.json gets npm run build, a CMakeLists.txt gets cmake --build build -j, and so on.

cd ~/code/my-app
$EDITOR superlog.conf     # topic prefix, dirs to watch, log files, services
./logging.sh start        # hub + viewer + this project's streams
./logging.sh status       # what is running, and where
./logging.sh stop         # stops what THIS project started - not the shared hub

Booting it with your build and run is the point of the last two commands. They start the logging first if it is not already up, so wiring them into what you already type is all it takes:

./logging.sh build        # your build, with its compiler diagnostics as events
./logging.sh run          # your program, output teed to the terminal AND the bench

build runs through the build wrapper, so warnings and errors arrive as WARN/ERROR with file:line, and sanitizer or valgrind findings arrive whole. run runs through superlog-tee, so stdout reaches your terminal byte for byte and the bench at the same time. Either is a drop-in for the command it wraps — alias b='./logging.sh build', or setup.sh --wire to add npm run log / log:stop / log:status to a Node project.

Sharing one bench between projects works the way you would hope. The second project's start finds the hub and viewer already up and only adds its own streams; each project's topics carry its own prefix, so they stay separable in the viewer; and stopping one project leaves the others running.

If you would rather wire an SDK into your code directly instead of watching from outside, that is the next section — but you do not have to, and for most projects the launcher is enough to see everything.

Putting your own apps on the bench

React Native / browser / Node (@super-log/client, zero dependencies):

import { createSuperLog } from '@super-log/client';

const slog = createSuperLog({
  url: 'http://192.168.1.20:7333',   // your bench machine
  topic: 'expo.ios.device',          // topics name streams - PROTOCOL.md
  app: 'my-app',
  development: __DEV__,              // exactly one of these two, or it throws
  production: !__DEV__,
  patchConsole: true,                // console.* now reaches the bench
  patchNetwork: true,                // ...and every HTTP call it makes
});

// One id for everything this action causes, on every tier it reaches
await slog.withTrace(async () => {
  slog.info('checkout mounted', { user: '42' });
  await fetch('https://api.example.com/v1/pay');   // header added for you
});

React trees can wrap once with @super-log/react: <SuperLogProvider> owns the client and an error boundary that logs the component stack — which a global handler can never see, because React swallows render errors.

C++ (header-only; compile with -DSUPERLOG_DEVELOPMENT or -DSUPERLOG_PRODUCTION):

superlog::transport_config cfg;
cfg.topic = "cpp.pricer";
auto bat = std::make_shared<superlog::batcher>(cfg);   // before the logger

superlog::origin who;
who.app = "pricer";
spdlog::default_logger()->sinks().push_back(
    std::make_shared<superlog::spdlog_sink_mt>(bat, who));

superlog::install_terminate_handler(bat, who);  // uncaught exceptions + stack

Plain C (one header, stb-style; the same two macros, and PRODUCTION compiles every call to nothing — strings the binary for /ingest/ to prove the logger absent, rather than trusting a flag):

#include "superlog.h"                    /* sdk/c, zero-alloc, POSIX sockets */

superlog_t lg;
superlog_init(&lg, "c.engine", "engine");
superlog_info(&lg, "engine up, port %d", 9000);
superlog_kv(&lg, "ERROR", "no rate", "symbol", "DOGE", NULL);
superlog_metric(&lg, "queue.depth", 17);
superlog_flush(&lg);

Python (standard library only; development= / production=, exactly one):

import logging, superlog

log = superlog.SuperLog(topic="python.pricer", app="pricer", development=True)

logging.getLogger().addHandler(log.handler())  # everything already logged
log.install_excepthook(capture_locals=True)    # and every crash, with locals

with log.traced():                 # ContextVars: async- and thread-safe,
    log.info("order received")     # and inherited by everything called inside
    stdlib_logger.debug("pricing") # ...including plain logging calls

Python gets two things the other SDKs cannot. logging.Handler means every line the program already logs reaches the bench with no call-site changes. And capture_locals attaches the local variables of the failing frames, so an exception says symbol='DOGE', n=7 rather than only where it happened — secret-looking names are redacted and values truncated.

Go (a log/slog handler, so existing calls need no changes):

log, _ := superlog.New(superlog.Config{
    Topic: "go.pricer", App: "pricer", Development: true,
})
defer log.Close()
slog.SetDefault(slog.New(log.SlogHandler(nil)))   // everything already logged

ctx, _ := superlog.WithTrace(context.Background(), "")
slog.InfoContext(ctx, "order received")           // ...on the tick's trace

go func() { defer log.Recover("worker"); work() }()  // panics, with stack

Trace lives in context.Context rather than a goroutine-local, because Go deliberately has none — so the id travels exactly where the context does. Recover logs the panic and re-panics: a logger that swallows a crash has changed the program it was meant to observe.

Java and Kotlin (java.util.logging bridge; InheritableThreadLocal trace, so a pooled task inherits its submitter's id):

var log = SuperLog.builder().topic("java.pricer").app("pricer")
                  .development(true).build();
Logger.getLogger("").addHandler(log.julHandler());   // everything already logged
log.installUncaughtHandler();                        // and every crash

log.traceScope(() -> {
    log.info("order received");
    pool.submit(log.wrap(() -> log.debug("settled")));  // same trace
});

Swift (@TaskLocal trace, inherited by child tasks):

let log = try SuperLog(topic: "swift.pricer", app: "pricer", development: true)
try SuperLog.withTrace {
    log.info("order received")
    Task { log.debug("pricing pass") }   // same trace, nothing passed in
}

There is deliberately no setTrace(): a TaskLocal binds to a scope and nothing else, which makes the usual leak — one request's id surviving into the next — impossible to express rather than merely discouraged.

Fortran (raw POSIX sockets through ISO_C_BINDING, no libcurl):

call sl_init(topic='fortran.solver', app='solver')
call sl_set_trace(sl_new_trace())
call sl_metric('solver.residual', residual)
if (residual /= residual) call sl_error('residual is NaN')
call sl_close()

A solver is the hardest program on the bench to observe: hours long, often somewhere you cannot attach, and the evidence is a slurm file nobody reads until the allocation is spent. DEVELOPMENT xor PRODUCTION is a preprocessor error like the C++ SDK, and SIGPIPE is ignored at init so a hub that goes away cannot kill a run twelve hours in.

Shell (any script, one line):

superlog-log --topic deploy "starting rollout"
tail -f /var/log/app.log | superlog-log --topic app.foo --level WARN

Rust (build with --features development or --features production):

let log = super_log::SuperLog::new(super_log::Config {
    topic: "rust.pricer".into(),
    app: "pricer".into(),
    ..Default::default()
});
log.install_panic_hook();                 // panics, with location
log.log(super_log::Level::Info, "engine up", None);
log.metric("fps", 58.9);

Ruby, and therefore Rails (stdlib only; mode from SUPERLOG_MODE):

require_relative "superlog"                        # sdk/ruby/superlog.rb
log = SuperLog.new(topic: "ruby.myapp", app: "myapp")
log.info("up", port: 3000)
log.metric("queue.depth", 17)

# Rails: everything it already logs, one assignment, no controller touched
config.logger = ActiveSupport::BroadcastLogger.new(
  ActiveSupport::Logger.new($stdout), log.logger_adapter)

Scala (no SDK, on purpose — the Java client, one import, zero glue, exactly as Kotlin uses it):

import com.snicholls.superlog.SuperLog

val log = SuperLog.builder().topic("scala.pricer").app("pricer")
  .development(true).production(false).build()
log.installUncaughtHandler()              // every thread's uncaught, chained
log.info("engine up", SuperLog.fields("port", Int.box(9000)))

OCaml (the unix library and nothing else; mode from SUPERLOG_MODE):

(* ocamlc -I +unix unix.cma superlog.ml myapp.ml -o myapp *)
let log = Superlog.create ~topic:"ocaml.solver" ~app:"solver" () in
Superlog.info log "solver up" ~fields:[ ("mesh", "1.2M cells") ];
Superlog.metric log "solver.residual" 1.6e-9;
Superlog.flush log

Haskell (GHC's boot libraries plus curl; the mode is compiled in — -DDEVELOPMENT or -DPRODUCTION, neither or both refuses to build):

import SuperLog
main = do
  lg <- newLog "haskell.pricer" "pricer"
  info lg "engine up" [("port", "9000")]
  metric lg "queue.depth" 17
  flushLog lg

Lean 4 (core IO plus curl; mode from SUPERLOG_MODE — proof search runs all night, and a metric per thousand goals beats a silent terminal. superlog-build -- lake build covers the build itself, with lake's [n/m] progress as a chartable build.progress_pct):

import Superlog

def main : IO Unit := do
  let lg ← Superlog.init "lean.search" "search"
  Superlog.info lg "search up" [("depth", "12")]
  Superlog.metric lg "goals.open" 4123
  Superlog.flush lg

C# (BCL only — HttpClient and System.Text.Json ship with .NET; mode from SUPERLOG_MODE). One file into any project: a console app, ASP.NET, a Unity project on a .NET profile, an Xbox Dev Mode build:

var log = new SuperLog(topic: "csharp.myapp", app: "myapp");
log.Info("up", new() { ["port"] = "3000" });
log.Metric("queue.depth", 17);
log.Flush();

Perl (core modules only — HTTP::Tiny and JSON::PP ship with Perl; mode from SUPERLOG_MODE), for the glue scripts and cron jobs that run half the world:

use lib "sdk/perl";
use SuperLog;
my $log = SuperLog->new(topic => "perl.backup", app => "backup");
$log->info("nightly backup starting", { host => "web1" });
$log->metric("backup.bytes", 48_211_233_792);
$log->flush;

Lua (any Lua 5.1+, PUC or LuaJIT; no sockets in Lua's stdlib, so it makes the same honest curl bargain the shell SDK makes):

package.path = "sdk/lua/?.lua;" .. package.path
local superlog = require "superlog"
local log = superlog.new{ topic = "lua.game", app = "game" }
log:info("level loaded", { level = "e1m1" })
log:metric("frame.ms", 6.9)
log:flush()

COBOL (yes, really — GnuCOBOL CALLs the header-only C SDK through demo/cobol/shim.c, ~20 lines, so the oldest business language on the bench inherits the C SDK whole, provably-compiled-out production included):

CALL "cobol_superlog_init"
    USING BY CONTENT Z"cobol.batch", Z"payroll"
CALL "cobol_superlog_log"
    USING BY CONTENT Z"INFO", Z"payroll run 4711 starting"

Zig needs even less: @cImport reads sdk/c/superlog.h directly (see demo/zig/clock.zig), with the header compiled once for the linker — cc -DSUPERLOG_API= -c impl.c — which is the same one-object route any language with a C FFI can take.

Machines, services, containers, chains (no app changes at all):

npm run tail:os -- --process MyApp          # this Mac's unified log
npm run tail:apps                           # what services log here
npm run tail:app -- postgres nginx redis    # ...then turn them on
npm run tail:app -- unity unreal            # engine editor logs, level-parsed
npm run tail:file -- /srv/app/production.log
npm run tail:ssh -- my-server               # a remote box, OS auto-detected
npm run tail:ssh -- db1 --app postgres      # ...or its postgres
npm run net -- 9000 http://localhost:3000   # every HTTP call through :9000
npm run grpc -- --listen 50052 --target localhost:50051  # every RPC, status from the trailer
npm run chain                               # watched addresses (see .env)
npm run gas                                 # operational key balances, alarmed (gas.json)
npm run gas -- --once                       # every key, one reading, right now
npm run sql -- --pg "postgres:///mydb"      # LISTEN superlog: NOTIFY from any trigger lands here
npm run sql -- --sqlite var/app.db          # an SQLite file, watched from outside the process
npm run prs -- --author you --repo owner/x  # PRs: alarms when one waits on YOU
npm run rpc                                 # RPC node health: block height, DOWN/STALLED, per provider
npm run starlink                            # the dish: latency, obstruction, alerts, outages
npm run otlp                                # OpenTelemetry inlet on :4318 (OTLP/JSON + protobuf)
npm run otlp-export -- --endpoint URL       # OpenTelemetry outlet: bench -> OTLP/HTTP JSON
OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318 your-app   # ...point any OTel exporter at it
npm run fix -- --file 'log/*.messages.*.log'  # FIX sessions: Reject/rejected-fill loud, fills INFO, heartbeats DEBUG
npm run mavlink -- --udp 14550              # a drone's MAVLink: battery/GPS metrics, failsafes and STATUSTEXT loud
npm run mavlink -- --tlog flight.tlog       # ...or replay a telemetry capture
npm run usb                                 # the USB device tree - is the phone connected?
npm run netstate                            # gateway, Wi-Fi, VPN, resolvers, ARP - changes only
npm run netstate -- --ping 1.1.1.1 --ping api.example.com|30   # path quality, traceroute on failure
npm run netstate -- --once                  # what does the network look like right now
npm run topology                            # the local network as a tree, in its own window
npm run topology -- --discover              # ping-sweep the /24 to show every device, not just known ones
npm run topology -- --to 1.1.1.1 --geo      # watch a route; AS/owner per public hop (opt-in outbound lookups)
npm run dns -- example.com --asn            # ...and which AS originates it (hijack watch)
npm run dns -- example.com --once           # every DNS record + cert, then exit
npm run dns -- example.com mail.example.com # ...or watch them for change
npm run ports -- --once                     # what is listening, and which process
npm run ports -- --ssh web1 --procs nginx   # ...on a server, watched
npm run connections                         # the OUTBOUND side: who this box is talking to
npm run connections -- --ssh web1           # ...on a server; a stuck SYN-SENT is a WARN (filtered/down)
npm run vitals -- --once                    # disk, memory, CPU, load
npm run vitals -- --ssh web1                # ...on a server, watched
npm run versions -- --once                  # every version under the bench: OS, compilers, runtimes, packages
npm run versions -- --ssh web1              # ...on a server (no node needed there), diffed over time
npm run versions -- --check-conflicts       # match a downloaded known-bad list, LOCALLY
npm run alert                               # rules from alerts.json
npm run alert -- --test                     # prove delivery without waiting
npm run alert -- --channels                 # the notification roster, and what is missing
npm run alarm                               # production's webhook door, tunnelled + tested
npm run alarm -- --provision endpoints.json # ...plus many public endpoints, from a manifest
                                            # incl. Stripe webhook testing: capture, verify
                                            # signatures, relay to your local handler
npm run build -- --label cxx -- cmake --build build -j
npm run build -- --label asan -- ./build/tests   # sanitizer findings, whole
npm run git                                 # this repo: commits, branches, conflicts
npm run git -- --ssh web1 --repo /srv/app   # ...a deployed checkout
npm run github -- --repo owner/name         # CI runs, PRs, releases
npm run watch -- --dir src                  # files created, modified, deleted
npm run watch -- --dir config --diff        # ...and the changed LINES, hunk by hunk
make 2>&1 | npx superlog tee --topic build.local # superlog-tee: a drop-in tee
npm run ws -- wss://stream.binance.com:9443/ws/btcusdt@trade
npm run serial -- --list                    # boards plugged in
npm run serial -- --port /dev/ttyUSB0       # the serial console, as events
npm run cf -- --worker my-api               # a Cloudflare Worker, live
npm run stripe -- --live --account acme     # payments, redacted by default
npm run socket -- --udp 5514                # syslog from routers, switches, NAS
npm run socket -- --tcp 5515                # ...or plain lines on a raw socket
npm run ros                                 # a robot's nodes, from /rosout
npm run ros -- --files                      # ...including past runs in ~/.ros/log
npm run gpu                                 # this machine's GPU, watched
npm run gpu -- --ssh trainer1               # ...or the box with the card in it
npm run power                               # watts, thermals, top energy hogs (macOS)
npm run power -- --once                     # one power reading, then exit
npm run sys                                 # crashes, panics, shutdown causes, volumes (macOS)
npm run sys -- --once --backfill 72h        # what has this machine suffered lately?
npm run bridge -- --ssh otherbench          # another hub's whole feed, into this one
npm run dl -- -- curl -LO https://host/model.safetensors   # a download, with progress
npm run dl -- --watch ~/models --size 140GB -- hf download org/model
npm run build -- --ssh web1 -- 'cd /srv/app && cargo build --release'

Infrastructure watches

dns, ports and netstate diff a snapshot rather than streaming, so the first poll is a silent baseline and only changes are reported — a watcher that announces everything it sees teaches you to ignore it. The rest publish readings as DEBUG metric events and raise their voice only on edge-triggered crossings.

superlog-topology renders the LAN as a tree — this host and its addresses, the gateway, and every device under it — in its own window in both viewers. --discover sweeps the subnet so devices the machine hasn't talked to still appear; --to <target> adds a watched route with per-hop RTT and AS/owner.

| git | git.<host>.<repo> — a checkout on the timeline: commits (subject + shortstat), branch switches, rewritten history, tags and merge conflicts. superlog-git watches repos (silent baseline, then only changes); superlog git install-hooks adds a post-commit hook that stamps each commit the instant it lands; superlog git recall <commit> replays every log the bench collected between that commit and the next — the build that ran, the tests, the services that flapped, correlated to the change that caused them. | A commit is INFO (its subject is the message); rewritten history is WARN — a rebase/amend/force-push, unremarkable on your own branch and the worst news of the day on a shared one, and the watcher cannot tell which; merge conflicts are WARN and resolution recovers; a checkout that vanished is ERROR, said once. recall reads the LOCAL journal (durable recall across sealed segments and other hosts is the Cloud console). |

dns queries one chosen resolver (1.1.1.1 by default) so a change means the record changed, not that a laptop moved networks and hit a different cache. build is transparent: it prints the output and exits with the build's own status, so it can sit inside a Makefile or a CI step unchanged.

dl wraps a fetch the way build wraps a compiler — output, stdin and exit status untouched — and it exists for the downloads everyone now does: pulling a large model or dataset from Hugging Face, shard by shard, for hours.

npm run dl -- -- curl -LO https://huggingface.co/Qwen/Qwen2.5-7B/resolve/main/model-00001-of-00004.safetensors
npm run dl -- --watch ~/.cache/huggingface --size 140GB -- hf download meta-llama/Llama-3.1-70B
npm run dl -- --watch /data/corpus --size 100GB     # a fetch some other process owns

The first form reads the tool's own bar (tqdm/hf, curl's meter, wget, or any bare NN%). The second is the one to reach for on a big multi-shard pull: hf download runs one tqdm bar per shard and each resets to 0%, so --watch measures the destination directory itself every tick — symlink-aware, so a Hugging Face cache of blobs and snapshot links counts each byte once — and --size turns that into the true overall percentage. The third form needs no command at all: it follows a download some other process owns, and exits when the size is reached.

Whole fleets

Eight servers with containers each is thirty tailers, and nobody runs thirty commands twice. Describe them once (fleet.json.example):

{ "url": "http://127.0.0.1:7333",
  "hosts": [
    { "ssh": "web1", "name": "web1", "os": true, "apps": ["nginx"] },
    { "ssh": "deploy@10.0.1.20", "name": "api", "os": true,
      "identity": "~/.ssh/id_ed25519",
      "docker": ["api", "worker"], "files": ["/srv/app/log/production.log"] }
  ] }

npm run fleet -- fleet.json starts every stream and restarts any that die. name is the topic name, so os.api reads better at 3am than os.ubuntu-4gb-nbg1-1. Everything is pulled over ssh — no agent, no open port, no route from production to the hub.

Many benches: superlog-bridge

A hub rebroadcasts everything it ingests on /ws, so hubs compose:

npm run bridge -- --ssh otherbench

subscribes to another machine's loopback hub over an ssh tunnel and re-ingests its whole feed here, verbatim — same topics, same bytes, so nothing downstream can tell a bridged stream from a local one. Neither hub ever listens on the network. One direction only: two hubs bridged at each other is a feedback loop, so pick one bench to be the bench.

Rebroadcast can be turned off. Composability is the default, not a mandate: a hub rebroadcasts on /ws because that is what makes it composable, and one environment variable takes it back. SUPER_LOG_NO_EGRESS names topics the hub accepts but serves to nothing — never on /ws, never from /recent — so a stream can exist on the bench and still never leave the machine:

SUPER_LOG_NO_EGRESS='secrets.*,vault.*' superlogd   # the scalpel: these topics only
SUPER_LOG_NO_EGRESS='*'                  superlogd   # the fire alarm: rebroadcast nothing

A pattern is an exact topic, a prefix.* glob, or a bare * for the whole hub; the hub says on the console at startup which topics are cut. It is a security control in MIT code you can read — an agent, a bridge, a journal, any /ws subscriber gets nothing for a cut topic, and because a cut topic never enters the rebroadcast ring, a subscriber that reconnects can't replay it either. One honest limit: served to nothing is the whole claim today — a cut topic is not journaled locally, because super-log's own journal is itself a /ws subscriber. Hub-internal journaling is the scheduled follow-up; until it lands, a cut topic is dropped, not persisted.

History

npm run journal                                   # capture everything, rotated
npm run search -- --since 3d --level ERROR --topic node.
npm run search -- --trace 9f1c0a2b7d4e5f60        # one action, days later
npm run replay -- --dir superlog-journal --speed 1

The window filters on hub arrival, not the producer's clock: arrival is monotonic, so the window is exact and the scan can stop early.

Modes and policies

Every SDK enforces DEVELOPMENT xor PRODUCTION — neither or both is a compile error (C++ and Fortran defines, Rust features) or a raised error (JS, Python, Go, Java, Swift). Each mode then ships what its policy allows: development everything, production nothing. Want crash triage from release builds? Say so explicitly — -DSUPERLOG_PROD_POLICY=ERROR, prod_policy: Policy::AtLeast(Level::Error), or productionPolicy: 'ERROR'. Below-policy events cost one compare; a policy of OFF compiles the transport to an inert shell — and says so once on the console, because a client that is silently doing nothing looks exactly like a broken one.

Log lines leaving a production box are a security decision, so nothing here makes it for you.

Security posture

The full model — the MIT tool's threat model and how it differs from super-log Cloud's — is docs/SECURITY_ARCHITECTURE.md. In short:

No auth, no TLS: anyone who can reach the port can read every stream and publish to any topic. The defaults are arranged so exposure is a choice, not an accident:

• The hub binds loopback only, demo or not; SUPER_LOG_LAN=1 or SUPER_LOG_BIND=0.0.0.0 open it up when real devices need it, on a network you trust.

• The ssh tailer and the fleet runner pull, so production logs reach the bench without production ever reaching the hub.

• Production builds forward nothing unless you loosened the policy.

• A topic can be cut from egress at the hub. SUPER_LOG_NO_EGRESS='secrets.*' (or * for the whole hub) makes the hub accept a stream but serve it to nothing — no /ws, no /recent, no replay — so it stays on the machine. Cut topics are dropped, not journaled; see Many benches above.

• Credentials are redacted, not logged: Authorization, Cookie and X-API-Key headers in the proxy, token-shaped query values in URLs, and the provider key inside an RPC endpoint.

• .env is gitignored — an RPC URL with a key in it is spendable.

• CSV exports defuse spreadsheet formula injection; viewers render log content as text, never markup.

SaaS and hosted services

The same idea, for the parts of a system you cannot attach a debugger to at all. Both drive the vendor's own CLI, so there is nothing to install in your service and no webhook to host.

Cloudflare Workers

npm run cf -- --worker my-api            # live, from now on
npm run cf -- --worker my-api --status error

Publishes to cf.<worker>. One invocation becomes several events sharing a trace — the request, every console line the handler wrote, and any exception — so /recent?trace=… returns one invocation end to end. Levels come from the Worker rather than from guesswork: console.error is ERROR, a 500 is an error whether or not the handler said so, and an outcome that is not ok is an error even when nothing was logged — exceededCpu kills the isolate silently, which is exactly the failure you cannot see from inside. CPU and wall time arrive as DEBUG metrics.

It uses wrangler's own login, so no API token is needed. It is live only: wrangler tail cannot reach backwards, and --since refuses with an explanation rather than quietly tailing from now and letting you believe you are looking at an hour ago.

Stripe

npm run stripe                                   # test mode, default account
npm run stripe -- --live                         # real money
npm run stripe -- --live --account acme --account beta

Publishes to stripe.<account>.<mode>, one process and one topic per account, so a busy account cannot delay a quiet one. Levels follow what an event means: a dispute is CRITICAL because it is money already gone plus a deadline, a failed payment is ERROR, a refund or cancelled subscription is WARN. Amounts arrive as a stripe.amount metric.

Redacted by allowlist, and this is the point. A payment_intent.payment_failed carries the customer's email, name, phone, full billing address, card brand, last four and fingerprint. That is a customer record, not log data, and this hub has no authentication. Only named fields ever leave the process — a blocklist would start leaking the day Stripe adds a field. What survives is what you would actually debug with: the decline code, the failure message, the amount, and the customer id. --unsafe-full turns it off and warns you first.

Setup is the Stripe CLI's own:

stripe login                        # the default account
stripe login --project-name acme    # a second account, then --account acme

stripe login grants test-mode keys; --live needs an account authorised for it. CLI keys also expire — if a stream goes quiet after a few months, re-run stripe login before suspecting the tailer.

Why the hub has no auth

(superlog login is a browser door to the optional Cloud — it adds no authentication to the local hub and makes no network call of its own. This is about the hub itself.)

The bar this aims at is deliberately modest and deliberately explicit: be no less safe than the logs a developer already has, and never more dangerous than them.

Normal logs are files under /var/log and ~/Library/Logs, adb logcat, the Metro and Xcode consoles, journalctl. Every one of them is local-only, enforced by the operating system, and none can be written to from another machine. Bound to loopback, this is the same thing: the OS is the authentication, and it is the same OS doing the same job it already does for your log files. Adding a password on top of that protects nothing that was not already protected.

So the honest answer to "shouldn't there be auth?" is that for the case this tool is actually used in, auth would be theatre. What matters is not adding a login; it is not quietly becoming reachable.

Two things follow, and they are the whole policy:

The default is the safe one. This was not always true. The hub used to bind 0.0.0.0 while this file claimed exposure was "a choice, not an accident" — true of the demo script, false of the binary these instructions tell you to run. A security claim the code did not honour is worse than either alone. It binds loopback now, and says so at startup, and says something louder when it is not.

A token would not fix the case people imagine it fixes. Without TLS a shared secret crosses the network in plaintext on every request, so anyone who can sniff that network has it after one request and keeps it. It would stop accidental access, not an attacker — while adding a real new leak, because browsers cannot set headers on a WebSocket and the viewer's token would have to travel in the URL, into browser history and Referer and every pasted link. That is a poor trade for something the OS already does properly one layer down.

Devices, without opening anything

The one genuine gap is a phone pushing logs, because a handset cannot reach loopback on your Mac. Use USB rather than the network:

adb reverse tcp:7333 tcp:7333          # Android - the phone's localhost is yours
iproxy 7333 7333                       # iOS, via libimobiledevice

That is parity with normal logs everywhere, with no new code and no open port. It is also strictly better than a LAN bind with a token, and simpler.

Containers, and other machines

Docker on macOS needs nothing. A container reaching host.docker.internal arrives on the host's loopback, so the repo's own Ubuntu producer keeps working against a loopback-bound hub — verified, not assumed. The counter-intuitive part is that --network host does not work on Docker Desktop: "host" there means the Linux VM, so the container's 127.0.0.1 is the VM's and not your Mac's. The permissive- sounding flag is the one that fails.

extra_hosts: ["host.docker.internal:host-gateway"]   # what the compose file does

Docker on Linux is the other way round: --network host shares the host's network namespace, so 127.0.0.1 in the container really is the host's loopback and a loopback-bound hub is reachable directly.

Either way, no GUI ever runs in the container — so there is no X11 socket to mount and no Wayland or waypipe forwarding to configure. Only HTTP crosses the container boundary; the viewer runs natively where your eyes are. A Linux workload in Docker on a Mac is visualised by the native macOS viewer, which is the hub/viewer split doing exactly the job it was designed for.

A Raspberry Pi, or any other machine, is the phone problem again — it cannot reach your loopback. Two answers, both already here and neither of which opens a port:

# 1. Pull. Nothing runs on the Pi, nothing is installed, no port is opened.
npm run tail:ssh -- pi4                     # its OS logs
npm run gpu -- --ssh pi4                    # its GPU, temperature and throttling
npm run build -- --ssh pi4 -- make          # a build on it

# 2. Push, through the ssh connection you already have, if an SDK runs there.
ssh -R 7333:127.0.0.1:7333 pi4              # then SUPER_LOG_URL=http://127.0.0.1:7333

The pull model is the better default and the reason the fleet support exists: logs travel to the bench over ssh, so the machine being watched never needs to reach the hub and the hub never needs to be reachable. A reverse tunnel covers the case where code on that machine wants to use an SDK directly — it carries the traffic over the ssh session you already trust, and is verified working against a real server.

If you must expose it

If devices really must reach it over the network, put the allowlist where allowlists belong — the firewall, not the application. An IP filter inside the process is reimplementing pf or nftables badly, and it is defeated by exactly the same attacker.

# macOS, /etc/pf.conf - only this handset may reach the bench
block in proto tcp to any port 7333
pass in proto tcp from 192.168.1.20 to any port 7333

# Linux, and you are probably already running it
ufw allow from 192.168.1.20 to any port 7333
ufw deny 7333

Then treat the bench as what it is: a development tool holding whatever your machines are saying. If that includes production access logs, ssh authentication failures or anything with a customer in it, the write side matters as much as the read side — nobody can forge lines into /var/log from across a network, and an open hub is the one place that stops being true.

See the Auth/TLS section of docs/ARCHITECTURE.md.

Requirements

• Hub + viewers: macOS or Linux (POSIX phase 1), a C++17 compiler, CMake ≥ 3.16. Windows machines join as producers (SDKs, ssh tailer). Plain C SDK: any C99 compiler with POSIX sockets.

• Displays, on Linux: the native viewer draws through GLFW, built here for X11 — which a Wayland desktop also runs via XWayland, so it should work there unchanged. Native Wayland output is a GLFW build switch (-DGLFW_BUILD_WAYLAND=ON plus the wayland/xkbcommon dev packages), not a code change. No Linux desktop has been on this bench, so treat the first run of either as bring-up; the web viewer needs only a browser and does not care what your compositor is.

• JS: Node ≥ 18 (≥ 22 for journal, chain watcher and MCP). Ruby: ≥ 3.0, stdlib only (Rails via the ::Logger adapter). OCaml: ≥ 4.14 (ocamlc -I +unix unix.cma). Haskell: GHC ≥ 9, boot libraries plus curl. Scala: ≥ 3, riding the Java SDK. Lean: 4.x via elan/lake, core IO plus curl. Perl: ≥ 5.14, core modules only (HTTP::Tiny, JSON::PP). Lua: any 5.1+ (PUC or LuaJIT) plus curl. COBOL: GnuCOBOL 3 and a C compiler, riding the C SDK via a 20-line shim. C#: .NET ≥ 8 (RollForward runs it on newer), BCL only. Zig: a recent Zig (verified on 0.16) and a C compiler, riding the C SDK via @cImport. SQL: the psql or sqlite3 CLI. Rust: any recent stable. Python: ≥ 3.8, standard library only. Go: ≥ 1.21 (log/slog). Java: ≥ 17, plain javac, no build tool. Swift: ≥ 5.9, SwiftPM. Fortran: gfortran or any compiler with -cpp. Shell: sh and curl, nothing else.

• On macOS, Go 1.21's internal linker omits LC_UUID, which current dyld rejects; build with -ldflags=-linkmode=external or use Go ≥ 1.22.

• ts-moveables provides the transport/fan-out/logging fabric. It is not a submodule: CMake uses a sibling ../TSMoveables checkout when one exists, and otherwise fetches a pinned SHA. Configure prints which of the two it chose, because a build quietly using someone's working copy is how "works on my machine" is made.

• spdlog + fmt, imgui + glfw, and nlohmann/json are pinned submodules in third_party/ (git submodule update --init) — versions chosen to work together, so system-installed ones are never trusted.

• Docker (optional) for the Ubuntu build-and-smoke image.

Layout

Status, honestly

Most of this is verified live on a real bench, and where something is merely written it is labelled as such. Verified: the hub, both C++ paths, Rust, the JS client in a real Expo app on simulators and hardware plus a real browser and Node, both viewers, the macOS and journald tailers, the adb tailer against a physical handset, ssh streaming from cloud hosts, a fleet of four servers, the chain watcher against Ethereum mainnet, search and replay over a 1 GB journal, correlation across tiers, the error hooks in all four SDKs, and the Docker image (which smoke-tests itself during docker build).

Also verified since: the Go, Java, Swift, Fortran and shell SDKs against a live hub; sanitizer and valgrind capture against real ASan/TSan/UBSan and valgrind output; the git and GitHub watchers (the latter catching real commits as they were pushed); the ROS tailer against genuine ROS 2 Jazzy /rosout; the socket inlet against real syslog datagrams; the serial tailer against a pty; and a 20-minute Binance soak that found the hub's replay ring holding 66 MB for one topic — leaks(1) confirmed no leak, the ring was bounded by chunk count rather than bytes, and it now peaks at 30 MB under the same load.

And since then: the power tailer in full root mode on two real Macs (a Mac Pro and a Ventura iMac), its wattage cross-checked against a hand-run powermetrics; the hub bridge relaying a second machine's loopback hub onto this bench byte for byte over ssh; and superlog-dl against both a live curl transfer and a real 100 GB Hugging Face dataset fetch, watched overnight from the machine next to it.

CI is green on every job, first run, which is worth stating precisely because it verifies things this bench cannot. It builds from a clean checkout on Linux under both gcc and clang and on macOS arm64 (the bench is x86_64), runs ThreadSanitizer on Linux where macOS's TSan is broken at the runtime level, builds the hub with no submodules at all, runs the POSIX shell producer inside Alpine with busybox ash, busybox awk and no GNU date, and runs verify-sdks.sh — so every SDK is proved to actually deliver events on a machine that is not the one they were written on.

Still written but not verified: the Windows event-log path (no Windows machine here), the iOS-hardware tailer, Kotlin (no kotlinc here), Swift on iOS, the serial tailer against real hardware at a real baud rate, the CUDA demo (no NVIDIA GPU and no nvcc here — treat its first build as a bring-up), gRPC against TLS and a real client library, and the OpenGL, D3D and WebGPU snippets. Not built yet: viewer "load session" and metric sparklines.

CI lives in .github/workflows/ci.yml; scripts/smoke.sh is the one smoke test that CI, the Docker image and your terminal all run identically.

Future directions

Deliberately not built yet, and the reasoning matters as much as the list:

• superlog-sys for Linux/Ubuntu. The macOS one reads DiagnosticReports, diskutil activity and kern.sleeptime; the Linux counterpart is coredumpctl and journalctl -k for crashes and OOM kills, udev/udisks events for volumes, and systemd-logind for suspend/resume. Same topic (sys.<host>), same event shapes, so the viewers and guide need not care which OS suffered. Waiting on a Linux desktop on the bench to verify against, per the house rule.

• A Grafana / Loki / OTLP forwarder. The obvious ask is "integrate Grafana", and the answer is a forwarder, not integration. Teaching the hub to be a Prometheus target or a Grafana datasource would make it depend on an ecosystem it does not need and would blur the line drawn above — the hub's job is to be the thing you can point anything at in ten seconds. A forwarder respects that line: one more subscriber on the firehose that re-posts into Loki or an OTLP collector, in the same shape as every tailer here, so the bench stays a bench and the long-term store stays separate. It is a small script, and it is the right bridge for anyone who wants yesterday's logs in Grafana and today's on the bench.

• FIX and the market-data binary protocols. A superlog-fix for the session and application layer (QuickFIX and FIX8 logs, or a tap on the session): heartbeats and gap-fills as DEBUG, a resend request or a sequence reset as WARN, a session logout mid-day as ERROR — the same edge discipline every watcher here keeps, applied to the one protocol whose silence costs the most. Below it, the exchange binary feeds — Nasdaq ITCH (market data) and OUCH (order entry), and their many venue cousins — are fixed-layout binary, so a bounded decoder in the log4j mould (lengths checked, content never evaluated) turns a captured session or a live tap into events: order-book adds/cancels as DEBUG readings, rejects and breaks as WARN/ERROR. Gated on a real session to verify against, per the house rule — a market protocol written but unproven is worse than none.

• Viewer "load session" and metric sparklines — the journal can already be searched and replayed, but neither viewer can open a saved session directly.

• A byte budget on the hub's replay ring, which belongs in ts-moveables rather than here; until it lands, SUPER_LOG_REPLAY_CHUNKS bounds the ring by count instead. See the comment in hub/src/main.cpp.

• zrok as a verified tunnel provider. The gateway is proven with Cloudflare (quick tunnels, the stable named tunnel, DNS provisioning, the endpoint factory) and ngrok (--tunnel ngrok — live round-trip verified on this bench: an alarm fired at the public ngrok URL from the internet landed on the hub; free-tier URLs rotate per restart, a paid domain is stable, and API callers send the ngrok-skip-browser-warning header). --tunnel zrok parses zrok's output and should work, but "should work" is not the house standard: it needs a zrok account on the bench and the same live treatment before the README claims it. The endpoint factory's quick tunnels are Cloudflare-only until then.

• Windows as a first-class host for the hub and viewers. Windows machines already work as producers, and the event-log tailer is written but unverified.

Contributing

This was built for my own bench and then it turned out to be useful, so here it is. Pull requests are welcome — new streams especially: if something on your desk emits logs and this cannot read it yet, that is the gap worth filling.

Two things make a change easy to accept. Run it against a real hub before you open the PR, and say in the description what you actually observed — this repo's habit is to label what is verified and what is merely written, and that habit is the reason it can be trusted. And keep the producer contract: bounded queue, drop oldest, count what was dropped, and never block the program you are observing.

Issues describing a stream you wish it read are useful too, even without a patch.

If you run a service that produces logs

An open invitation. If you build a SaaS product that emits logs, events or metrics that developers debug against, a reader for it belongs here and you are the person best placed to write it — you know the event shapes, the severities that actually matter, and which fields are the ones people need at three in the morning.

There are readers here for Cloudflare Workers, Stripe, GitHub Actions, Postgres, nginx, Redis, Kafka, Docker, ROS and a couple of dozen others. Every one of them is a single file of a few hundred lines that spawns your CLI or calls your API and turns the output into events. There is no plugin API to learn and no interface to implement — read one of them and you have seen the whole pattern.

Three things a good one does, and they are the whole review:

Levels mean what your product means. Not what the event is called. A Stripe dispute is CRITICAL because it is money already gone plus a deadline; a Cloudflare Worker that exceeded CPU is an error even though it logged nothing at all. You know which of your events are the three-in-the-morning ones. Encode that, and a developer's level filter starts doing real work.

Redact by allowlist, not blocklist. This hub has no authentication and is read by anyone who can reach it. A payment_intent.payment_failed carries an email, a name, a phone number, a billing address and a card fingerprint — that is a customer record, not log data. The Stripe reader names the fields that may leave and drops everything else, because a blocklist starts leaking silently the day you add a field. If yours carries personal data, do the same.

Keep the producer contract. Bounded queue, drop oldest, count the drops, and never block or kill the program being observed. A logger that can take down the thing it is watching has no business being installed.

If that sounds like your product, open an issue or a PR — or just tell us what the API is and we will have a go. Streams nobody has thought of yet are the most interesting kind.

Releases

Tagged releases carry notes saying what is verified and what is not — CHANGELOG.md. There is no package registry: the clone is the distribution, so pin a tag if you want a stable base.

git clone --recurse-submodules --shallow-submodules --branch v0.1.0 \
  https://github.com/saxonnicholls/super-log ~/dev/super-log

Licence

MIT. Every source file carries the copyright line and an SPDX-License-Identifier: MIT, so the licence travels with the file when someone copies one out — which, for a repo made of self-contained single-file tools, is how most of it will actually be reused.

Copyright 2026 Saxon Herschel Nicholls.