Kingfisher Streaming Inference

Incremental model scoring over an append-only event log, with a hash-chained audit trail so every decision can be reviewed and independently re-verified. Built for Quarryman Labs (Project Kingfisher).

Why it is shaped this way

The governing constraint is that every decision must carry a reviewable audit trail. That drives the architecture:

• Events land in an append-only log (kingfisherstreaming/log.py). There is no update or delete path; duplicate ids are rejected.

• Each event is scored incrementally (kingfisherstreaming/scoring.py) using Welford's online mean/variance. Scoring one event is O(1) and never rescans the log, which is what makes the stream tractable over time.

• Every decision is written to a hash-chained audit log (kingfisherstreaming/audit.py). Each record's SHA-256 covers its contents and the previous record's hash, so any later edit, reorder, or deletion is detectable with verify().

Model behaviour sits behind a provider interface (kingfisherstreaming/providers/base.py) with a deterministic, offline stub (stub.py) used by the tests and an HTTP-backed implementation (real.py). The full test suite runs offline with no API key.

Related MCP server: DCL Evaluator

Layout

kingfisherstreaming/
  types.py          domain types (Event, Score, Decision, ...)
  log.py            append-only event log (in-memory + optional JSONL)
  scoring.py        Welford running stats + incremental scorer (core algorithm)
  audit.py          hash-chained, tamper-evident audit log
  engine.py         wires log + scorer + audit into one ingest path
  providers/
    base.py         ScoringProvider interface
    stub.py         deterministic offline provider
    real.py         HTTP-backed provider (optional 'real' extra)
tests/

Install and test

make install      # creates .venv and installs with the dev extra
make test         # runs pytest

Override the interpreter if you manage your own venv:

make test PY=python

Usage

from kingfisherstreaming import StreamingInferenceEngine, Event
from kingfisherstreaming.providers.stub import StubProvider

engine = StreamingInferenceEngine(StubProvider(), threshold=3.0, min_samples=30)

score, record = engine.ingest(
    Event(event_id="lease-001", stream="office-cbd", timestamp=0.0, value=100.0)
)
print(score.decision, score.normalized_score)
assert engine.verify_audit()   # re-checks the entire chain

During cold start (fewer than min_samples observations on a stream) the decision is REVIEW, not a guessed pass/flag: with too little history a z-score is not trustworthy, and saying so is part of the audit trail.

To score against a real endpoint:

pip install '.[real]'
export KINGFISHER_SCORING_ENDPOINT=https://scoring.internal/score
export KINGFISHER_API_KEY=...

Running the MCP server

The server speaks JSON-RPC 2.0 over stdio (one JSON object per line), implemented with the standard library only. Logs are JSON on stderr; stdout carries only protocol frames.

python -m kingfisherstreaming serve

Tools exposed: ingest_event, verify_audit, stream_stats, export_audit. verify_audit and export_audit exist so a reviewer can pull and re-check the hash chain through the same interface that produced the decisions.

Example exchange (request on stdin, response on stdout):

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"ingest_event","arguments":{"event_id":"lease-001","stream":"office-cbd","timestamp":0.0,"value":100.0}}}

Batch scoring a file

python -m kingfisherstreaming score events.jsonl

One event object per line. Malformed lines are logged and skipped, not fatal; the exit code is non-zero only if the resulting audit chain fails verification.

Configuration

All configuration is read from the environment and validated at startup:

Invalid configuration exits non-zero with a message on stderr rather than a traceback.