# Kingfisher Streaming Inference

Incremental model scoring over an append-only event log, with a hash-chained
audit trail so every decision can be reviewed and independently re-verified.
Built for Quarryman Labs (Project Kingfisher).

## Why it is shaped this way

The governing constraint is that **every decision must carry a reviewable audit
trail**. That drives the architecture:

- Events land in an **append-only log** (`kingfisherstreaming/log.py`). There is
  no update or delete path; duplicate ids are rejected.
- Each event is scored **incrementally** (`kingfisherstreaming/scoring.py`)
  using Welford's online mean/variance. Scoring one event is O(1) and never
  rescans the log, which is what makes the stream tractable over time.
- Every decision is written to a **hash-chained audit log**
  (`kingfisherstreaming/audit.py`). Each record's SHA-256 covers its contents
  and the previous record's hash, so any later edit, reorder, or deletion is
  detectable with `verify()`.

Model behaviour sits behind a provider interface
(`kingfisherstreaming/providers/base.py`) with a deterministic, offline stub
(`stub.py`) used by the tests and an HTTP-backed implementation (`real.py`).
The full test suite runs offline with no API key.

## Layout

```
kingfisherstreaming/
  types.py          domain types (Event, Score, Decision, ...)
  log.py            append-only event log (in-memory + optional JSONL)
  scoring.py        Welford running stats + incremental scorer (core algorithm)
  audit.py          hash-chained, tamper-evident audit log
  engine.py         wires log + scorer + audit into one ingest path
  providers/
    base.py         ScoringProvider interface
    stub.py         deterministic offline provider
    real.py         HTTP-backed provider (optional 'real' extra)
tests/
```

## Install and test

```
make install      # creates .venv and installs with the dev extra
make test         # runs pytest
```

Override the interpreter if you manage your own venv:

```
make test PY=python
```

## Usage

```python
from kingfisherstreaming import StreamingInferenceEngine, Event
from kingfisherstreaming.providers.stub import StubProvider

engine = StreamingInferenceEngine(StubProvider(), threshold=3.0, min_samples=30)

score, record = engine.ingest(
    Event(event_id="lease-001", stream="office-cbd", timestamp=0.0, value=100.0)
)
print(score.decision, score.normalized_score)
assert engine.verify_audit()   # re-checks the entire chain
```

During cold start (fewer than `min_samples` observations on a stream) the
decision is `REVIEW`, not a guessed pass/flag: with too little history a z-score
is not trustworthy, and saying so is part of the audit trail.

To score against a real endpoint:

```
pip install '.[real]'
export KINGFISHER_SCORING_ENDPOINT=https://scoring.internal/score
export KINGFISHER_API_KEY=...
```

## Running the MCP server

The server speaks JSON-RPC 2.0 over stdio (one JSON object per line),
implemented with the standard library only. Logs are JSON on stderr; stdout
carries only protocol frames.

```
python -m kingfisherstreaming serve
```

Tools exposed: `ingest_event`, `verify_audit`, `stream_stats`, `export_audit`.
`verify_audit` and `export_audit` exist so a reviewer can pull and re-check the
hash chain through the same interface that produced the decisions.

Example exchange (request on stdin, response on stdout):

```
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"ingest_event","arguments":{"event_id":"lease-001","stream":"office-cbd","timestamp":0.0,"value":100.0}}}
```

## Batch scoring a file

```
python -m kingfisherstreaming score events.jsonl
```

One event object per line. Malformed lines are logged and skipped, not fatal;
the exit code is non-zero only if the resulting audit chain fails verification.

## Configuration

All configuration is read from the environment and validated at startup:

| Variable | Default | Meaning |
|---|---|---|
| `KINGFISHER_PROVIDER` | `stub` | `stub` (offline) or `real` (HTTP) |
| `KINGFISHER_THRESHOLD` | `3.0` | z-score magnitude that flags an event |
| `KINGFISHER_MIN_SAMPLES` | `30` | per-stream samples before scoring leaves REVIEW |
| `KINGFISHER_LOG_PATH` | unset | JSONL path to persist events |
| `KINGFISHER_AUDIT_PATH` | unset | JSONL path to persist the audit chain |
| `KINGFISHER_MAX_EVENT_BYTES` | `65536` | reject events larger than this |
| `KINGFISHER_MAX_FEATURES` | `256` | reject events with more features than this |
| `KINGFISHER_LOG_LEVEL` | `INFO` | logging level |

Invalid configuration exits non-zero with a message on stderr rather than a
traceback.