streamsieve
This server has a remotely hosted endpoint. Open its connector listing for connection details, authentication requirements, and setup instructions:
streamsieve
SIMD C++20/AVX2 anytime-valid streaming inference, zero-echo CRDT belief fusion, and Byzantine-safe multi-agent calibration — zero runtime dependencies, <5 ms native CLI, 7-tool MCP server, and 99.99% LLM token compression with bit-exact raw drill-down.
AI agent fleets break in four predictable ways on live data streams:
Context-window blowup — dumping 1,024-channel telemetry into prompts burns millions of tokens and gets trapped by high-variance "loud channels."
Continuous-peeking p-hacking — polling rolling z-scores or Welch $t$-tests every few ticks inflates false alarms from 5% to 20–83% under heavy tails or DC offsets.
Echo-chamber double-counting — gossiping findings across $W$ agents multiplies log-odds by 5–10× or causes 93–97% last-write-wins overwrite churn.
Delayed-label & Byzantine trust collapse — uncalibrated "bloviators", poisoned agents, and sudden regime flips wreck naive voting when ground truth arrives late and out of order.
streamsieve solves all four in a single zero-dependency C++20/AVX2 engine exposed via a <5 ms native CLI, a zero-dependency Python SDK (ctypes + zero-copy memoryview/NumPy support), a 7-tool JSON-RPC 2.0 stdio MCP server, and drop-in Claude Code / Cursor plugin skills.
Verified Benchmarks (StreamSwarmBench)
Measured live on an Intel Xeon 2.30 GHz Linux VM (2 vCPUs, AVX2+FMA, Python 3.12.16) against river 0.27.0, scipy 1.18.1, and numpy 2.5.5. Full structured output: benchmarks/results.json | Detailed report: docs/benchmarks.md.
1. Throughput, Incumbent Speedups & Startup Latency
Workload |
| Incumbent Baseline | Verified Speedup |
Native AVX2 CLI ( | 160.75 M ch-ticks/s ( | — | Native C++20 SIMD |
Native AVX2 CLI ( | 93.15 M ch-ticks/s ( | — | Native C++20 SIMD |
Python batch vs | 1.95 ms ( |
| 17.1× faster |
Python batch vs | 1.95 ms ( |
| 54.4× faster |
Python batch + online e-BH vs NumPy/SciPy z+BH ( | 13.70 ms ( |
| 5.65× faster |
Cold startup ( | 4.03 ms (CLI) / 24.90 ms ( |
| 68× faster ( |
2. Continuous-Peeking Calibration & Rare Zero-Mean Burst Detection
Benchmark (nominal $\alpha = 0.05$) |
| Naive | Peeking Welch+BH |
|
|
Null Peeking FAR (Gaussian / Student-$t(3)$ / LogNormal / $10^{10}$ DC+1000×) | 0.004 – 0.010 | 0.196 – 0.834 | 0.476 – 0.542 | 0.000 – 0.004 | 0.108 – 1.000 |
256-Ch Firehose (4 shifts + 4 bursts + 32 loud 5×): Emp. FDR / F1 | 0.000 / 0.883 ( | 0.966 / 0.060 | 0.321 / 0.800 ( | 0.054 / 0.862 ( | — |
Zero-Mean | 0.65–0.70 / 0.900 | 0.30 / 0.102 | 0.00 / 0.578 | 0.00 / 0.667 | — |
Sequential Stopping ($H_0$ FAR / $H_1$ Power / Samples Saved) | 0.015 / 1.00 / 70.0% | — | 0.440 / 1.00 / 74.7% | — | — |
3. Multi-Agent Coordination, Token Compression & Real NOAA Weather
Mechanism |
| Verified Result vs Baseline |
Token Compression ( |
| 3,844,353 → 325 tokens (99.99% / 11,829× reduction), 1.00/1.00 Prec/Rec, 0 loud FPs, |
Anti-Echo Gossip ($W = 8$ agents, 40 msgs/trial, 200 trials) |
| 0.0% overwrite churn, 1.00× inflation (Brier skill |
Wire Budget (256 unique findings across 8 hypotheses) |
| 10,428 B binary ( |
Firehose Attention (2,048 ch, 256 sites, $W = 4$, 10 seeds) |
| +0.702 Brier skill, 78.3% Recall@Top, 0.11% inter-agent overlap vs |
Byzantine + Trust Flip (4-epoch delay, flip @ 60, 20 seeds) |
| 100.0% Byzantine quarantine, post-flip Brier 0.1840 vs |
Real NOAA Seattle |
| 30.39% asymmetric cost reduction ( |
Related MCP server: collab-charter
Quickstart (3 Commands)
1. Build Native AVX2 Core & CLI
git clone https://github.com/pgg898200-ship-it/streamsieve.git && cd streamsieve
make all test-cpp2. Sift Any CSV/TSV Stream from the Terminal (<5 ms cold start)
# Compact agent-ready brief with lossless drill-down handles (numeric columns 3-6)
cut -d, -f3-6 examples/noaa_weather.csv | ./build/streamsieve sift -
# Machine-readable RFC-8259 JSON brief from stdin
cat metrics.csv | ./build/streamsieve sift --alpha 0.05 --top 10 --json -
# Native AVX2 throughput self-benchmark
./build/streamsieve bench --channels 1024 --ticks 40003. Zero-Dependency Python SDK
pip install --no-build-isolation -e . # or: PYTHONPATH=python python3 ...import streamsieve as ss
# 1. Anytime-valid firehose sift + bit-exact lossless drill-down
est, brief = ss.sift_matrix(matrix, names=col_names, alpha=0.05, ring_capacity=256)
print(brief.to_compact_text()) # ~320 tokens regardless of channel count
raw_ticks = est.fetch_raw(brief["top"][0]["handle"]) # exact (tick, float) tuples
# 2. Zero-echo CRDT multi-agent belief & Vovk-Wang e-value fusion
lat_a, lat_b = ss.EvidenceLattice(), ss.EvidenceLattice()
lat_a.observe("outage", "ch:3@t=180..600", llr=2.4, log_e=3.2, agent_id=0, timestamp_ns=100)
lat_b.observe("outage", "ch:3@t=180..600", llr=2.4, log_e=3.2, agent_id=1, timestamp_ns=200) # echo!
lat_a.merge(lat_b) # commutative, associative, idempotent
print(lat_a.query("outage")) # unique_evidence=1, echoes_suppressed=1
# 3. Zero-overlap hierarchical VOI multi-agent read allocation
sifter = ss.VOISifter(n_channels=2048, n_sites=256, ticks_per_epoch=8)
sifter.set_noise_scales(channel_stds) # loud-channel immune
plan = sifter.allocate(budget_per_agent=4, n_agents=4)
# 4. Delayed-label Platt calibration, Ville quarantine & asymmetric-cost AgACI action
tc = ss.TrustCalibrator(n_agents=4, conformal_alpha=0.10)
dec = tc.predict_scalar("day_42", [18.1, 17.8, 18.4, 999.0], underage_cost=4.0, overage_cost=1.0)
tc.resolve("day_42", outcome=18.2) # strictly causal delayed resolution
# 5. Drop-in 1-line replacement for river.drift.ADWIN / PageHinkley (zero river dep!)
from streamsieve import EStreamDrift as ADWIN
det = ADWIN(alpha=0.05) # catches shifts + zero-mean +-3.5 sigma bursts
for x in stream:
if det.update(x).drift_detected:
print(f"Drift ({det.mode}) e={det.e_value:.1f}")4. Drop-In MCP Server & Claude Code / Cursor Skill
Run the zero-dependency JSON-RPC 2.0 stdio MCP server exposing 7 stateful tools (sift_stream, fetch_raw_ticks, lattice_observe, lattice_merge, lattice_query, voi_allocate, trust_step):
PYTHONPATH=python python3 -m streamsieve.mcp_serverAdd to .mcp.json / Claude Desktop / Cursor MCP config:
{
"mcpServers": {
"streamsieve": {
"command": "python3",
"args": ["-m", "streamsieve.mcp_server"],
"env": { "PYTHONPATH": "/path/to/streamsieve/python" }
}
}
}Plugin manifests and agent skill instructions are pre-packaged at SKILL.md, .claude-plugin/plugin.json, and .cursor-plugin/plugin.json.
Architecture: Four Co-Designed Strictly Causal Engines
flowchart LR
RAW["High-Frequency Stream\n(N ch × T ticks)"] --> E1["1. EStreamEngine\nAVX2 4-wide SoA\nDual-Res E-Process + Online e-BH\nLossless Raw Ring"]
E1 -->|"Compact Brief (~320 tok)\n+ Raw Drill-Down Handles"| E3["3. VOISifter\nSite × Channel × Tick\nSparse-Burst LLR\nZero-Overlap Multi-Agent VOI"]
E1 -->|"Provenance-Tagged\nChannel E-Values"| E2["2. EvidenceLattice\nJoin-Semilattice CRDT\nBayes Log-Odds + Vovk-Wang E\nSSLT v1 Binary / Base85 Wire"]
E3 -->|"Non-Overlapping\nAgent Inspections"| E2
E2 -->|"Echo-Free Beliefs"| E4["4. TrustCalibrator\nCausal Delayed-Label Buffer\nNon-Negative Platt + Fixed-Share BOA\nVille Quarantine + Delayed AgACI"]
E4 -->|"Agent Trust Weights\n& Quarantine Masks"| E2Every engine obeys bit-exact zero-lookahead causality verified by assert_causal() (poisoning all future ticks $s > t$ with +1e12, -1e12, NaN, and +Inf produces 0.0 bit diff on outputs at $\le t$).
Verification, Adversarial Testing & Honest Scope
C++20 Core + Sanitizers:
make all test-cpp test-sanbuilds-O3 -march=native -ffp-contract=off -Wall -Wextra -Wpedantic(AVX2) and-fsanitize=address,undefined(scalar) with 0 compiler warnings, 0 sanitizer errors, and bit-exact AVX2-vs-scalar parity.56/56 Zero-Dependency Python Tests + Adversarial Breaker Suite:
PYTHONPATH=python python3 -m unittest discover -s tests -p "test_*.py" -vruns 29 unit/adapter tests + 27 independent adversarial torture tests (tests/test_breaker_adversarial.py) covering non-SIMD channel widths (1, 3, 5, 7, 13, 17),±1e300spikes on constant-zero channels,1e17tick-0 warmup spikes, 6 heavy-tailed/skewed/bimodal/loud continuous-peeking distributions, 600 randomized CRDT axiom trials, 500× echo storms, 40% Byzantine poisoning + trust flips, strict zero-lookahead causality across all 4 engines, drop-inEStreamDriftparity, and MCP/CLI JSON security fuzzing.Reproduce all benchmarks:
PYTHONPATH=python .bench-venv/bin/python benchmarks/run_all.py
Honest Limitations (Where NOT to Use streamsieve)
Non-stationary raw series (strong trends / diurnal seasonality):
EStreamguarantees anytime-valid null calibration under stationary/exchangeable nulls (including heavy tails, skew, and huge DC offsets). Feed one-step prediction residuals or deseasonalized differences for trending/seasonal metrics.Short baselines on ultra-wide panels for rare bursts: Online e-BH at $M = 1{,}000$ with 1 active channel requires $E \ge 20{,}000$; rare bursts ($p_b \le 0.05$) need enough pre-onset baseline ticks ($n_0$) for the universal-inference Beta-Binomial denominator to separate the rate, or hierarchical triage via
VOISifter.Single-tick Python loops: Calling
EStream.update_row()one row at a time from Python incursctypesper-call overhead (5.67 M ch-ticks/s, still1.47×faster thanriver.ADWIN). Batch rows withupdate_batch()(65–87 M/s) or use the native CLI/C ABI (93–161 M/s).Sybil provenance forgery:
EvidenceLatticeprevents accidental or gossip-induced double counting when identical observations carry identical content-hashedprovenance_ids; deliberate Sybil adversaries forging fresh IDs are mitigated byTrustCalibratorVille quarantine.
Documentation & Comparisons
Technical Specification & Math:
docs/architecture.mdFull Benchmark Report (
StreamSwarmBench):docs/benchmarks.md|benchmarks/results.jsonHonest Head-to-Head Comparisons:
Agent / LLM Machine Index:
llms.txt|llms-full.txt|SKILL.md
License
Apache License 2.0 — see LICENSE.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
AI Reasoning Cache & Consensus Layer with 11 MCP tools via Streamable HTTP.
Pseudonymous message network for agents. Claims, marketplace, 20 tools, verifiable seal.
49 deterministic tools for text integrity, agent control, and contextual quality evidence.
ERC-8004 agent identity, on-chain subscriptions, reputation, A2A tasks and chat over 53 tools.
Related MCP Servers
- FlicenseAqualityBmaintenanceExactly-once execution for irreversible agent actions: an agent claims the right to run an effect, and a retry after a lost response returns the sealed result instead of charging again. 12 tools over stdio, including a gateway mode where the agent holds a single-use ticket and never the provider key.964 PyPI-
- AlicenseNot gradedqualityBmaintenanceExposes 14 tools that let an agent query charter clauses, route and execute an 8-stage collaboration pipeline, arbitrate conflicts with cited clause IDs, enforce safety gates, run receipt self-checks, deposit and recall memory, solve symbolic anchoring constraints, and mine a 330-entry knowledge base plus semantic-field analysis. It also diagnoses image-generation prompts against a formal feature-cluster grammar, all over a zero-dependency stdio JSON-RPC interface.1MIT
- AlicenseBqualityCmaintenanceExposes 12 memory and multi-agent tools over stdio JSON-RPC so agents can write episodes and extracted facts, then recall them with hybrid retrieval, bi-temporal state tracking, and evidence-chain decomposition under a token budget. It also supports agent registration, team-shared memory with access grants, and an event bus for multi-agent collaboration.12Apache 2.0
- AlicenseNot gradedqualityBmaintenanceExposes ten tools that let agents and humans coordinate through a deterministic control plane: starting tasks, acquiring and releasing typed file leases, writing and searching permissioned memory, requesting approvals, polling structured events, and submitting work through a merge gate. Every action is checked against signed mandates and policy before it succeeds, so multi-agent teams can share a repo and an audit trail without a supervisor model watching every session.MIT