nightshift-grid
This server lets you monitor and operate a local Nightshift Grid for distributed extraction and local graph querying.
grid_status: inspect worker capacity, tiers, states, models, throughput, queue depth, and recent job progress.dispatch_mapreduce_text: submit text chunks for distributed entity/relationship extraction; wait for results or get a job id.ingest_directory_to_graph: build a knowledge graph from local files under allowed roots, producingoutput_graph.jsonland resolvedgraph.json.query_local_graph: search a local knowledge graph by entity or keyword and retrieve related triples with evidence, without a model call.
Uses local Ollama instances on worker nodes as the inference backend, running quantized small models for batch GraphRAG chunk processing, structured JSON extraction, and preemptible generation that can be cancelled when a user returns.
Nightshift Grid ⚡️
Turn idle laptops, office PCs and legacy mini-PCs into a zero-CapEx, fault-tolerant local AI batch and GraphRAG cluster.
pipx install nightshift-grid # or: uv tool install nightshift-grid
nightshift up --mock --workers 3 # coordinator + 3 workers + live dashboard
A real (mock-inference) run recorded with scripts/record_demo.py. Benchmarks so far are from a single Mac; see Known limitations.
Nightshift Grid is a lightweight distributed compute engine ("SETI@home meets local enterprise MapReduce"). It harvests idle cycles across mixed local machines without slowing down the people using them.
It doesn't slice one giant model across slow LAN links, where Wi-Fi jitter destroys token
throughput and one closed laptop lid kills the forward pass. Instead, every machine is a
preemptible, fenced micro-worker running its own quantized small model (qwen3:4b,
qwen2.5:3b, phi4-mini) via Ollama. A central coordinator hands out leased chunks.
When a human touches the machine, the node yields immediately and its chunk moves to another node.
submit 24 chunks output_graph.jsonl
client ──────────────────────▶ Coordinator ───────────────────────▶ (1 record / chunk)
│ ▲ lease(task_id, lease_id) / heartbeat 3s / complete / preempt
┌───────────────┼──┼────────────────┐
▼ ▼ │ ▼
node-office-pc node-legacy-laptop node-mac-studio each: worker_daemon.py
(Tier 2) (Tier 1) (Tier 3) + local OllamaWhy this exists
Most teams have gigabytes to terabytes of unstructured PDFs, logs and tickets that can't go to cloud APIs for cost or privacy reasons. Meanwhile the office PCs, laptops and mini-PCs that could process them sit mostly idle all day and fully idle overnight.
The common ways to pool local machines leave a gap for batch pipelines:
Approach | Typical failure mode for batch work | Nightshift Grid |
Tensor/layer splitting (one big model sharded across machines) | Bound by LAN serialization. One node dropping mid-pass stalls the whole request. | Each node runs a whole small model on independent chunks. A dropped node costs one chunk retry. |
Stateless request routing (load-balance chat requests across nodes) | No durable job state, so a node dying mid-request fails that request. | Stateful MapReduce queue with fenced leases. Every chunk completes or is explicitly dead-lettered. |
Dedicated inference pools | Machines must be handed over to the cluster. | Good-citizen harvesting: the human always wins and the grid backs off in milliseconds. |
Capability tiers decide what each node gets: Tier 1 (<8GB free) does embeddings and chunking, Tier 2 (8–24GB) does 3B–8B INT4 entity/relationship extraction, and Tier 3 (>24GB, or Apple Silicon with ≥16GB free) does heavy verification and synthesis.
Related MCP server: omni-rag-mcp
Key highlights and live benchmarks
1. Sub-250ms GPU yield, including during model load and prompt eval
A common trap when wrapping Ollama or llama.cpp in Python is breaking out of the token loop
without closing the underlying socket. The server keeps generating, and the GPU stays pegged
until the response finishes. A second trap is checking for user activity only when a token
arrives. That leaves a blind spot while the model is loading weights or evaluating a long prompt.
Nightshift runs inference in its own asyncio task. An out-of-band activity watcher polls the
chaos trigger file every 100ms and OS HID idle time every 500ms: IOKit HIDIdleTime read in-process
on macOS (ioreg -c IOHIDSystem as a fallback), GetLastInputInfo on Windows, D-Bus or xprintidle
on Linux. On activity it calls task.cancel(),
which unwinds async with client.stream(...) and closes the connection, so Ollama stops
generating.
Measured with qwen3:4b on Apple Silicon (GPU utilisation from IOKit, 250ms samples):
Worker cancellation method | GPU utilisation afterwards |
Stop the read loop, leave the response open | 84–94% until the model finishes |
Error raised inside | 0% by the first sample; next request ready in 0.04s |
Out-of-band | 0% by the first sample |
2. End-to-end chaos benchmarks
Three workers (qwen3:4b, about 130 tok/s decode) processed a 24-chunk industrial-maintenance
GraphRAG extraction job per phase, with auth enabled (python nightshift.py chaos-test --model qwen3:4b). All three workers ran on one Mac sharing one Ollama
instance. Multi-machine runs are still to do.
Chaos scenario / metric | Measured result |
Live human preemption (user returns mid-job on | 24/24 chunks in 47.3s, 0 failures |
Activity → coordinator reclaims the lease (trigger file, 100ms poll) | 0.01–0.10s across runs (0.011s in the final run) |
Lease reclaimed → re-assigned to another node | 0.8s |
Activity → preempted chunk finished on the backup node | 5.2s |
Work sent to the paused node while the human is active | 0 chunks |
Worker crash ( | 24/24 chunks in 45.8s |
| 11.9s → +1.1s → 16.8s total |
Coordinator crash ( | 24/24 chunks in 46.4s, 24 attempts for 24 chunks (0 wasted) |
Restart → ready; in-flight leases finished by their original holders | 0.23s; 2/2 |
Host CPU spike (busy-loop on all 16 cores mid-job, | 24/24 chunks in 54.4s; all 3 workers yielded ( |
Unintended preemptions from the grid's own Ollama load (phases 1–3) | 0 |
Auth (anonymous submit / worker token on client API / forged completion) | 401 / 401 / 401 |
Output integrity (4 jobs) | 96/96 records, 0 missing, 0 duplicates, 434 triples |
With --native-idle-seconds, detection latency is bounded by the 500ms HID poll rather than the
100ms file poll. The HID path is covered by unit tests and live IOKit and ioreg reads. An end-to-end
benchmark with real keyboard input is still to do, because synthesising HID events needs macOS
Accessibility permission.
Core guarantees
Guarantee | Mechanism |
No chunk lost | Every lease has a 12s TTL, extended by 3s heartbeats, and a reaper re-queues expired leases. Preemption and graceful |
Fenced leases (no split-brain) | Each assignment carries a unique |
Race-free pull and pause | If a human returns while a pull is in flight, the new lease goes straight back. Preempt messages name their lease, so a late one can't revoke a newer lease. A paused node gets no work until it reports idle. |
Dead-letter protection | Errors and lease expiries count toward |
Clean graph output | Ollama structured outputs (a JSON schema) constrain decoding. |
Idempotent sink |
|
Capability-aware scheduling | A worker serves the highest tier it's capable of first, so Tier 3 nodes aren't spent on Tier 1 work while Tier 3 work waits. Recovered chunks go to the front of the queue. |
Survives coordinator crashes | Every state transition is journaled to SQLite (WAL) in one transaction. On restart, queue order, results and dead letters are recovered, and in-flight leases get a fresh TTL so workers that kept computing can still complete them. Workers retry completions for 30s to ride out the restart. About 11k durable transitions/s; a 50k-task job recovers in 0.3s. |
Least-privilege auth | Separate bearer tokens for workers ( |
Good citizen beyond the keyboard | Workers also pause on battery power ( |
Fail-closed idle sensing | With |
Zero-config LAN discovery |
|
Live vector index (optional) |
|
Preemption without lost work (Baton relay, opt-in) | With |
Live fidelity (qwen3:4b, 3 chunks × 25/50/75% cuts, 2 runs):
18/18 resumed outputs are valid; 6/18 are identical to a fresh run.
Mean triple overlap with the fresh run is 0.70, against 0.89 between two fresh runs.
So resumed extractions drift more than run-to-run noise, mainly because an ungrammared continuation sometimes closes the object early.
Relay therefore stays opt-in. It suits slow nodes and long outputs, where restarting costs most. | | Resilient workers | Non-JSON or 5xx responses are retried, not fatal. If the activity watcher or heartbeat task dies, the worker exits loudly instead of running unprotected. |
Quickstart
1. Install
As a tool (isolated environment; puts nightshift, nightshift-coordinator, nightshift-worker, nightshift-mcp,
nightshift-dashboard and nightshift-credential on your PATH):
pipx install nightshift-grid # or: uv tool install nightshift-grid; latest main: uv tool install git+https://github.com/redd34/nightshift-grid
ollama pull qwen3:4b # any small chat model works; pass it with --modelFrom a checkout (for development; every command below also works as python <script>.py):
git clone https://github.com/redd34/nightshift-grid && cd nightshift-grid
python3 -m venv .venv && source .venv/bin/activate # or: uv venv .venv
pip install -e ".[dev]"2. Try it in 30 seconds
nightshift up --mock --workers 3 # coordinator + 3 workers + live dashboard, one terminal
nightshift submit --wait # (second terminal) 24-chunk demo job, zero config
nightshift status # one-shot table; `down` stops the grid and cleans up a crashed session
nightshift up --model qwen3:4b # same, with real inferenceup keeps tokens (0600), per-worker credentials, the state DB, output and logs in a state directory (.nightshift/
in a checkout, ~/.nightshift for an installed tool, or $NIGHTSHIFT_STATE_DIR), so submit/status need no flags.
3. Verify
python -m pytest -q tests # 379 tests: state machine, durability fuzz, guards, auth, TLS, relay, MCP...
python nightshift.py chaos-test --mock # 4-phase chaos benchmark, Markdown table to stdout (~2.5 min)
python nightshift.py chaos-test --model qwen3:4b # same, against live Ollama4. Run a grid by hand
python coordinator.py --output output_graph.jsonl --db grid_state.db # loopback, no auth: local dev
python worker_daemon.py --id node-office-pc --tier 2 --model qwen3:4b
python worker_daemon.py --id node-legacy-laptop --tier 1 --model qwen3:4b --native-idle-seconds 60
python worker_daemon.py --id node-mac-studio --tier 3 --model qwen3:4b
python dashboard.py # live rich TUI
python demo_submit.py --wait # submit 24 chunks and wait for the summary
touch /tmp/preempt_node-legacy-laptop # simulate a human; rm the file to release5. From documents to an answerable graph (map → reduce → query)
python ingest.py docs/ --tier 2 --wait # chunk .txt/.md/.log/.csv/.jsonl/.pdf → grid job(s) + manifest
python reduce_graph.py output_graph.jsonl --out graph.json --graphml graph.graphml \
--qdrant http://localhost:6333 --embed-model nomic-embed-text # entity resolution + nightshift_* collections
python reduce_graph.py query graph.json "which bearing assemblies were replaced?" --qdrant http://localhost:6333ingest.py keeps provenance in a sidecar <job>.manifest.jsonl, not in the prompt, so workers never see file paths.
reduce_graph.py merges surface variants such as Compressor turbine/Compressor: turbine, but never merges names
whose numbers differ (B-101 ≠ B-102, 4.2 ≠ 4.5 mm/s). It only ever writes Qdrant collections prefixed nightshift_,
and it never deletes or recreates a collection.
6. Drive it from an AI IDE (MCP)
Claude Code plugin (MCP server + a skill that teaches Claude when and how to use it):
/plugin install nightshift-grid --marketplace redd34/nightshift-gridNeeds nightshift-mcp on your PATH (pipx install nightshift-grid). With no settings, the server finds the grid that
nightshift up started (URL and client token from its state dir) and lets Claude read files under the project directory.
Any MCP client, by hand:
claude mcp add nightshift -- nightshift-mcp # optional: -e NIGHTSHIFT_MCP_ROOTS=/abs/docs:/abs/projectThe skill on its own lives in plugin/skills/nightshift-grid/SKILL.md; copy it to
~/.claude/skills/ if you register the server yourself.
The server exposes 4 tools: grid_status, dispatch_mapreduce_text, ingest_directory_to_graph and query_local_graph.
File access is limited to NIGHTSHIFT_MCP_ROOTS (symlinks resolved, secret files refused), and tools warn when data
would leave the machine. See docs/MCP.md for Claude Desktop config and the security model.
7. Deploy as services
deploy/gen_token.sh --env > coordinator.env # GRID_WORKER_TOKEN / GRID_CLIENT_TOKEN
# Coordinator (Linux): hardened systemd unit, steps in deploy/README.md §2
# Worker on macOS (LaunchAgent) / Linux (systemd --user); preview first with --dry-run:
printf '%s\n' "$GRID_WORKER_TOKEN" | deploy/install_worker.sh \
--coordinator http://10.0.0.5:8000 --model qwen3:4b --tier 2 --token-stdin
# Windows (at-logon task; add -WhatIf to preview):
# deploy\windows\install_worker.ps1 -Coordinator http://10.0.0.5:8000 -TokenFile worker.token
# Remove: deploy/install_worker.sh --uninstall [--purge] · Ops guide: deploy/README.mdWorkers run as per-user services (a LaunchAgent, systemd --user, or an at-logon task), never as system daemons.
Only a per-user service can see the logged-in user's keyboard and mouse, which preemption depends on.
Useful worker flags:
--mocksimulates the token stream.--tier Noverrides hardware fingerprinting.--native-idle-seconds Npauses the worker while HID idle time is under N seconds.--unload-on-preempt auto|always|neversendskeep_alive: 0to free the model's RAM.autoonly does this under memory pressure.--allow-battery,--min-battery-percent P,--ignore-thermaland--max-temp-c Ccontrol the power and thermal guards.--num-ctx N(default 4096, envNIGHTSHIFT_NUM_CTX) is sent with every Ollama request. It is never left unset: Ollama would otherwise load the model at its own default context, which for qwen3 is 262k tokens (~40 GB).--max-host-cpu-pct P(default 65, 0 = off) and--cpu-busy-samples Ncontrol the CPU guard.--min-free-ram-gb G(default 2.0) controls the RAM guard.--allow-no-idle-sensorkeeps the worker running when no HID idle sensor is available.--discover(with--discovery-port) finds the coordinator on the LAN, and learns its TLS fingerprint through the authenticated reply.--token-file PATHsets the worker credential. Otherwise it comes from$GRID_WORKER_TOKEN,$GRID_TOKEN,$NIGHTSHIFT_TOKEN_FILEor$NIGHTSHIFT_TOKEN. With a per-worker credential,--idis optional, because the credential names its id.--tls-fingerprint sha256:…pins the coordinator's certificate.--ca-file PEMverifies it against a CA instead. The env equivalents are$NIGHTSHIFT_TLS_FINGERPRINTand$NIGHTSHIFT_CA_FILE, which the dashboard,demo_submit.py, the CLI and the MCP server also read.
Coordinator flags:
--hostdefaults to127.0.0.1.--db grid_state.dbsets the state file (none= in-memory).--outputsets the JSONL sink.--worker-token-fileand--client-token-fileset the tokens. Otherwise they come from$GRID_WORKER_TOKEN/$GRID_CLIENT_TOKEN, or$GRID_TOKENfor both.--auto-tlsserves HTTPS with a self-signed certificate. It's minted on the first run next to the db and reused afterwards, and its fingerprint is printed at startup. Alternatively, use--tls-cert/--tls-key. A non-loopback bind refuses plain HTTP unless you pass--allow-plaintext, which is for networks that are already encrypted, such as Tailscale/WireGuard.--worker-auth auto|per-worker|sharedsets how workers authenticate. The default,auto, meansper-workeron a LAN bind: each worker presents a credential bound to its own id, minted withpython grid_auth.py issue --id node-x --worker-token-file worker.token --out node-x.credential. The master worker token then never leaves the coordinator host.--allow-insecurepermits a LAN bind without tokens or TLS.--auto-tokengenerates and persists 0600 tokens on the first run.--advertise/--no-advertiseand--discovery-portcontrol LAN discovery. It's on by default for non-loopback binds.--qdrant-url URLor--qdrant-memory, plus--qdrant-collection nightshift_graphand--qdrant-embedder auto|ollama|hash, enable the live vector sink.python qdrant_sink.py backfillre-indexes from the JSONL.
API
With tokens configured, /worker/* requires Authorization: Bearer <worker credential> and /v1/* requires the client token. A per-worker credential is nsw1.<id>.<HMAC(master, id)>. It only works for its own worker_id: using it for any other id gets a 403, and it can't mint new ids.
Endpoint | Purpose |
| Liveness probe (unauthenticated, reveals nothing). |
| Join, or re-join. A re-join re-queues the node's old leases. |
| Lease the next chunk → |
| Extend the lease and report telemetry plus progress ( |
|
|
| Fenced by |
| Submit |
| Progress, attempts, preemptions, dead letters and results. |
| Dashboard snapshot and structured event log. Sequence numbers keep increasing across restarts. |
Project layout
File | Role |
| FastAPI control plane. |
| SQLite (WAL) journal: one transaction per state transition, plus recovery. |
| Token resolution (file or env), bearer headers, and fail-fast 401 handling. |
| Preemptible async worker: hardware fingerprinting, cancellable Ollama streaming, heartbeats. |
|
|
| Map-side input: walks files, chunks on paragraph/sentence boundaries with overlap, dedupes, submits jobs, writes a provenance manifest. |
| Reduce side: entity resolution (key normalization, guarded embedding merge, union-find), graph.json/GraphML, Qdrant sink, and |
| One-command CLI: |
| MCP server (stdio) exposing the grid to Claude Code, Claude Desktop, Cursor and Kiro. See |
| UDP LAN discovery with an HMAC proof of token possession. |
| Live Qdrant sink: background batching, disk spool, embedder consistency, and |
| Service installers and templates: macOS LaunchAgent, systemd, Windows Task Scheduler, plus a token generator. See |
| Extraction prompt, JSON schema, normalizer and idempotent JSONL sink. |
| Baton token relay helpers: JSON progress scanner and online validator, verified ChatML raw-prompt renderer, grammar-per-item structured resume. |
| E2E harness with auth on: human-preemption, worker-SIGKILL and coordinator-SIGKILL phases, with zero-loss and zero-duplicate checks. |
| TUI and batch submitter. |
Known limitations and roadmap
Single-host benchmarks so far. All three workers shared one Mac and one Ollama. The next milestone is a multi-machine pilot. Latency metrics compare worker and coordinator wall clocks, which is exact on one host but subject to NTP skew across hosts.
Transport and identity. A LAN coordinator serves TLS (
--auto-tlsor your own certificate), and workers pin its fingerprint. Each worker holds a credential that's valid only for its own id, and every call is also bound to the current registration's session. What remains: a compromised worker can still submit bad results for chunks it leases (results aren't cross-checked yet), credentials are revoked only by rotating the master worker token, and the self-signed certificate's private key is only as safe as the coordinator host.Single coordinator. It's durable, but not highly available. Finished jobs are never pruned, so the state DB grows with history.
Mixed failure policy. HID idle sensing fails closed. Battery, thermal, CPU and RAM sensors fail open with one warning each.
Qdrant sink and SIGKILL. Records still in the sink's in-memory queue when the coordinator is SIGKILLed (under a second's worth) are in the JSONL but not in Qdrant until
python qdrant_sink.py backfillruns.Noisy hosts. The default 65% CPU guard pauses workers on a busy developer machine. That's intended, but tune
--max-host-cpu-pctfor build servers.Not validated on real hardware: thermal throttling and battery unplugging (unit-tested with fakes only), the Linux and Windows installers (rendered and inspected, but never run on those OSes), and
systemd-analyze,shellcheckandpwshlinting.Extraction quality from 3–4B models is unmeasured. Next: a small labelled set, plus Tier 3 verification/consensus on a sample.
Privacy: Nightshift Grid collects nothing; your documents stay on your machines. See PRIVACY.md.
Available Tools
4 toolsdispatch_mapreduce_textA
Submit text chunks to the grid for entity/relationship extraction (one model call per chunk, spread across idle workers). With wait=true (default) blocks until done, reporting progress, and returns counts, elapsed time, failed chunks, the merged top entities and up to 200 [subject, RELATION, object] triples. With wait=false returns the job_id at once. required_tier: 1 any worker, 2 workers with 8-24GB free RAM, 3 the largest machines.
Inputs: chunks (required), job_name (required), wait, timeout_s, +1 more. Returns: object
grid_statusARead-only
Show Nightshift Grid capacity: each worker's id, capability tier (1 embeddings, 2 JSON extraction, 3 heavy), state, model, live tok/s and completed/preempted counts; task queue depth by state; and the most recent jobs with progress.
Inputs: no named parameters. Returns: object
ingest_directory_to_graphA
Build a knowledge graph from local files: chunk every file under directory_path that matches glob_pattern (~chunk_size characters per chunk, with overlap), extract entities/relations on the grid, then write /<job_id>/output_graph.jsonl and the resolved graph.json. Returns their paths, top entities/relations, timings, and every skipped file with the reason. Only reads inside the allowed roots; refuses secret files (.env, keys, credentials). Relative paths are resolved against the first allowed root. With wait=false returns after dispatch; graph.json appears when the job finishes.
Inputs: directory_path (required), wait, chunk_size, glob_pattern, +1 more. Returns: object
query_local_graphARead-only
Look up facts in a local Nightshift knowledge graph (no LLM call): entities matching entity_or_keyword (exact name/alias, substring, or keyword overlap) and the [subject, RELATION, object] triples around them - hop 1 touches a match, hop 2 touches a neighbour - strongest first, each with evidence (job_id, chunk_index, source file span). Searches the newest graph.json in the output dir unless graph_path names a graph.json or a coordinator output_graph.jsonl (inside the allowed roots).
Inputs: entity_or_keyword (required), hops, limit, graph_path. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool occupies a distinct stage of the workflow: status inspection, text-chunk dispatch, directory ingestion, and local graph querying. The only potential overlap is dispatch_mapreduce_text vs. ingest_directory_to_graph, but they clearly differ by input type and output behavior.
All names use snake_case and are descriptive; the three action tools follow verb_object phrasing, while grid_status is noun-first. This is a minor deviation but remains readable and predictable.
Four tools are well scoped for a grid/KG pipeline: inspect capacity, dispatch extraction, ingest files, and query results. No tool feels redundant or missing for the core purpose.
The surface covers the main lifecycle: status, extraction dispatch, file-to-graph ingestion, and graph lookup. Minor gaps exist around explicit job management, such as checking or canceling a job by job_id and listing available graphs, but core workflows are covered.
Maintenance
Related MCP Connectors
Reach your own computers from any AI app: files, search, commands, approvals and outage alerts.
Read and search folders on your own computer with explicit consent. Files are never uploaded.
End-to-end agent-managed company brain. Docs, diagrams, plans, databases, agents, Knowledge Graph.
Give your AI agent a persistent map of your project's structure, dependencies, and bugs.
Related MCP Servers
- FlicenseAqualityDmaintenanceEnables AI agents to query a local knowledge graph built from document collections using hybrid search (BM25 + vector fusion) and entity-relationship extraction. Supports privacy-first, offline operation with tools for semantic search, entity graph exploration, and corpus statistics.3-
- AlicenseBqualityDmaintenanceEnables token-efficient semantic search and analysis over any directory of files through hybrid search, directory overview, structural analysis, and dependency graphs.14MIT
- AlicenseNot gradedqualityBmaintenanceConnects Claude Code to a local GraphRAG knowledge base (LightRAG + Ollama) for private, offline document retrieval. Provides tools to search, add documents, and check the RAG status, enabling Claude to reason over user's own notes, code, and papers without cloud dependencies.MIT
- FlicenseNot gradedqualityCmaintenanceEnables Claude to interact with local document vaults as searchable knowledge graphs, providing MCP tools for ingestion, search, and memory consolidation without cloud dependencies.-