# PROTOTYPE: MIDNIGHT MCP KNOWLEDGE WORKSPACE

**Author:** `@noelrim`

**Date:** `March 2026`

**Github Repository:** `https://github.com/shieldedtech/product/tree/prototypes/prototypes/midnight-mcp`

## Gap
Midnight knowledge is spread across source repositories, Confluence, Google Drive, and strategy/process documents. Answering product or technical questions reliably requires switching between many systems, and generic LLM workflows tend to confuse documented intent with implemented reality. In practice, teams either burn tokens by stuffing large context into prompts, or repeatedly upload the same material across sessions and tools.

## Purpose
This prototype is an LLM-first Midnight knowledge workspace built around an MCP server, a local index backend, and a one-command installer. Its purpose is to make the broadest useful Midnight context available to any LLM client with minimal user effort, without repeatedly uploading the same documents or paying token costs to restate large context windows on every question. It lets Codex or similar clients answer Midnight questions using indexed evidence from code, Confluence, and Drive while enforcing explicit source integrity rules.

**Data Sources:**
- **Code Index:** Midnight repositories and selected repo documentation.
- **Confluence Index:** exported internal workspace pages.
- **Google Drive Index:** exported internal docs such as PRDs, strategy docs, and research.
- **Pinned Source Repos:** optional local clones aligned to the exact indexed commits.

## Features
- **MCP-First Midnight Retrieval**
  - Exposes a dedicated Midnight MCP server with retrieval, status, and preference tools.
  - Shapes answers by audience mode (`executive`, `product`, `mixed`, `engineering`, `forensic`) and verbosity.
  - Returns source-integrity metadata so code-backed implementation can be separated from docs-only intent.
  - Makes retrieval reusable across sessions, so LLMs query the index instead of re-ingesting the same raw context repeatedly.

- **Two-Pass Evidence Model**
  - Runs a broad retrieval pass for cross-source context.
  - Runs a forced code verification pass when implementation checking is enabled.
  - Uses alias-expanded fallback queries to handle naming drift such as `Technical Authority` vs `Technical Committee`.

- **One-Command Installer**
  - Installs MCP config, Codex skill, indexes, raw docs bundles, and optional pinned repos.
  - Pulls private artifacts from Google Drive by default.
  - Includes live progress bars, ETA, and auth/session reuse.
  - Optimizes for low-friction LLM enablement rather than manual context assembly.

- **Reproducible Local Context**
  - Ships `repo-fingerprints.json` so indexed repositories can be cloned and checked out at the exact indexed commit.
  - Remaps workspace-relative paths so retrieved evidence remains explorable after install.

## Technical Highlights
- **MCP Layer:** Node.js server wrapping a local retrieval backend and enforcing source-policy logic.
- **Indexer Backend:** FastAPI + LlamaIndex persisted storage served via `/health`, `/search`, and `/doc`.
- **Embedding Stack:** local HuggingFace embeddings using `BAAI/bge-base-en-v1.5`.
- **Model Flexibility:** indexes are versioned per embedding model slug, so multiple LlamaIndex-compatible embedding models can coexist and be installed on demand.
- **Chunking Profile:** `2200` chunk size with `80` overlap across code, docs, and drive indexes.
- **Artifact Distribution:** private Drive-based bundle delivery for indexes and raw docs, with GitHub release support as fallback.

## Design Notes
- This is primarily an LLM context-delivery system, not just an MCP wrapper.
- Code and tests are treated as implementation truth when implementation evidence is expected.
- Documentation is preserved as valuable product/process context, but not accepted as implementation proof by default.
- The retrieval contract is backend-agnostic at the API level, even though the current persisted indexes use LlamaIndex.

For deeper architecture and rationale, see `ARCHITECTURE.md`.