Aegis — Agent-Ready Knowledge Compiler

Aegis is a durable, evidence-grounded knowledge compiler that transforms instructional video content (starting with YouTube playlists in v1) into versioned, structured, and validated knowledge packages exposed to AI agents via the Model Context Protocol (MCP 2026-07-28).

Key Capabilities

• Deterministic 10-Stage Pipeline: Discover $\rightarrow$ Acquire $\rightarrow$ Evidence Extraction $\rightarrow$ Segmentation $\rightarrow$ Structured Extraction $\rightarrow$ Evidence Binding $\rightarrow$ Enhancement $\rightarrow$ Validation $\rightarrow$ Vector Indexing $\rightarrow$ Atomic Publication.

• PostgreSQL 18 + pgvector Single Source of Truth: All stage executions, worker leases, idempotent operation identities, evidence citations, and knowledge items reside in PostgreSQL.

• Content-Addressed Immutable Artifacts: All intermediate outputs and raw transcripts are hashed (SHA256) and stored in S3/MinIO.

• Hybrid Semantic & Lexical Retrieval: Combines dense text-embedding-3-small vector similarity with PostgreSQL full-text search and evidence-support boosting.

• Read-Only MCP 2026-07-28 Server: 5 read-only tools exposed over stdio and Streamable HTTP for seamless integration with Claude Desktop, Cursor, and custom agent SDKs.

• Operator Console: Next.js 16 + React 19 web dashboard for ingestion, run tracking, evidence timeline exploration, and human-in-the-loop review queue resolution.

Related MCP server: youtube-ai

Monorepo Layout

aegis/
├── apps/
│   ├── api/          # FastAPI 0.139 Control REST API (:8000)
│   ├── worker/       # Background processing daemon & reconciler
│   ├── mcp/          # Read-only MCP 2026-07-28 Server (:8001)
│   └── web/          # Next.js 16 + React 19 Operator Console (:3000)
├── packages/
│   ├── domain/       # Pure domain models (zero framework dependencies)
│   ├── schemas/      # Pydantic v2 validation models & extraction schemas
│   ├── database/     # SQLAlchemy 2.0 async models (16 tables) & repositories
│   ├── pipeline/     # Idempotency engine, lease manager, 10-stage graph
│   ├── providers/    # External AI (OpenAI GPT-5.6) & YouTube (yt-dlp) adapters
│   ├── retrieval/    # Hybrid vector + lexical search engine
│   └── storage/      # Content-addressed S3/MinIO storage backend
├── migrations/       # Alembic database migrations
├── infra/
│   ├── compose/      # Docker Compose definition & overrides
│   ├── docker/       # Production multi-stage Dockerfiles
│   └── scripts/      # Backup, restore, migration, and healthcheck utilities
├── tests/            # Unit, integration, contract, evaluation, and crash recovery tests
└── docs/             # Architecture, REST API, operations runbook, and MCP guide

Technology Stack

Quickstart

Option A: One-Liner Production Installation (curl)

Bootstrap dependencies, virtual environment, and configuration with a single command:

# Automated installer (fetches uv, clones/syncs repo, initializes .env with secure keys, checks Docker)
curl -fsSL https://raw.githubusercontent.com/Demi8-patch/aegis/main/install.sh | bash

Or execute locally from the repository root:

./install.sh

Option B: Quickstart with uv

# 1. Clone and Bootstrap Environment
cp .env.example .env
uv sync
pnpm install

# 2. Verify System Health & Diagnostics
uv run aegis doctor

# 3. Start PostgreSQL 18 (with pgvector) and MinIO
docker compose -f infra/compose/docker-compose.yml up -d

# 4. Apply Database Migrations
uv run aegis migrate

Unified Aegis CLI Reference

Aegis provides a comprehensive command-line tool aegis accessible via uv run aegis <command> (or directly as aegis when installed):

Launching Development Services

# Terminal 1: FastAPI Control API (:8000)
uv run aegis api --port 8000 --reload

# Terminal 2: Aegis Background Worker & Reconciler
uv run aegis worker

# Terminal 3: Read-Only MCP Server (:8001 / stdio)
uv run aegis mcp

# Terminal 4: Operator Web Console (:3000)
pnpm --filter aegis-web dev

The Operator Console will be live at http://localhost:3000, the Control API at http://localhost:8000, and the MCP endpoint at http://localhost:8001/mcp.

MCP Tools Reference

The Aegis MCP server exposes 5 read-only tools conforming to the MCP 2026-07-28 specification:

For client integration setup (Claude Desktop, Cursor, Windsurf, custom agents), see the MCP Client Integration Guide.

Testing & Quality Assurance

Aegis includes a comprehensive test suite with 82+ tests covering unit logic, integration flows, crash recovery, prompt injection defense, and retrieval quality.

# Run full pytest suite (82 passed)
uv run pytest tests/

# Run Ruff linter and style checks
uv run ruff check .
uv run ruff format --check .

# Build Wheel & Source Distribution
uv build

# Build Next.js Operator Web Console
pnpm --filter aegis-web build

Production Operations & Disaster Recovery

• Database & Artifact Backup:

  ./infra/scripts/backup.sh

• Database & Artifact Restore:

  ./infra/scripts/restore.sh backups/aegis_db_YYYYMMDD_HHMMSS.sql.gz backups/aegis_artifacts_YYYYMMDD_HHMMSS.tar.gz

• Comprehensive Documentation:

  • Architecture Specification

  • REST API Reference

  • Operations Runbook

  • MCP Client Guide

License

Apache-2.0