<div align="center">
  <h1>Distill v2</h1>
  <p><strong>Model Context Protocol (MCP) Server for Massive Context Engineering & Compression</strong></p>
  
  [![CI](https://github.com/yatinkoul/distill/actions/workflows/ci.yml/badge.svg)](https://github.com/yatinkoul/distill/actions)
  [![PyPI version](https://img.shields.io/pypi/v/distill-mcp-v2.svg)](https://pypi.org/project/distill-mcp-v2/)
  [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
  [![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
</div>

---

## ⚡ Overview

`distill-mcp-v2` is a high-performance, network-dependency-free Python **FastMCP** server designed to aggressively optimize Large Language Model (LLM) context windows. It provides specialized tools for compressing and analyzing massive AI‑agent payloads without losing critical semantic information.

By filtering noise, stabilizing cache prefixes, and running multi-model token cost estimations locally, Distill v2 dramatically reduces API costs and preserves LLM reasoning abilities when dealing with heavy payloads like infinite logs, monolithic API schemas, and multi-agent war room transcripts.

## 🚀 Performance & Token Compression

Our rigorous, independent stress-testing benchmarks (audited via `pytest` and chaos blueprints) prove that `distill-mcp-v2` achieves **up to 99.7% token compression** while retaining 100% of the crucial context.

| Scenario | Payload Profile | Raw Tokens | Distilled Tokens | Savings % |
| :--- | :--- | :--- | :--- | :--- |
| **Trace Avalanche** | Heavy Java Stacktraces | 150,027 | 546 | **99.6%** |
| **Schema Monolith** | Massive Microservice JSON | 56,588 | 1,355 | **97.6%** |
| **Incident War Room** | Multi-Agent Chat Logs | 117,952 | 371 | **99.7%** |

*(Tested against Claude-3-Opus budgets. Scenario 1 reduced costs from $2.25/call to $0.008/call.)*

> Read the full [Benchmark & Execution Report](docs/benchmark_report.md) for deeper insights.

## 🛠 Features & Toolset

Distill v2 exposes 8 precise tools to agents via the Model Context Protocol:

1. `distill_json` — Compresses raw JSON payloads, retaining anomalies, exceptions, and errors.
2. `distill_logs` — Compresses raw `.log` files, preserving head/tail contexts and stack traces.
3. `distill_schema` — Compacts massive MCP tool catalogs and JSON schemas to structural parameters.
4. `distill_response` — Progressively prunes, minifies, and truncates outputs to fit strict token budgets.
5. `distill_conversation` — Extracts goals, decisions, blockers, and actions from multi-agent transcripts without leaking raw ISO timestamps.
6. `stabilize_for_cache` — Maps chaotic raw identifiers (UUIDs, hex IDs) to sequential placeholders to stabilize LLM prompt caching.
7. `analyze_tokens` — Accurately estimates token counts using `tiktoken` (cl100k_base).
8. `compare` — Computes detailed diffs and cost-savings analyses between raw and distilled payloads.

## 📦 Quick Start

### Installation

Distill v2 requires Python 3.10+. We recommend using `uv` or `pip` in an isolated virtual environment.

```bash
# Clone the repository
git clone https://github.com/yatinkoul/distill.git
cd distill

# Create a virtual environment and activate it
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .
```

### Running the Server

Run the FastMCP server, which natively binds to a stateless HTTP endpoint.

```bash
# Run the FastMCP server (default host: 0.0.0.0, port: 8000)
distill --host 0.0.0.0 --port 8000
# Or using uvicorn directly:
.venv/bin/uvicorn src.main:app --host 0.0.0.0 --port 8000
```

The server exposes a JSON‑RPC endpoint at `http://localhost:8000/mcp`.

## 🧪 Stress Testing & Development

The repository ships with an exhaustive **stress-test runner** that dynamically allocates ports, executes deterministic payload scenarios, and validates the integrity of the distilled outputs.

To run the complete test suite (118/118 passing):

```bash
# Install development dependencies
pip install -e ".[dev]"

# Run the test suite
pytest
```

To manually execute the stress-testing blueprints and regenerate the benchmark report:

```bash
python stress_tests_blueprints/run_blueprints.py
```

## 📖 Documentation

- **[Project Goals & Strategy](docs/PROJECT.md)** – High-level strategy and milestones.
- **[Benchmark Report](docs/benchmark_report.md)** – Exhaustive stress-test results.
- **[Testing Infrastructure](docs/TEST_INFRA.md)** – Notes on the testing architecture and CI/CD readiness.

## 🤝 Contributing

Contributions, issues, and feature requests are welcome! 
Please read our `CONTRIBUTING.md` (coming soon) for guidelines on how to propose improvements. Ensure all tests and linting (`ruff`) pass before submitting pull requests.

## 📜 License

This project is licensed under the MIT License – see the [LICENSE](LICENSE) file for details.