# Needle Extract MCP

A production-ready, ultra-lightweight Model Context Protocol (MCP) server for persistent conversational memory.

## Overview

Needle Extract MCP enables small local models (<=4B parameters) to persist and retrieve long-term user facts reliably. It eliminates the need for heavy VRAM usage, external vector databases, or complex multi-step tool logic. By employing a lightweight extraction logic and SQLite's built-in FTS5 text search, this server acts as a rapid, lightweight memory store for any AI agent compatible with the Model Context Protocol (MCP) standard.

## Features

- **FastMCP Protocol**: Fully compliant with standard MCP. Exposes `remember` and `recall` tools for LLM integration.
- **Smart Chunking Engine**: Automatically preprocesses and chunks incoming prompts into short sentence blocks (under 50 words) to guarantee it fits safely within small KV windows (such as Needle 2's 256-token limit).
- **Lightweight DB**: Uses `sqlite3` with FTS5 (Full-Text Search) and the Porter stemmer for lightning-fast BM25 keyword matching and fact retrieval. No heavy vector database required.
- **Regex Fallback**: Built-in rule-based fallback parsing to ensure uninterrupted server execution if extraction logic fails.
- **Strict Validation**: Leverages `pydantic` to maintain strict memory schema parsing and validation.

## Architecture

- **`server.py`**: The FastMCP server handling standard I/O communication and exposing the MCP tools.
- **`chunker.py`**: Performs sentence-level windowing, stripping noise like code/markdown blocks, and filtering trivial chunks to minimize model passes.
- **`extractor.py`**: Handles schema decoding and mapping conversational snippets to structured `MemoryFact` objects.
- **`db.py`**: Manages SQLite FTS5 virtual tables, transaction batching, and BM25 searching.

## Requirements

- Python 3.11+
- CPU-only execution (sub-50MB RAM footprint).
- Zero cloud dependencies.

## Installation

```bash
git clone https://github.com/BhargavGadekar/needle-extract-mcp.git
cd needle-extract-mcp
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

## Running the MCP Server

To configure your MCP Client (like Antigravity or LM Studio) to use this server, add the following to your MCP client's configuration file (e.g., `mcp_config.json`):

```json
{
  "mcpServers": {
    "needle-extract": {
      "command": "/absolute/path/to/needle-extract-mcp/.venv/bin/python",
      "args": ["/absolute/path/to/needle-extract-mcp/server.py"]
    }
  }
}
```

*Note: Replace `/absolute/path/to/needle-extract-mcp` with the actual path to your repository on your system.*

## Available Tools

### 1. `remember`
- **Inputs**: `conversation_snippet` (string)
- **Description**: Extracts facts from a conversation snippet and stores them in persistent memory. It chunks the text, extracts atomic facts, assigns topics/tags, and commits them in a batch transaction.
- **Returns**: A confirmation message with a bulleted list of extracted facts, or a "no new facts detected" message.

### 2. `recall`
- **Inputs**: `query` (string)
- **Description**: Searches persistent memory for relevant facts based on a text query. It uses SQLite FTS5 for BM25 ranking and handles wildcard tokens.
- **Returns**: A formatted, scannable bullet list of facts found in the database.

## Testing

Run the automated test suite with `pytest` to ensure chunking limits, extraction heuristics, and storage/search mechanisms are working properly.

```bash
pytest test_memory.py
```

## License

This project is licensed under the Apache 2.0 License. See the `LICENSE` file for details.