agent-memory
Supports using Google Gemini models for fact extraction, conflict reconciliation, and embedding generation in the memory pipeline.
Supports running fully offline local models via Ollama for fact extraction, reconciliation, and embedding generation.
Supports using OpenAI models as the LLM provider for extracting atomic facts, reconciling memory conflicts, and generating embeddings.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@agent-memory Remember that I moved to San Francisco and now work at Stripe."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
π§ Self-Learning AI Agent: Long-Term Memory Engine
A production-grade Long-Term Memory Microservice & Model Context Protocol (MCP) Server for AI Agents. Enables continuous, cross-session learning by dynamically extracting atomic user facts, resolving state conflicts (
ADD,UPDATE,DELETE,NOOP), and persisting vectors in Qdrant.
π Why This Architecture?
Traditional RAG and naive chat-history appending have critical flaws:
Context Window Bloat: Appending full raw transcripts increases latency, token costs, and attention drift.
Contradictions & Stale State: If a user says "I live in San Francisco" and 3 months later says "I moved to Tokyo", naive RAG retrieves both chunks, causing hallucinations.
My Solution: A Two-Phase Memory Pipeline where an LLM parses atomic facts and performs explicit state mutations (
ADD,UPDATE,DELETE,NOOP) directly on the vector store.
Related MCP server: Memsolus MCP Server
ποΈ System Architecture
flowchart TD
subgraph Ingestion ["1. Client Interfaces & Ingestion"]
A["π¬ User / Agent Chat"] --> C["β‘ FastAPI REST API (/v1/chat)"]
B["π€ Claude Desktop / Cursor IDE"] --> D["π MCP 2.x Server (stdio)"]
C --> E["π₯ Async Ingestion Queue (asyncio.Queue)"]
D --> E
end
subgraph TwoPhase ["2. Two-Phase Agentic State Machine"]
E --> F["π§ Phase 1: Fact Extractor (LLM)"]
F -->|Atomic Facts & Triples| G["π Candidate Facts"]
G --> H["π Semantic Search Candidates"]
Q1[("πΎ Qdrant Vector Store (Current State)")] -.->|Existing User Memories| H
H --> I["βοΈ Phase 2: Conflict Reconciler (LLM)"]
G --> I
I -->|State Mutation Decision| J{"Operation"}
J -->|ADD| K["β¨ Insert New Vector Point"]
J -->|UPDATE| L["π Overwrite Stale Vector & Payload"]
J -->|DELETE| M["ποΈ Delete Vector Point"]
J -->|NOOP| N["βΈοΈ Ignore Redundant Duplicate"]
end
subgraph Storage ["3. Storage & Knowledge Graph Layer"]
K --> Q2[("πΎ Qdrant Vector DB (Synchronized State)")]
L --> Q2
M --> Q2
Q2 --> O["β³ Ebbinghaus Temporal Decay & Spaced Reinforcement"]
Q2 --> P["πΈοΈ GraphRAG Topological Entity Visualizer"]
end⨠Key Features
π§ Dynamic Two-Phase Lifecycle:
Extraction: Extracts durable, self-contained third-person facts while discarding conversational noise.
Reconciliation: Semantic lookup finds candidate conflicts; LLM decides
ADD,UPDATE,DELETE, orNOOP.
πΈοΈ Interactive GraphRAG Knowledge Visualizer:
Live topological node-link graph visualizer built on HTML5 Canvas force physics.
Automatically extracts Entity-Relation Triples (
Subject -> Relation -> Object) for multi-hop associative retrieval.
β‘ Sub-150ms Asynchronous Ingestion Queue:
Event-driven background queue (
asyncio.Queue) offloads fact extraction and vector synchronization, enabling instant conversational replies.
ποΈ Multi-Tier Scoped Memory:
Hierarchical isolation across
user(persistent),session(ephemeral/task-level), andworkspace(shared team conventions).
β³ Cognitive Temporal Decay (Ebbinghaus Forgetting Curve):
Mathematical recency weighting: $\text{Score} = (1 - w) \cdot \text{Similarity} + w \cdot e^{-\lambda \Delta t}$.
Spaced reinforcement: Automatically touches and refreshes retention every time a memory is recalled.
π Interactive Visual Memory Explorer & AI Chat Playground:
Full-featured dark-mode web dashboard (
/dashboard) with live chat playground, real-time memory bank feed, and similarity confidence meters.
π Dual Serving Interfaces:
FastAPI REST Endpoints: High-performance HTTP service with OpenAPI docs (
/docs).Model Context Protocol (MCP 2.x): Plug-and-play tools (
remember_conversation,recall_memories,forget_memory) for Claude Desktop, Cursor, and agentic workflows.
πΎ Hybrid Qdrant Support: Runs via Docker or automatic embedded local disk mode (
./qdrant_data) with zero cloud cost.π Multi-Provider Support: Seamlessly swappable across Google Gemini (
gemini-3.5-flash-lite), OpenAI (gpt-4o-mini), or 100% offline local models via Ollama.π‘οΈ Production Hardened: Adaptive exponential backoff retry handler parsing upstream rate-limit windows (429/503).
π Repository Structure
βββ src/
β βββ api/ # FastAPI REST API routes & controllers
β β βββ __init__.py
β β βββ routes.py # /v1/memories/process, /search, /user, /delete
β βββ db/ # Vector database layer
β β βββ __init__.py
β β βββ qdrant.py # Qdrant client manager & schema initializers
β βββ llm/ # Unified LLM provider client (Gemini / OpenAI)
β β βββ __init__.py
β β βββ client.py # Structured JSON generation & embedding generation
β βββ memory/ # Two-Phase Memory Pipeline Core
β β βββ __init__.py
β β βββ models.py # Pydantic schemas (Fact, Operation, MemoryRecord)
β β βββ extractor.py # Phase 1: Atomic fact extractor
β β βββ reconciler.py # Phase 2: Conflict reconciler
β β βββ service.py # High-level memory orchestrator
β βββ mcp_server/ # Model Context Protocol (MCP 2.x) integration
β β βββ __init__.py
β β βββ server.py # Standardized MCP server & tools
β βββ config.py # Pydantic Settings management
β βββ main.py # FastAPI ASGI entrypoint & lifecycle
βββ scripts/
β βββ verify_setup.py # Setup & database connectivity verifier
β βββ demo_pipeline.py # Interactive visual lifecycle demo
βββ tests/
β βββ test_phase1.py # Infrastructure & DB tests
β βββ test_phase2.py # Two-phase pipeline & lifecycle tests
β βββ test_phase3_api.py # FastAPI REST endpoint tests
β βββ test_phase3_mcp.py # MCP tools test suite
βββ docker-compose.yml # Multi-container orchestration (API + Qdrant)
βββ Dockerfile # Multi-stage production container build
βββ requirements.txt # Project dependencies
βββ pytest.ini # Pytest configurationπ Quickstart Guide
1. Clone & Setup Environment
# Clone the repository
git clone https://github.com/your-username/self-learning-ai-agent.git
cd self-learning-ai-agent
# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt2. Configure Environment Variables
cp .env.example .envEdit .env with your API key and preferred models:
# Google Gemini (Free Tier / Zero Cloud Cost)
OPENAI_API_KEY=AIzaSy...
PROVIDER=gemini
EMBEDDING_MODEL=gemini-embedding-2
EMBEDDING_DIMENSION=3072
EXTRACTION_MODEL=gemini-3.5-flash-lite
RECONCILIATION_MODEL=gemini-3.5-flash-lite
# Vector DB Settings (Automatically falls back to local disk if Docker is off)
QDRANT_HOST=localhost
QDRANT_PORT=6333
SIMILARITY_THRESHOLD=0.603. Verify Setup & Run Interactive Demo
# Verify vector DB connectivity
python scripts/verify_setup.py
# Run visual multi-turn memory evolution demo
python scripts/demo_pipeline.py4. Run Automated Test Suite
pytest -vπ Running the Services
Option A: Local FastAPI Server
uvicorn src.main:app --host 0.0.0.0 --port 8000 --reloadInteractive Swagger API documentation: http://localhost:8000/docs
Option B: Full Stack Docker Compose
docker compose up --build -dOption C: Launch MCP Server (stdio)
python -m src.mcp_server.serverπ Model Context Protocol (MCP) Integration
Connect this memory engine directly to Claude Desktop, Cursor IDE, or Antigravity.
Add to your claude_desktop_config.json (or Cursor MCP settings):
{
"mcpServers": {
"agent-memory": {
"command": "/path/to/self-learning-ai-agent/.venv/bin/python",
"args": ["-m", "src.mcp_server.server"],
"cwd": "/path/to/self-learning-ai-agent"
}
}
}Exposed MCP Tools:
remember_conversation(user_id, conversation_text): Extracts facts and reconciles them into memory.recall_memories(user_id, query, limit): Retrieves semantically relevant facts with similarity scores.list_user_memories(user_id, limit): Lists all active facts for the user.forget_memory(memory_id): Manually removes a memory point.
π‘ REST API Reference
1. Ingest Conversation & Update Memory State
POST /v1/memories/process
curl -X POST http://localhost:8000/v1/memories/process \
-H "Content-Type: application/json" \
-d '{
"user_id": "alex_01",
"conversation": "I am a Senior AI engineer based in San Francisco. I switched my primary language from Python to Rust."
}'2. Semantic Memory Search
GET /v1/memories/search?user_id=alex_01&query=What+languages+does+the+user+code+in%3F
curl "http://localhost:8000/v1/memories/search?user_id=alex_01&query=What+languages+does+the+user+code+in%3F&limit=3"3. List All User Memories
GET /v1/memories/user/{user_id}
curl http://localhost:8000/v1/memories/user/alex_014. Delete Memory by ID
DELETE /v1/memories/{memory_id}
curl -X DELETE http://localhost:8000/v1/memories/c7b2049e-648b-4b10-a24e-b5f7cf839a82πΌ Resume & Technical Impact Highlights
If you include this project in your portfolio or resume, here are production-oriented bullet points:
Engineered a self-learning long-term memory microservice in Python (FastAPI) utilizing Qdrant vector search to provide AI agents with persistent, cross-session user context.
Implemented a dynamic Two-Phase Memory Pipeline that prompts an LLM to extract atomic facts and programmatically execute state mutations (
ADD,UPDATE,DELETE,NOOP), eliminating stale fact contradictions and optimizing LLM token utilization.Packaged the memory layer into a Model Context Protocol (MCP 2.x) Server, enabling native, zero-latency tool-use integration across IDEs and AI client agents (Cursor, Claude Desktop).
Architected a modular provider layer supporting Gemini, OpenAI, and local Ollama, featuring automated schema validation and adaptive rate-limit backoff retry handlers.
Containerized the full stack with multi-stage Docker builds and automated test suites achieving 100% test pass rates across unit and end-to-end integration flows.
π License
MIT License. Free for open-source and commercial use.
Available Tools
4 toolsforget_memoryA
Deletes a specific memory record by its unique ID.
Args: memory_id: The UUID of the memory record to remove.
Inputs: memory_id (required). Returns: result
list_user_memoriesA
Lists all active stored memories for a user.
Args: user_id: The unique identifier for the user. limit: Maximum number of memories to return (default: 50).
Inputs: user_id (required), limit. Returns: result
recall_memoriesA
Retrieves semantically relevant memories from vector storage given a natural language query.
Args: user_id: The unique identifier for the user. query: The topic, question, or context to retrieve facts for. limit: Maximum number of memory records to return (default: 5).
Inputs: query (required), user_id (required), limit. Returns: result
remember_conversationA
Extracts durable facts from conversation text and reconciles them into long-term memory (ADD, UPDATE, DELETE, NOOP).
Args: user_id: The unique identifier for the user. conversation_text: The conversation transcript or user statement to remember.
Inputs: user_id (required), conversation_text (required). Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogβ¦
TDQS
Scored across 4 tools
Each tool has a distinct core action: list, recall, remember, and forget. The only potential confusion is between list_user_memories and recall_memories, but the descriptions clearly separate 'all active memories' from 'semantically relevant' retrieval.
All tool names follow a consistent verb_noun snake_case pattern: list_user_memories, forget_memory, remember_conversation, recall_memories. The verbs are meaningful and align with their actions.
Four tools is a well-scoped size for a memory server, covering ingestion, retrieval, listing, and deletion. Each tool maps to a clear capability without redundant or excessive surface area.
The set covers the core memory lifecycle: adding memories, recalling them, listing them, and deleting them. A direct get-by-ID or explicit update tool is missing, but remember_conversation's reconciliation covers updates through its ADD/UPDATE/DELETE/NOOP behavior.
Maintenance
Related MCP Connectors
Persistent memory for AI agents. Search and store durable facts, preferences and decisions.
Persistent memory and knowledge management for AI agents with semantic search and 50+ tools.
Universal memory for AI agents and tools. Save, organize and search context anywhere.
Persistent memory for AI agents, for code, research and everyday life. Stale facts retire.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceProvides AI agents with persistent, searchable memory that survives across conversations using semantic search, temporal versioning, and smart organization. Enables long-term context retention and cross-session continuity for AI assistants.14-

Memsolus MCP Serverofficial
AlicenseAqualityDmaintenanceProvides persistent long-term memory for AI agents through semantic search and automated knowledge graph extraction. It enables agents to store, recall, and reason over facts, preferences, and relationships across multiple conversations and sessions.146 npmMIT- AlicenseAqualityCmaintenanceProvides persistent, searchable memory for AI agents, enabling them to retain, recall, and reflect on information across conversations.1932 PyPI1MIT
- AlicenseNot gradedqualityAmaintenanceProvides persistent, shared memory for AI agents by capturing conversations verbatim, distilling facts and summaries, and enabling retrieval through search, timeline, details, and explicit remember tools.MIT