MCP Server

### Document Purpose **PLANNING.md:** Comprehensive guide to implementing the platform - Lists tasks and subtasks required to implement the site - Documents if a task has been completed with a checkbox - Describes the setup locally and on Render --- # PICKING UP WHERE WE LEFT OFF: - Still working on some tests related to token blacklisting now that the correct table is in the database. # Picard MCP Implementation Plan (MVP) This document outlines the step-by-step implementation plan for the Picard MCP project MVP, focusing on simplicity and essential functionality. ## Phase 1: Project Setup and Configuration ### 1.1 Environment Setup - [x] Create project repository structure - [x] Set up virtual environments for both MCP server and Django client - [x] Create `.env` files with required environment variables - [x] Set up Docker and Docker Compose configuration ### 1.2 Database Configuration - [x] Configure PostgreSQL with pgvector extension for MCP server - [x] Configure simple PostgreSQL for Django client - [x] Create minimal database schemas and initial migrations ### 1.3 Docker Configuration - [x] Create Dockerfile for MCP server - [x] Create Dockerfile for Django client - [x] Create docker-compose.yml for local development - [x] Test Docker setup with basic applications ## Phase 2: Initial Server and Client Implementation ### 2.1 Core Server Setup - [x] Set up FastAPI application structure - [x] Configure ASGI server (Uvicorn) - [x] Implement basic server health check endpoints - [x] Set up minimal logging ### 2.2 Database Models - [x] Implement User model - [x] Implement Memory model with vector storage - [x] Implement OAuth Client model - [x] Implement Token model - [x] Create database migrations - [x] Create unit tests for models ### 2.3 Basic Django Client Setup - [x] Set up Django project structure - [x] Configure settings and environment variables - [x] Set up URL routing - [x] Create simple base templates with Bootstrap ### 2.4 Django User Management - [x] Implement basic user registration - [x] Implement user authentication - [x] Implement minimal user profile - [x] Create simple login/logout flow ## Phase 3: OAuth 2.0 Implementation ### 3.1 OAuth Core Setup - [x] Set up OAuth routes and controller structure - [x] Implement OAuth schema validation - [x] Create OAuth error handling utilities - [x] Implement basic OAuth security measures - [x] Implement OAuth endpoint tests ### 3.2 Client Registration - [x] Implement client registration endpoint - [x] Create secure client credentials generation - [x] Implement client management system - [x] Add client registration script for Django client ### 3.3 Authorization Flow - [x] Implement authorization endpoint - [x] Create user consent page - [x] Implement CSRF protection for OAuth flows - [x] Add PKCE support ### 3.4 Token Management - [x] Implement token exchange endpoint - [x] Implement token refresh endpoint - [x] Implement token validation and introspection - [x] Create token storage and retrieval system ### 3.5 Django OAuth Client Implementation - [x] Implement OAuth 2.0 client with PKCE - [x] Create token storage in database - [x] Implement token refresh logic - [x] Set up basic error handling ### 3.6 Permission System - [ ] Implement scope-based permission system - [ ] Set up middleware for resource protection - [ ] Create role-based access controls - [ ] Implement token blacklisting for revocation ## Phase 4: Memory Management and Search ### 4.1 Memory Management API - [ ] Implement memory creation endpoint - [ ] Implement memory retrieval endpoints - [ ] Implement memory update endpoint - [ ] Implement memory deletion endpoint - [ ] Implement permission-based access control - [ ] Implement basic memory encryption at rest ### 4.2 Vector Embedding and Search - [ ] Integrate OpenAI API for text embeddings using LangChain - [ ] Implement vector storage with pgvector - [ ] Implement basic semantic search functionality ### 4.3 LLM Integration - [ ] Implement simple persona query functionality - [ ] Set up basic context management for LLM queries (using LangChain) - [ ] Implement memory-based query processing ### 4.4 API Documentation - [ ] Set up Swagger/OpenAPI documentation - [ ] Document essential endpoints with examples ## Phase 5: Django Client UI Implementation ### 5.1 Memory Management UI - [ ] Create simple memory creation form - [ ] Create basic memory listing page - [ ] Create simple memory editing form - [ ] Create memory deletion functionality - [ ] Implement basic permission toggle ### 5.2 Search and Query UI - [ ] Implement basic search form - [ ] Create simple persona query form - [ ] Implement basic results display ### 5.3 API Integration - [ ] Create MCP server API client - [ ] Implement basic error handling for API calls ## Phase 6: Testing and Deployment ### 6.1 MCP Server Tests - [ ] Write basic tests for critical endpoints - [ ] Test OAuth flow - [ ] Test core memory operations ### 6.2 Django Client Tests - [ ] Test OAuth client integration - [ ] Test basic memory operations ### 6.3 End-to-End Tests - [ ] Create simple test script for system testing - [ ] Test OAuth flow from client to server - [ ] Test memory creation, retrieval, and querying ### 6.4 Local Deployment with Docker - [ ] Finalize docker-compose.yml - [ ] Test full system with Docker Compose ### 6.5 Render Deployment - [ ] Create render.yaml blueprint - [ ] Configure environment variables for Render - [ ] Deploy MCP server to Render - [ ] Deploy Django client to Render - [ ] Test deployed application ## Implementation Timeline | Phase | Estimated AI Prompting Sessions | Dependencies | |-------|-------------------|--------------| | Phase 1: Project Setup | 2-3 prompts | None | | Phase 2: Initial Server and Client | 2-3 prompts | Phase 1 | | Phase 3: OAuth 2.0 Implementation | 3-5 prompts | Phase 2 | | Phase 4: Memory Management and Search | 2-3 prompts | Phase 3 | | Phase 5: Django Client UI | 2-3 prompts | Phase 4 | | Phase 6: Testing and Deployment | 2-3 prompts | Phase 5 | ## Getting Started ### Local Development 1. Clone the repository: ```bash git clone https://github.com/yourusername/picard_mcp.git cd picard_mcp ``` 2. Create `.env` files for both MCP server and Django client by copying from the example files: ```bash # Copy example files to create your actual .env files cp .env.example .env cp django_client/.env.example django_client/.env ``` Then edit the `.env` files to add your actual sensitive values. The example files contain placeholders for all required environment variables: - `.env.example` - For the MCP server environment variables - `django_client/.env.example` - For the Django client environment variables These example files are committed to the repository for documentation, while the actual `.env` files are excluded via `.gitignore` to keep sensitive information out of version control. 3. Start the services using Docker Compose: ```bash docker-compose up -d ``` 4. Register the Django client with the MCP server: ```bash docker exec picard_mcp-django_client-1 python register_oauth_client.py ``` 5. Access the Django client at http://localhost:8000 and the MCP server at http://localhost:8001 ### Render Deployment 1. Create a `render.yaml` file in the project root: ```yaml services: # MCP Server service - type: web name: picard-mcp-server env: python plan: starter buildCommand: cd mcp_server && pip install -r requirements.txt startCommand: cd mcp_server && uvicorn app.main:app --host 0.0.0.0 --port $PORT envVars: - key: POSTGRES_USER sync: false - key: POSTGRES_PASSWORD sync: false - key: POSTGRES_DB value: picard_mcp - key: POSTGRES_HOST fromService: type: pserv name: picard-mcp-db property: host - key: POSTGRES_PORT value: 5432 - key: MCP_SERVER_NAME value: Picard MCP - key: MCP_SERVER_HOST value: 0.0.0.0 - key: MCP_SERVER_PORT value: 8000 - key: MCP_ISSUER_URL fromService: type: web name: picard-mcp-server property: url - key: OAUTH_CLIENT_ID generateValue: true - key: OAUTH_CLIENT_SECRET generateValue: true - key: OAUTH_REDIRECT_URI fromService: type: web name: picard-mcp-client property: url suffix: /oauth/callback - key: OPENAI_API_KEY sync: false healthCheckPath: /health autoDeploy: false # Django Client service - type: web name: picard-mcp-client env: python plan: starter buildCommand: cd django_client && pip install -r requirements.txt && python manage.py migrate startCommand: cd django_client && gunicorn django_client.wsgi:application envVars: - key: DEBUG value: false - key: DJANGO_SECRET_KEY generateValue: true - key: DB_NAME value: django_client - key: DB_USER sync: false - key: DB_PASSWORD sync: false - key: DB_HOST fromService: type: pserv name: picard-mcp-client-db property: host - key: DB_PORT value: 5432 - key: MCP_SERVER_URL fromService: type: web name: picard-mcp-server property: url - key: MCP_SERVER_INTERNAL_URL fromService: type: web name: picard-mcp-server property: url - key: OAUTH_CLIENT_ID fromService: type: web name: picard-mcp-server envVarKey: OAUTH_CLIENT_ID - key: OAUTH_CLIENT_SECRET fromService: type: web name: picard-mcp-server envVarKey: OAUTH_CLIENT_SECRET - key: OAUTH_REDIRECT_URI fromService: type: web name: picard-mcp-client property: url suffix: /oauth/callback - key: OAUTH_SCOPES value: memories:read memories:write healthCheckPath: /health autoDeploy: false # MCP Server Database - type: pserv name: picard-mcp-db env: docker plan: starter disk: name: data mountPath: /var/lib/postgresql/data sizeGB: 10 image: ankane/pgvector:latest envVars: - key: POSTGRES_USER sync: false - key: POSTGRES_PASSWORD sync: false - key: POSTGRES_DB value: picard_mcp # Django Client Database - type: pserv name: picard-mcp-client-db env: docker plan: starter disk: name: data mountPath: /var/lib/postgresql/data sizeGB: 10 image: postgres:14 envVars: - key: POSTGRES_USER sync: false - key: POSTGRES_PASSWORD sync: false - key: POSTGRES_DB value: django_client ``` 2. Create a Render account and connect it to your GitHub repository. 3. In the Render dashboard, click on "Blueprint" and select your repository. 4. Set the required environment variables that are marked with `sync: false`: - `POSTGRES_USER` - `POSTGRES_PASSWORD` - `OPENAI_API_KEY` - `DB_USER` - `DB_PASSWORD` 5. Deploy the blueprint, which will create all the services defined in the `render.yaml` file. 6. After deployment, run the client registration script: - Go to the "Shell" tab of the Django client service in the Render dashboard - Run: `python register_oauth_client.py` 7. Access your deployed applications using the URLs provided in the Render dashboard. ## Next Steps 1. ✅ Complete Client Registration (Phase 3.2) 2. ✅ Complete Authorization Flow (Phase 3.3) 3. ✅ Complete Token Management (Phase 3.4) 4. ✅ Implement Django OAuth Client (Phase 3.5) 5. Implement Permission System (Phase 3.6) 6. Implement Memory Management API (Phase 4.1)