Google Jules MCP

# Comprehensive Plan: Python Program for Google Jules Job Management ## Executive Summary This document provides a detailed, step-by-step plan for building a Python program that manages Google Jules jobs. The plan is divided into two main phases: 1. **MVP (Minimum Viable Product)**: A functional job management system with core features 2. **Production Enhancement**: Converting the MVP into a robust, production-ready application The program will interact with the existing Google Jules MCP server (TypeScript/Node.js) to manage Jules tasks, monitor their status, and perform operations like approving plans, sending messages, and committing branches. --- ## Part 1: MVP (Minimum Viable Product) Creation ### 1.1 MVP Objectives Create a functional Python program that can: - Connect to the Google Jules MCP server - List all Jules tasks with their current status - Get detailed information about specific tasks - Monitor task status changes - Send messages/instructions to Jules tasks - Approve execution plans - Resume paused tasks - Create new tasks programmatically ### 1.2 MVP Architecture Overview ``` ┌─────────────────────────────────────────────────────────┐ │ Python Job Manager │ │ ┌───────────────────────────────────────────────────┐ │ │ │ CLI Interface (argparse) │ │ │ └───────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────────────────────────────────────────────────┐ │ │ │ Job Manager Core (job_manager.py) │ │ │ │ - Task operations │ │ │ │ - Status monitoring │ │ │ │ - Message handling │ │ │ └───────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────────────────────────────────────────────────┐ │ │ │ MCP Client (mcp_client.py) │ │ │ │ - JSON-RPC communication │ │ │ │ - Tool invocation │ │ │ └───────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────────────────────────────────────────────────┐ │ │ │ Data Storage (storage.py) │ │ │ │ - Local JSON cache │ │ │ │ - Task history │ │ │ └───────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ Google Jules MCP Server (Node.js) │ │ - Browser automation (Playwright) │ │ - Jules.google.com interaction │ │ - Task management │ └─────────────────────────────────────────────────────────┘ ``` ### 1.3 MVP Technology Stack **Core Dependencies:** - Python 3.9+ - `subprocess` (built-in) - For MCP server communication - `json` (built-in) - For JSON-RPC and data handling - `argparse` (built-in) - For CLI interface - `pathlib` (built-in) - For file path handling - `typing` (built-in) - For type hints - `dataclasses` (built-in) - For data models **Optional Dependencies (for MVP):** - `rich` - For enhanced terminal output (optional but recommended) - `python-dotenv` - For environment variable management ### 1.4 MVP Project Structure ``` jules_job_manager/ ├── src/ │ ├── __init__.py │ ├── main.py # CLI entry point │ ├── mcp_client.py # MCP communication layer │ ├── job_manager.py # Core job management logic │ ├── models.py # Data models (Task, Status, etc.) │ └── storage.py # Local data persistence ├── tests/ │ ├── __init__.py │ ├── test_mcp_client.py │ └── test_job_manager.py ├── config/ │ └── config.json # Configuration file ├── data/ │ └── tasks.json # Local task cache ├── requirements.txt ├── setup.py ├── .env.example └── README.md ``` ### 1.5 MVP Data Models **Task Model** (`models.py`): ```python from dataclasses import dataclass from typing import List, Optional from datetime import datetime from enum import Enum class TaskStatus(Enum): PENDING = "pending" IN_PROGRESS = "in_progress" COMPLETED = "completed" PAUSED = "paused" WAITING_APPROVAL = "waiting_approval" @dataclass class SourceFile: filename: str url: str status: str # 'modified', 'created', 'deleted', 'unchanged' @dataclass class ChatMessage: timestamp: str content: str type: str # 'user', 'jules', 'system' @dataclass class JulesTask: id: str title: str description: str repository: str branch: str status: TaskStatus created_at: datetime updated_at: datetime url: str chat_history: List[ChatMessage] source_files: List[SourceFile] ``` ### 1.6 MVP Core Components #### 1.6.1 MCP Client (`mcp_client.py`) **Purpose**: Handle JSON-RPC communication with the MCP server **Key Functions**: ```python class MCPClient: def __init__(self, server_path: str): """Initialize MCP client with path to Node.js server""" def call_tool(self, tool_name: str, arguments: dict) -> dict: """Call an MCP tool and return the response""" def list_tools(self) -> List[dict]: """Get list of available tools from MCP server""" def _send_request(self, method: str, params: dict) -> dict: """Send JSON-RPC request to MCP server""" ``` **Communication Method**: - Use `subprocess.Popen` to start the Node.js MCP server - Communicate via stdin/stdout using JSON-RPC 2.0 protocol - Handle request/response correlation using request IDs #### 1.6.2 Job Manager (`job_manager.py`) **Purpose**: High-level job management operations **Key Functions**: ```python class JobManager: def __init__(self, mcp_client: MCPClient, storage: Storage): """Initialize job manager with MCP client and storage""" def list_tasks(self, status_filter: Optional[str] = None) -> List[JulesTask]: """List all tasks, optionally filtered by status""" def get_task(self, task_id: str) -> JulesTask: """Get detailed information about a specific task""" def create_task(self, description: str, repository: str, branch: str = "main") -> JulesTask: """Create a new Jules task""" def send_message(self, task_id: str, message: str) -> bool: """Send a message to a Jules task""" def approve_plan(self, task_id: str) -> bool: """Approve the execution plan for a task""" def resume_task(self, task_id: str) -> bool: """Resume a paused task""" def monitor_task(self, task_id: str, interval: int = 30) -> None: """Monitor a task and print status updates""" ``` #### 1.6.3 Storage (`storage.py`) **Purpose**: Local data persistence and caching **Key Functions**: ```python class Storage: def __init__(self, data_path: str): """Initialize storage with path to data file""" def save_task(self, task: JulesTask) -> None: """Save or update a task in local storage""" def get_task(self, task_id: str) -> Optional[JulesTask]: """Retrieve a task from local storage""" def list_tasks(self) -> List[JulesTask]: """List all tasks from local storage""" def delete_task(self, task_id: str) -> bool: """Delete a task from local storage""" ``` ### 1.7 MVP CLI Interface **Command Structure**: ```bash # List all tasks python -m jules_job_manager list [--status pending|in_progress|completed|paused] # Get task details python -m jules_job_manager get <task_id> # Create a new task python -m jules_job_manager create --repo owner/repo --description "Task description" [--branch main] # Send message to task python -m jules_job_manager message <task_id> "Your message here" # Approve task plan python -m jules_job_manager approve <task_id> # Resume paused task python -m jules_job_manager resume <task_id> # Monitor task status python -m jules_job_manager monitor <task_id> [--interval 30] ``` ### 1.8 MVP Configuration **Configuration File** (`config/config.json`): ```json { "mcp_server": { "path": "../google-jules-mcp/dist/index.js", "node_path": "node" }, "storage": { "data_path": "./data/tasks.json" }, "monitoring": { "default_interval": 30, "max_retries": 3 } } ``` **Environment Variables** (`.env`): ```bash # MCP Server Configuration MCP_SERVER_PATH=../google-jules-mcp/dist/index.js NODE_PATH=node # Jules MCP Configuration (passed to Node.js server) HEADLESS=true TIMEOUT=30000 DEBUG=false SESSION_MODE=cookies GOOGLE_AUTH_COOKIES="your_cookies_here" JULES_DATA_PATH=~/.jules-mcp/data.json ``` ### 1.9 MVP Error Handling Strategy **Error Categories**: 1. **Connection Errors**: MCP server not running or unreachable 2. **Authentication Errors**: Jules authentication failed 3. **Task Errors**: Task not found, invalid task ID 4. **Operation Errors**: Operation failed (approve, resume, etc.) **Handling Approach**: - Use try-except blocks for all external operations - Log errors to stderr - Return meaningful error messages to user - Exit with appropriate exit codes (0=success, 1=error) ### 1.10 MVP Development Timeline **Phase 1: Foundation (Days 1-2)** - Set up project structure - Implement data models - Create basic MCP client with JSON-RPC communication - Test connection to MCP server **Phase 2: Core Features (Days 3-4)** - Implement JobManager class - Add list_tasks, get_task, create_task functions - Implement local storage - Basic error handling **Phase 3: Advanced Operations (Days 5-6)** - Add send_message, approve_plan, resume_task - Implement task monitoring - Enhanced error handling **Phase 4: CLI Interface (Day 7)** - Build argparse-based CLI - Add all commands - User-friendly output formatting **Phase 5: Testing & Documentation (Day 8)** - Write unit tests - Create README with usage examples - Test end-to-end workflows **Total MVP Timeline: 8 days** --- ## Part 2: Converting MVP to Robust Production Program ### 2.1 Production Enhancement Objectives Transform the MVP into a production-ready application with: - Comprehensive error handling and recovery - Advanced logging and monitoring - Configuration management - Performance optimization - Security enhancements - Extensive testing - Documentation - Deployment automation ### 2.2 Enhanced Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Production Job Manager │ │ ┌───────────────────────────────────────────────────────────┐ │ │ │ CLI + Web API (FastAPI) │ │ │ └───────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────────────────────────────────────────────────────────┐ │ │ │ Job Manager Core + Async Operations │ │ │ └───────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌──────────────┬──────────────┬──────────────┬──────────────┐ │ │ │ MCP Client │ Retry Logic │ Logging │ Metrics │ │ │ └──────────────┴──────────────┴──────────────┴──────────────┘ │ │ │ │ │ ┌──────────────┬──────────────┬──────────────┐ │ │ │ PostgreSQL │ Redis │ File Cache │ │ │ └──────────────┴──────────────┴──────────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` ### 2.3 Production Technology Stack **Additional Dependencies**: - `asyncio` - Asynchronous operations - `aiohttp` - Async HTTP client - `fastapi` - Web API framework - `uvicorn` - ASGI server - `pydantic` - Data validation - `sqlalchemy` - Database ORM - `alembic` - Database migrations - `redis` - Caching and pub/sub - `prometheus-client` - Metrics collection - `structlog` - Structured logging - `tenacity` - Retry logic - `pytest` - Testing framework - `pytest-asyncio` - Async testing - `pytest-cov` - Code coverage - `black` - Code formatting - `mypy` - Type checking - `ruff` - Linting ### 2.4 Enhanced Features #### 2.4.1 Comprehensive Error Handling **Custom Exception Hierarchy**: ```python class JulesManagerError(Exception): """Base exception for Jules Manager""" class MCPConnectionError(JulesManagerError): """MCP server connection failed""" class AuthenticationError(JulesManagerError): """Jules authentication failed""" class TaskNotFoundError(JulesManagerError): """Task not found""" class OperationFailedError(JulesManagerError): """Operation failed""" ``` **Retry Logic**: - Use `tenacity` library for automatic retries - Exponential backoff for transient failures - Circuit breaker pattern for persistent failures - Configurable retry policies per operation #### 2.4.2 Advanced Logging **Structured Logging with `structlog`**: ```python import structlog logger = structlog.get_logger() logger.info("task_created", task_id=task.id, repository=task.repository, branch=task.branch) ``` **Log Levels**: - DEBUG: Detailed diagnostic information - INFO: General informational messages - WARNING: Warning messages for recoverable issues - ERROR: Error messages for failures - CRITICAL: Critical failures requiring immediate attention **Log Destinations**: - Console (stdout/stderr) - File rotation (daily/size-based) - Centralized logging (e.g., ELK stack, CloudWatch) #### 2.4.3 Configuration Management **Hierarchical Configuration**: 1. Default configuration (hardcoded) 2. Configuration file (config.yaml) 3. Environment variables 4. Command-line arguments **Configuration Schema** (`config.yaml`): ```yaml mcp_server: path: ../google-jules-mcp/dist/index.js node_path: node startup_timeout: 30 health_check_interval: 60 storage: backend: postgresql # or 'json', 'sqlite' connection_string: postgresql://user:pass@localhost/jules cache_backend: redis redis_url: redis://localhost:6379/0 monitoring: default_interval: 30 max_retries: 3 retry_backoff: exponential circuit_breaker_threshold: 5 logging: level: INFO format: json file: logs/jules_manager.log rotation: daily retention_days: 30 api: enabled: true host: 0.0.0.0 port: 8000 cors_origins: ["*"] metrics: enabled: true port: 9090 ``` #### 2.4.4 Database Integration **Schema Design** (PostgreSQL): ```sql CREATE TABLE tasks ( id VARCHAR(255) PRIMARY KEY, title TEXT NOT NULL, description TEXT NOT NULL, repository VARCHAR(255) NOT NULL, branch VARCHAR(255) NOT NULL, status VARCHAR(50) NOT NULL, created_at TIMESTAMP NOT NULL, updated_at TIMESTAMP NOT NULL, url TEXT NOT NULL, metadata JSONB ); CREATE TABLE chat_messages ( id SERIAL PRIMARY KEY, task_id VARCHAR(255) REFERENCES tasks(id), timestamp TIMESTAMP NOT NULL, content TEXT NOT NULL, type VARCHAR(50) NOT NULL ); CREATE TABLE source_files ( id SERIAL PRIMARY KEY, task_id VARCHAR(255) REFERENCES tasks(id), filename TEXT NOT NULL, url TEXT NOT NULL, status VARCHAR(50) NOT NULL ); CREATE TABLE task_events ( id SERIAL PRIMARY KEY, task_id VARCHAR(255) REFERENCES tasks(id), event_type VARCHAR(100) NOT NULL, event_data JSONB, timestamp TIMESTAMP NOT NULL ); CREATE INDEX idx_tasks_status ON tasks(status); CREATE INDEX idx_tasks_repository ON tasks(repository); CREATE INDEX idx_task_events_task_id ON task_events(task_id); ``` #### 2.4.5 Caching Strategy **Redis Caching**: - Cache task details (TTL: 5 minutes) - Cache task lists (TTL: 1 minute) - Invalidate cache on task updates - Use Redis pub/sub for real-time updates #### 2.4.6 Web API (FastAPI) **API Endpoints**: ```python # Task Management GET /api/v1/tasks # List tasks GET /api/v1/tasks/{task_id} # Get task details POST /api/v1/tasks # Create task PUT /api/v1/tasks/{task_id} # Update task DELETE /api/v1/tasks/{task_id} # Delete task # Task Operations POST /api/v1/tasks/{task_id}/message # Send message POST /api/v1/tasks/{task_id}/approve # Approve plan POST /api/v1/tasks/{task_id}/resume # Resume task # Monitoring GET /api/v1/tasks/{task_id}/status # Get status GET /api/v1/tasks/{task_id}/events # Get event history # Health & Metrics GET /health # Health check GET /metrics # Prometheus metrics ``` ### 2.5 Testing Strategy #### 2.5.1 Unit Tests - Test individual functions and methods - Mock external dependencies (MCP server, database) - Aim for 80%+ code coverage #### 2.5.2 Integration Tests - Test interaction with MCP server - Test database operations - Test API endpoints #### 2.5.3 End-to-End Tests - Test complete workflows - Test error scenarios - Test concurrent operations #### 2.5.4 Performance Tests - Load testing with multiple concurrent tasks - Stress testing with high request rates - Memory leak detection ### 2.6 Security Enhancements **Authentication & Authorization**: - API key authentication for web API - Role-based access control (RBAC) - Secure storage of credentials (environment variables, secrets manager) **Data Security**: - Encrypt sensitive data at rest - Use HTTPS for API communication - Sanitize user inputs - Implement rate limiting **Audit Logging**: - Log all operations with user context - Track access to sensitive data - Maintain audit trail for compliance ### 2.7 Monitoring & Observability **Metrics Collection** (Prometheus): ```python from prometheus_client import Counter, Histogram, Gauge # Counters tasks_created = Counter('jules_tasks_created_total', 'Total tasks created') tasks_completed = Counter('jules_tasks_completed_total', 'Total tasks completed') errors_total = Counter('jules_errors_total', 'Total errors', ['error_type']) # Histograms task_duration = Histogram('jules_task_duration_seconds', 'Task duration') api_request_duration = Histogram('jules_api_request_duration_seconds', 'API request duration') # Gauges active_tasks = Gauge('jules_active_tasks', 'Number of active tasks') ``` **Health Checks**: - MCP server connectivity - Database connectivity - Redis connectivity - Disk space availability **Alerting**: - Alert on high error rates - Alert on task failures - Alert on service unavailability - Integration with PagerDuty, Slack, etc. ### 2.8 Deployment **Containerization** (Docker): ```dockerfile FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY src/ ./src/ COPY config/ ./config/ CMD ["python", "-m", "jules_job_manager"] ``` **Orchestration** (Docker Compose): ```yaml version: '3.8' services: jules-manager: build: . ports: - "8000:8000" - "9090:9090" environment: - DATABASE_URL=postgresql://postgres:password@db:5432/jules - REDIS_URL=redis://redis:6379/0 depends_on: - db - redis db: image: postgres:15 environment: - POSTGRES_DB=jules - POSTGRES_PASSWORD=password volumes: - postgres_data:/var/lib/postgresql/data redis: image: redis:7-alpine volumes: - redis_data:/data volumes: postgres_data: redis_data: ``` ### 2.9 Documentation **User Documentation**: - Installation guide - Configuration guide - Usage examples - API reference - Troubleshooting guide **Developer Documentation**: - Architecture overview - Code structure - Contributing guidelines - Development setup - Testing guide **API Documentation**: - OpenAPI/Swagger specification - Interactive API documentation (FastAPI auto-generated) - Code examples in multiple languages ### 2.10 Production Timeline **Phase 1: Error Handling & Logging (Week 1)** - Implement custom exceptions - Add retry logic with tenacity - Set up structured logging - Add comprehensive error messages **Phase 2: Database Integration (Week 2)** - Design database schema - Implement SQLAlchemy models - Create migration scripts - Add database operations **Phase 3: Caching & Performance (Week 3)** - Integrate Redis caching - Implement cache invalidation - Optimize database queries - Add connection pooling **Phase 4: Web API (Week 4)** - Build FastAPI application - Implement all endpoints - Add authentication - Create API documentation **Phase 5: Testing (Week 5)** - Write unit tests - Write integration tests - Write end-to-end tests - Set up CI/CD pipeline **Phase 6: Monitoring & Metrics (Week 6)** - Implement Prometheus metrics - Add health checks - Set up alerting - Create dashboards (Grafana) **Phase 7: Security & Compliance (Week 7)** - Implement authentication/authorization - Add audit logging - Security audit - Penetration testing **Phase 8: Deployment & Documentation (Week 8)** - Create Docker images - Set up orchestration - Write documentation - Production deployment **Total Production Timeline: 8 weeks** --- ## Appendix A: Key Challenges & Mitigation Strategies ### Challenge 1: MCP Server Communication **Issue**: Reliable communication with Node.js MCP server via stdin/stdout **Mitigation**: - Implement robust JSON-RPC client with request/response correlation - Add connection health checks - Implement automatic reconnection logic ### Challenge 2: Asynchronous Task Monitoring **Issue**: Jules tasks run asynchronously and may take hours **Mitigation**: - Implement polling-based monitoring with configurable intervals - Use webhooks if available in future Jules versions - Store task state locally for offline access ### Challenge 3: Authentication Management **Issue**: Google authentication cookies may expire **Mitigation**: - Implement cookie refresh mechanism - Detect authentication failures and prompt for re-authentication - Support multiple authentication modes (cookies, Browserbase, etc.) ### Challenge 4: Concurrent Operations **Issue**: Multiple users/processes accessing the same tasks **Mitigation**: - Use database transactions for consistency - Implement optimistic locking - Add distributed locks (Redis) for critical operations ### Challenge 5: Error Recovery **Issue**: Handling partial failures and maintaining consistency **Mitigation**: - Implement idempotent operations - Use event sourcing for audit trail - Add rollback mechanisms for failed operations --- ## Appendix B: Example Usage Scenarios ### Scenario 1: Automated Bug Fix Workflow ```bash # Create task for bug fix task_id=$(python -m jules_job_manager create \ --repo mycompany/webapp \ --description "Fix login bug with special characters" \ --branch main) # Monitor task until plan is ready python -m jules_job_manager monitor $task_id --until waiting_approval # Approve the plan python -m jules_job_manager approve $task_id # Monitor until completion python -m jules_job_manager monitor $task_id --until completed # Get final status python -m jules_job_manager get $task_id ``` ### Scenario 2: Batch Task Creation ```python # Python script for batch task creation from jules_job_manager import JobManager manager = JobManager() tasks = [ {"repo": "mycompany/frontend", "description": "Add dark mode support"}, {"repo": "mycompany/backend", "description": "Optimize database queries"}, {"repo": "mycompany/api", "description": "Add rate limiting"}, ] for task_data in tasks: task = manager.create_task( repository=task_data["repo"], description=task_data["description"] ) print(f"Created task {task.id} for {task.repository}") ``` ### Scenario 3: Status Dashboard ```python # Display real-time status of all tasks from jules_job_manager import JobManager from rich.table import Table from rich.console import Console manager = JobManager() console = Console() while True: tasks = manager.list_tasks() table = Table(title="Jules Tasks Status") table.add_column("ID", style="cyan") table.add_column("Repository", style="magenta") table.add_column("Status", style="green") table.add_column("Updated", style="yellow") for task in tasks: table.add_row( task.id[:8], task.repository, task.status.value, task.updated_at.strftime("%Y-%m-%d %H:%M") ) console.clear() console.print(table) time.sleep(30) ``` --- ## Appendix C: Configuration Examples ### Development Configuration ```yaml # config/development.yaml mcp_server: path: ../google-jules-mcp/dist/index.js node_path: node storage: backend: json data_path: ./data/tasks.json logging: level: DEBUG format: text api: enabled: true host: localhost port: 8000 ``` ### Production Configuration ```yaml # config/production.yaml mcp_server: path: /opt/jules-mcp/dist/index.js node_path: /usr/bin/node startup_timeout: 60 storage: backend: postgresql connection_string: ${DATABASE_URL} cache_backend: redis redis_url: ${REDIS_URL} logging: level: INFO format: json file: /var/log/jules/manager.log api: enabled: true host: 0.0.0.0 port: 8000 cors_origins: ["https://dashboard.example.com"] metrics: enabled: true port: 9090 ``` --- ## Conclusion This comprehensive plan provides a clear roadmap for building a Python program to manage Google Jules jobs, starting with a functional MVP and evolving into a robust production system. The MVP can be completed in approximately 8 days, while the full production enhancement requires an additional 8 weeks. The modular architecture allows for incremental development and deployment, ensuring that each phase delivers value while building toward the final production-ready system.