Taiwan Stock Agent

# TODO: tw-stock-agent MCP Server Improvements ## Executive Summary This TODO list outlines the roadmap for transforming the tw-stock-agent into a production-ready MCP server. The project currently has architectural inconsistencies with dual FastAPI/FastMCP implementations and lacks several critical production features. **Key Goals:** - Consolidate to pure FastMCP architecture - Implement comprehensive error handling and validation - Add database persistence and monitoring - Achieve production-ready quality standards ## Priority 0: Critical Architecture Fixes (Week 1) ### P0.1 Consolidate MCP Server Architecture - [ ] **Remove FastAPI Implementation** - Delete `tw_stock_agent/main.py` FastAPI server - Consolidate all endpoints into `mcp_server.py` using FastMCP - Remove FastAPI dependencies from `pyproject.toml` - **Risk**: Breaking changes for any existing FastAPI consumers - **Mitigation**: Document migration path, provide compatibility layer if needed - **Acceptance Criteria**: Single MCP server runs with `uv run python mcp_server.py` - [ ] **Fix Parameter Naming Consistency** - Standardize on `stock_code` across all tools (not `stock_id`) - Update tool schemas and documentation - **Acceptance Criteria**: All tools use consistent parameter names - [ ] **Implement Async/Await Throughout** - Convert all tool functions to async - Make `StockService` methods async with proper I/O handling - Use `aiohttp` instead of `requests` for external API calls - **Acceptance Criteria**: No blocking I/O operations in main event loop ### P0.2 Basic MCP Compliance Fixes - [ ] **Implement Proper Tool Schemas** - Add input validation using Pydantic models - Define proper JSON schemas for all tools - **Acceptance Criteria**: All tools have valid JSON schemas and input validation - [ ] **Fix Async Context Management** - Implement proper lifespan context for resource initialization - Add graceful shutdown handling - **Acceptance Criteria**: Server starts/stops cleanly with proper resource management ## Priority 1: Production Essentials (Weeks 2-5) ### P1.1 Database & Persistence - [ ] **Add Database Layer** ```sql -- Suggested schema CREATE TABLE stock_cache ( stock_code VARCHAR(10) PRIMARY KEY, data_type VARCHAR(50), data JSONB, expires_at TIMESTAMP, created_at TIMESTAMP DEFAULT NOW() ); CREATE TABLE api_requests ( id SERIAL PRIMARY KEY, endpoint VARCHAR(255), stock_code VARCHAR(10), timestamp TIMESTAMP DEFAULT NOW(), success BOOLEAN, response_time_ms INTEGER ); ``` - Implement SQLite for development, PostgreSQL for production - Add database migration scripts - **Acceptance Criteria**: Persistent caching with configurable TTL - [ ] **Implement Structured Output** ```python class StockDataResponse(BaseModel): stock_code: str company_name: str industry: str market_cap: Optional[float] current_price: float change_percent: float updated_at: datetime ``` - Create Pydantic models for all response types - Support both structured and unstructured output - **Acceptance Criteria**: All tools return typed, validated responses ### P1.2 Comprehensive Error Handling - [ ] **Custom Exception Classes** ```python class StockNotFoundError(Exception): pass class APIRateLimitError(Exception): pass class DataSourceUnavailableError(Exception): pass ``` - Implement centralized error handling - Return proper MCP error responses - **Acceptance Criteria**: All error conditions handled gracefully with meaningful messages - [ ] **Input Validation & Security** - Validate stock codes (4-6 digit format) - Sanitize all string inputs - Implement parameter length limits - **Acceptance Criteria**: No invalid inputs crash the server ### P1.3 Logging & Monitoring - [ ] **Structured Logging Implementation** ```python logger.info("Stock data fetched", extra={ "stock_code": stock_code, "response_time_ms": response_time, "cache_hit": cache_hit }) ``` - Use JSON-formatted logs - Add request tracing with correlation IDs - **Acceptance Criteria**: Comprehensive logging for debugging and monitoring - [ ] **Health Checks & Metrics** - Add `/health` endpoint - Implement basic metrics collection - Monitor external API response times - **Acceptance Criteria**: Observable service health and performance ### P1.4 Resource Implementation - [ ] **Dynamic Resource Discovery** ```python @mcp.resource("stock://info/{stock_code}") @mcp.resource("stock://price/{stock_code}") @mcp.resource("stock://realtime/{stock_code}") ``` - Implement resource templates - Add completion support for stock codes - **Acceptance Criteria**: MCP clients can discover and access stock resources ## Priority 2: Quality & Enhancement (Weeks 6-9) ### P2.1 Testing Infrastructure - [ ] **Comprehensive Unit Tests** - Test all tools with mocked dependencies - Achieve >90% test coverage - Use `anyio` for async testing - **Acceptance Criteria**: Full test suite passes with high coverage - [ ] **Integration Testing** - Test complete MCP server lifecycle - Mock external APIs for reliable testing - **Acceptance Criteria**: End-to-end tests verify MCP protocol compliance ### P2.2 Performance Optimization - [ ] **Connection Pooling** - Implement `aiohttp` session pooling - Add background data refresh jobs - Optimize cache key strategies - **Acceptance Criteria**: Improved response times and resource utilization - [ ] **Rate Limiting Integration** - Connect existing `RateLimiter` to MCP server - Add per-client rate limiting - **Acceptance Criteria**: Respectful API usage with proper backoff ### P2.3 Documentation & Developer Experience - [ ] **Comprehensive Documentation** - Add detailed README with MCP integration examples - Document all tool parameters and return values - Create API reference documentation - **Acceptance Criteria**: New developers can understand and use the server easily - [ ] **Development Tools** - Create development server script - Add data seeding scripts for testing - Implement mock data providers - **Acceptance Criteria**: Streamlined development workflow ## Priority 3: Advanced Features (Weeks 10-14) ### P3.1 Authentication & Security - [ ] **OAuth 2.1 Implementation** - Add `TokenVerifier` for protected resources - Support both public and authenticated endpoints - **Acceptance Criteria**: Secure access to premium features - [ ] **Advanced Security Features** - Implement request signing - Add audit logging - Support API key authentication - **Acceptance Criteria**: Enterprise-ready security features ### P3.2 Advanced MCP Features - [ ] **Prompts Implementation** ```python @mcp.prompt() def analyze_stock(stock_code: str, analysis_type: str = "technical") -> str: """Generate stock analysis prompt""" return f"Analyze {stock_code} using {analysis_type} analysis..." ``` - Create interactive prompt templates - Add parameter completion - **Acceptance Criteria**: Rich prompt-based interactions - [ ] **Sampling & LLM Integration** - Add market commentary generation - Implement narrative analysis - **Acceptance Criteria**: AI-powered stock insights ### P3.3 Multi-Source Data Integration - [ ] **Additional Data Providers** - Integrate multiple Taiwan stock exchanges - Add economic indicators - Support cryptocurrency data - **Acceptance Criteria**: Comprehensive financial data coverage ## Implementation Guidelines ### Development Workflow 1. **Branch Strategy**: Feature branches with PR reviews 2. **Code Quality**: Pre-commit hooks with `ruff` and `mypy` 3. **Testing**: TDD approach with automated testing 4. **Documentation**: Update docs with each feature ### Technology Stack Recommendations - **FastMCP**: Primary MCP server framework - **Pydantic**: Data validation and serialization - **SQLAlchemy + Alembic**: Database ORM and migrations - **aiohttp**: Async HTTP client - **pytest + anyio**: Testing framework - **structlog**: Structured logging ### Success Metrics - [ ] Server starts without errors - [ ] All tools return valid responses - [ ] >90% test coverage maintained - [ ] <500ms average response time - [ ] Zero security vulnerabilities - [ ] Complete MCP protocol compliance ## Risk Assessment & Mitigation ### High Risk Items 1. **FastAPI to FastMCP Migration** - Risk: Breaking existing integrations - Mitigation: Phased migration with compatibility testing 2. **Database Schema Changes** - Risk: Data loss during migration - Mitigation: Backup procedures and rollback plans 3. **External API Dependencies** - Risk: Rate limiting and availability - Mitigation: Robust caching and fallback mechanisms ### Medium Risk Items 1. **Performance Optimization** - Risk: Premature optimization complexity - Mitigation: Profile before optimizing, measure improvements 2. **Authentication Implementation** - Risk: Security vulnerabilities - Mitigation: Use proven OAuth libraries, security audits ## Timeline Estimate **Weeks 1-2**: P0 Critical fixes (2 developers) **Weeks 3-6**: P1 Production essentials (2-3 developers) **Weeks 7-10**: P2 Quality & enhancement (2 developers) **Weeks 11-14**: P3 Advanced features (1-2 developers) **Total Effort**: ~20-30 developer weeks for complete implementation ## Conclusion This roadmap prioritizes architectural consistency and production readiness while maintaining a clear path toward advanced MCP features. The consolidation to FastMCP in P0 is critical as it unblocks effective development of all subsequent features. Regular checkpoints should assess progress against success metrics and adjust priorities based on business requirements and technical discoveries during implementation.