Nara Market FastMCP Server

# CLAUDE.md - Developer Guide **AI-Assisted Development Guide for Nara Market FastMCP Server** This document provides comprehensive technical guidance for Claude Code when working with this Korean government procurement data collection server. ## 🎯 Project Context **Primary Function:** Large-scale, memory-safe collection of Korean government procurement (G2B/Nara Market) data **Architecture:** Dual-server design (FastMCP + FastAPI) with window-based resumable crawling **Key Innovation:** Direct-to-disk storage preventing LLM context overflow ## 🚀 Quick Development Commands ### Setup & Run ```bash # Development setup pip install -r requirements.txt echo "NARAMARKET_SERVICE_KEY=your_key" > .env python src/main.py # Package installation pip install -e ".[dev]" naramarket-mcp # HTTP server mode uvicorn src.api.app:app --reload ``` ### Testing & Quality ```bash # Run tests pytest pytest tests/test_api.py -v pytest --cov=src --cov-report=html # Type checking mypy src/ --ignore-missing-imports ``` ### Docker Operations ```bash # Quick build & run docker build -t naramarket-mcp . docker run --rm -e NARAMARKET_SERVICE_KEY=key naramarket-mcp # Production deployment docker build --target production -t naramarket-prod . ``` ## 🏗 Technical Architecture ### Core Design Patterns **Dual Server Architecture:** - `src/main.py` → FastMCP server (AI tool integration) - `src/api/app.py` → FastAPI server (HTTP/REST interface) **Memory-Safe Processing:** - Never return large datasets to MCP context - Direct CSV/Parquet writes bypass memory - Streaming NDJSON for intermediate storage **Window-Based Collection:** ```python # Resumable pattern result = crawl_to_csv(category="computers", total_days=365, max_windows_per_call=2) while result["incomplete"]: result = crawl_to_csv( total_days=result["remaining_days"], anchor_end_date=result["next_anchor_end_date"], append=True ) ``` ### Key Module Organization ```text src/core/ # Infrastructure (client, config, models) src/services/ # Business logic (crawler, file_processor) src/tools/ # MCP tool wrappers src/api/ # HTTP endpoints ``` ## ⚠️ Critical Implementation Guidelines ### Memory Safety Rules - **NEVER** return large `products` arrays to MCP context - Use `crawl_to_csv` for production data collection (returns metadata only) - Stream large datasets directly to disk (CSV/Parquet) ### Context Protection (컨텍스트 보호) - Remote Server Optimized - **자동 응답 크기 제한**: 50,000자 초과 시 자동 압축 적용 - **핵심 필드 추출**: 서비스별 중요 필드만 선별 반환 (입찰번호, 계약금액 등) - **아이템 수 제한**: 기본 5개 아이템으로 제한 (컨텍스트 윈도우 보호) - **페이징 가이드**: 대용량 데이터 탐색을 위한 자동 페이징 안내 제공 - **리모트 서버 최적화**: 파일 저장 없이 효율적인 데이터 접근 방법 제공 ### Configuration Constants ```python MAX_RETRIES = 3 # API retry attempts DEFAULT_DELAY_SEC = 0.1 # Request throttling TIMEOUT_LIST = 20 # List API timeout TIMEOUT_DETAIL = 15 # Detail API timeout ``` ### Error Handling Strategy 1. Network errors → Retry with backoff 2. API errors → Log and continue 3. Data errors → Track in counters 4. Critical errors → Raise with partial results ## 🔧 Development Workflow ### Adding New MCP Tools 1. Implement business logic in `src/services/` 2. Create MCP wrapper in `src/tools/naramarket.py` 3. Register with `@mcp.tool()` decorator 4. Add corresponding API endpoint (optional) ### Environment Variables ```bash # Required NARAMARKET_SERVICE_KEY=your_api_key # Optional FASTMCP_TRANSPORT=stdio # or sse, http LOG_LEVEL=INFO ``` ### Key API Endpoints (FastAPI mode) - `GET /api/v1/health` - Health check - `POST /api/v1/crawl/list` - Product listings - `POST /api/v1/crawl/csv` - Large-scale export - `GET /api/v1/files` - File management ### Context-Protected MCP Tools Usage Examples (Remote Server) ```python # 기본 사용 (자동 컨텍스트 보호) call_public_data_standard_api( operation="getDataSetOpnStdBidPblancInfo", num_rows=5, # 작은 데이터셋 bid_notice_start_date="202401010000" ) # 대용량 데이터 탐색 (페이징 가이드 포함) call_api_with_pagination_support( service_type="procurement_statistics", operation="getTotlPubPrcrmntSttus", num_rows=10, # 적당한 크기 search_base_year="2024" ) # 데이터 탐색 전략 가이드 get_data_exploration_guide( service_type="shopping_mall", operation="getMASCntrctPrdctInfoList", expected_data_size="large" # 탐색 전략 제공 ) ``` ### Testing Strategy ```bash pytest tests/test_api.py -v # API validation pytest --cov=src --cov-report=html # Coverage report ``` ## 🤖 AI Assistant Guidelines ### For Claude Code Integration **Complex Task Handling:** - Create specialized subagents for multi-step modifications - Use parallel processing for independent tasks - Focus on: FastMCP upgrades, API integration, architecture, testing **Key Subagent Roles:** - `@agent-fastmcp-migration-expert` - FastMCP version upgrades - `@agent-architecture-refactoring-expert` - Memory optimization & structure - `@agent-core-function-optimizer` - API patterns & performance - `@agent-testing-validation-coordinator` - Test suites & validation ### Resource Management - Sync for MCP tools, Async for FastAPI - Monitor memory usage (2G limit) - Use streaming for large datasets ## 🔍 Troubleshooting ### Common Issues 1. **Service key error** → Set `NARAMARKET_SERVICE_KEY` 2. **Timeout errors** → Reduce `window_days` parameter 3. **Memory errors** → Use `crawl_to_csv` (not memory tools) 4. **Column mismatch** → Set `fail_on_new_columns=False` ### Debug Commands ```bash LOG_LEVEL=DEBUG python src/main.py python -c "from src.services.crawler import crawler_service; print(crawler_service.crawl_list('computers', days_back=1))" ``` ### Health Monitoring - MCP: `server_info` tool - HTTP: `/api/v1/health` endpoint - Logs: `data/logs/` directory