Nanonets MCP Server

# Nanonets MCP Server An MCP (Model Context Protocol) server that exposes Nanonets OCR functionality for converting images to structured markdown. ## Features - **Advanced OCR**: Convert documents to structured markdown using Nanonets-OCR-s (3.75B parameter model) - **Multi-format Support**: Handles images, PDFs, Word documents, and Excel spreadsheets - **Images**: PNG, JPEG, BMP, TIFF, WEBP - **Documents**: PDF, DOCX, XLSX - **PDF Processing**: Complete multi-page PDF document processing with page-by-page OCR - **Office Document Processing**: Direct text extraction from Word and Excel files - **Intelligent Recognition**: Detects and converts: - Text and paragraphs - Tables with structure preservation - LaTeX equations - Images with descriptions - Signatures and watermarks - Checkboxes - Complex layouts - Multi-page documents with proper page separation - Word document headings and formatting - Excel worksheets and data tables ## Installation ### Option 1: Docker (Recommended with GPU) ```bash # Clone the repository git clone <repository-url> cd nanonets_mcp # Build and run with Docker Compose (requires NVIDIA Docker runtime) docker-compose up --build ``` **Prerequisites for GPU support:** - NVIDIA GPU with CUDA support - [NVIDIA Docker runtime](https://github.com/NVIDIA/nvidia-docker) installed - Docker Compose v3.8+ ### Option 2: Local Installation ```bash # Clone the repository git clone <repository-url> cd nanonets_mcp # Install dependencies with uv uv pip install -e . ``` ## Usage ### Running the Server #### With Docker: ```bash # Start with Docker Compose docker-compose up # Or run directly with Docker docker run --gpus all -p 8000:8000 nanonets-mcp:latest ``` #### Local Installation: ```bash # Start the MCP server nanonets-mcp # Or run directly python -m nanonets_mcp.server ``` ### Available Tools #### `ocr_image_to_markdown` Convert an image to structured markdown format. **Parameters:** - `image_data` (string): Image data as base64 string, data URL, or file path - `image_format` (optional string): Format hint (png, jpg, etc.) **Returns:** Structured markdown representation of the document #### `ocr_pdf_to_markdown` Convert an entire PDF document to structured markdown format. **Parameters:** - `pdf_data` (string): PDF data as base64 string, data URL, or file path **Returns:** Structured markdown representation of the entire PDF document with page separators #### `process_word_to_markdown` Convert a Word document (.docx) to structured markdown format. **Parameters:** - `docx_data` (string): Word document data as base64 string, data URL, or file path **Returns:** Structured markdown representation of the Word document with headings and tables #### `process_excel_to_markdown` Convert an Excel file (.xlsx) to structured markdown format. **Parameters:** - `excel_data` (string): Excel file data as base64 string, data URL, or file path **Returns:** Structured markdown representation of all worksheets in the Excel workbook #### `get_supported_formats` Get information about supported formats and capabilities. **Returns:** Dictionary with supported formats, input methods, capabilities, and processing options ### Available Resources #### `nanonets://model-info` Provides detailed information about the Nanonets OCR model, including capabilities and specifications. ## Examples ### Basic OCR Usage #### Image Processing ```python # Using file path result = await ocr_image_to_markdown("/path/to/document.png") # Using base64 data with open("document.jpg", "rb") as f: image_b64 = base64.b64encode(f.read()).decode() result = await ocr_image_to_markdown(image_b64) # Using data URL data_url = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..." result = await ocr_image_to_markdown(data_url) ``` #### PDF Processing ```python # Process entire PDF document result = await ocr_pdf_to_markdown("/path/to/document.pdf") # Using base64 PDF data with open("document.pdf", "rb") as f: pdf_b64 = base64.b64encode(f.read()).decode() result = await ocr_pdf_to_markdown(pdf_b64) # Result includes all pages with separators # Example output: # # PDF Document # *Total pages: 3* # # --- # # Page 1 # [Content of page 1] # # --- # # Page 2 # [Content of page 2] # ... ``` #### Word Document Processing ```python # Process Word document result = await process_word_to_markdown("/path/to/document.docx") # Using base64 Word document data with open("document.docx", "rb") as f: docx_b64 = base64.b64encode(f.read()).decode() result = await process_word_to_markdown(docx_b64) # Result includes text, headings, and tables # Example output: # # Word Document # # # Main Title # # This is a paragraph of text. # # ## Section Header # # More content here. # # | Name | Age | City | # | --- | --- | --- | # | John | 30 | NYC | ``` #### Excel Spreadsheet Processing ```python # Process Excel file result = await process_excel_to_markdown("/path/to/spreadsheet.xlsx") # Using base64 Excel data with open("spreadsheet.xlsx", "rb") as f: excel_b64 = base64.b64encode(f.read()).decode() result = await process_excel_to_markdown(excel_b64) # Result includes all worksheets as tables # Example output: # # Excel Workbook # # ## Sheet: Employee Data # # | Name | Department | Salary | # | --- | --- | --- | # | Alice | Engineering | 75000 | # | Bob | Marketing | 65000 | # # ## Sheet: Financial Data # # | Quarter | Revenue | Expenses | # | --- | --- | --- | # | Q1 | 150000 | 120000 | ``` ### Integration with Claude Desktop Add to your Claude Desktop configuration: ```json { "mcpServers": { "nanonets-ocr": { "command": "nanonets-mcp" } } } ``` ## Model Information - **Model**: nanonets/Nanonets-OCR-s - **Parameters**: 3.75B (based on Qwen2.5-VL-3B-Instruct) - **Input**: Images up to 2048x2048 pixels (recommended) and PDF documents - **Output**: Structured markdown with semantic tagging - **PDF Processing**: 200 DPI conversion, all pages processed sequentially ## Requirements ### Core Dependencies - Python ≥3.10 - PyTorch ≥2.0.0 - Transformers =4.53.0 - PIL/Pillow ≥10.0.0 - MCP ≥1.0.0 ### Optional Dependencies - pdf2image ≥1.16.0 (for PDF support) - PyMuPDF ≥1.23.0 (for PDF support) - python-docx ≥0.8.11 (for Word document support) - openpyxl ≥3.1.0 (for Excel support) - pandas ≥2.0.0 (for Excel support) ## Development ### Testing #### Docker Testing: ```bash # Test Docker build docker-compose build # Run health check docker-compose up -d docker-compose ps # View logs docker-compose logs -f nanonets-mcp # Stop services docker-compose down ``` #### Local Testing: ```bash # Test with MCP Inspector mcp dev nanonets_mcp/server.py # Install for development uv pip install -e . ``` ### Docker Management ```bash # Rebuild image after changes docker-compose build --no-cache # View resource usage docker stats nanonets-mcp-server # Access container shell docker-compose exec nanonets-mcp bash # Clean up volumes and images docker-compose down -v docker image prune -f ``` ## License [Add your license information here]