Florence-2 MCP Server

Florence-2 MCP Server brings local vision intelligence and Optical Character Recognition (OCR) to your AI assistants via the Model Context Protocol (MCP). Powered by Microsoft's Florence-2 foundation model, it lets your LLMs inspect, caption, and extract text from local images, web URLs, and multi-page PDF documents — entirely offline and on your own machine.

✨ Key Features

• 🔒 100% Local & Private: All image and document processing runs on your local machine. No external vision API fees, token limits, or data sharing.

• 📄 Multi-Page PDF & Image Support: Process common image formats (JPEG, PNG, WebP, etc.) as well as multi-page PDF documents page-by-page.

• ⚡ Native Hardware Acceleration: Automatically detects and leverages Apple Silicon (MPS), NVIDIA GPU (CUDA), or CPU.

• 🧹 Zero VRAM Waste: By default, inference runs in an on-demand subprocess to release VRAM immediately after completion. A --cache-model flag is available for low-latency continuous usage.

• 🎯 Rich Vision Tasks: Perform high-precision text OCR, detailed descriptive captioning, and open-ended visual queries using native Florence-2 prompt tokens (e.g. object detection, region descriptions).

• 📦 One-Click Installs: Ready-to-use .mcpb bundle for Claude Desktop, 1-click deeplink for LM Studio, and extension support for Goose.

Related MCP server: 🪄 ImageSorcery MCP

🚀 Quick Installation

Choose your preferred MCP client below:

Claude Desktop

Option 1: One-Click Extension Bundle (Recommended)

1. Download the latest mcp-florence2.mcpb bundle from the Releases page.

2. Double-click the downloaded .mcpb file, or drag-and-drop it into Claude Desktop's Settings window.

Option 2: Manual Configuration

Add the following configuration to your claude_desktop_config.json:

{
  "mcpServers": {
    "florence-2": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/jkawamoto/mcp-florence2",
        "mcp-florence2"
      ]
    }
  }
}

For more details, see Connect to local MCP servers.

Goose

Option 1: One-Click URL

Click the link below to launch the Goose extension installer:

goose://extension?cmd=uvx&arg=--from&arg=git%2Bhttps%3A%2F%2Fgithub.com%2Fjkawamoto%2Fmcp-florence2&arg=mcp-florence2&id=florence2&name=Florence-2&description=An%20MCP%20server%20for%20processing%20images%20using%20Florence-2

Option 2: Manual Configuration

Add the following to ~/.config/goose/config.yaml:

extensions:
  florence2:
    name: Florence-2
    cmd: uvx
    args: [ --from, git+https://github.com/jkawamoto/mcp-florence2, mcp-florence2 ]
    enabled: true
    type: stdio

For more details, refer to Using Extensions | goose.

LM Studio

Click the button below to install directly into LM Studio:

Claude Code, Cursor, Cline & Other MCP Clients

You can connect any client that supports stdio MCP servers using uvx:

uvx --from git+https://github.com/jkawamoto/mcp-florence2 mcp-florence2

🛠️ Tools

The server automatically equips your AI assistant with the following vision capabilities. You can interact with them using natural language:

1. ocr

Extracts all visible text from images or multi-page PDF documents.

• Example user request: "Please extract all text from /path/to/invoice.pdf."

• Multi-page support: Automatically iterates through every page in a PDF document.

2. caption

Generates comprehensive, detailed captions describing what is depicted in the image.

• Example user request: "Describe what is shown in https://example.com/photo.jpg."

• Returns granular descriptions of objects, spatial relationships, colors, and background scenery.

3. process

Executes custom Florence-2 task prompts directly against the input image or PDF for advanced computer vision workflows (such as object detection or region captioning).

• Example user request: "Detect all objects in this image and return their bounding boxes."

Parameters

Supported Florence-2 Task Prompts

⚙️ Configuration & CLI Options

The server can be customized using command-line arguments:

mcp-florence2 [OPTIONS]

Using a Smaller / Faster Model

If you have limited GPU memory or want faster execution, switch to Florence-2-base:

{
  "mcpServers": {
    "florence-2": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/jkawamoto/mcp-florence2",
        "mcp-florence2",
        "--model",
        "florence-community/Florence-2-base"
      ]
    }
  }
}

🛡️ Security Considerations

When processing URLs starting with http:// or https://, outbound HTTP requests are initiated directly from the server process. In environments where untrusted users or prompts may supply URLs, this could potentially expose the server to Server-Side Request Forgery (SSRF) targeting internal network services or cloud metadata endpoints.

Since Python's requests library automatically respects standard environment variables, proxy settings can be configured directly in your client configuration:

• HTTP_PROXY / http_proxy: Proxy URL for HTTP requests.

• HTTPS_PROXY / https_proxy: Proxy URL for HTTPS requests.

• NO_PROXY / no_proxy: Comma-separated list of hosts to bypass the proxy.

Claude Desktop

Add the env block to claude_desktop_config.json:

{
  "mcpServers": {
    "florence-2": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/jkawamoto/mcp-florence2",
        "mcp-florence2"
      ],
      "env": {
        "HTTP_PROXY": "http://proxy.example.com:8080",
        "HTTPS_PROXY": "http://proxy.example.com:8080"
      }
    }
  }
}

Goose

Add the env block to ~/.config/goose/config.yaml:

extensions:
  florence2:
    name: Florence-2
    cmd: uvx
    args: [ --from, git+https://github.com/jkawamoto/mcp-florence2, mcp-florence2 ]
    env:
      HTTP_PROXY: "http://proxy.example.com:8080"
      HTTPS_PROXY: "http://proxy.example.com:8080"
    enabled: true
    type: stdio

💻 Development

Setup & Run Tests

This project uses uv for fast, reliable dependency management.

# Clone the repository
git clone https://github.com/jkawamoto/mcp-florence2.git
cd mcp-florence2

# Install dependencies and run tests
uv run pytest

# Run pre-commit checks
uv run pre-commit run --all-files

📄 License

This application is released under the MIT License.