CanvasXpress MCP Server

🎨 Model Context Protocol (MCP) server for generating CanvasXpress visualizations from natural language.

This MCP server provides AI assistants like Claude Desktop with the ability to generate CanvasXpress JSON configurations from natural language descriptions. It uses Retrieval Augmented Generation (RAG) with 132 few-shot examples (66 human + 66 GPT-4 descriptions) and semantic search.

📄 Based on: Smith & Neuhaus (2024) - CanvasXpress NLP Research
🔌 Supports: Azure OpenAI (BMS Proxy) or Google Gemini
🚀 Built with - The modern standard for Python MCP servers

🚀 Quick Start (5 Minutes)

Prerequisites

• ✅ Docker installed and running (or Python 3.10+ for local venv)

• ✅ API key: BMS Azure OpenAI or Google Gemini

• ✅ 8GB RAM for local embeddings, or 2GB for cloud embeddings

• ✅ No GPU required

Step 1: Clone Repository

Step 2: Configure Environment

For Azure OpenAI (BMS) with local embeddings:

For Google Gemini (fully cloud-based) ⭐ Lightweight:

Step 3: Choose Your Setup

Option A: Docker (Full) - Default

Option B: Local Venv (Lightweight) ⭐ For smaller servers

Note: Option B requires EMBEDDING_PROVIDER=gemini or EMBEDDING_PROVIDER=openai in .env

Step 4: Test the Server

That's it! 🎉

Your server is running at http://localhost:8000/mcp

Quick Reference:

📊 Server Modes

HTTP Mode (Network Access) - Recommended

• Access at: http://localhost:8000/mcp

• Use for: Remote access, web-based AI assistants, multiple clients

STDIO Mode (Local Only)

• Use for: Claude Desktop integration on same machine

• Single client only

🧪 Testing Options

CLI Client (Quick & Easy) ⭐

HTTP MCP Client

Vector Database Test

Python API

Next Steps:

• 🌐 HTTP Client Guide: See HTTP_CLIENT_GUIDE.md for HTTP/network usage

• 📖 Python API: See PYTHON_USAGE.md for API reference

• 🔌 Claude Desktop: Configure MCP integration (see below)

• 📘 Technical Details: See TECHNICAL_OVERVIEW.md for architecture

🌟 Features

• 🎯 High Accuracy: 93% exact match, 98% similarity (peer-reviewed methodology)

• 🔍 Semantic Search: BGE-M3 embeddings with vector database (or cloud embeddings)

• 🤖 Multi-Provider LLM: Azure OpenAI (BMS) or Google Gemini

• 📊 Multi-Provider Embeddings: Local BGE-M3, Azure OpenAI, or Google Gemini

• 🐳 Docker or Local: Run in Docker containers or Python virtual environment

• 🔌 FastMCP 2.0: Modern, Pythonic MCP server framework with HTTP & STDIO support

• 🌐 Network Access: HTTP mode for remote deployment and multiple concurrent clients

• 📊 132 Few-Shot Examples: 66 chart types × 2 description styles (human + GPT-4)

⚙️ Provider Configuration

The server supports multiple LLM and embedding providers. Configure in .env:

LLM Providers

Embedding Providers

ONNX Model Options

Set ONNX_EMBEDDING_MODEL to use different models:

Example Configurations

Azure OpenAI + Local BGE-M3 (highest accuracy, requires ~8GB RAM):

Gemini + ONNX (lightweight local embeddings, ~500MB RAM) ⭐ Recommended for small servers:

Google Gemini + Local BGE-M3 (best accuracy with Gemini):

Full Gemini (no local model needed, faster startup):

Note: If you change EMBEDDING_PROVIDER, you must reinitialize the vector database (make init or make init-local) since different providers have different embedding dimensions.

📋 Prerequisites

• Docker OR Python 3.10+

• API key: BMS Azure OpenAI or Google Gemini

• Linux/macOS (tested on Amazon Linux 2023)

• 8GB RAM for local embeddings, or 2GB for cloud embeddings

• No GPU required (embeddings run on CPU)

🐍 Local Development (Alternative to Docker)

If you prefer running without Docker, you can use a Python virtual environment.

Prerequisites

⚠️ Requires Python 3.10+ (FastMCP requirement)

If Python 3.10+ is not installed, install it:

Step 1: Configure Python Path (if needed)

The Makefile uses python3.12 by default. If your Python 3.10+ has a different name:

Step 2: Create Virtual Environment

Option A: Full Installation (~8GB) - Local BGE-M3 embeddings

This creates a ./venv/ directory and installs all dependencies (~5-10 minutes for PyTorch/BGE-M3). Use this if you want EMBEDDING_PROVIDER=local (default, highest accuracy).

Option B: Lightweight Installation (~500MB) - Cloud embeddings only ⭐

This installs only cloud-compatible dependencies (no PyTorch, no BGE-M3). Perfect for lightweight servers that will use Gemini or OpenAI for embeddings.

⚠️ If using venv-light: You MUST set EMBEDDING_PROVIDER=gemini (or openai) in your .env file before running make init-local. The local BGE-M3 model is not installed.

Step 3: Configure Environment

For lightweight venv, use this configuration:

Step 4: Initialize Vector Database

Creates ./vector_db/canvasxpress_mcp.db with 132 embedded examples.

Step 5: Start Server

Step 6: Test

Cleanup

Troubleshooting Local Setup

"python3.12: command not found"

• Edit PYTHON_BIN in the Makefile to match your Python 3.10+ executable name

• Common alternatives: python3.11, python3.10, python3

"No module named 'venv'" or venv creation fails

• Install the venv module: sudo apt install python3.11-venv (Ubuntu) or sudo dnf install python3.11 (RHEL/Amazon Linux)

Permission denied on vector_db

• If you previously ran Docker, the vector_db/ directory may be owned by root

• Fix with: sudo rm -rf vector_db && mkdir vector_db

Import errors when running

• Make sure you activated the venv: source venv/bin/activate

• Verify dependencies installed: pip list | grep fastmcp

"No module named 'FlagEmbedding'" (with venv-light)

• You used make venv-light but have EMBEDDING_PROVIDER=local in .env

• Fix: Set EMBEDDING_PROVIDER=gemini (or openai) in .env

• The lightweight venv doesn't include the BGE-M3 model

BGE-M3 model download fails

• Ensure you have ~2GB free disk space

• Check network connectivity (may need VPN for some networks)

• The model downloads to ~/.cache/huggingface/

🔧 Detailed Setup (Docker)

If you need more control or want to customize the Docker setup:

Build Docker Image

Initialize Vector Database

Start MCP Server

Configure Claude Desktop

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

Restart Claude Desktop.

Test in Claude

Open Claude Desktop and try:

"Use the CanvasXpress tool to create a bar chart showing sales by region with blue bars and a legend on the right"

📖 Usage

Transport Modes

The MCP server supports two transport modes:

1. HTTP Mode (Default) - Network Access 🌐

• URL: http://localhost:8000/mcp

• Access: From anywhere on network/internet

• Clients: Multiple simultaneous connections

• Use cases:

  • Remote AI assistants (ChatGPT, Claude web)

  • Cloud deployment (AWS, GCP, Azure)

  • Team collaboration

  • Production services

2. STDIO Mode - Local Only 💻

• Access: Same machine only

• Clients: Single local client (Claude Desktop)

• Use cases:

  • Claude Desktop integration

  • Local development

  • Private/offline usage

To switch modes, edit .env:

CLI Client (mcp_cli.py)

Command-line interface for querying the HTTP MCP server with custom requests.

Basic Usage

Inside Docker

Available Options

Note: The CLI client connects to the running HTTP MCP server. Make sure the server is running with make run-http first.

Available Tool

The server provides one tool: generate_canvasxpress_config

Parameters:

• description (required): Natural language description of visualization

• headers (optional): Column names from your dataset

• temperature (optional): LLM temperature, default 0.0

Example:

Supported Chart Types

Bar, Boxplot, Scatter, Line, Heatmap, Area, Dotplot, Pie, Venn, Network, Sankey, Genome, Stacked, Circular, Radar, Bubble, Candlestick, and 40+ more.

🔧 Configuration

Environment Variables

Configure your .env file with your BMS credentials:

Available Azure Models

BMS Proxy Details

• Endpoints URL: https://bms-openai-proxy-eus-prod.azu.bms.com/openai-urls.json

• Retry Logic: Automatically rotates endpoints on failures (429, connection errors)

• Max Retries: 3 attempts per request

• API Version: 2024-02-01

🛠️ Development

Makefile Commands

Testing Utilities

Vector Database Testing

Test and inspect the Milvus vector database:

What it tests:

• ✓ Database connection and collection info

• ✓ Row count (should be 132 examples)

• ✓ Sample data display (first 3 examples)

• ✓ Chart type distribution (30 unique types)

• ✓ Vector dimensions (1024 for BGE-M3)

• ✓ Semantic search with 3 sample queries

Example output:

This is useful for:

• Verifying database initialization

• Understanding what examples are available

• Testing semantic search quality

• Debugging RAG retrieval issues

Python Usage Examples

Test the generator as a Python library:

See examples_usage.py and PYTHON_USAGE.md for detailed usage patterns.

Directory Structure

Local Testing (without Docker)

For local development, use the virtual environment setup:

See the "Local Development" section above for detailed instructions.

📊 Methodology

Based on peer-reviewed research (Smith & Neuhaus, 2024):

1. Embedding Model: BGE-M3 (1024d, local) or cloud alternatives (OpenAI 1536d, Gemini 768d)

2. Vector DB: Milvus-lite (local SQLite-based storage)

3. Few-Shot Examples: 132 examples (66 configs with human + GPT-4 descriptions)

4. Retrieval: Top 25 most similar examples per query

5. LLM: Azure OpenAI (BMS proxy) or Google Gemini

6. Retry Logic: Endpoint rotation on failures (BMS proxy pattern)

Performance (with BGE-M3 + Azure OpenAI):

• 93% exact match accuracy

• 98% similarity score

• Handles 30+ chart types

• Automatic failover across Azure regions

� Data Files

The data/ directory contains the files used for RAG and prompt generation:

Switching Few-Shot Files:

To use the expanded examples, create a symlink:

Then reinitialize the vector database:

Note: The expanded file is ~5MB vs ~40KB for the default. It provides better coverage but takes longer to embed on first run.

�🐛 Troubleshooting

Server won't start

Vector database errors

Note: The vector database files are created by Docker with root ownership, so you need sudo to delete them.

Empty database after init

If make init shows success but make test-db reports 0 rows:

This ensures the database is populated with all 132 examples.

Testing semantic search

API key issues

BMS Proxy Issues

Claude Desktop not connecting

1. Check config file path: ~/Library/Application Support/Claude/claude_desktop_config.json

2. Verify server is running: docker ps | grep canvasxpress

3. Restart Claude Desktop

4. Check Claude Desktop logs: ~/Library/Logs/Claude/

📚 Resources

• Original Research: Smith & Neuhaus (2024) - CanvasXpress NLP Preprint

• FastMCP 2.0: https://gofastmcp.com/

• CanvasXpress: https://www.canvasxpress.org/

• MCP Protocol: https://modelcontextprotocol.io/

• BGE-M3 Model: https://huggingface.co/BAAI/bge-m3

📝 License

Based on reference implementation (JOSS publication). MCP server implementation: MIT License

🤝 Contributing

Issues and pull requests welcome!

📧 Support

For issues related to:

• MCP Server: Open GitHub issue

• CanvasXpress: See https://www.canvasxpress.org/documentation.html

• MCP Protocol: See https://modelcontextprotocol.io/docs

Built with ❤️ using