Skip to main content
Glama
StanleyChanH

VLLM MCP Server

by StanleyChanH
OverviewSchemaRelated ServersScoreDiscussions
Python
Remote

VLLM MCP Server

MIT License Python 3.11+ uv

A Model Context Protocol (MCP) server that enables text models to call multimodal models. This server supports both OpenAI and Dashscope (Alibaba Cloud) multimodal models, allowing text-only models to process images and other media formats through standardized MCP tools.

GitHub Repository: https://github.com/StanleyChanH/vllm-mcp

Features

  • Multi-Provider Support: OpenAI GPT-4 Vision and Dashscope Qwen-VL models

  • Multiple Transport Options: STDIO, HTTP, and Server-Sent Events (SSE)

  • Flexible Deployment: Docker, Docker Compose, and local development

  • Easy Configuration: JSON configuration files and environment variables

  • Comprehensive Tooling: MCP tools for model interaction, validation, and provider management

Related MCP server: mcp-image-analyzer

Quick Start

Prerequisites

  • Python 3.11+

  • uv package manager

  • API keys for OpenAI and/or Dashscope (阿里云)

Installation & Setup

  1. Clone the repository:

    git clone https://github.com/StanleyChanH/vllm-mcp.git
cd vllm-mcp

  2. Set up environment:

    cp .env.example .env
# Edit .env with your API keys
nano .env  # or use your preferred editor

  3. Configure API keys (in .env file):

    # Dashscope (阿里云) - Required for basic functionality
DASHSCOPE_API_KEY=sk-your-dashscope-api-key

# OpenAI - Optional
OPENAI_API_KEY=sk-your-openai-api-key

  4. Install dependencies:

    uv sync

  5. Verify setup:

    uv run python test_simple.py

Running the Server

  1. Start the server (STDIO transport - default):

    ./scripts/start.sh

  2. Start with HTTP transport:

    ./scripts/start.sh --transport http --host 0.0.0.0 --port 8080

  3. Development mode with hot reload:

    ./scripts/start-dev.sh

Testing & Verification

  1. List available models:

    uv run python examples/list_models.py

  2. Run basic tests:

    uv run python test_simple.py

  3. Test MCP tools:

    uv run python examples/client_example.py

Docker Deployment

  1. Build and run with Docker Compose:

    # Create .env file with your API keys
cp .env.example .env

# Start the service
docker-compose up -d

  2. Build manually:

    docker build -t vllm-mcp .
docker run -p 8080:8080 --env-file .env vllm-mcp

Configuration

Environment Variables

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key
OPENAI_BASE_URL=https://api.openai.com/v1  # Optional
OPENAI_DEFAULT_MODEL=gpt-4o
OPENAI_SUPPORTED_MODELS=gpt-4o,gpt-4o-mini,gpt-4-turbo,gpt-4-vision-preview

# Dashscope Configuration
DASHSCOPE_API_KEY=your_dashscope_api_key
DASHSCOPE_DEFAULT_MODEL=qwen-vl-plus
DASHSCOPE_SUPPORTED_MODELS=qwen-vl-plus,qwen-vl-max,qwen-vl-chat,qwen2-vl-7b-instruct,qwen2-vl-72b-instruct

# Server Configuration (optional)
VLLM_MCP_HOST=localhost
VLLM_MCP_PORT=8080
VLLM_MCP_TRANSPORT=stdio
VLLM_MCP_LOG_LEVEL=INFO

Configuration File

Create a config.json file:

{
  "host": "localhost",
  "port": 8080,
  "transport": "stdio",
  "log_level": "INFO",
  "providers": [
    {
      "provider_type": "openai",
      "api_key": "${OPENAI_API_KEY}",
      "base_url": "${OPENAI_BASE_URL}",
      "default_model": "gpt-4o",
      "max_tokens": 4000,
      "temperature": 0.7
    },
    {
      "provider_type": "dashscope",
      "api_key": "${DASHSCOPE_API_KEY}",
      "default_model": "qwen-vl-plus",
      "max_tokens": 4000,
      "temperature": 0.7
    }
  ]
}

MCP Tools

The server provides the following MCP tools:

generate_multimodal_response

Generate responses from multimodal models.

Parameters:

  • model (string): Model name to use

  • prompt (string): Text prompt

  • image_urls (array, optional): List of image URLs

  • file_paths (array, optional): List of file paths

  • system_prompt (string, optional): System prompt

  • max_tokens (integer, optional): Maximum tokens to generate

  • temperature (number, optional): Generation temperature

  • provider (string, optional): Provider name (auto-detected if not specified)

Example:

result = await session.call_tool("generate_multimodal_response", {
    "model": "gpt-4o",
    "prompt": "Describe this image",
    "image_urls": ["https://example.com/image.jpg"],
    "max_tokens": 500
})

list_available_providers

List available model providers and their supported models.

Example:

result = await session.call_tool("list_available_providers", {})

validate_multimodal_request

Validate if a multimodal request is supported by the specified provider.

Parameters:

  • model (string): Model name to validate

  • image_count (integer, optional): Number of images

  • file_count (integer, optional): Number of files

  • provider (string, optional): Provider name

Supported Models

OpenAI

  • gpt-4o

  • gpt-4o-mini

  • gpt-4-turbo

  • gpt-4-vision-preview

Dashscope

  • qwen-vl-plus

  • qwen-vl-max

  • qwen-vl-chat

  • qwen2-vl-7b-instruct

  • qwen2-vl-72b-instruct

Model Selection

Using Environment Variables

You can configure default models and supported models through environment variables:

# OpenAI
OPENAI_DEFAULT_MODEL=gpt-4o
OPENAI_SUPPORTED_MODELS=gpt-4o,gpt-4o-mini,gpt-4-turbo

# Dashscope
DASHSCOPE_DEFAULT_MODEL=qwen-vl-plus
DASHSCOPE_SUPPORTED_MODELS=qwen-vl-plus,qwen-vl-max

Listing Available Models

Use the list_available_providers tool to see all available models:

result = await session.call_tool("list_available_providers", {})
print(result.content[0].text)

Model Selection Examples

# Use specific OpenAI model
result = await session.call_tool("generate_multimodal_response", {
    "model": "gpt-4o-mini",  # Specify exact model
    "prompt": "Analyze this image",
    "image_urls": ["https://example.com/image.jpg"]
})

# Use specific Dashscope model
result = await session.call_tool("generate_multimodal_response", {
    "model": "qwen-vl-max",  # Specify exact model
    "prompt": "Describe what you see",
    "image_urls": ["https://example.com/image.jpg"]
})

# Auto-detect provider based on model name
# OpenAI models (gpt-*) will use OpenAI provider
# Dashscope models (qwen-*) will use Dashscope provider

Model Configuration File

You can also configure models in config.json:

{
  "providers": [
    {
      "provider_type": "openai",
      "api_key": "${OPENAI_API_KEY}",
      "default_model": "gpt-4o-mini",
      "supported_models": ["gpt-4o-mini", "gpt-4-turbo"],
      "max_tokens": 4000,
      "temperature": 0.7
    },
    {
      "provider_type": "dashscope",
      "api_key": "${DASHSCOPE_API_KEY}",
      "default_model": "qwen-vl-max",
      "supported_models": ["qwen-vl-plus", "qwen-vl-max"],
      "max_tokens": 4000,
      "temperature": 0.7
    }
  ]
}

Client Integration

Python Client

import asyncio
from mcp.client.session import ClientSession
from mcp.client.stdio import StdioServerParameters, stdio_client

async def main():
    server_params = StdioServerParameters(
        command="uv",
        args=["run", "python", "-m", "vllm_mcp.server"],
        env={"PYTHONPATH": "src"}
    )

    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            # Generate multimodal response
            result = await session.call_tool("generate_multimodal_response", {
                "model": "gpt-4o",
                "prompt": "Analyze this image",
                "image_urls": ["https://example.com/image.jpg"]
            })

            print(result.content[0].text)

asyncio.run(main())

MCP Client Configuration

Add to your MCP client configuration:

{
  "mcpServers": {
    "vllm-mcp": {
      "command": "uv",
      "args": ["run", "python", "-m", "vllm_mcp.server"],
      "env": {
        "PYTHONPATH": "src",
        "OPENAI_API_KEY": "${OPENAI_API_KEY}",
        "DASHSCOPE_API_KEY": "${DASHSCOPE_API_KEY}"
      }
    }
  }
}

Development

Project Structure

vllm-mcp/
├── src/vllm_mcp/
│   ├── __init__.py
│   ├── server.py          # Main MCP server
│   ├── models.py          # Data models
│   └── providers/
│       ├── __init__.py
│       ├── openai_provider.py
│       └── dashscope_provider.py
├── scripts/
│   ├── start.sh           # Production startup
│   └── start-dev.sh       # Development startup
├── examples/
│   ├── client_example.py  # Example client
│   └── mcp_client_config.json
├── docker-compose.yml
├── Dockerfile
├── config.json
└── README.md

Adding New Providers

  1. Create a new provider class in src/vllm_mcp/providers/

  2. Implement the required methods:

    • generate_response()

    • is_model_supported()

    • validate_request()

  3. Register the provider in src/vllm_mcp/server.py

  4. Update configuration schema

Running Tests

# Install development dependencies
uv add --dev pytest pytest-asyncio

# Run tests
uv run pytest

Deployment Options

STDIO Transport (Default)

Best for MCP client integrations and local development.

vllm-mcp --transport stdio

HTTP Transport

Suitable for web service deployments.

vllm-mcp --transport http --host 0.0.0.0 --port 8080

SSE Transport

For real-time streaming responses.

vllm-mcp --transport sse --host 0.0.0.0 --port 8080

Troubleshooting

Common Issues

  1. Import Error: No module named 'vllm_mcp'

    # Make sure you're in the project root and run:
uv sync
export PYTHONPATH="src:$PYTHONPATH"

  2. API Key Not Found

    # Ensure your .env file is properly configured:
cp .env.example .env
# Edit .env with your actual API keys

  3. Dashscope API Errors

    • Verify your API key is valid and active

    • Check if you have sufficient quota

    • Ensure network connectivity to Dashscope services

  4. Server Startup Issues

    # Check for port conflicts:
lsof -i :8080

# Try a different port:
./scripts/start.sh --port 8081

  5. Docker Issues

    # Rebuild Docker image:
docker-compose down
docker-compose build --no-cache
docker-compose up -d

Debug Mode

Enable debug logging for troubleshooting:

./scripts/start.sh --log-level DEBUG

Getting Help

  • Check SETUP_GUIDE.md for detailed setup instructions

  • Run uv run python test_simple.py to verify basic functionality

  • Review logs for error messages and warnings

License

MIT License

Contributing

  1. Fork the repository

  2. Create a feature branch

  3. Make your changes

  4. Add tests if applicable

  5. Submit a pull request

Support

Acknowledgments

Available Tools

3 tools
generate_multimodal_responseC

Generate response from multimodal model.

        Args:
            model: Model name to use
            prompt: Text prompt
            image_urls: Optional list of image URLs
            file_paths: Optional list of file paths
            system_prompt: Optional system prompt
            max_tokens: Maximum tokens to generate
            temperature: Generation temperature
            provider: Optional provider name (openai, dashscope)

        Returns:
            Generated response text
ParametersJSON Schema
NameRequiredDescriptionDefault
modelYes
promptYes
image_urlsNo
file_pathsNo
system_promptNo
max_tokensNo
temperatureNo
providerNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the basic action ('generate response') without detailing behavioral traits like rate limits, authentication needs, error handling, or what happens with invalid inputs. For a complex tool with 8 parameters, this lack of context is a significant gap, though it doesn't contradict any annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, starting with the core purpose followed by parameter details in a structured format. Every sentence serves a purpose, with no wasted words, though the parameter explanations could be more concise. It efficiently conveys essential information without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 parameters, no annotations, but with an output schema), the description is moderately complete. It covers the purpose and parameters but lacks behavioral context and usage guidelines. The output schema handles return values, so the description doesn't need to explain those, but it should provide more operational details to fully guide an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description lists all 8 parameters with brief explanations (e.g., 'Model name to use', 'Text prompt'), adding meaning beyond the input schema, which has 0% description coverage. However, the explanations are minimal and don't cover details like format constraints or examples. With low schema coverage, this partially compensates but doesn't fully address the complexity, warranting a baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Generate response from multimodal model.' This specifies the verb ('generate response') and resource ('multimodal model'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'list_available_providers' or 'validate_multimodal_request', which serve different purposes (listing vs. validation vs. generation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or any context for choosing this tool over others, such as for generating outputs versus validating requests. Without such guidance, an AI agent might struggle to select the appropriate tool in a given scenario.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_available_providersB

List available model providers and their configurations.

        Returns:
            JSON string of available providers and their models
ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns a JSON string of providers and models, which is useful, but doesn't cover other behavioral traits like rate limits, authentication needs, error handling, or whether it's a read-only operation. The description adds some value but leaves significant gaps for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately concise with two sentences: one stating the purpose and one specifying the return format. It's front-loaded with the main action. However, the formatting includes extra indentation and a blank line, which slightly detracts from structure but doesn't impact clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 0 parameters, 100% schema coverage, and an output schema exists, the description is moderately complete. It explains what the tool does and the return format, but with no annotations, it should ideally cover more behavioral aspects like safety or performance. The output schema reduces the need to detail return values, but the description could better address gaps from missing annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so the schema fully documents the lack of inputs. The description doesn't need to add parameter details, and it correctly avoids discussing parameters. This meets the baseline for 0 parameters, as the description focuses on output without redundancy.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('List') and resource ('available model providers and their configurations'). It distinguishes from sibling tools like 'generate_multimodal_response' and 'validate_multimodal_request' by focusing on listing rather than generating or validating. However, it doesn't explicitly differentiate itself from potential alternative listing tools beyond the scope of providers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context for usage, or compare it to sibling tools. The only implied usage is to retrieve provider information, but this is basic and lacks explicit when/when-not instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_multimodal_requestB

Validate if a multimodal request is supported.

        Args:
            model: Model name to validate
            image_count: Number of images in request
            file_count: Number of files in request
            provider: Optional provider name

        Returns:
            Validation result
ParametersJSON Schema
NameRequiredDescriptionDefault
modelYes
image_countNo
file_countNo
providerNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It only states the basic validation function without describing what 'supported' means (e.g., capability checks, rate limits, authentication requirements, error conditions, or what happens when validation fails). This leaves significant behavioral gaps for a tool that likely interfaces with external services.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and well-structured: a clear purpose statement followed by parameter and return value documentation in a standard format. Every sentence earns its place with no redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (4 parameters, validation logic) and the presence of an output schema (which handles return values), the description is minimally adequate. However, with no annotations and 0% schema description coverage, it should provide more behavioral context about what validation entails and error scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It lists all 4 parameters with brief explanations, adding meaning beyond the bare schema. However, it doesn't provide format details (e.g., model name patterns), constraints (e.g., valid ranges for counts), or how parameters interact (e.g., if provider affects model validation).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose as validating if a multimodal request is supported, which is a specific verb (validate) applied to a specific resource (multimodal request). However, it doesn't explicitly differentiate from sibling tools like 'generate_multimodal_response' or 'list_available_providers' beyond the validation focus.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, typical use cases, or relationships to sibling tools like 'generate_multimodal_response' (which might require validation first) or 'list_available_providers' (which might help select a provider).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv1.0.0
    • First observedgenerate_multimodal_response
    • First observedlist_available_providers
    • First observedvalidate_multimodal_request
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.4/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose with no overlap. generate_multimodal_response handles actual generation, list_available_providers provides configuration information, and validate_multimodal_request performs validation checks. An agent can easily distinguish between these three functions.

Naming Consistency5/5

All three tools follow a consistent verb_noun naming pattern (generate_multimodal_response, list_available_providers, validate_multimodal_request). The naming is uniform, predictable, and clearly communicates each tool's function without any style mixing or deviations.

Tool Count3/5

With only 3 tools, this server feels somewhat thin for a multimodal generation service. While the tools cover core functionality, typical MCP servers in this domain would include additional operations like model management, conversation history, or specialized generation modes. The count is borderline but functional.

Completeness4/5

The tool set covers the essential workflow: checking available providers, validating requests, and generating responses. However, there are minor gaps such as no tool for managing conversation context, handling streaming responses, or providing model-specific configuration options that would enhance the agent's ability to work effectively with multimodal models.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/StanleyChanH/vllm-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server