vLLM MCP Server

Instructions

Implementation Reference

• src/vllm_mcp_server/tools/models.py:11-37 (handler)

  The main list_models handler function that retrieves available models from the vLLM server and formats them as TextContent. It handles errors and returns formatted markdown output with model details (id, owned_by, created).

  async def list_models() -> list[TextContent]:
      """
      List all available models on the vLLM server.
  
      Returns:
          List of TextContent with model information.
      """
      try:
          async with VLLMClient() as client:
              models = await client.list_models()
  
              if not models:
                  return [TextContent(type="text", text="No models available on the vLLM server.")]
  
              # Format model list
              model_list = []
              for model in models:
                  model_id = model.get("id", "unknown")
                  owned_by = model.get("owned_by", "unknown")
                  created = model.get("created", "unknown")
                  model_list.append(f"- **{model_id}** (owned by: {owned_by}, created: {created})")
  
              result = f"## Available Models ({len(models)} total)\n\n" + "\n".join(model_list)
              return [TextContent(type="text", text=result)]
  
      except VLLMClientError as e:
          return [TextContent(type="text", text=f"Error listing models: {str(e)}")]

• src/vllm_mcp_server/server.py:131-138 (registration)

  Tool registration that defines the list_models tool with its name, description, and input schema (empty object, meaning no parameters required).

  Tool(
      name="list_models",
      description="List all available models on the vLLM server",
      inputSchema={
          "type": "object",
          "properties": {},
      },
  ),

• src/vllm_mcp_server/server.py:346-347 (handler)

  Handler invocation in the call_tool function that routes the list_models tool name to the actual handler function.

  elif name == "list_models":
      return await list_models()

• src/vllm_mcp_server/utils/vllm_client.py:84-100 (helper)

  VLLMClient helper method that makes the actual HTTP GET request to the vLLM server's /models endpoint and returns the model data as a list of dictionaries.

  async def list_models(self) -> list[dict[str, Any]]:
      """List available models."""
      session = await self._get_session()
      try:
          async with session.get(
              f"{self.base_url}/models",
              headers=self.headers,
          ) as response:
              if response.status != 200:
                  body = await response.text()
                  raise VLLMAPIError(
                      f"Failed to list models: {response.status}",
                      response.status,
                      body,
                  )
              data = await response.json()
              return data.get("data", [])

• src/vllm_mcp_server/server.py:25-25 (registration)

  Import statement that brings the list_models function into scope for tool registration and invocation.

  from vllm_mcp_server.tools.models import get_model_info, list_models