Skip to main content
Glama
Sign In
deenesjakoruzh
mlx-serve-mcp

mlx-serve-mcp

by Congenital
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

MCP Badge

An MCP (Model Context Protocol) server that gives any MCP client two things at once:

  1. A bridge to a remote mlx-serve instance — Apple Silicon's OpenAI-compatible local inference server — exposing its media generation endpoints (image, image-edit, speech, music, video, 3D) and model management as MCP tools.

  2. The full local agent toolset — files, shell, background processes, web search, memory, and scheduled tasks — so the same server can also drive the machine it runs on.

It speaks MCP using the official Python SDK (mcp.server.fastmcp.FastMCP) over stdio by default, or SSE / streamable-HTTP if you prefer a network transport.

This is a ground-up reimplementation. The previous revision hand-rolled the JSON-RPC protocol, which is fragile and drifts from the spec. This version delegates the protocol to the official SDK and adds the local toolset on top.

Install

With uv (recommended):

uv sync                      # creates .venv and installs deps (add --extra dev for tests)
uv run mlx-serve-mcp       # launch the server

Important: after uv sync (or uv pip install), the mlx-serve-mcp console script exists only inside the project's .venv — start it with uv run mlx-serve-mcp, not a bare mlx-serve-mcp (that works only after uv tool install / pipx install, or a pip install).

Or with plain pip:

pip install -e ".[dev]"
mlx-serve-mcp              # or: python -m mlx_serve_mcp

Requires Python ≥ 3.10 (uv run picks a suitable interpreter automatically). ffmpeg comes from the imageio-ffmpeg dependency (a system ffmpeg on PATH is used preferentially when present).

Related MCP server: media-gen-mcp

Run

# stdio (default) — for MCP hosts that spawn the server (Claude Desktop, etc.)
uv run mlx-serve-mcp

# Native deps avoided entirely: `cryptography` is excluded via
# `[tool.uv] override-dependencies` (uv sync will not install it; `pyjwt`
# stays on the pure-Python `2.10` line, and nothing in the code requires it)

# point it at a specific mlx-serve instance
uv run mlx-serve-mcp --url http://127.0.0.1:11234 --api-key "$MLX_SERVE_API_KEY"

# network transport (SSE / streamable-HTTP on 127.0.0.1:8765)
uv run mlx-serve-mcp --transport sse --host 127.0.0.1 --port 8765

If you installed with plain pip, drop the uv run prefix (or use python -m mlx_serve_mcp).

Configure an MCP host

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (Windows: %APPDATA%\Claude\claude_desktop_config.json) and restart Claude:

{
  "mcpServers": {
    "mlx-serve": {
      "command": "uv",
      "args": ["run", "--directory", "/absolute/path/to/mlx-serve-mcp", "mlx-serve-mcp"],
      "env": {
        "MLX_SERVE_URL": "http://127.0.0.1:11234",
        "MLX_SERVE_API_KEY": "your-key"
      }
    }
  }
}

After uv tool install mlx-serve-mcp (or pipx install mlx-serve-mcp) you can instead use "command": "mlx-serve-mcp" with no uv run wrapper.

LobeHub

The repo ships lhm.plugin.json — upload it as a plugin. Its stdio deployment option spawns mlx-serve-mcp and reads MLX_SERVE_URL / MLX_SERVE_API_KEY from the deployment environment.

Smoke-test without a host

uv sync --extra dev
uv run python mcp_client.py --list
uv run python mcp_client.py --call recall_memory
uv run python mcp_client.py --call generate_image --args '{"prompt": "a red fox in the snow"}'

mcp_client.py spawns the server itself (auto-detecting mlx-serve-mcp or python -m mlx_serve_mcp, and adding src/ to PYTHONPATH when running from a source checkout) and speaks MCP over stdio.

Toolset

Generation (proxied to the remote mlx-serve instance)

Tool

What it does

generate_image

Text → image (saved to disk; returns the saved path)

edit_image

Image + prompt → edited image (image-to-image, saved to disk)

generate_speech

Text → spoken audio (TTS, saved as WAV)

generate_music

Style description (+ optional lyrics) → music (saved as WAV)

generate_video

Prompt → short video clip (muxed to MP4 via ffmpeg, saved to disk)

generate_3d

Prompt → 3D mesh (GLB)

Local (run on the machine hosting the server)

Tool

What it does

read_file / write_file / edit_file

Filesystem read / write / edit

search_files / list_files

Content search (regex) and directory listing (glob)

shell

Run a command (foreground, or background with a handle)

list_processes / read_process_output / kill_process

Manage background processes

web_search

DuckDuckGo web search

browse

Fetch-based page reading (navigate / readText / extractText / readHTML)

save_memory / recall_memory / clear_memory

A small persistent note store

create_task / list_tasks / cancel_task

Background / scheduled command jobs

Resources & prompts

  • Resources — mlx-serve://status (live health), mlx-serve://models (model inventory with capability flags), mlx-serve://guidance (model-selection advice).

  • Prompts — one-click recipes: portrait, poster, lofi_track, song, short_video.

Configuration

CLI flags are layered over MLX_SERVE_* environment variables (flag wins):

Flag

Env var

Default

--url / --base-url

MLX_SERVE_URL

http://127.0.0.1:11234

--api-key

MLX_SERVE_API_KEY

(none)

--output-dir

MLX_SERVE_OUTPUT_DIR

./output

--timeout

MLX_SERVE_TIMEOUT

1800 (seconds)

--image-model

MLX_SERVE_IMAGE_MODEL

ddalcu/Mage-Flow-Turbo-MLX-Serve-8bit

--image-edit-model

MLX_SERVE_IMAGE_EDIT_MODEL

Runpod/FLUX.2-klein-4B-mflux-4bit

--tts-model

MLX_SERVE_TTS_MODEL

mlx-community/Kokoro-82M-v1.0-mlx

--music-model

MLX_SERVE_MUSIC_MODEL

mlx-community/musicgen-small-mlx

--video-model

MLX_SERVE_VIDEO_MODEL

mlx-community/HunyuanVideo-mlx

--mesh-model

MLX_SERVE_MESH_MODEL

mlx-community/trellis-mlx

--working-dir

MLX_SERVE_WORKING_DIR

current directory

--data-dir

MLX_SERVE_DATA_DIR

~/.mlx-serve-mcp

--transport

MLX_SERVE_TRANSPORT

stdio

--host / --port

MLX_SERVE_HOST / MLX_SERVE_PORT

127.0.0.1 / 8765

Generated artifacts land under --output-dir (images/, audio/, video/, mesh/); the memory store lives under --data-dir.

Layout

src/mlx_serve_mcp/
  server.py        # composition root: builds the FastMCP app
  config.py        # CLI flags over MLX_SERVE_* env vars
  client.py        # async HTTP client for the remote mlx-serve instance
  video.py         # raw frames + PCM -> H.264/AAC MP4 via ffmpeg
  state.py         # process-lifetime state (cwd, processes, tasks, memory)
  prompts.py       # one-click prompt recipes
  resources.py     # live resources (status / models / guidance)
  tools/
    generation.py  # image / edit / speech / music / video / 3d
    files.py       # read / write / edit / search / list
    shell.py       # shell (foreground + background)
    processes.py   # list / read / kill background processes
    web.py         # web_search + browse
    memory.py      # save / recall / clear memory
    tasks.py       # create / list / cancel background jobs

SDK note (v1 pin)

This package pins mcp>=1.9,<2.0 and uses mcp.server.fastmcp.FastMCP. The MCP Python SDK v2 removed the bundled FastMCP (it is now MCPServer, and the maintained high-level framework is the standalone fastmcp package). Upgrading is a one-line import change — from fastmcp import FastMCP — plus repointing the content imports; the tool / resource / prompt decorators and run() are unchanged. The construction in server.py is deliberately defensive so it keeps working across SDK constructor variance.

Tests

uv sync --extra dev
uv run pytest            # unit tests (no network / mlx-serve required)
uv run pytest -m live    # live tests (need a running mlx-serve + ffmpeg)

Available Tools

10 tools
edit_imageA

Edit an existing image with a text prompt.

Args: image_path: Absolute or ~-relative path to the source image (PNG/JPEG). prompt: Description of the desired edit. model: Model id. Defaults to MLX_SERVE_IMAGE_EDIT_MODEL. size: Output dimensions. strength: Denoising strength 0..1 (how much to change the image). seed: Optional random seed.

Inputs: prompt (required), image_path (required), seed, size, +2 more.

generate_3dA

Generate a textured 3D mesh (GLB) from a subject image on the remote server.

The image should be a clean cutout of the subject with real alpha transparency (an opaque photo also works — the server composites it on white). Returns a GLB file path you can open in Blender / three.js / Quick Look.

Args: image_path: Absolute or ~-relative path to a local PNG/JPEG of the subject. model: 3D mesh model id. Defaults to MLX_SERVE_MESH_MODEL. steps: Shape sampling steps (default 30). octree_resolution: Mesh grid resolution in [64, 512] (default 256). guidance_scale: Shape guidance in [0, 20] (default 5). texture: Request the texture-paint stage for a textured GLB. Requires the paint weights to be installed server-side, else a named error. texture_steps: Texture painting steps in [1, 100] when texture=true. seed: Optional seed.

Inputs: image_path (required), seed, model, steps, +4 more.

generate_imageA

Generate an image from a text prompt on the remote server.

The image is saved as a local PNG file; the returned text includes the absolute path of the saved file.

Args: prompt: Text description of the image to generate. model: Model id (from list_models). Defaults to MLX_SERVE_IMAGE_MODEL (or the built-in default if unset). size: Output dimensions, e.g. "512x512" or "1024x1024". seed: Optional random seed for reproducibility. steps: Sampling steps (backend-specific). cfg_scale: Classifier-free guidance scale (SDXL/Flux style). guidance_scale: Guidance scale (MAGE-Flow style).

Inputs: prompt (required), seed, size, model, +3 more.

generate_musicA

Generate a music track from a style prompt; returns a local WAV path.

Args: prompt_style: Style description, e.g. "lo-fi hip hop, mellow piano". model: Music model id. Defaults to MLX_SERVE_MUSIC_MODEL. lyrics: Optional lyrics for vocal tracks. instrumental: True for no vocals (default true for text2music). duration_seconds: Target length 10..600 (default 60). bpm: Tempo, e.g. 120. keyscale: Musical key, e.g. "C major" / "E minor". timesignature: Time signature, e.g. "4/4". vocal_language: Vocal language code for sung lyrics, e.g. "en", "zh". task: "text2music" (default) | "cover" | "complete". src_audio_path: Local WAV (10-600 s) source for cover/complete tasks; its length becomes the output length. cover_strength: 0..1 how strongly to follow the cover source (default 1). seed: Optional seed for reproducibility. steps: Optional sampling steps (music3 backend only).

Inputs: prompt_style (required), bpm, seed, task, +10 more.

generate_videoA

Generate a short video from a text prompt; returns a local MP4 path.

mlx-serve answers with raw frames (+ optional soundtrack); this tool encodes them into an H.264 MP4 (AAC audio when present) locally via ffmpeg, so the returned file is playable anywhere. Generation is slow — minutes per clip depending on frame count and resolution.

Args: prompt: Scene description for the video. model: Video model id. Defaults to MLX_SERVE_VIDEO_MODEL. num_frames: Frame count. LTX backends use an 8N+1 ladder (default 9; e.g. 9/25/33/49/57/81...); MiniMax-H3 uses 17k+5 (default 56). width: Pixel width (defaults: LTX 384, H3 256). Two-stage pipelines need both dimensions divisible by 64. height: Pixel height (defaults: LTX/H3 256). steps: Sampling steps (backend-specific defaults). turbo: Use the distilled 4-step turbo path where the model pack provides it. seed: Optional seed. cfg_scale: Guidance scale. first_frame_image_path: Optional local image to condition the first frame. last_frame_image_path: Optional local image to condition the final frame. audio_path: Optional local WAV to mix as soundtrack (must match frame duration).

Inputs: prompt (required), seed, model, steps, +8 more.

health_checkA

Check connectivity to the remote mlx-serve instance.

Pings GET /health on the configured server. Use this first to verify that the ip:port is reachable before running any generation.

Inputs: no named parameters.

list_modelsA

List models available on the remote mlx-serve server with their capabilities.

Returns each model's id plus capability flags: chat/vision, image engine, speech (TTS) engine, music backend, video engine, and 3D mesh engine. Pick ids from this list for the model argument of generation tools; a media tool without model uses whatever matching model the server has loaded or configured as default.

Inputs: no named parameters.

load_modelA

Explicitly cold-load a model on the remote mlx-serve server into GPU memory.

Args: model: Model id as returned by list_models (or an absolute path to a model directory on the SERVER machine). make_default: Also promote it to the server-wide default model. Leave false for side-loaded media models so chat traffic keeps its current default.

Inputs: model (required), make_default.

text_to_speechA

Synthesize speech from text; returns a local WAV file path.

Args: text: The text to speak. model: TTS model id. Defaults to MLX_SERVE_TTS_MODEL. voice: Voice name or id (backend-specific). speed: Playback speed multiplier (0.25..4.0, default 1.0).

Inputs: text (required), model, speed, voice.

unload_modelA

Free a model's GPU memory on the remote mlx-serve server.

The model stays registered (it can be reloaded later); only its resident weights are evicted. Useful after heavy media generation to reclaim unified memory.

Inputs: model (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    A4.2/5.0

    Scored across 10 tools

    Disambiguation5/5

    Each tool targets a distinct modality or administrative action: TTS, music, video, 3D, image generation, image editing, health check, and model management. No two tools overlap in purpose, and even the two audio tools are clearly separated by speech vs. music.

    Naming Consistency4/5

    The set mostly follows a verb_noun pattern: generate_music, generate_video, generate_image, edit_image, list_models, load_model, unload_model. text_to_speech breaks the pattern slightly, but it is still readable and the deviation is minor.

    Tool Count5/5

    Ten tools is well-scoped for a media-generation server: creative generation/edit tools, TTS, and operational tools for health and model management. Each tool earns its place and none feel redundant.

    Completeness4/5

    Core media generation workflows are well covered: image generation/edit, video, music, TTS, and 3D mesh generation, plus model lifecycle management. The main gap is that list_models advertises chat/vision capability, but no chat or vision tool is exposed, leaving that part of mlx-serve unreachable.

    Resources

    Related MCP Connectors

    Related MCP Servers