# Codex Local Vision Router

![Codex using DeepSeek V4 through DwarfStar while Qwen3.5 describes an image through the Local Vision Router](docs/assets/codex-ds4-qwen35-vision-demo.png)

*DeepSeek V4 runs the Codex session through DwarfStar while Qwen3.5 in LM
Studio handles image inspection through the local MCP tool.*

Use one model as the main Codex agent and a separate local vision-language
model for image inspection.

This project exposes a read-only MCP tool that:

1. reads an image from an absolute local path;
2. sends it to an OpenAI-compatible vision endpoint such as LM Studio;
3. returns only a bounded text observation to Codex.

It is intended for custom Codex providers that work well with text and tools
but reject `input_image` content.

## Architecture

```text
user prompt with local path
        |
        v
primary Codex model (text + tools)
        |
        | MCP: local_vision.describe_image
        v
Local Vision Router
        |
        | image bytes over loopback HTTP
        v
LM Studio + local vision model
        |
        | plain-text observation
        v
primary Codex model
```

## Privacy defaults

- The default endpoint is `http://127.0.0.1:1234/v1`.
- Non-loopback endpoints are rejected unless
  `LOCAL_VISION_ALLOW_REMOTE=1` is explicitly set.
- Image bytes, base64 payloads, prompts, and API responses are not logged.
- The returned tool result does not include the local image path.
- No credentials, local configuration, model files, source images, or session
  data are included. The documentation contains only the intentionally
  published demonstration screenshot above.

A configured remote endpoint can receive image contents. Read
[SECURITY.md](SECURITY.md) before enabling one.

## Requirements

- Node.js 20 or newer
- Codex CLI
- LM Studio, or another compatible `/v1/chat/completions` server
- A loaded vision model

The tested model is
[TracNetwork/Qwen3.5-4B-4bit-mlx](https://huggingface.co/TracNetwork/Qwen3.5-4B-4bit-mlx),
loaded through LM Studio. LM Studio exposes it to the local API as
`qwen3.5-4b-mlx` in the tested setup; use the exact ID returned by
`/v1/models` if yours differs. Other vision models can be selected with
`--model`.

## Quick start

### 1. Start the vision endpoint

In LM Studio:

1. Load a vision-capable model.
2. Open **Developer**.
3. Start the local server on port `1234`.

Confirm the model is visible:

```bash
curl http://127.0.0.1:1234/v1/models
```

### 2. Clone and check the router

```bash
git clone <repository-url>
cd codex-local-vision-router
node scripts/doctor.mjs
```

### 3. Register the MCP server

For normal Codex:

```bash
node scripts/register.mjs --model qwen3.5-4b-mlx
```

For an isolated custom-provider installation, set the same `CODEX_HOME` used by
its launcher:

```bash
CODEX_HOME="$HOME/.codex-custom" \
  node scripts/register.mjs --model qwen3.5-4b-mlx
```

PowerShell:

```powershell
$env:CODEX_HOME = "$HOME\.codex-custom"
node .\scripts\register.mjs --model qwen3.5-4b-mlx
```

The registration command refuses to replace an existing `local_vision` entry
unless `--replace` is supplied.

### 4. Mark the primary model as text-only

Copy the root keys from
[templates/text-only-primary-config.toml](templates/text-only-primary-config.toml)
into the active `$CODEX_HOME/config.toml`. Place them before the first TOML
table.

This disables the built-in image tool and instructs Codex to use
`local_vision.describe_image`.

### 5. Start a new Codex session

Do not resume a session that already contains an incompatible image block.

Run `/mcp` and confirm `local_vision` is connected. Then test:

```text
Use local_vision.describe_image to inspect /absolute/path/to/image.jpg.
Describe the main subject, composition, visible text, and uncertainty.
```

Use a filesystem path rather than attaching or dragging the image into a
text-only session.

## End-to-end diagnostics

```bash
node scripts/doctor.mjs \
  --image /absolute/path/to/image.jpg \
  --prompt "Transcribe the largest visible heading."
```

## Configuration

| Variable | Default | Purpose |
| --- | --- | --- |
| `LOCAL_VISION_BASE_URL` | `http://127.0.0.1:1234/v1` | Compatible API base URL |
| `LOCAL_VISION_MODEL` | `qwen3.5-4b-mlx` | Vision model ID |
| `LOCAL_VISION_TIMEOUT_MS` | `120000` | Request timeout |
| `LOCAL_VISION_MAX_IMAGE_BYTES` | `52428800` | Maximum image size |
| `LOCAL_VISION_MAX_OUTPUT_TOKENS` | `4096` | Generation ceiling |
| `LOCAL_VISION_MAX_ANSWER_CHARS` | `2000` | Structured answer limit |
| `LOCAL_VISION_ALLOW_REMOTE` | unset | Set to `1` to permit remote endpoints |
| `LOCAL_VISION_API_KEY` | unset | Optional endpoint bearer token |

Supported extensions: PNG, JPEG, WebP, GIF, and AVIF.

## Remove

```bash
codex mcp remove local_vision
```

Remove the text-only routing block from `config.toml` only if the primary
provider can safely receive images.

## Development

```bash
npm run check
npm test
```

The MCP server has no runtime npm dependencies.

See [docs/SETUP.md](docs/SETUP.md) for the detailed provider workflow and
[docs/PUBLISHING.md](docs/PUBLISHING.md) for publishing this repository.

## Codex plugin bundle

The repository is also a valid Codex plugin bundle. A marketplace can point to
this repository, and `codex plugin add` will install both the routing skill and
the bundled MCP server. The plugin defaults to the loopback LM Studio endpoint
and `qwen3.5-4b-mlx`.

Use the clone-and-register flow above when a different endpoint, model ID, or
`CODEX_HOME` needs to be selected at install time.

## License

MIT