Skip to main content
Glama
phykn

llm-local

by phykn
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

local-models

Run local models on an Apple Silicon Mac with MLX, use them from Claude Code over MCP, and start or stop them from a Mac app.

  • Kev (jaredpalmer/kev): decision models that return a calibrated answer, not generated text (choose, yes_no, score).

  • LLM: Qwen3.8-27B and the faster Qwen3.6-35B-A3B (4-bit mxfp4) for drafting, translating, summarizing or a second opinion.

Servers listen only on 127.0.0.1, with no authentication.

Install

Requires an Apple Silicon Mac and uv.

git clone https://github.com/phykn/local-models.git && cd local-models
git clone https://github.com/jaredpalmer/kev.git vendor/kev && git -C vendor/kev checkout kev-1.0
uv sync --python 3.13
.venv/bin/llmctl pull qwen3.8-27b                 # weights go to weight/
HF_HUB_OFFLINE=0 .venv/bin/kevctl start kev-4b    # first start downloads Kev

Memory: Kev-4B ~16 GB, Kev-0.8B ~4 GB, Qwen3.8-27B ~21 GB, Qwen3.6-35B-A3B ~22 GB.

Related MCP server: claude-token-saver-mcp

Use

Mac app — one section for Kev, one for the LLMs; each runs one model at a time.

scripts/build-app.sh && open LOCAL-MODELS.app

Claude Code — register the MCP servers, then ask e.g. "Use kev to decide whether this ticket is urgent".

claude mcp add kev --scope user -- "$PWD/.venv/bin/kev-mcp"
claude mcp add llm-local --scope user -- "$PWD/.venv/bin/llm-mcp"

  • kev: choose, yes_no, score, decide, list_models. Check answers marked low_confidence.

  • llm-local: ask (pass think: true for harder problems), list_models.

Command line — kevctl and llmctl share the same commands.

.venv/bin/kevctl start kev-4b      # also: stop, status, list, port <model> <port>

HTTP — the LLMs serve an OpenAI-compatible API (/v1/chat/completions, default port 8082). Set model to the loaded id from /v1/models and send "enable_thinking": false to skip thinking. Kev serves POST /v1/systemone (see the Kev README).

Configure

kev.toml and llm.toml register the models (weights, port, memory, sampling). Pin run to a commit; unknown keys are an error.

Test

.venv/bin/pytest                                                   # fake servers
.venv/bin/pytest -m live                                           # running models
.venv/bin/python scripts/evaluate.py --data evals/ko/smoke.jsonl   # Kev accuracy and calibration

License

MIT. Kev, its weights and the Qwen weights are Apache-2.0.

This server cannot be deployed

Shareable, User-Owned Context BackendPeople, applications, and agents read, write, and coordinate together in one context lake. Reach it over MCP, a CLI, a Python SDK, or REST.fulcradynamics.comAd

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • Ollama MCP Server

    Jadael
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables Claude to delegate coding tasks to local Ollama models, reducing API token usage by up to 98.75% while leveraging local compute resources. Supports code generation, review, refactoring, and file analysis with Claude providing oversight and quality assurance.
    363 npm
    25
    AGPL 3.0

  • claude-token-saver-mcp

    BlackFoil
    A
    license
    A
    quality
    C
    maintenance
    Enables Claude Code to offload routine code generation and text processing tasks to a local Ollama LLM, saving Cloud API tokens and costs with automatic model selection and security features.
    11
    596 npm
    4
    Apache 2.0

  • localagents

    ccebelenski
    A
    license
    A
    quality
    B
    maintenance
    Enables Claude Code to delegate agent tasks to local LLMs via llama.cpp or vLLM servers on your own hardware, reducing API usage while maintaining full agent capabilities.
    11
    13
    MIT