Skip to main content
Glama
naka-koma

local-llm-mcp

by naka-koma
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

local-llm-mcp

English | 日本語

https://github.com/naka-koma/local-llm-mcp

A thin MCP server for delegating lightweight tasks from Claude Code (or any other MCP-compatible client) to local or cloud LLMs via LiteLLM.

Designed as a general-purpose tool with no dependency on any specific project. Set it up once, and you can reuse the same configuration from any project on your machine.

What is this for?

Claude Code's Task tool (the subagent mechanism) only lets you choose between sonnet, opus, and haiku as the model. It isn't designed to let you use other vendors' models (such as local LLMs) directly as subagents.

Instead of "adding more subagents" to work around this limitation, this tool adds "one more tool" to enable delegation to any LLM.

Claude Code ──(MCP)──> local-llm-mcp ──(HTTP)──> LiteLLM ──> Ollama (local) / cloud API

It's intended for offloading lightweight tasks that the main agent doesn't need to handle itself — summarizing test results, pre-reading logs, simple classification, and so on.

Related MCP server: litellm-mcp

Prerequisites

  • Python 3.10+

  • Ollama running locally (if using a local model)

  • LiteLLM running as a proxy

This repository provides only the MCP server itself. Setting up Ollama or LiteLLM is out of scope.

Setup

1. Install dependencies

pip install -r requirements.txt

2. Configure environment variables

Refer to .env.example and set the LiteLLM endpoint and model name (if you use a .env file, you'll need something like python-dotenv to load it separately; setting the variables directly in your environment also works).

Variable

Description

Default

LITELLM_BASE_URL

LiteLLM proxy endpoint

http://localhost:4000

LITELLM_MODEL

The model_name registered in LiteLLM's config.yaml

local-gemma

LITELLM_TIMEOUT_SECONDS

Response timeout (seconds)

30

3. Register with Claude Code

Registering with the -s user scope makes the tool available from any project on this machine.

Run via uvx (no clone needed — for embedding in other projects)

If uv is installed, you can run it directly without cloning the repository. This is the recommended approach when embedding into another project's setup steps (e.g. Flowrite).

claude mcp add local-llm -s user -- uvx --from git+https://github.com/naka-koma/local-llm-mcp local-llm-mcp

On first run, uv automatically builds the environment including dependencies, so pip install isn't needed.

Use the setup script (recommended for local development)

Runs dependency installation and claude mcp add together.

git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp

# macOS / Linux / Git Bash
bash scripts/setup.sh

# Windows PowerShell
pwsh scripts/setup.ps1

Register manually

git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp
pip install -r requirements.txt
claude mcp add local-llm -s user -- python /path/to/local-llm-mcp/server.py

On Windows, if python's PATH is unreliable, we recommend using the py launcher (C:\Windows\py.exe, which has system-wide PATH set up).

claude mcp add local-llm -s user -- py D:\path\to\local-llm-mcp\server.py

Provided tool

ask_local_llm(prompt: str) -> str

Sends a prompt to an LLM via LiteLLM and returns the response text. On connection errors or timeouts, it doesn't raise an exception — it returns a string describing the error instead.

How to use it in your project

We recommend adding a conditional instruction like the following to the consuming project's CLAUDE.md (or equivalent):

## Using a local LLM

If an MCP tool called `local-llm` is available, lightweight tasks such as
summarizing test results or pre-reading logs may be delegated to it.

Even in an environment where the tool doesn't exist, this has no impact
on this project's behavior (delegation is "use it if it's there," not a
required dependency).

License

MIT License. See LICENSE for details.

Available Tools

1 tool
ask_local_llmA

LiteLLM経由でローカル/クラウドLLMにプロンプトを投げ、返答のテキストを受け取る。

要約・分類・ログの下読みなど、呼び出し元のメインエージェントが 自分でやるまでもない軽量なタスクをオフロードしたいときに使う。

Args: prompt: LLMに渡すプロンプト文字列。

Returns: LLMからの返答テキスト。接続エラーやタイムアウトの場合は、 その旨を説明する文字列を返す（例外は投げない）。

ParametersJSON Schema
NameRequiredDescriptionDefault
promptYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses that on connection errors or timeouts, the tool returns an explanation string instead of throwing an exception—valuable behavioral context beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured: a summary sentence, usage context, and documented args/returns. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no annotations, the description covers what it does, when to use it, and the return behavior including error handling. It lacks explicit prerequisites or model specifics, but these are not essential for basic usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% with only one required 'prompt' parameter. The description's Args section explains that prompt is the string passed to the LLM, adding a small but useful layer of meaning to the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool sends a prompt to a local/cloud LLM via LiteLLM and returns the response text. The verb '投げ' (send) and resource 'LLM' are specific, and the purpose is unambiguous even without sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool for offloading lightweight tasks like summarization, classification, or log pre-reading. It implies not for tasks the main agent should handle, but lacks explicit when-not conditions or named alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev0.1.0
    • First observedask_local_llm
Deploy Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.3/5.0

Scored across 1 tool

Disambiguation5/5

With only one tool, there is no possibility of confusion or overlap. The tool's purpose is clearly defined and isolated.

Naming Consistency5/5

The single tool name 'ask_local_llm' follows a clear verb_object pattern. Naming consistency is trivially maintained with only one tool.

Tool Count2/5

The server has only one tool, which falls below the typical well-scoped range of 3-15 tools. Even though the tool is functional and not trivial, a single-tool server feels incomplete and lacks breadth.

Completeness4/5

For the stated purpose of offloading lightweight LLM queries, the tool covers the main use case and handles errors gracefully. However, there is no tool to list models or configure parameters, which could be considered minor gaps.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers