MCP Local LLM Server

A FastMCP server that exposes a locally-hosted llama.cpp language model as MCP tools, plus a suite of utility tools for weather, news, web fetching, file I/O, stock data, summarization, and more.

Overview

This server lets any MCP-compatible client (e.g. Claude Desktop, Cursor) use a local GGUF model for text generation and chat, without sending data to an external API. The model is served by a llama-server subprocess; the MCP server communicates with it over a local HTTP port.

src/
├── llm_server.py        # Entry point, argument parsing, server startup
├── model.py             # llama-server lifecycle, token generation
├── upload.py            # POST /upload endpoint for PDF and Markdown file uploads
├── resources.py         # MCP resource: llm://info
└── tools/
    ├── generate.py      # Tool: generate
    ├── chat.py          # Tool: chat
    ├── weather.py       # Tool: get_weather
    ├── date_time.py     # Tool: get_datetime
    ├── fetch_url.py     # Tool: fetch_url
    ├── news.py          # Tool: news_headlines
    ├── read_pdf.py      # Tool: read_pdf
    ├── read_markdown.py # Tool: read_markdown
    ├── create_file.py   # Tool: create_file
    ├── list_directory.py# Tool: list_directory
    ├── stock_price.py   # Tool: get_stock_price
    ├── summarize.py     # Tool: summarize_text
    ├── agent.py         # Tool: run_agent (autonomous ReAct agent)
    ├── explain_code.py  # Tool: explain_code (coding tutor)
    ├── review_code.py   # Tool: review_code  (coding tutor)
    ├── coding_tutor.py  # Tool: coding_tutor (orchestrating tutor agent)
    ├── transcribe_audio.py # Tool: transcribe_audio
    ├── text_to_speech.py   # Tool: text_to_speech
    ├── word_definition.py  # Tool: define_word
    └── random_joke.py      # Tool: get_random_joke

Related MCP server: MCP Tool Server

Requirements

• Python 3.10+

• A llama-server binary from llama.cpp

• A GGUF model file (e.g. models/Qwen2.5-7B-Instruct-Q4_K_M.gguf)

• CUDA-capable GPU recommended for large models

Install dependencies:

pip install -r requirements.txt

On Windows, IANA timezone data is not bundled with Python. Install it for the get_datetime tool to support non-UTC timezones:

pip install tzdata

The transcribe_audio tool requires faster-whisper, which is listed in requirements.txt but has an optional CUDA-accelerated variant. For GPU inference, install the matching PyTorch CUDA build first:

# CPU-only (default)
pip install faster-whisper

# GPU (CUDA 12)
pip install faster-whisper
pip install torch --index-url https://download.pytorch.org/whl/cu128

Configuration

Some tools require API keys. Create a .env file in the project root (already gitignored):

NEWSAPI_KEY=your_key_here

The server loads this file automatically on startup.

All other tools work without any API key.

Starting the Server

stdio transport (for MCP clients like Claude Desktop)

python src/llm_server.py \
  --model models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
  --llama-server /path/to/llama-server

HTTP transport (for network clients or testing with curl)

python src/llm_server.py \
  --model models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
  --llama-server /path/to/llama-server \
  --transport http \
  --port 5174

CLI flags

HTTP Endpoints

These endpoints are only available when using --transport http.

POST /upload

Upload a PDF or Markdown file to the server and receive an upload_id to pass to run_agent.

Supported types: .pdf, .md, .markdown

Request: multipart/form-data with a single field named file.

Response:

{
  "upload_id": "3f8a1c...",
  "filename": "report.pdf",
  "size": 84210
}

Uploaded files are stored in uploads/ at the project root and deleted when the server shuts down.

Example:

curl -X POST http://localhost:5174/upload \
  -F "file=@/path/to/report.pdf"

curl -X POST http://localhost:5174/upload \
  -F "file=@/path/to/notes.md"

Tools

generate

Generate text from a raw prompt. The input prompt is not included in the returned text.

Parameters

Returns: The generated text as a plain string.

Example

{
  "prompt": "The capital of France is",
  "max_new_tokens": 50,
  "temperature": 0
}

chat

Chat with the local LLM using a conversation history.

Parameters

Returns: The assistant's reply as plain text.

Example

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",   "content": "What is the Eiffel Tower?"}
  ],
  "temperature": 0.7,
  "max_new_tokens": 256
}

get_weather

Fetch current weather for any location using the free Open-Meteo API. No API key required.

Parameters

Returns: Current conditions, temperature, humidity, and wind speed.

Example output

Weather in London, England, United Kingdom:
  Conditions:  Partly cloudy
  Temperature: 12.3°C
  Humidity:    74%
  Wind:        18.5 km/h

get_datetime

Return the current date and time for any IANA timezone. No API key required.

Parameters

Returns: A formatted date/time string, e.g. "2025-03-01 14:30:00 EST (UTC-0500)".

Windows note: Non-UTC timezones require pip install tzdata.

Example

{"timezone": "America/Chicago"}

fetch_url

Fetch the content of any URL and return it as plain text. HTML pages are stripped of tags; JSON and plain-text responses are returned as-is. No API key required.

Parameters

Returns: Extracted page text, truncated to max_chars if needed. Returns a descriptive error string on connection failure rather than raising an exception.

Example

{
  "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
  "max_chars": 2000
}

news_headlines

Fetch the latest news headlines, optionally filtered by topic. Requires a free NewsAPI key.

Parameters

Returns: Numbered list of headlines with source, publication date, and URL.

Example

{"topic": "artificial intelligence", "max_results": 3}

Example output

1. [BBC News] OpenAI releases new model
   Published: 2025-03-01
   https://bbc.co.uk/...

2. [Reuters] ...

read_pdf

Extract and return the text content of a PDF file, organised by page. Image-only (scanned) PDFs return a clear message rather than empty output.

Parameters

Returns: Extracted text organised by page, truncated to max_chars if needed.

Example

{"file_path": "/home/user/documents/report.pdf", "max_chars": 10000}

read_markdown

Read and return the contents of a Markdown file.

Parameters

Returns: The file's text content, truncated to max_chars if needed.

Example

{"file_path": "/home/user/documents/notes.md"}

create_file

Create a new file with the given name and content. Rejects path traversal attempts and validates the filename before writing.

Parameters

Returns: A JSON object with keys:

• success — true if the file was created

• file_path — absolute path to the created file (null on failure)

• error — error message if success is false

• message — human-readable status

Examples

{"file_name": "notes.md", "content": "# My Notes\n"}

{
  "file_name": "config.json",
  "content": "{\"debug\": true}",
  "directory": "src",
  "overwrite": true
}

list_directory

List the files and directories at a given path, with optional glob filtering and recursive traversal.

Parameters

Returns: A formatted listing showing [DIR] and [FILE] entries with file sizes, plus a summary count. Truncation is noted if max_results is reached.

Examples

{"path": "/home/user/projects"}

{
  "path": "/home/user/projects/mcp-server",
  "pattern": "*.py",
  "recursive": true
}

Example output

Directory: /home/user/projects/mcp-server/src

  [DIR]  tools/
  [FILE] llm_server.py  (4 KB)
  [FILE] model.py       (7 KB)
  [FILE] resources.py   (1 KB)
  [FILE] upload.py      (2 KB)

4 item(s) shown

get_stock_price

Get the current stock price and key market data for a ticker symbol via the Yahoo Finance API. No API key required.

Parameters

Returns: Current price, daily change, day range, 52-week range, volume, and market cap.

Example

{"ticker": "NVDA"}

Example output

NVIDIA Corp (NVDA)
  Price:       875.40 USD
  Change:      +12.30 (+1.43%)
  Prev close:  863.10 USD
  Day range:   860.00 – 879.50 USD
  52-wk range: 410.00 – 974.00 USD
  Volume:      42,381,200
  Market cap:  2.16T USD

Tip: Crypto pairs are also supported (e.g. "BTC-USD", "ETH-USD").

summarize_text

Summarize a block of text using the local LLM. Long texts are automatically split into chunks, each summarized independently, then merged into a single coherent summary.

Parameters

Returns: A concise summary of the input text.

Examples

{"text": "... (long article) ..."}

{
  "text": "... (meeting transcript) ...",
  "focus": "action items",
  "max_length": 150
}

{
  "text": "... (technical document) ...",
  "focus": "key risks",
  "max_length": 300
}

transcribe_audio

Transcribe an audio file to text using a local Whisper model via faster-whisper. Runs entirely on-device — no API key or internet connection required. Models are downloaded automatically on first use and cached locally.

Supported formats: mp3, mp4, wav, flac, ogg, m4a, webm, and most ffmpeg-readable formats.

Parameters

Returns: A JSON object with keys:

• success — true if transcription succeeded

• text — the transcribed text (null on failure)

• language — detected or forced language code

• duration_seconds — audio duration in seconds

• error — error message if success is false

Examples

{"audio_path": "/home/user/recordings/meeting.mp3"}

{
  "audio_path": "/home/user/recordings/lecture.wav",
  "model_size": "small",
  "language": "en"
}

Tip: Use "tiny" or "base" for fast transcription of short clips. Use "small" or higher for better accuracy on noisy audio or non-English speech.

run_agent

Run an autonomous ReAct agent powered by the local LLM. The agent reasons step by step and calls tools as many times as needed before producing a final answer.

How it works

User goal
   ↓
LLM decides: call a tool or answer?
   ↓ (if tool)
Tool executes → result fed back to LLM
   ↓
LLM decides again … (repeats up to max_steps)
   ↓ (when done)
FINAL answer returned

Parameters

Returns: The agent's final answer as plain text.

If you see Agent stopped after N steps without a FINAL answer, the agent exhausted its iterations. Either increase max_steps or simplify the goal.

Tools available to the agent

Examples

{"goal": "What should I wear in Paris today?"}

{"goal": "Compare the weather in London and Tokyo, then tell me which city is warmer."}

{"goal": "Summarise this document", "upload_id": "3f8a1c..."}

{
  "goal": "What are the top AI news stories today?",
  "max_steps": 5
}

text_to_speech

Convert text to an MP3 audio file using Google Text-to-Speech (gTTS). Requires an internet connection. No API key required.

Dependency: Install gTTS before using this tool:

pip install gtts

Parameters

Returns: A JSON object with keys:

• success — true if the file was saved successfully

• output_path — absolute path to the saved MP3 (null on failure)

• error — error message if success is false

Examples

{
  "text": "Hello, world!",
  "output_path": "output/hello.mp3"
}

{
  "text": "Bonjour le monde",
  "output_path": "output/bonjour.mp3",
  "lang": "fr",
  "slow": true
}

define_word

Look up the definition, phonetics, synonyms, and antonyms of an English word using the free Dictionary API. No API key required.

Parameters

Returns: A JSON object with keys:

• success — true if the word was found

• word — the normalised word that was looked up

• results — list of meanings, each containing:

  • phonetic — IPA phonetic spelling (may be null)

  • part_of_speech — e.g. "noun", "verb", "adjective"

  • definitions — up to 3 definitions, each with definition, example (may be null), synonyms (up to 5), antonyms (up to 5)

  • synonyms — up to 5 synonyms for this part of speech

  • antonyms — up to 5 antonyms for this part of speech

• error — error message if success is false

Example

{"word": "ephemeral"}

Example output (abbreviated)

{
  "success": true,
  "word": "ephemeral",
  "results": [
    {
      "phonetic": "/ɪˈfɛm(ə)r(ə)l/",
      "part_of_speech": "adjective",
      "definitions": [
        {
          "definition": "Lasting for a very short time.",
          "example": "fashions are ephemeral",
          "synonyms": ["transitory", "transient", "fleeting"],
          "antonyms": ["permanent", "eternal"]
        }
      ],
      "synonyms": ["transitory", "transient"],
      "antonyms": ["permanent"]
    }
  ],
  "error": null
}

get_random_joke

Fetch a random joke from the free JokeAPI. No API key required.

Parameters

Returns: A JSON object with keys:

• success — true if a joke was returned

• category — the category the joke belongs to

• type — "single" or "twopart"

• joke — the joke text; two-part jokes are formatted as "setup\n\n— delivery"

• error — error message if success is false

Note: The "Dark" category is unavailable when safe_mode is true.

Examples

{"category": "Programming"}

{
  "category": "Pun",
  "joke_type": "twopart",
  "safe_mode": true
}

Coding Tutor Tools

Three tools that turn the server into an interactive programming tutor. The high-level entry point is coding_tutor; the two supporting tools (explain_code, review_code) can also be called directly.

coding_tutor

An autonomous ReAct agent specialised for teaching. It reasons step by step, calling explain_code, review_code, and fetch_url as needed, then produces a pedagogical response.

Parameters

Returns: A teaching response as plain text.

Tools available to the tutor

Examples

{"question": "Why does my list comprehension give the wrong result?\n\nresult = [x * 2 for x in [1, 2, 3] if x > 1]"}

{"question": "Explain the difference between a shallow copy and a deep copy in Python, with examples."}

Tip: The tutor infers skill level from your question — use plain language for beginner explanations, technical terminology for advanced ones.

explain_code

Explain a code snippet using the local LLM, tailored to the learner's skill level.

Parameters

Returns: A plain-text explanation of the code.

Example

{
  "code": "result = {k: v for k, v in zip(keys, values)}",
  "language": "python",
  "level": "beginner"
}

review_code

Review a code snippet for issues. Outputs a structured report: overall impression, numbered issues with severity, positives, and a top recommendation.

Parameters

Returns: A structured review with four sections: Overall Impression, Issues Found, Positives, and Top Recommendation.

Example

{
  "code": "def get_user(id):\n    return db.execute(f'SELECT * FROM users WHERE id={id}')",
  "language": "python",
  "focus": "security"
}

Resources

llm://info

Returns metadata about the currently loaded model (path, context size, GPU layers).

Configuring with Claude Desktop

Add the server to your claude_desktop_config.json:

{
  "mcpServers": {
    "local-llm": {
      "command": "python",
      "args": [
        "src/llm_server.py",
        "--model", "models/Qwen2.5-7B-Instruct-Q4_K_M.gguf",
        "--llama-server", "/path/to/llama-server"
      ],
      "cwd": "/absolute/path/to/mcp-server"
    }
  }
}

Notes

• The model is loaded once at startup via llama-server and held in memory for the lifetime of the server.

• GPU offloading is controlled by --gpu-layers; -1 offloads all layers.

• --context-size sets the total token budget shared between prompt and generated output. Increase it if you experience truncation on long responses.

• The HTTP transport binds to 0.0.0.0 by default, making it accessible from other machines on the network. Use --host 127.0.0.1 to restrict to localhost. CORS is enabled for all origins — restrict allow_origins before exposing to untrusted networks.

• Uploaded files (POST /upload) are stored in uploads/ at the project root and automatically deleted on server shutdown.

Running a standalone llama-server (e.g. for opencode)

llama-server \
  --model /path/to/model.gguf \
  --port 8000 \
  --host 127.0.0.1 \
  --n-gpu-layers -1 \
  --ctx-size 16384 \
  --no-mmap