Local AI with Ollama, WebUI & MCP on Windows

A self-hosted AI stack combining Ollama for running language models, Open WebUI for user-friendly chat interaction, and MCP for centralized model management—offering full control, privacy, and flexibility without relying on the cloud.

This sample project provides an MCP-based tool server for managing employee leave balance, applications, and history. It is exposed via OpenAPI using mcpo for easy integration with Open WebUI or other OpenAPI-compatible clients.

🚀 Features

• ✅ Check employee leave balance

• 📆 Apply for leave on specific dates

• 📜 View leave history

• 🙋 Personalized greeting functionality

📁 Project Structure

📋 Prerequisites

1. Windows 10 or later (required for Ollama)

2. Docker Desktop for Windows (required for Open WebUI and MCP)

   • Install from: Docker Desktop for Windows

🛠️ Workflow

1. Install Ollama on Windows

2. Pull the deepseek-r1 model

3. Clone the repository and navigate to the project directory

4. Run the docker-compose.yml file to launch services

Install Ollama

➤ Windows

1. Download the Installer:

   • Visit Ollama Download and click Download for Windows to get OllamaSetup.exe.

   • Alternatively, download from Ollama GitHub Releases.

2. Run the Installer:

   • Execute OllamaSetup.exe and follow the installation prompts.

   • After installation, Ollama runs as a background service, accessible at: http://localhost:11434.

   • Verify in your browser; you should see:

     Ollama is running

   

3. Start Ollama Server (if not already running):

   ollama serve

   • Access the server at: http://localhost:11434.

Verify Installation

Check the installed version of Ollama:

Expected Output:

Pull the deepseek-r1 Model

1. Pull the Default Model (7B):

Using PoweShell

To Pull Specific Versions:

2. List Installed Models:

Expected:

Expected Output:

4. Alternative Check via API:

Expected Output: A JSON response listing installed models, including deepseek-r1:latest.

4. Test the API via PowerShell:

Expected Response: A JSON object containing the model's response to the "Hello, world!" prompt.

5. Run and Chat the Model via PowerShell:

• This opens an interactive chat session with the deepseek-r1 model.

• Type /bye and press Enter to exit the chat session.

🐳 Run Open WebUI and MCP Server with Docker Compose

1. Clone the Repository:

   git clone https://github.com/ahmad-act/Local-AI-with-Ollama-Open-WebUI-MCP-on-Windows.git cd Local-AI-with-Ollama-Open-WebUI-MCP-on-Windows

2. To launch both the MCP tool and Open WebUI locally (on Docker Desktop):

   docker-compose up --build

   

This will:

• Start the Leave Manager (MCP Server) tool on port 8000

• Launch Open WebUI at http://localhost:3000

🌐 Add MCP Tools to Open WebUI

The MCP tools are exposed via the OpenAPI specification at: http://localhost:8000/openapi.json.

1. Open http://localhost:3000 in your browser.

2. Click the Profile Icon and navigate to Settings.

3. Select the Tools menu and click the Add (+) Button.

4. Add a new tool by entering the URL: http://localhost:8000/.

💬 Example Prompts

Use these prompts in Open WebUI to interact with the Leave Manager tool:

• Check Leave Balance:

  Check how many leave days are left for employee E001

  

• Apply for Leave:

  Apply 

• View Leave History:

  What's the leave history of E001?

  

• Personalized Greeting:

  Greet me as Alice

  

🛠️ Troubleshooting

• Ollama not running: Ensure the service is active (ollama serve) and check http://localhost:11434.

• Docker issues: Verify Docker Desktop is running and you have sufficient disk space.

• Model not found: Confirm the deepseek-r1 model is listed with ollama list.

• Port conflicts: Ensure ports 11434, 3000, and 8000 are free.

📚 Additional Resources

• Ollama Documentation

• Open WebUI Documentation

• Docker Desktop Documentation

• MCP Documentation

• OpenAPI Tool Servers

• mcpo - Works with OpenAPI tools, SDKs, and UIs