AI Voice Assistant with MCP Tool Calling

A fully free, end-to-end voice-enabled AI assistant built with Python.

Features

• 🎙️ Voice Input — speak naturally; the assistant understands you

• 🔊 Voice Output — responses are read aloud via offline TTS

• 🤖 Gemini AI — context-aware, multi-turn conversation

• 🛠️ 7 MCP Tools available to the AI:

  1. Calculator — safe math expression evaluator

  2. Web Search — DuckDuckGo (no API key)

  3. Weather — real-time via wttr.in (no API key)

  4. Date/Time — current date and time

  5. Read File — read any local file

  6. Write File — write/append to a local file

  7. List Directory — browse local folders

• 💬 Multi-turn memory — remembers conversation context

• ⌨️ Text mode — works without a microphone (--text flag)

Related MCP server: TOOL4LM

Prerequisites

• Python 3.11+

• A free Gemini API key — get one at https://aistudio.google.com/app/apikey

• Internet connection (for speech recognition, Gemini API, and web tools)

• Microphone (optional — text mode works without one)

Installation

1. Clone / download the project

# If using git:
git clone <your-repo-url>
cd "AI voice assistant"

# Or just open the folder in your terminal
cd "C:\Users\sasid\Downloads\AI voice assistant"

2. Create and activate a virtual environment (recommended)

python -m venv .venv

# Windows:
.venv\Scripts\activate

# macOS / Linux:
source .venv/bin/activate

3. Install PyAudio (Windows — required for microphone)

PyAudio on Windows needs a pre-built binary. The easiest way:

pip install pipwin
pipwin install pyaudio

Or download the correct .whl from https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio and install with:

pip install PyAudio‑0.2.14‑cpXX‑cpXX‑win_amd64.whl

4. Install remaining dependencies

pip install -r requirements.txt

5. Set your Gemini API key

Option A — .env file (recommended):

copy .env.example .env
# Then open .env and replace "your_gemini_api_key_here" with your actual key

Option B — edit config.py directly:

Open config.py and change:

GEMINI_API_KEY: str = os.getenv("GEMINI_API_KEY", "YOUR_GEMINI_API_KEY_HERE")

to:

GEMINI_API_KEY: str = "your_actual_api_key"

Running

Voice mode (default — microphone + TTS)

python main.py

Text-only mode (no microphone needed)

python main.py --text

List available TTS voices

python main.py --list-voices

Example Interactions

Project Structure

AI voice assistant/
├── main.py          # Entry point — CLI, banner, main loop
├── assistant.py     # Gemini + MCP integration (agentic tool-call loop)
├── speech.py        # SpeechRecognition (STT) + pyttsx3 (TTS)
├── mcp_server.py    # MCP tool server with 7 built-in tools
├── config.py        # All settings and API key placeholder
├── requirements.txt # Python dependencies
├── .env.example     # API key template
└── README.md        # This file

Customisation

Change the AI's personality

Edit SYSTEM_PROMPT in config.py.

Adjust microphone sensitivity

Edit MIC_ENERGY_THRESHOLD in config.py (lower = more sensitive).

Change TTS voice or speed

Edit TTS_RATE and TTS_VOICE_PREFERENCE in config.py. Run python main.py --list-voices to see available voice names.

Add more MCP tools

Open mcp_server.py, add a new function, then register it in list_tools() and call_tool().

Gemini Free Tier Limits

Get your key at: https://aistudio.google.com/app/apikey

Troubleshooting

"No module named 'pyaudio'" → See PyAudio installation step above.

"Could not understand audio" → Speak clearly; adjust MIC_ENERGY_THRESHOLD lower in config.py.

"Speech recognition service error" → Check your internet connection (Google Web Speech API requires internet).

Gemini 429 / rate limit error → You've hit the free tier limit. Wait a minute and try again.

Assistant doesn't speak / TTS silent → Check system audio / volume. Try python main.py --list-voices to verify pyttsx3 works.

License

MIT — free to use, modify, and distribute.