Skip to main content
Glama
unscene

Gemini Audio Upload

by unscene
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

Gemini Multimodal Audio Upload

CodeRabbit Pull Request Reviews

This project provides a Model Context Protocol (MCP) server that enables audio analysis using Google's Gemini models. It allows you to upload audio files, provide optional context (JSON), and receive detailed analysis based on your prompts.

Features

  • Audio Analysis: Upload and analyze audio files (WAV, MP3, etc.) using Google Gemini.

  • Multimodal Context: Support for providing additional context via JSON files or strings.

  • System Instructions: Ability to provide system instructions (e.g., "Gem" definitions) to guide the model's behavior.

  • MCP Server: Exposes functionality as an MCP tool, making it compatible with MCP clients like Claude Desktop or VS Code extensions.

Related MCP server: MCP Server Whisper

Prerequisites

  • Python 3.10 or higher

  • A Google Cloud Project with the Gemini API enabled.

  • An API key for the Gemini API.

Installation

  1. Clone the repository:

    git clone https://github.com/unscene/gemini-audio-upload.git
cd gemini-audio-upload

  2. Install dependencies with uv:

    uv sync

Configuration

  1. Create a .env file in the root directory:

    cp .env.example .env # If .env.example exists, otherwise create new

  2. Add your Google API key to the .env file:

    GOOGLE_API_KEY=your_api_key_here

Usage

Running the MCP Server

You can run the MCP server directly using uv:

uv run gemini_audio/mcp_server.py

However, it is typically run by an MCP client.

MCP Tool: analyze_audio

The server exposes a single tool: analyze_audio.

Arguments:

  • audio_path (string, required): The absolute path to the audio file you want to analyze.

  • prompt (string, optional): The prompt to guide the analysis. Default: "Describe this audio."

  • json_path (string, optional): Path to a JSON file containing context data.

  • json_context (string, optional): A JSON string containing context data (overrides json_path).

  • instruction_file (string, optional): Path to a text file containing system instructions.

  • model (string, optional): The Gemini model to use. Default: "gemini-1.5-pro".

Example Usage (Conceptual)

If you are using an MCP client, you might ask:

"Analyze the audio file at C:\path\to\recording.wav and tell me if the speaker sounds happy."

The client would call the analyze_audio tool with:

  • audio_path: C:\path\to\recording.wav

  • prompt: "Tell me if the speaker sounds happy."

Client Configuration

Claude Desktop App

To use this server with the Claude Desktop App, add the following configuration to your claude_desktop_config.json file.

Windows Location: %APPDATA%\Claude\claude_desktop_config.json macOS Location: ~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "gemini-audio": {
      "command": "uv",
      "args": [
        "--directory",
        "/absolute/path/to/gemini-audio-upload",
        "run",
        "gemini_audio/mcp_server.py"
      ],
      "env": {
        "GOOGLE_API_KEY": "your_api_key_here"
      }
    }
  }
}

Note: Replace /absolute/path/to/gemini-audio-upload with the actual path to where you cloned this repository. You can also set the GOOGLE_API_KEY in the .env file in the project directory instead of the config JSON, provided uv picks it up correctly or you use the python executable directly.

VS Code (MCP Extension)

If you are using an MCP extension in VS Code (like the official "Model Context Protocol" extension), you can typically configure it in your VS Code settings.json:

"mcp.servers": {
    "gemini-audio": {
        "command": "uv",
        "args": [
            "--directory",
            "C:\\absolute\\path\\to\\gemini-audio-upload",
            "run",
            "gemini_audio/mcp_server.py"
        ],
        "env": {
            "GOOGLE_API_KEY": "your_api_key_here"
        }
    }
}

License

MIT

Available Tools

1 tool
analyze_audioC

Analyze an audio file using Google Gemini.

Args: audio_path: Path to the audio file (wav, mp3, etc.) prompt: The prompt to send to Gemini. json_path: Optional path to a JSON file to provide as context. json_context: Optional JSON string to provide as context (overrides json_path if provided). instruction_file: Optional path to a text file containing system instructions. model: The Gemini model to use.

ParametersJSON Schema
NameRequiredDescriptionDefault
audio_pathYes
promptNoDescribe this audio.
json_pathNo
json_contextNo
instruction_fileNo
modelNogemini-3-pro-preview

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. While 'analyze' implies a read-only operation, the description doesn't clarify permissions, rate limits, costs, response format, or error handling. It mentions using Google Gemini but lacks details on what analysis entails or behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded with the core purpose in the first sentence. The parameter list is structured but could be more concise; some explanations are brief yet clear. No redundant information is present, making it efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 6 parameters with 0% schema coverage and no annotations, the description provides basic parameter info but lacks behavioral context, usage guidelines, and output details. An output schema exists, so return values needn't be explained, but for a complex tool with multiple inputs, more completeness on constraints and interactions would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It lists all 6 parameters with brief explanations (e.g., 'Path to the audio file', 'The prompt to send to Gemini'), adding basic semantics beyond schema titles. However, it doesn't detail formats (e.g., audio file types beyond 'wav, mp3, etc.'), constraints, or interactions between parameters like json_context overriding json_path.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Analyze an audio file using Google Gemini.' It specifies the verb ('analyze'), resource ('audio file'), and technology ('Google Gemini'), making the function unambiguous. However, with no sibling tools provided, there's no opportunity to differentiate from alternatives, preventing a score of 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, prerequisites, or typical use cases. It simply lists parameters without contextual advice. With no sibling tools mentioned, there's no comparison, but general usage context is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool update
    • First observedanalyze_audio
Deploy Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.1/5.0

Scored across 1 tool

Disambiguation5/5

With only one tool, there is no possibility of ambiguity or overlap between tools. The tool 'analyze_audio' has a single, clear purpose: analyzing audio files using Google Gemini, so an agent cannot misselect between non-existent alternatives.

Naming Consistency5/5

The single tool name 'analyze_audio' follows a clear verb_noun pattern, and with no other tools to compare, there is no inconsistency. The naming is straightforward and readable, adhering to snake_case conventions without deviation.

Tool Count2/5

A single tool is too few for the server's apparent scope of audio analysis, as it lacks basic operations like listing available models, uploading files, or managing audio resources. This minimal set may force agents into dead ends or require workarounds for common tasks.

Completeness2/5

The tool set is severely incomplete for audio analysis; it only provides analysis but lacks creation, retrieval, update, or deletion of audio files or results. There are obvious gaps, such as no way to handle audio preprocessing, batch operations, or result storage, which will likely cause agent failures in broader workflows.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers