mcp-lab

Companion repo for the "MCP" post (#14) on karthiksolution.wordpress.com, in the AI/agents series. Builds on the tracer/drift-monitor infra established across earlier posts, most directly structured-outputs-lab.

What's in here

A real MCP (Model Context Protocol) server, mcp_server/server.py, exposing all 3 of the protocol's primitives around a small ops- knowledge domain:

• Tools: search_incidents(query), get_runbook(service)

• Resource: incident://{incident_id}, a full incident record addressed by URI

• Prompt: postmortem_summary(incident_id), a reusable, pre-filled prompt template

Two ways to connect a Gemini-based client to it:

• mcp_client_native: google-genai's own experimental built-in MCP bridge, pass a live MCP session directly into generation config and automatic function calling drives it. Tools only, async client only, and (see Pitfalls) requires passing config as a plain dict, not a types.GenerateContentConfig object, or it crashes.

• mcp_client_manual: a hand-written bridge covering all 3 primitives, converting MCP tool schemas to Gemini function declarations, running the tool-call loop by hand, and manually fetching the resource and prompt the native bridge can't touch.

common/evaluator.py's ToolCallEvaluator checks structural correctness against real MCP call history (response.automatic_function_calling_history for the native bridge, the manual loop's own return value for the other), not an LLM judge, this post is about protocol plumbing, not prompt content quality.

Related MCP server: devops-mcp-server

4 real bugs found while building this, all in Pitfalls

1. A subprocess launched with the bare string "python" instead of sys.executable used the system Python, not the venv, and couldn't find mcp at all.

2. GenerateContentConfig(tools=[session]) crashes with TypeError: cannot pickle '_asyncio.Task' object, generate_content unconditionally deep-copies its config before reaching any MCP-specific handling. Passing config as a plain dict avoids it entirely.

3. Schema.from_json_schema() expects a JSONSchema object, not the plain dict Tool.input_schema actually is, JSONSchema.model_validate(...) first is required.

4. Replaying a function_call part back into conversation history for a hand-written multi-turn loop needs the original part's thought_signature preserved, reconstructing a new Part from just the extracted call drops it and the API rejects the next turn.

Setup (Windows / PowerShell)

python -m venv venv
.\venv\Scripts\Activate.ps1
pip install -r requirements.txt
copy .env.example .env

Edit .env and set GEMINI_API_KEY to your real key.

Running the demo

python run_demo.py

Running the offline validation (no API key required beyond a placeholder)

python -m tests_offline.test_data_mock
python -m tests_offline.test_bridge_conversion_mock
python -m tests_offline.test_evaluator_mock
python -m tests_offline.test_llm_utils_config_mock

Running the evaluator

python run_eval.py --save-baseline
python run_eval.py

License

MIT, see LICENSE.