web-search-mcp

An MCP server that gives IBM BOB live web search via Gemini's built-in grounding, registered as an upstream on the LiteLLM MCP Gateway.

BOB  ──►  LiteLLM proxy (MCP Gateway)  ──►  this server  ──►  LiteLLM proxy  ──►  Gemini (grounding)

LiteLLM connects to this server as an MCP client and re-exposes its tool at LiteLLM's own /mcp endpoint, so BOB gains search through a config entry rather than a code change.

Instead of returning raw search results, the tool calls a Gemini model through the LiteLLM proxy with the googleSearchRetrieval grounding tool. Gemini searches the web internally and returns a synthesised answer with source citations.

One tool is exposed:

1. Credentials

Three values are required; the server refuses to start without them.

1. Gemini API key — from aistudio.google.com → Get API key (starts with AIza). Set in your LiteLLM config as GEMINI_API_KEY. → used by LiteLLM, not this server directly.

2. LiteLLM API key — the master key set in your LiteLLM proxy config. → LITELLM_API_KEY

3. MCP auth token — a shared secret you generate yourself:

   python -c "import secrets; print(secrets.token_urlsafe(32))"

   → MCP_AUTH_TOKEN (this server) and WEB_SEARCH_MCP_TOKEN (LiteLLM env)

Related MCP server: Ask Google MCP Server

2. Run it locally

cp .env.example .env   # fill in LITELLM_BASE_URL, LITELLM_API_KEY, MCP_AUTH_TOKEN
uv sync
uv run web-search-mcp

In a second terminal, start the LiteLLM proxy (run from the project root):

export GEMINI_API_KEY=AIzaSy...
export WEB_SEARCH_MCP_TOKEN=<same value as MCP_AUTH_TOKEN>
uvx --from 'litellm[proxy]==1.94.1' litellm --config litellm/config.example.yaml

Verify the MCP server:

curl -s localhost:8000/healthz   # {"status":"ok"} -- no auth needed
curl -i localhost:8000/mcp       # 401, because no token was sent

Then inspect the tool interactively:

npx @modelcontextprotocol/inspector

Connect with transport Streamable HTTP to http://127.0.0.1:8000/mcp, add header Authorization: Bearer <MCP_AUTH_TOKEN>, then call web_search from the Tools tab.

3. Register with LiteLLM

Merge the mcp_servers block from litellm/config.example.yaml into your LiteLLM config:

mcp_servers:
  web_search:
    url: "https://your-host/mcp"
    transport: http
    auth_type: bearer_token
    auth_value: os.environ/WEB_SEARCH_MCP_TOKEN
    allowed_tools: ["web_search"]

web_search should appear in LiteLLM's tool list (UI → MCP Servers, or through the proxy's /mcp endpoint). Then ask BOB something that requires current information and check that it calls the tool and cites URLs.

4. Deploy

docker build -t web-search-mcp .
docker run -p 8000:8000 --env-file .env web-search-mcp

The image is platform-agnostic — Code Engine, OpenShift, Kubernetes, Cloud Run. It binds 0.0.0.0 and honours an injected PORT. Point /healthz at your platform's liveness probe; it is intentionally unauthenticated so probes work without a token.

Supply LITELLM_BASE_URL, LITELLM_API_KEY, and MCP_AUTH_TOKEN as platform secrets, never baked into the image.

Layout

src/web_search_mcp/
  config.py         environment settings; fails fast when required values are missing
  gemini_client.py  LiteLLM/Gemini grounding client -- knows Gemini, not MCP
  cache.py          TTL/LRU cache to avoid redundant Gemini calls
  server.py         MCPServer, the web_search tool, /healthz, ASGI assembly
  auth.py           bearer-token ASGI middleware
  __main__.py       uvicorn entrypoint

Run the tests (fully mocked — no network, no API calls):

uv run pytest

Notes for whoever maintains this

• Gemini grounding returns an answer, not a list of results. The tool response shape is {answer, sources} rather than {results[]}. Bob reads the answer and cites the source URLs directly.

• sources may be empty. Gemini sometimes omits grounding citations even when it used search internally. This is not an error.

• Error messages tell the model whether retrying can help. A rate limit says wait; a bad API key says an operator must intervene; a proxy timeout says retry shortly.

• Auth middleware is pure ASGI, not Starlette's BaseHTTPMiddleware, because the MCP endpoint streams and BaseHTTPMiddleware buffers.

• streamable_http_app() is used directly, not mounted. It already serves at /mcp and runs its own session-manager lifespan; mounting it under /mcp would nest the path to /mcp/mcp and drop that lifespan.

• LiteLLM version: the proxy has a known incompatibility with mcp>=2.0 so it cannot be installed in the same venv as this server. Use uvx --from 'litellm[proxy]==1.94.1' to run it in isolation. Version 1.95.0 has a FastAPI import bug; stay on 1.94.1 until that is resolved upstream.