ask-llm-mcp

Give your coding agent a cheap LLM to delegate to.

ask-llm-mcp is a minimal MCP server that exposes raw text-in / text-out LLM calls. No agent runtime, no tools, no context injection — you send a prompt, you get text back.

The point: your main agent (Claude / Codex / CodeBuddy / …) is running on an expensive model. Summarising a file, classifying an issue, drafting a commit message, extracting a JSON blob — none of that needs the expensive model. Hand those subtasks to ask_llm and keep the good model for real reasoning.

Contents

• Why

• Features

• Requirements

• Quick start

• Client configuration

• Tools

• Configuration

• How it works

• Troubleshooting

• Project layout

• Contributing

• License

English | 中文文档 | Install guide

Why

Running devin -p "<prompt>" spawns a full agent runner: it loads a system prompt, registers tools, and can burn minutes deciding it doesn't want to answer a one-line question. That's the wrong shape for "classify this log line".

ask-llm-mcp calls the backend GetChatMessage endpoint directly over Connect-RPC. One HTTP request, one text response. That's it.

The second half of the design is keeping the answer out of your context window. ask_llm does not return the LLM's text inline — it returns a path. Your agent reads the file only if it actually needs the content, and can ignore it, grep it, or pass it along without paying for the tokens twice.

Features

• Three tools: ask_llm, list_models, list_sessions.

• Model choice with pricing — pick a cheap model per call, or discover options at runtime.

• Multi-turn conversations via session_id.

• File-based results — result_file (answer) + log_file (timing, errors, prompt preview) under a configurable data directory.

• No MCP SDK dependency — hand-rolled JSON-RPC over stdio; the only runtime dependency is requests.

• Two stdio transports — newline-delimited JSON and Content-Length framing, auto-detected per message.

• Offline test suite — protobuf/framing round-trip tests, no credentials needed.

Requirements

The server reads your API key from ~/.local/share/devin/credentials.toml (written by devin auth login / devin login). No key is ever passed through MCP arguments or env vars.

list_models and list_sessions additionally shell out to the devin CLI (devin models list --format json, devin list --format json). ask_llm does not.

Quick start

# 1. get the code
git clone https://github.com/agentming/ask-llm-mcp.git
cd ask-llm-mcp

# 2. install (creates an isolated venv + console script)
./install.sh

# 3. sanity check — should print the three tool definitions
echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | ask-llm-mcp

# 4. sanity check with real credentials
python3 devin_api.py "Say hello in one sentence."

./install.sh --help shows the options (--dev, --uninstall, --prefix DIR).

Alternative installs

# pipx — no venv management for you
pipx install git+https://github.com/agentming/ask-llm-mcp.git

# plain pip
pip install git+https://github.com/agentming/ask-llm-mcp.git

# run straight from the checkout, no install at all
python3 /path/to/ask-llm-mcp/server.py

Then register it with your MCP client — see below.

Client configuration

Claude Code / Codex CLI

claude mcp add ask-llm -- /absolute/path/to/ask-llm-mcp
# or, without installing:
claude mcp add ask-llm -- python3 /absolute/path/to/ask-llm-mcp/server.py

Claude Desktop

~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "ask-llm": {
      "command": "/absolute/path/to/ask-llm-mcp",
      "env": {
        "DEVIN_MODEL": "swe-1-7"
      }
    }
  }
}

CodeBuddy Code / other JSON-config clients

.codebuddy/mcp.json or ~/.codebuddy/mcp.json:

{
  "mcpServers": {
    "ask-llm": {
      "command": "python3",
      "args": ["/absolute/path/to/ask-llm-mcp/server.py"]
    }
  }
}

More copy-pasteable variants live in examples/.

Use absolute paths. MCP clients launch the server with an arbitrary working directory.

Tools

ask_llm

Raw text-in / text-out LLM call.

Returns structured data, not the answer text:

{
  "status": "ok",
  "session_id": "0f9c...",
  "result_file": "/home/you/.local/share/ask-llm/results/20260908_113000_ab12cd34.txt",
  "log_file": "/home/you/.local/share/ask-llm/logs/20260908_113000_ab12cd34.json",
  "error": null,
  "call_id": "20260908_113000_ab12cd34"
}

Read result_file for the answer. If the model emitted reasoning, it is prepended and separated by a --- divider. On failure status is "error", error carries the message, and result_file contains [ERROR] ….

Multi-turn example — just feed the returned session_id back in:

turn 1: ask_llm(prompt="My function returns None. Here it is: ...")
        -> session_id = "0f9c..."
turn 2: ask_llm(prompt="Now show the fix.", session_id="0f9c...")

list_models

Lists model families with per-1M-token pricing, context window and cost tier.

## SWE (aliases: swe)
  swe-1-7  —  SWE-1.7  [200K ctx, cheap, $0.25/$0.03/$1.00]

list_sessions

Lists recent Devin sessions so you can recover a session_id.

Configuration

All configuration is via environment variables (set them in your MCP client's env block).

Result files are never deleted automatically — clear $ASK_LLM_DATA_DIR/results yourself if it grows.

How it works

MCP client ──stdin (JSON-RPC)──► server.py ──► devin_api.py ──HTTPS──► server.codeium.com
                                     │                                  (Connect-RPC)
                                     └──► $ASK_LLM_DATA_DIR/{results,logs}/

• devin_api.py hand-rolls the protobuf request (GetChatMessageRequest) and wraps it in a single uncompressed Connect-RPC frame. Responses are streamed back as gzipped frames whose delta_text / delta_thinking fields are concatenated into the final answer.

• server.py implements just enough of the MCP protocol to advertise and serve the three tools: initialize, tools/list, tools/call. It accepts both newline-delimited JSON and Content-Length framing on stdin.

Because the request field numbers were calibrated against live traffic rather than a published schema, they are the most likely thing to break upstream. If calls start failing with an HTTP error, look there first.

Troubleshooting

Logs for every call live in $ASK_LLM_DATA_DIR/logs/*.json and include the timestamp, model, elapsed seconds, prompt length/preview and the error.

Project layout

server.py       MCP JSON-RPC server + the three tool implementations
devin_api.py    Connect-RPC / protobuf client for the backend (no MCP knowledge)
tests/          Offline tests for the wire helpers
examples/       MCP client config snippets
install.sh      Isolated-venv installer

Contributing

See CONTRIBUTING.md. The short version: keep changes small, never add tests that hit the real API, never commit credentials.

License

MIT © agentming