cloudloop

A hand-rolled agentic assistant with real tool use, a real confirmation gate on the one tool that matters, and two independent access paths - a CLI agent and a standards-compliant MCP server - both calling the exact same underlying code.

Built as a companion project to ragsentry: where ragsentry is about measuring whether a RAG system is right, cloudloop is about giving a local LLM the ability to act - safely, with a human in the loop wherever an action isn't purely read-only.

What's actually in here

• Agent loop (agent.py) - a ReAct-style loop written from scratch, no LangGraph or agent framework. Uses Ollama's native OpenAI-compatible tool-calling (Qwen2.5-7B), with defensive handling for the model's occasional malformed tool-call output, an iteration cap, and a real CLI confirmation prompt gating the one mutating tool.

• Tools (tools.py) - four tools shared identically by both access paths: search_docs (reuses ragsentry's Pinecone index), estimate_cost (deterministic AWS cost calculator, explicitly illustrative not live pricing), check_latest_updates (AWS's real, live "What's New" RSS feed), and create_budget_alert (a simulated write action - no real AWS API is ever called - that exists specifically to demonstrate the confirmation pattern).

• MCP server (mcp_server.py) - the same four tools exposed as real MCP tools via MCPServer (mcp SDK 2.0.0), with read_only_hint/destructive_hint annotations so any compliant client applies its own caution UI, and native ctx.elicit() for the mutating tool's confirmation instead of a CLI prompt, since there's an actual client to delegate that to here.

• Observability - MLflow autolog on the agent's OpenAI calls, same pattern as ragsentry.

Related MCP server: AWS Documentation MCP Server

Why these four tools

search_docs ties this project to ragsentry rather than starting from zero. estimate_cost and check_latest_updates were chosen to force genuine tool-selection reasoning - one is pure local computation, one hits a live external feed, one is static indexed knowledge - so the agent has to actually distinguish between them, not just always call the same tool. create_budget_alert is the one mutating tool, deliberately simulated so the guardrail pattern has something real to gate without any actual risk.

Setup

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Needs Ollama with qwen2.5:7b-instruct pulled (see ragsentry's README for install steps), and a .env with the same PINECONE_API_KEY/PINECONE_INDEX_NAME as ragsentry, since search_docs queries that same index:

cp ../ragsentry/.env .

Running it

# single question
python src/agent.py "What would 10 million Lambda invocations a month cost, at 500ms and 512MB?"

# interactive chat - also the more robust mode, see engineering notes below
python src/agent.py

# MCP server (stdio transport)
python src/mcp_server.py

Notable engineering decisions & bugs found along the way

• GPU backend regression, mid-project - the exact same Ollama + Qwen2.5 setup that worked reliably throughout ragsentry suddenly failed with vk::PhysicalDevice::createDevice: ErrorInitializationFailed when this project started. Root cause: an Ollama update had switched to attempting a Vulkan GPU backend instead of the CUDA backend that actually works on this hardware - confirmed as a known, recently-reported issue (not something in this project's code) before fixing it with a permanent OLLAMA_VULKAN=0 systemd override, applied to the actual service rather than an ad-hoc foreground process (which has its own, separate empty model directory - a second, unrelated near-miss along the way).

• A five-alarm shell-quoting bug that looked like a model bug - create_budget_alert consistently extracted threshold_usd: 0 from requests like "...for $50...", reproducibly, across a schema-description fix and a full preprocessing normalization layer built specifically to address it. Neither fix worked, because neither was the actual cause: bash itself was silently mangling $50 into 0 inside double-quoted command-line arguments ($5 as an empty positional parameter, followed by the literal 0) before Python ever started. Confirmed directly by inspecting sys.argv. The real fix was single-quoting the shell argument - nothing in this project's code was ever wrong. Left as a documented example of a misdiagnosis corrected by going back to first-principles evidence rather than continuing to patch downstream of the wrong root cause.

• Defense in depth on the guardrail tool anyway - even though the "$" bug turned out to be shell-side, create_budget_alert still validates threshold_usd > 0 internally, independent of the confirmation prompt. The reasoning holds regardless of that specific bug's real cause: a confirmation prompt only works if a human actually reads it carefully, and a tool shouldn't rely on that as its only safeguard against a bad value slipping through.

• MCP SDK 2.0.0 broke its own quick-start API - FastMCP (the class every existing tutorial and even this repo's own bundled MCP-builder skill reference) was renamed to MCPServer, and elicit()'s signature changed from (prompt, input_type) to (message, schema) with a Pydantic model. Found by direct inspection (inspect.signature, reading the SDK's own source) rather than trusting cached documentation - the same lesson as ragsentry's Ragas dependency saga, applied to a different library.

• A real MCP server, proven correct independent of flaky tooling - MCPServer.list_tools() and the tool-wrapper logic were verified two ways: direct unit tests against the underlying functions, and a genuine MCP protocol handshake performed via the SDK's own client (stdio_client + ClientSession), spawning the real server subprocess and confirming all four tools respond correctly - independent of @modelcontextprotocol/inspector, which proved unreliable in this environment for reasons unrelated to this project (an outdated system Node.js, a known npm optional-dependency bug, and an unexplained connection hang that a from-scratch protocol-level test ruled out as a server-side issue). Knowing when a piece of tooling is the actual problem - and proving it with an independent test rather than continuing to debug the tool - mattered as much here as fixing an actual bug would have.

What's not independently verified

ctx.elicit() rendering as an actual confirmation dialog in a live MCP client (e.g. Claude Desktop) has not been observed directly - Inspector's unreliability in this environment made that specific check impractical to complete. Everything upstream of that (the protocol handshake, tool registration, schema generation, the wrapper logic) is proven; the equivalent confirmation concept is fully proven live via the CLI agent's input() path. To check this yourself, add the following to Claude Desktop's MCP server config and restart it:

{
  "mcpServers": {
    "cloudloop": {
      "command": "/absolute/path/to/cloudloop/venv/bin/python",
      "args": ["/absolute/path/to/cloudloop/src/mcp_server.py"]
    }
  }
}

(Config file location varies by OS - search "Claude Desktop MCP config location" for your platform if the path isn't already familiar.)

Status

• Tools: built and unit-tested (all four, including the guardrail validation)

• Agent loop: hand-rolled ReAct, verified live end-to-end across all four tools including the confirmation flow

• MCP server: built, tool registration and wrapper logic verified, real protocol handshake proven via direct SDK client testing

• elicit() verified against a live first-party MCP client (see above - not blocking, lower priority than it might first appear)