# cloudloop

A hand-rolled agentic assistant with real tool use, a real confirmation gate on the one tool that matters, and two independent access paths - a CLI agent and a standards-compliant MCP server - both calling the exact same underlying code.

Built as a companion project to [ragsentry](../ragsentry): where ragsentry is about *measuring* whether a RAG system is right, cloudloop is about giving a local LLM the ability to *act* - safely, with a human in the loop wherever an action isn't purely read-only.

## What's actually in here

- **Agent loop** (`agent.py`) - a ReAct-style loop written from scratch, no LangGraph or agent framework. Uses Ollama's native OpenAI-compatible tool-calling (Qwen2.5-7B), with defensive handling for the model's occasional malformed tool-call output, an iteration cap, and a real CLI confirmation prompt gating the one mutating tool.
- **Tools** (`tools.py`) - four tools shared identically by both access paths: `search_docs` (reuses ragsentry's Pinecone index), `estimate_cost` (deterministic AWS cost calculator, explicitly illustrative not live pricing), `check_latest_updates` (AWS's real, live "What's New" RSS feed), and `create_budget_alert` (a simulated write action - no real AWS API is ever called - that exists specifically to demonstrate the confirmation pattern).
- **MCP server** (`mcp_server.py`) - the same four tools exposed as real MCP tools via `MCPServer` (mcp SDK 2.0.0), with `read_only_hint`/`destructive_hint` annotations so any compliant client applies its own caution UI, and native `ctx.elicit()` for the mutating tool's confirmation instead of a CLI prompt, since there's an actual client to delegate that to here.
- **Observability** - MLflow autolog on the agent's OpenAI calls, same pattern as ragsentry.

## Why these four tools

`search_docs` ties this project to ragsentry rather than starting from zero. `estimate_cost` and `check_latest_updates` were chosen to force genuine tool-selection reasoning - one is pure local computation, one hits a live external feed, one is static indexed knowledge - so the agent has to actually distinguish between them, not just always call the same tool. `create_budget_alert` is the one mutating tool, deliberately simulated so the guardrail pattern has something real to gate without any actual risk.

## Setup

```bash
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

Needs Ollama with `qwen2.5:7b-instruct` pulled (see ragsentry's README for install steps), and a `.env` with the same `PINECONE_API_KEY`/`PINECONE_INDEX_NAME` as ragsentry, since `search_docs` queries that same index:

```bash
cp ../ragsentry/.env .
```

## Running it

```bash
# single question
python src/agent.py "What would 10 million Lambda invocations a month cost, at 500ms and 512MB?"

# interactive chat - also the more robust mode, see engineering notes below
python src/agent.py

# MCP server (stdio transport)
python src/mcp_server.py
```

## Notable engineering decisions & bugs found along the way

- **GPU backend regression, mid-project** - the exact same Ollama + Qwen2.5 setup that worked reliably throughout ragsentry suddenly failed with `vk::PhysicalDevice::createDevice: ErrorInitializationFailed` when this project started. Root cause: an Ollama update had switched to attempting a Vulkan GPU backend instead of the CUDA backend that actually works on this hardware - confirmed as a known, recently-reported issue (not something in this project's code) before fixing it with a permanent `OLLAMA_VULKAN=0` systemd override, applied to the actual service rather than an ad-hoc foreground process (which has its own, separate empty model directory - a second, unrelated near-miss along the way).
- **A five-alarm shell-quoting bug that looked like a model bug** - `create_budget_alert` consistently extracted `threshold_usd: 0` from requests like `"...for $50..."`, reproducibly, across a schema-description fix and a full preprocessing normalization layer built specifically to address it. Neither fix worked, because neither was the actual cause: bash itself was silently mangling `$50` into `0` inside double-quoted command-line arguments (`$5` as an empty positional parameter, followed by the literal `0`) *before Python ever started*. Confirmed directly by inspecting `sys.argv`. The real fix was single-quoting the shell argument - nothing in this project's code was ever wrong. Left as a documented example of a misdiagnosis corrected by going back to first-principles evidence rather than continuing to patch downstream of the wrong root cause.
- **Defense in depth on the guardrail tool anyway** - even though the "$" bug turned out to be shell-side, `create_budget_alert` still validates `threshold_usd > 0` internally, independent of the confirmation prompt. The reasoning holds regardless of that specific bug's real cause: a confirmation prompt only works if a human actually reads it carefully, and a tool shouldn't rely on that as its only safeguard against a bad value slipping through.
- **MCP SDK 2.0.0 broke its own quick-start API** - `FastMCP` (the class every existing tutorial and even this repo's own bundled MCP-builder skill reference) was renamed to `MCPServer`, and `elicit()`'s signature changed from `(prompt, input_type)` to `(message, schema)` with a Pydantic model. Found by direct inspection (`inspect.signature`, reading the SDK's own source) rather than trusting cached documentation - the same lesson as ragsentry's Ragas dependency saga, applied to a different library.
- **A real MCP server, proven correct independent of flaky tooling** - `MCPServer.list_tools()` and the tool-wrapper logic were verified two ways: direct unit tests against the underlying functions, and a genuine MCP protocol handshake performed via the SDK's own client (`stdio_client` + `ClientSession`), spawning the real server subprocess and confirming all four tools respond correctly - independent of `@modelcontextprotocol/inspector`, which proved unreliable in this environment for reasons unrelated to this project (an outdated system Node.js, a known npm optional-dependency bug, and an unexplained connection hang that a from-scratch protocol-level test ruled out as a server-side issue). Knowing when a piece of tooling is the actual problem - and proving it with an independent test rather than continuing to debug the tool - mattered as much here as fixing an actual bug would have.

## What's *not* independently verified

`ctx.elicit()` rendering as an actual confirmation dialog in a live MCP client (e.g. Claude Desktop) has not been observed directly - Inspector's unreliability in this environment made that specific check impractical to complete. Everything upstream of that (the protocol handshake, tool registration, schema generation, the wrapper logic) is proven; the equivalent confirmation *concept* is fully proven live via the CLI agent's `input()` path. To check this yourself, add the following to Claude Desktop's MCP server config and restart it:

```json
{
  "mcpServers": {
    "cloudloop": {
      "command": "/absolute/path/to/cloudloop/venv/bin/python",
      "args": ["/absolute/path/to/cloudloop/src/mcp_server.py"]
    }
  }
}
```

(Config file location varies by OS - search "Claude Desktop MCP config location" for your platform if the path isn't already familiar.)

## Status

- [x] Tools: built and unit-tested (all four, including the guardrail validation)
- [x] Agent loop: hand-rolled ReAct, verified live end-to-end across all four tools including the confirmation flow
- [x] MCP server: built, tool registration and wrapper logic verified, real protocol handshake proven via direct SDK client testing
- [ ] `elicit()` verified against a live first-party MCP client (see above - not blocking, lower priority than it might first appear)