Skip to main content
Glama

agent-platform

CI

A small multi-agent pipeline that turns a product idea into a validated technical spec, running entirely on a local Ollama model (no paid API, no data leaving the machine). It can be used from the command line or as an MCP server from Claude Code, Claude Desktop or any MCP client.

idea ─▶ Product Manager ─▶ Tech Architect ─▶ QA (LLM review + deterministic gate) ─┐
                               ▲                                                   │ not approved:
                               └──────────── patch + QA feedback, up to 3 rounds ◀─┘

Every run and each iteration (spec, QA report, approval) is stored in SQLite.

What is different from a plain "prompt chain"

  • Structured outputs. Each agent asks Ollama for JSON that matches a pydantic schema, so decoding is constrained instead of parsed out of free text.

  • Failing loudly. If the architect cannot produce a valid spec it retries once and then raises. It does not return a canned spec, and the validators do not invent endpoints.

  • A deterministic gate next to the LLM reviewer. QAGate checks things a model should not be trusted with: valid and unique endpoints, components that only use technologies listed in the stack, spec identity matching the product, risks with a mitigation. When it fails, the score is capped and a critical issue is added.

  • Measured, not assumed. eval/ runs the pipeline on 20 fixed ideas and reports approval rate, iterations and coherence.

Related MCP server: Foundry MCP

Quick start

Requires Python 3.12 and a running Ollama with the llama3.1 model (ollama pull llama3.1).

python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,mcp]"
python -m src.main "Booking app for padel courts with online payments"
pytest -q

The database is agent_platform.db in the project root; set AGENT_PLATFORM_DB to use another file. The agent prompts are written in Spanish and the specs come out in Spanish.

MCP server

Five tools, all local:

Tool

What it does

generate_tech_spec(idea)

Runs the whole pipeline (1 to 3 minutes). Returns {approved, run_id, tech_spec}

check_tech_spec(product_spec, tech_spec)

Deterministic gate, instant, no LLM

review_tech_spec(product_spec, tech_spec)

LLM review plus the gate: score, issues, suggestions, patch

list_runs(limit)

Recent runs and whether QA approved a final spec

get_run(run_id)

One run with its product spec, final spec and the QA score of each iteration

Register it in Claude Code:

claude mcp add agent-platform --scope user -- \
  /path/to/agent-platform/.venv/bin/python /path/to/agent-platform/scripts/run_mcp.py

scripts/run_mcp.py works from any directory. Some clients time out before a long generate_tech_spec call returns; check_tech_spec and review_tech_spec are fast.

Evaluation

20 fixed ideas (eval/ideas.json), llama3.1, 3 repetitions per version (python -m eval.run_eval, python -m eval.report):

version

approved

final specs passing the strict gate

mean iterations

original pipeline

53% (50% to 55%)

88% (85% to 90%)

2.38

this version

95% (95% to 95%)

95% (95% to 95%)

1.20

Read it with care (details in eval/RESULTS.md):

  • The approval gain mixes three changes that were not measured separately: temperature 0, approval that no longer uses the reviewer's own is_valid flag, and the rest (structured outputs, gate, neutral prompts).

  • Temperature 0 makes this version's three repetitions nearly identical, so they say nothing about its variability.

  • The gap in final-spec coherence (88% vs 95%) is much smaller than the gap in approval.

  • Silent fallbacks and invented content never occurred in the 120 runs, so those safeguards were not exercised.

  • One model, one idea set.

Layout

src/agents/        product manager, tech architect, QA agent and their validators
src/schemas/       pydantic schemas used both for validation and as the JSON schema sent to Ollama
src/services/      RunService: the pipeline loop and approval rule
src/database/      SQLAlchemy models and repository (SQLite)
src/mcp_server/    the MCP server
eval/              idea set, runner, report and results

Known limits

  • Only Ollama is supported, and the package is imported as src (not yet a proper installable package name).

  • Approval is score >= 0.8 with no critical issue; that threshold is not calibrated against human judgement.

  • The one idea that is never approved (the financial KPI dashboard, in all three repetitions) fails the same way every time: a component uses D3.js, which is missing from the stack, and the architect does not fix it in three rounds. The gate catches it correctly; the repair loop does not resolve it.

License

MIT

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    A Spec-Driven Development toolkit that transforms LLMs into development agents by providing expert-crafted prompts for generating structured specifications and validating documents across the Requirements → Design → Tasks → Code workflow.
    1
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables spec-driven development workflows with AI assistants, providing tools for managing specification lifecycles, task dependencies, code navigation, testing, and automated reviews through a unified CLI and MCP interface.
    4
    MIT