# Agent Farm v3.4 - Chunked Write Edition
AI organism evolution and parallel task execution with tool-enabled agents.
Now with **Chunked Write Pattern** for generating large documents and code files!
## What's New in v3.4
- **Chunked Write Pattern**: Bugs write sections in parallel, Python assembles directly
- **chunked_write**: Generate large markdown/text documents (unlimited size)
- **chunked_code_gen**: Generate multi-function code files in parallel
- **chunked_analysis**: Multi-perspective analysis with synthesis
- **Bypasses 500-char limit**: Each bug writes small chunks, combined output is unlimited
## Performance
- **8.6x faster** than v3.0 (103s -> 12s for 4-task swarm)
- **1 iteration** per task (was 3-5)
- **100% success rate** with real tool data
- **Local synthesis** - qwen2.5:14b synthesizes results (no cloud tokens!)
## Models
| Role | Model | VRAM | Purpose |
|------|-------|------|---------|
| Scout | qwen3:4b | 2.5GB | Reconnaissance |
| Worker | qwen3:4b | 2.5GB | Task execution |
| Memory | qwen3:4b | 2.5GB | Context retention |
| Guardian | qwen3:4b | 2.5GB | System monitoring |
| Learner | qwen3:4b | 2.5GB | Pattern acquisition |
| **Synthesizer** | **qwen2.5:14b** | **8.99GB** | **Result synthesis** |
## MCP Tools (30)
### Colony Management
- `spawn_colony` - Create bug colony (standard/fast/heavy/hybrid)
- `list_colonies` - List active colonies
- `colony_status` - Detailed colony info
- `quick_colony` - Quick health check
- `dissolve_colony` - Remove colony
- `cleanup_idle` - Remove idle colonies
- `farm_stats` - Comprehensive statistics
### Swarm Deployment
- `deploy_swarm` - Deploy tasks to colony
- `quick_swarm` - One-shot spawn + deploy
### Specialized Swarms
- `code_review_swarm` - 4-perspective code review
- `code_gen_swarm` - Generate code + tests + docs
- `file_swarm` - Parallel file operations
- `exec_swarm` - Parallel shell commands
- `api_swarm` - Parallel HTTP requests
- `kmkb_swarm` - Multi-angle knowledge queries
### Tool-Enabled Agents
- `tool_swarm` - Deploy bugs with real system tools
- `system_health_swarm` - Quick system health check
- `recon_swarm` - Directory/codebase reconnaissance
- `deep_analysis_swarm` - Deep disk/file analysis
- `worker_task` - Single worker with full tools
### Direct Operations
- `heavy_write` - Direct file write (bypasses LLM for large content)
- `synthesize` - Standalone synthesis of any JSON results
### Chunked Write Pattern (NEW)
- `chunked_write` - Generate large documents via parallel section writing
- `chunked_code_gen` - Generate code files with functions written in parallel
- `chunked_analysis` - Multi-perspective analysis with synthesis
## Bug Tool Permissions
| Role | Tools |
|------|-------|
| Scout | read_file, list_dir, file_exists, system_status, process_list, disk_usage, check_service, exec_cmd |
| Worker | read_file, write_file, list_dir, exec_cmd, http_get, http_post, system_status, disk_usage, check_service |
| Memory | read_file, kmkb_search, kmkb_ask, list_dir, system_status, process_list, disk_usage, check_service, exec_cmd |
| Guardian | system_status, process_list, disk_usage, check_service, read_file, list_dir, exec_cmd |
| Learner | read_file, analyze_code, list_dir, kmkb_search, system_status, process_list, disk_usage, check_service, exec_cmd |
## Structured Output Details
Agent Farm v3.3 uses Ollama's structured output feature to enforce JSON schemas on model responses:
```python
# Bug responds with guaranteed-valid JSON:
{"tool": "system_status", "arg": ""}
{"tool": "exec_cmd", "arg": "df -h"}
{"tool": "check_service", "arg": "ollama"}
```
The constrained decoding (GBNF grammar) masks invalid tokens during generation, ensuring:
- Always valid JSON
- Correct tool names
- Proper argument structure
- No parsing failures
Results now include a `mode` field showing which method was used:
- `structured` - JSON schema enforced
- `structured+autoformat` - JSON + simple result formatting
- `structured+deep` - JSON with multi-step reasoning
- `regex` - Fallback regex parsing
- `regex+autoformat` - Regex + simple result formatting
## Chunked Write Pattern
The chunked write pattern solves the ~500 char output limitation of small models by decomposing large tasks:
```
1. PLANNER BUG (qwen2.5:14b)
|-- Creates structured JSON outline
|-- {"sections": [{"title": "...", "description": "..."}]}
2. WORKER BUGS (qwen3:4b) - IN PARALLEL
|-- Each writes one section (~300-500 chars)
|-- 4 workers = 4 sections simultaneously
3. PYTHON CONCATENATION (NO LLM)
|-- header + separator.join(sections)
|-- Zero token cost, instant assembly
4. DIRECT FILE WRITE (NO LLM)
|-- tool_write_file() saves result
|-- Bypasses any output corruption
```
### Performance
| Tool | Output Size | Sections | Time |
|------|-------------|----------|------|
| chunked_write | 9.6 KB | 5 | 78s |
| chunked_code_gen | 1.9 KB | 4 functions | 88s |
| chunked_analysis | Varies | 4 perspectives | ~60s |
### Why It Works
- Small models excel at focused, short outputs
- Each section is within the "safe zone" (<500 chars)
- Python handles assembly (no LLM token cost)
- Parallel execution via ThreadPoolExecutor
- Structured output ensures reliable planning
## Usage Examples
### System Health Check
```
agent-farm:system_health_swarm
```
### Custom Task Swarm
```
agent-farm:tool_swarm
colony_type: "heavy"
tasks: [
{"prompt": "Check CPU temperature"},
{"prompt": "List top 5 memory processes"},
{"prompt": "Check if docker is running"}
]
```
### Large File Write (Direct)
```
agent-farm:heavy_write
path: "/tmp/large_output.txt"
content: "... large content ..."
```
### Codebase Reconnaissance
```
agent-farm:recon_swarm
target_path: "/home/kyle/repos/my-project"
```
### Generate Large Document (Chunked)
```
agent-farm:chunked_write
output_path: "/tmp/security_guide.md"
spec: "Linux server security hardening guide"
num_sections: 5
doc_type: "markdown"
```
Output: 9KB+ document with 5 coherent sections
### Generate Code File (Chunked)
```
agent-farm:chunked_code_gen
output_path: "/tmp/utils.py"
spec: "File utilities: read, write, copy, delete"
language: "python"
num_functions: 4
```
Output: Complete Python module with 4 functions
### Multi-Perspective Analysis
```
agent-farm:chunked_analysis
target: "/home/kyle/repos/project"
question: "What are the architectural patterns?"
num_perspectives: 4
```
Output: Analysis from Structure, Patterns, Quality, Performance perspectives
## Installation
```bash
cd ~/repos/agent-farm
uv venv
uv pip install -e .
```
## Claude Desktop Config
```json
{
"mcpServers": {
"agent-farm": {
"command": "/home/kyle/repos/agent-farm/.venv/bin/python",
"args": ["-m", "agent_farm.server"]
}
}
}
```
## Changelog
### v3.4.0 (2026-01-23)
- **Chunked Write Pattern** - Bugs write sections in parallel, Python assembles
- **chunked_write** - Generate unlimited-size documents (tested: 9.6KB in 78s)
- **chunked_code_gen** - Generate multi-function code files in parallel
- **chunked_analysis** - Multi-perspective analysis with synthesis
- Bypasses 500-char bug limitation via task decomposition
- Planner uses structured JSON output for reliable outlines
### v3.3.0 (2026-01-23)
- **Ollama Structured Output** - JSON schema enforcement via constrained decoding
- **Reliable tool parsing** - No more regex failures
- **Mode tracking** - Results show parsing method used
- **Regex fallback** - Legacy parsing still available as backup
- All roles get exec_cmd for complex shell queries
### v3.2.0 (2026-01-22)
- **Synthesizer role** - qwen2.5:14b for accurate result synthesis
- **synthesize parameter** - Added to tool_swarm, system_health_swarm, recon_swarm, deep_analysis_swarm
- **synthesize tool** - Standalone synthesis of any JSON results
- **No more Claude synthesis tax** - bugs do ALL the work locally
### v3.1.0 (2026-01-20)
- 8.6x speed improvement (103s -> 12s)
- Auto-format results skip redundant LLM calls
- Reject invalid tools instantly
- Force tool usage before answers
- Complex shell command support fixed
- All roles upgraded to qwen3:4b minimum
### v3.0.0 (2026-01-19)
- Linux rebuild from Windows version
- Tool-enabled agents with role permissions
- System health, recon, worker swarms
- TRUE PARALLEL via ThreadPoolExecutor
