# Yala Toolkit

<div align="center">
<pre>
██╗   ██╗ █████╗ ██╗      █████╗ 
╚██╗ ██╔╝██╔══██╗██║     ██╔══██╗
 ╚████╔╝ ███████║██║     ███████║
  ╚██╔╝  ██╔══██║██║     ██╔══██║
   ██║   ██║  ██║███████╗██║  ██║
   ╚═╝   ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝
</pre>
<b>T O O L K I T</b> &nbsp;·&nbsp; 570 tools, one prompt away
</div>

A suite of **570 standalone, config-driven** utilities that cover the whole product development cycle — **plan → develop → QA → feedback automation → deploy** — usable as plain CLI tools, or spoken to in plain English through Claude, Codex, and Cursor. Every tool is self-contained, configured via flags and environment variables (no host paths or private identifiers baked in). Python · Go · Bash.

> 🧭 **Two ways to use it:** talk to it through an AI assistant (setup below), or run the tools directly from your shell (see *Prefer the command line?* further down). Same tools either way.

---

## 🔄 One toolkit, the whole product cycle

Yala isn't a grab-bag of scripts — it covers **every stage of shipping software**, so one toolkit (and one AI assistant wired to it) can carry a feature from idea to production and keep it healthy after:

```mermaid
flowchart LR
    P["🗺️ Plan"] --> D["🔨 Develop"] --> Q["🔍 QA"] --> F["🔁 Feedback automation"] --> Y["🚀 Deploy"] --> P
```

Below, a demo pass around the loop. Every command is a real tool in this repo (each honors `--help`); every quoted line is a real prompt you can give Claude once the MCP server is connected (setup in the next section).

### 🗺️ Plan — know what to build

> *"Research best practices for idempotent webhook receivers and write me a cited report."*
> *"Give me an architecture report of this repo — hot files, coupling, who owns what."*

```bash
# discover the work already hiding in the codebase (TODO/FIXME/secrets/skipped tests),
# ranked by severity × age × churn, into a living TODO.md
python src/project-autopilot/autopilot_backlog.py --repo . --write-todo TODO.md
```

### 🔨 Develop — build it

> *"Scaffold a new Python HTTP service called billing-api and verify it."*
> *"Which packages are affected by my change vs main? Generate a CI matrix for just those."*
> *"Generate an OpenAPI spec from this HAR capture, then a mock server on port 8400."*

```bash
python src/development/scaffold_runner.py new billing-api                   # bootstrap + verify
python src/turborepo/turbo_affected.py --base main                          # build only what changed
```

### 🔍 QA — prove it's good

> *"Run a risk score on my diff; if it's high, do a full automated review and only show verified findings."*
> *"Lint this migration for table rewrites and non-concurrent indexes before I merge it."*
> *"Load-test the staging endpoint and give me p95/p99 — fail if worse than the baseline."*

```bash
python src/code-review/review_risk.py --from main --json                    # deterministic risk score
python src/code-review/review_gate.py --findings findings.json             # policy gate for CI
```

### 🔁 Feedback automation — let it maintain itself

The [`project-autopilot`](src/project-autopilot/) category turns the loop 24/7: a supervisor runs guardrail-gated jobs that keep tests green, triage production errors into issues, patch new CVEs, review PRs, nudge coverage, and mine new backlog — all **PR-only** (never pushing a protected branch), with a kill switch, a daily action budget, and a morning digest that pages you only when a human is needed.

```bash
cd your-project
python <yala>/src/project-autopilot/autopilot_supervisor.py init --repo .   # generates autopilot.yaml
python <yala>/src/project-autopilot/autopilot_supervisor.py run             # ...and walk away
python <yala>/src/project-autopilot/autopilot_report.py --state-dir .autopilot --since 24h
```

> *"Analyze this A/B test — is the lift real or am I peeking?"* · *"Alert #ops on IRC when error rate spikes."*

### 🚀 Deploy — ship it safely

> *"Lint my GitHub Actions workflow for unpinned actions and script injection."*
> *"Gate this deploy: coverage ≥ 80, no critical CVEs, tests green."*
> *"Roll out the canary metric-gated and auto-rollback on SLO breach."*

```bash
python src/cicd/deploy_gate.py --gate cmd:pytest --gate git_clean           # CI promotion gate
```

…and the autopilot's triggers notice the new deploy's errors, which become backlog, which becomes the next plan. **The loop closes.**

---

## 🤖 Use it from Claude (the easy way)

Instead of remembering hundreds of tool names, you just **describe what you want** and Claude finds and runs the right tool. No tool floods your context — the toolkit exposes four small "meta-tools" (`find_tools`, `tool_info`, `run_tool`, `list_categories`) that let the assistant search and execute on demand.

**Setup — about 2 minutes:**

```bash
# 1. get the toolkit and install the shared Python packages
cd yala-toolkit
python -m venv .venv && source .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -r requirements.txt

# 2. register the MCP server with Claude Code (one time)
claude mcp add yala -- python3 "$(pwd)/src/claude-mcp/yala_mcp_server.py"

# 3. check it connected
claude mcp list        # you should see:  yala ... ✔ Connected
```

Now open a **new** Claude Code session and just ask — see the prompt examples below.

<details>
<summary><b>Codex, Cursor, and other MCP clients</b></summary>

The same stdio server works with any MCP client:

```bash
# Codex CLI
codex mcp add yala -- python3 "$(pwd)/src/claude-mcp/yala_mcp_server.py"

# Cursor — add to ~/.cursor/mcp.json:
# { "mcpServers": { "yala": { "command": "python3",
#   "args": ["/ABSOLUTE/PATH/yala-toolkit/src/claude-mcp/yala_mcp_server.py"] } } }
```

Opening this repo in an editor that reads a project `.mcp.json` offers the server automatically. See [`src/claude-mcp/`](src/claude-mcp/) for details and a `--self-test`.
</details>

---

## 💬 Talk to it — prompt examples

Once the MCP server is connected, these are real things you can type to Claude. You never name the tool — describing the task is enough; Claude searches, reads the tool's help, and runs it.

**Just browsing**
- *"What yala tools do I have for working with SQL databases?"*
- *"List the yala tool categories."*
- *"Is there a yala tool to check TLS certificate expiry?"*

**System, files & everyday chores**
- *"Use yala to show me what's eating disk space in my home folder."*
- *"Find duplicate files under ./Downloads with yala and show me the biggest ones."*
- *"Pick the best compression for this folder and tell me how much I'd save."*
- *"Give me a snapshot of this machine — CPU, memory, disk."*

**Security & secrets**
- *"Scan this repo for hardcoded secrets with yala."*
- *"Check my dependencies for known CVEs."*
- *"Audit file permissions under /etc/myapp for anything world-writable."*

**Git & code review** *(the `code-review` category)*
- *"Run a risk score on my current git diff and tell me if it needs careful review."*
- *"Do an automated code review of my changes vs main and only show me real, verified findings."*
- *"Who should review this PR? Check git blame and flag any bus-factor risk."*
- *"Run the review checklist on my staged changes — secrets, debug prints, missing tests."*

**Turborepo / monorepos** *(the `turborepo` category)*
- *"What's the critical path in this turborepo's build?"*
- *"Which packages are affected if I change against main? Generate a CI matrix for just those."*
- *"Lint my turbo.json — is my cache configured correctly?"*
- *"Audit my workspaces for dependency cycles and version drift."*

**Start a new project** *(the `development` + scaffold tools)*
- *"Scaffold a new Python HTTP service called billing-api with yala and verify it."*
- *"What project shapes can the scaffold kit make?"*

**API work** *(the `api-docs` / `api-versioning` categories)*
- *"Generate an OpenAPI spec from this HAR file."*
- *"Diff these two OpenAPI specs and tell me if anything is a breaking change."*
- *"Spin up a mock server from openapi.yaml on port 8400."*

**IRC ops & bots** *(the `irc-ops` category)*
- *"Stand up a local ergo IRC server and put a logging agent in #ops."*
- *"Is my bot still connected to #alerts? Health-check the IRC server."*

**Backups, data & infra**
- *"Are any of my backups overdue? Check against a 24-hour RPO."*
- *"Profile this CSV — types, null rates, and anything that looks off."*
- *"Bump 1.2.3 to the next minor version."*

> 💡 **Tip:** if a tool can change things (delete files, post comments, run a server), it's flagged as mutating and supports `--dry-run` — ask Claude to *"do a dry run first"* and it will.

---

## ⌨️ Prefer the command line?

Every tool is a normal script — no AI required:

```bash
cd yala-toolkit
python -m venv .venv && source .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -r requirements.txt

python src/system-process/system_info.py             # your first tool
```

Every tool prints its options with `--help`, and **every category folder has its own README** with plain-English explanations, setup steps, and copy-paste examples. Browse the [categories](#categories) below.

- **Python tools** (`.py`): `python src/<category>/<tool>.py --help`
- **Go tools** (`.go`, in `go-tools/`): `go run src/go-tools/<tool>.go -h`
- **Bash tools** (`.sh`, in `bash/`): `bash src/bash/<tool>.sh`
- **Libretto workflows** (`.ts`, in `libretto/`): copy into your Libretto workflows folder and run `libretto run <slug>`

Some categories need extra software (ffmpeg, Docker, a database, an LLM, …). Each category README's **"Before you start"** section lists the exact install commands.

## Layout

Tools are grouped into category folders under [`src/`](src/):

```
src/<category>/<tool>.py      # or .go / .sh
src/<category>/README.md      # beginner-friendly docs: intro, setup, examples per tool
```

Example: `python src/network-recon/port_scan.py 10.0.0.5 --ports 1-1024`

## Categories — by stage

Every category, organized by where it lives in the cycle. Each folder has its own beginner-friendly README with setup steps and per-tool examples.

### 🗺️ Plan & research

Understand the codebase, research the problem, and discover the work before writing code.

| Category | Tools | Description |
|----------|------:|-------------|
| [Deep Research Agent](src/deep-research/) | 1 | LLM agent that plans, gathers across all reachable local data sources, and writes a cited report |
| [Web Search Spider Tools](src/web-search/) | 4 | target a SearXNG JSON API via `SEARXNG_URL` (default `http://127.0.0.1:8181`), or `--public` for no-setup search via random public instances (searx.neocities.org, best-effort) |
| [News & Trends Tools](src/news-trends/) | 5 | trends via `TRENDRADAR_URL`, default `http://127.0.0.1:3333`; news via a NewsAPI-compatible service with `NEWS_API_KEY`/`NEWS_API_BASE`. Reports redact source/outlet identities to stable aliases by default — `--show-sources` reveals |
| [Real-Time Market Data Tools](src/market-data/) | 8 | stocks via Yahoo Finance, crypto via CoinGecko/Binance/Coinbase — no API keys required; read-only, no trading/execution |
| [Code Intelligence Reports](src/code-intelligence/) | 6 | compose the `gitnexus` and `codegraph` CLIs + git history into markdown reports |
| [Vector DB & Page Index Tools](src/vector-pageindex/) | 6 | vector DB: `RUVECTOR_URL`, default `http://127.0.0.1:6333`; page index: `PAGEINDEX_URL`, default `http://127.0.0.1:8190`; embeddings via local Ollama |
| [Natural Language to SQL](src/nl2sql/) | 6 | Turn plain-English questions into safe, schema-aware **PostgreSQL** — introspect the schema, LLM-generate, guard read-only, EXPLAIN-validate, self-repair, then run read-only; plus a batch runner, interactive shell, static SQL safety guard, and an LLM query optimizer |

### 🔨 Develop

Scaffold, write, build — projects, APIs, frontends, data, and the git workflow around them.

| Category | Tools | Description |
|----------|------:|-------------|
| [Development Tools](src/development/) | 12 | Git helpers, TODO extraction, license/dep checks, Docker cleanup, env validation, pre-commit |
| [Next.js Development Tools](src/nextjs/) | 8 | Next.js dev-stack bootstrap, env wizard, migration checks, bundle analysis, deploy preflight |
| [React & React Native Dev Tools](src/react-dev/) | 15 | toolchain reset, codebase audits (unused exports, coverage gaps, assets, i18n, complexity), dep upgrades, and codegen |
| [React & React Native Component Libraries](src/component-libs/) | 8 | scaffold, edit, catalog, version, and publish publishable component libraries |
| [GitHub Tools](src/github/) | 8 | require the `gh` CLI, authenticated via `gh auth login` |
| [GitLab Tools](src/gitlab/) | 8 | require `--url`/`--token` or `GITLAB_URL`/`GITLAB_TOKEN` |
| [Turborepo Monorepo Tooling](src/turborepo/) | 8 | Analyze the **task graph + critical path** (from `--dry=json`, no execution), compute **affected workspaces** for fast CI + generate an **affected-only CI matrix** (GHA/GitLab/shards), **lint turbo.json** for cache-killing misconfigs (build tasks with no `outputs`, missing `dependsOn`), a **cache-effectiveness report**, an **internal-dependency audit** (cycles/version-drift/mismatch), **run-summary** timings/failures/regressions, and **workspace package.json** consistency — stdlib, npm/yarn/bun + pnpm |
| [API Documentation & OpenAPI Generation](src/api-docs/) | 8 | Generate an OpenAPI 3 spec from **observed traffic** (HAR/NDJSON) or from **route source** (Flask/FastAPI/Express/Fastify), render a spec to a self-contained **HTML/Markdown** docs page, a **breaking-change diff** (semver gate), a **mock server** straight from a spec, **curl/Postman/.http** request examples, multi-file **bundle/dereference** ($ref flattening), and a typed, stdlib-only **Python client-SDK** generator |
| [Advanced jq / JSON Wrangling](src/jq/) | 8 | Powerful **jq**-based scripts — a cookbook of advanced recipes (group/pivot/top-N/dedup), flatten/unflatten, structural path diff, JSON→CSV, deep-merge, shape profiling, path listing, and streaming huge files (NDJSON / giant arrays) |
| [Bash Tools](src/bash/) | 13 | Portable Bash utilities for certs, disk/memory alerts, service waits, backups, and audits |
| [AI Localization & Translation](src/localization/) | 7 | An audited game/software i18n pipeline — extract strings from many formats (JSON/YAML/Android/iOS/gettext/XLIFF/CSV), LLM-translate with enforced placeholder + terminology preservation, deterministically **validate** every unit (a wrong `{name}`/`%d` is a crash), independent AI QA review, and merge back; incremental diff + glossary tools |
| [Document Conversion Tools](src/document-conversion/) | 2 | Convert any document (PDF/DOCX/PPTX/XLSX/HTML/…) to Markdown, single or recursive |
| [Data Pipeline & ETL Orchestration](src/data-pipeline/) | 8 | A **DAG pipeline runner** (deps/parallel/retries/manifest), a declarative **extract→transform→load** engine (csv/json/ndjson/sqlite), data-quality **validation** (Great-Expectations-lite), dataset **profiling**, **schema inference** (DDL/JSON-Schema/rules), watermark-based **incremental/CDC extraction**, **lineage + freshness** tracking, and dataset **diff/reconciliation** — all on plain files + SQLite, no warehouse/Airflow needed |
| [Database Tools](src/database/) | 12 | PostgreSQL DBA dashboards (health, slow queries, bloat, indexes, locks, vacuum, sizes, connections), schema diff, guarded pg_dump, plus a PG/SQLite query runner and SQLite analyzer |
| [Go Tooling for Hardware & Software](src/go-tools/) | 12 | Stdlib-only **Go programs** (`go run <file>.go`) for hardware/runtime — host telemetry, I2C bus scan, GPIO chip inspect, serial/UART monitor with send-and-expect, TinyGo build/flash, concurrent HTTP load — plus **Python** wrappers around the Go toolchain: module/dependency audit, cross-compile matrix, structured test runner with flaky detection, lint aggregation, binary analysis, and pprof summaries |

### 🔍 QA — test, review & secure

Everything that decides whether a change is good enough to ship.

| Category | Tools | Description |
|----------|------:|-------------|
| [Code Review Automation](src/code-review/) | 8 | Deterministic **diff-risk scoring**, an **LLM reviewer with an adversarial verify pass** (filters hallucinated/noisy findings), a rule-driven **review checklist**, **blame-based reviewer suggestion** + bus-factor flags, a **policy merge-gate** over findings/SARIF, **publish to GitHub/GitLab/Markdown/SARIF**, orchestration of the Alibaba **`ocr`** CLI across commits/repos, and a self-contained **HTML review packet** — shared findings contract, every gate CI-ready |
| [Load Testing & Performance Benchmarking](src/load-testing/) | 8 | An **authorized-use** HTTP load generator with latency percentiles (closed/open models), multi-step user-journey load tests (virtual users, value chaining), command + Python-code microbenchmarks with A/B compare, latency histogram + tail analysis, process CPU/memory/FD sampling, endurance/soak testing with **leak + latency-creep detection**, and a CI performance-regression gate vs a baseline |
| [Chaos Engineering & Fault Injection](src/chaos-engineering/) | 8 | System-level fault injectors — CPU/memory/IO pressure, TCP network chaos (latency/partition, no root), process/container kill/pause with blast limits, and disk fill/inode/read-only — plus a **steady-state chaos-experiment orchestrator** with guaranteed rollback, a blast-radius gate + dead-man's-switch, a scheduled chaos monkey, and a **game-day exercise runner** with scorecard |
| [Database Migrations & Schema Management](src/db-migrations/) | 8 | A migration runner (Postgres + SQLite) with checksum drift detection + rollback, migration scaffolding, a **safe-migration linter** (table rewrites, non-concurrent indexes, drops/renames, unbounded backfills), normalized JSON schema snapshots + snapshot diffing that **generates DDL**, integrity/order/drift verification for CI, idempotent seed-data upserts, and batched resumable column backfills |
| [API Debugging & Hardening Tools](src/api-tools/) | 12 | request inspector with phase timings, endpoint probe/latency/diff/replay, plus **authorized-use** hardening: security-header/CORS/auth/rate-limit scans, JWT audit, GraphQL introspection, OpenAPI lint |
| [Security & Compliance](src/security/) | 5 | Secret scanning, dependency CVE checks, file integrity, permission audits, password generation |
| [SQL Injection Defense](src/sqli-defense/) | 6 | Detect + prevent SQL injection — scan source (7 languages) and an AST-precise Python linter for non-parameterized queries, a WAF-style payload detector, a log-attempt hunter, an identifier allowlist for cases parameters can't cover, and an **authorized-use** endpoint tester |
| [Antivirus & Malware Defense](src/antivirus/) | 7 | Defensive endpoint tooling — static heuristic triage, a signature scanner (built-in + custom rules, optional YARA), hash-reputation/denylist checks, ELF/PE binary analysis, a quarantine manager, a directory watcher, and a ClamAV front-end; pure-Python where no engine is installed, verified with the EICAR test file |
| [AI-Generated Content Detection](src/ai-detection/) | 6 | Surface provenance + signals for AI-made images/video/text/code — C2PA Content Credentials, Stable Diffusion/ComfyUI/EXIF generator metadata (strong evidence), plus clearly-labeled low-confidence pixel/text/code heuristics; a dispatcher routes any file to the right detector |
| [Secure Encryption & PGP](src/crypto/) | 6 | Authenticated file encryption (AES-256-GCM / ChaCha20-Poly1305, scrypt-wrapped keys, chunked AEAD with tamper + truncation detection) using the audited `cryptography` library; true-random/hardware-TRNG key & passphrase generation with an RNG quality tester; and OpenPGP keygen/encrypt/sign/verify/keyring management via GnuPG |
| [Network Recon Tools](src/network-recon/) | 6 | authorized/owned networks ONLY |

### 🔁 Feedback automation

The loops that watch what you shipped, learn from it, and fix or file work automatically.

| Category | Tools | Description |
|----------|------:|-------------|
| [Project Autopilot — 24/7 Automation](src/project-autopilot/) | 10 | A **supervisor** that runs agent loops on cron/interval/**event triggers** (crash-restart, no overlap), a **safety layer** (kill switch + daily-action budget + attempt caps + escalation), edge-triggered **event detection** (new commit / red CI / new error / new CVE), a **morning digest** (what ran/fixed/needs-you), and 5 ready **PR-only, guardrail-gated jobs** — keep-tests-green, error-triage, CVE-watch, auto-review, coverage-nudge; delegates fixes to a pluggable `--agent-cmd` |
| [Autonomous Coding Loops](src/coding-loops/) | 11 | Long-running iterate-until-goal loops for a codebase — quality-streak gate, coverage-to-target, production error sweep, docs drift, git changelog, flaky-test stabilizer, logging-coverage, error-message rewrite, dependency-CVE burndown, restartable handoff, workspace health — deterministic detection with a pluggable `--agent-cmd`/`--llm` fix step |
| [Feature Flags & A/B Testing](src/feature-flags/) | 7 | A dependency-free experimentation stack — a consistent-hash flag evaluation engine (targeting, rollouts, variants, prerequisites, kill switch) + a hot-reloading flag server, deterministic A/B/n assignment, from-scratch statistics (two-proportion z-test + Wilson CIs, Welch t-test, no scipy), sample-size/power planning with Bonferroni, monotonic percentage-rollout control, and config+code hygiene auditing |
| [Webhook Handling & Delivery](src/webhooks/) | 6 | Both sides of webhooks — HMAC sign/verify for GitHub/Stripe/Slack/Shopify/generic schemes (constant-time, replay windows), a verifying+de-duplicating receiver, reliable signed delivery with backoff/jitter retries + `Retry-After` + dead-letters, a local request-bin inspector, replay of captured/dead-lettered events, and a verify-and-fan-out relay with per-destination retry + live `/stats` |
| [Structured Logging & Log Aggregation](src/logging/) | 8 | An end-to-end log pipeline — emit JSON logs with bound context + secret redaction, parse any format (JSON/logfmt/nginx/syslog) into NDJSON, query with level+time awareness, aggregate (level rates, top-N, latency percentiles, timechart), live multi-file tail with rotation, PII/secret redaction, ship to Loki/Elasticsearch/HTTP (batched + retry), and alert on threshold/spike/absence |
| [Distributed Tracing & Observability](src/observability/) | 8 | The traces + metrics pillars — W3C trace-context propagation, instrument shell commands into OpenTelemetry spans (nested calls form a trace tree), export to OTLP/Jaeger/Zipkin (batched + retry), analyze trace trees + critical paths + latency percentiles, render ASCII trace waterfalls, a Prometheus metrics library + `/metrics` scrape server + Pushgateway, query Prometheus / parse exposition, and synthetic HTTP/TCP SLO monitoring |
| [Observability Dashboards & Alerting](src/dashboards-alerting/) | 8 | Generate **Grafana dashboards**, Prometheus **alert rules**, multi-window **SLO burn-rate alerts**, and **Alertmanager routing** from specs; test alert rules against metric values (promtool-lite, CI assertions); send notifications to **Slack/PagerDuty/Alertmanager/webhook**; a live terminal dashboard (gauges/sparklines); and a self-contained **HTML status page** |
| [Cloud Cost Optimization & Billing](src/cloud-cost/) | 8 | Analyze a billing export by service/team/tag (+ trend + movers), detect cost **anomalies/spikes**, **forecast** period-end spend vs budget, tag-based **showback/chargeback** allocation with coverage, **budget** enforcement (actual + forecast), **rightsizing** recommendations, unused/orphaned-resource reclaim, and **Reserved-Instance / Savings-Plan** commitment advice — all on your billing/usage exports, no cloud API |
| [IRC Server Ops & Agent Fleet](src/irc-ops/) | 8 | Provision/manage **ergo IRC servers** in docker, deploy **resilient channel agents** (pipe/exec/log modes), supervise a whole **agent fleet** from a spec with heartbeat-based restart, **operator/admin** moderation controls, channel **monitoring** (silence/flood/keyword/departure alerts), a **health-gate** that verifies expected bots are in their channels, a scriptable **announcer**, and **CHATHISTORY export** (NDJSON/text/HTML) — stdlib-only IRC, tested against real ergo |
| [Messaging Tools](src/messaging/) | 3 | email via any SMTP server; SMS/calls via a Twilio-compatible API — set `TWILIO_ACCOUNT_SID`/`TWILIO_AUTH_TOKEN`/`TWILIO_FROM_NUMBER` |

### 🚀 Deploy & operate

Ship safely, keep it configured, and keep it running.

| Category | Tools | Description |
|----------|------:|-------------|
| [CI/CD Pipeline Automation](src/cicd/) | 8 | Lint GitHub Actions / GitLab CI configs for correctness + security (unpinned actions, script injection, hardcoded secrets), generate a pipeline from a detected stack, run pipelines locally with a stage/needs DAG + matrix, expand build matrices, compute versions from Conventional Commits, generate Keep-a-Changelog entries, gate deployments (coverage/CVE/tests/tagging), and package/verify build artifacts with provenance |
| [Progressive Delivery & Release Automation](src/progressive-delivery/) | 8 | An automated **metric-gated rollout controller** with auto-rollback, canary analysis scoring (promote/hold/rollback), blue-green + ring-based deployment orchestration, an **SLO watchdog** that auto-rolls-back on sustained breach, an error-budget deploy-freeze gate (SRE burn-rate), a rollout-strategy planner (blast-radius), and an operational **kill switch** with audit trail |
| [Infrastructure as Code & Provisioning](src/iac-provisioning/) | 8 | Analyze Terraform plans for **destructive/data-loss** changes, inspect state inventory (+ secrets in state), an IaC **security linter** (tfsec-lite), **policy-as-code** enforcement (fail-closed), monthly **cost estimation** with plan deltas (infracost-lite), cloud-init user-data generation + real-schema validation, Ansible inventory build/convert (ini/yaml/json), and multi-stack plan/apply/destroy orchestration with confirm-guards + locking + audit |
| [Container Orchestration & Kubernetes](src/kubernetes/) | 8 | Manifest **security/reliability linter** (kube-linter-lite), hardened manifest generator (that passes the linter), resource-request/QoS/right-sizing calculator, per-environment overlay renderer (**mini-kustomize**), probe config check + live probe exec, rolling-update simulator (min-availability), **NetworkPolicy** generator + open-workload analyzer, and a Dockerfile linter + image inspector — all manifest-native (no cluster/kubectl needed) |
| [API Gateway & Service Mesh](src/api-gateway/) | 8 | A configurable edge gateway (routing/auth/LB/CORS/rewrite), canary traffic splitting with live stable-vs-canary metrics, API aggregation (BFF) parallel fan-out/merge, a service-discovery registry (TTL + heartbeat + eviction), a mesh sidecar with retries/timeouts/**circuit-breaking + outlier ejection**, a fault-injection proxy for chaos testing, **OpenAPI contract enforcement** at the edge, and mTLS service identities (CA + SPIFFE leaf certs) |
| [API Versioning & Deprecation](src/api-versioning/) | 8 | A **version-aware reverse proxy** (route by path/header/Accept/query to per-version upstreams), a proxy that injects **RFC 8594 Deprecation/Sunset/Link/Warning** headers (+ 410-on-sunset), a **version-lifecycle manifest** validator + timeline, a **sunset audit** CI-gate with live header probing, **deprecated-endpoint usage** analysis from access logs by consumer, a **Markdown migration-guide** generator between two specs, staged **T-minus deprecation notifications**, and a standalone **version-negotiation** resolver/tester |
| [Configuration Management & Secrets Rotation](src/config-secrets/) | 8 | Layered config rendering + `${VAR}`/`{{key}}` templating, JSON-Schema/rules validation, cross-environment drift diffing (secret-masked), `.env` hygiene (lint/sync/convert/redact), at-rest AES-256-GCM value encryption (git-safe `ENC[...]`), a safe two-phase secret-rotation lifecycle (generate→apply→verify→retire + rollback), rotation/expiry + TLS-cert auditing, and deploy-time `ref:env\|file\|cmd` secret resolution |
| [HashiCorp Vault Ops](src/vault/) | 8 | Host-agnostic Vault admin — run a command with only scoped secrets in a clean env, set up least-privilege AppRoles, hash-verified key rotation, unseal, encrypted backup + disaster restore, and health/KV-export over the HTTP API; configured entirely by `VAULT_ADDR`/`VAULT_TOKEN` env vars |
| [API Rate Limiting & Throttling](src/rate-limiting/) | 6 | All 5 classic limiter algorithms (token/leaky bucket, fixed/sliding window) as a library + CLI, a distributed limiter via atomic Redis Lua, an enforcing reverse proxy (429 + `Retry-After` + `RateLimit-*` headers), a self-throttling HTTP client (rate + concurrency + adaptive backoff), a resilient retry wrapper with jitter + circuit breaker, and an **authorized-use** ramp load-test to find the real ceiling |
| [Backup & Disaster Recovery](src/backup-dr/) | 8 | A full/**incremental** encrypted (AES-256-GCM) backup engine, a **chain-resolving restore** with per-file checksum verification, integrity/chain **verification** + deep test-restore, **GFS retention** pruning, database backup/restore (SQLite/Postgres/MySQL), **offsite replication** with lag tracking, a **DR-runbook** runner with RTO tracking + drills, and backup-freshness monitoring (RPO/missing/shrink alerts) |
| [Message Queue & Event Streaming](src/messaging-streams/) | 8 | Produce/consume events with consumer groups + dead-letter handling on **Redis Streams and Kafka**, monitor consumer lag (threshold alerts), manage/redrive DLQs, a versioned event-schema registry with backward/forward compatibility checks, a transactional **outbox relay** (DB→stream), event replay/reprocessing by id/time range, and a read-only live stream tap |
| [Message Queue Tools](src/message-queue/) | 10 | RabbitMQ management-API tools: health dashboard, queue inventory/health, consumer/connection audits, dead-letter monitor, topology export, message peek, publish test, and guarded purge |
| [Redis Tools](src/redis/) | 10 | Health dashboard, memory-by-prefix, big keys, TTL/config audits, slowlog, client audit, latency, hot/cold keys, and keyspace export/restore |
| [GPU & Local Stack Tools](src/gpu-stack/) | 7 | NVIDIA GPU/CUDA monitoring, Ollama/LiteLLM health, and a one-command stack dashboard |
| [SSH Tools](src/ssh/) | 8 | SSH config linting, multi-host checks, known_hosts/authorized_keys hygiene, parallel exec, tunnels |
| [Network Tools](src/network/) | 6 | authorized hosts only |
| [Network Tunneling Tools](src/network-tunneling/) | 8 | authorized/owned infrastructure ONLY |

### 🤖 AI, agents & web automation

Building blocks for the assistants and automations that drive the cycle.

| Category | Tools | Description |
|----------|------:|-------------|
| [Claude & MCP Integration](src/claude-mcp/) | 2 | An **MCP stdio server** that exposes the whole toolkit to Claude as four meta-tools (**ranked tool search**, per-tool detail with live `--help`, **safe argv execution** with timeouts + output caps, category map) driven by the generated manifest — plus a generic **MCP client/smoke-tester** for debugging any stdio server; stdlib-only, `.mcp.json` included |
| [Agentic Loop Tools](src/agent-loops/) | 23 | Reusable agent-loop patterns — ReAct, plan-execute, reflexion, self-consistency, debate, tree-of-thought, tool-routing (text + native function-calling), budget/map-reduce, verifier & code-interpreter loops, constitutional revise, LLM-judge, supervisor/worker orchestration, role pipelines, cross-model ensembles, long-term memory, context compaction, RAG-with-citations, schema-extract, gated shell operator, and FSM guardrails — against any OpenAI-compatible chat API |
| [Agent Harness & Tool Infrastructure](src/agent-harness/) | 4 | A **sandboxed subprocess tool-runner** (timeout/retry/rlimit CPU+memory, always-JSON output), **JSON-Schema** input/output validation + schema generation from an example, a shell/SQL **command safety checker** for dangerous patterns, and **semantic tool-similarity search** over descriptions — all stdlib-only, CLI + importable |
| [LLM Prompt/Inference Tuning Tools](src/llm-tuning/) | 8 | target an Ollama-compatible local API by default: `http://127.0.0.1:11434` |
| [Browser Automation Tools](src/browser-automation/) | 8 | require `playwright install chromium` after `pip install -r requirements.txt` |
| [Crawlee Web Crawling](src/crawlee/) | 8 | Advanced crawlers on the **Crawlee** framework (request queue, autoscaling, retries) — deep site crawl, list->detail router scraping, paginated JSON APIs, SEO/broken-link audit, CSS+XPath (Parsel), sitemap crawl, JS rendering + screenshots (Playwright), and a resilient/proxied crawler |
| [Libretto Browser Automation](src/libretto/) | 8 | Advanced Playwright-driven **Libretto** workflows (TypeScript, run via `libretto run <slug>`) — capture/reuse logged-in sessions, harvest infinite-scroll & paginated listings, record hidden API/XHR traffic, scripted JSON action flows, form fill, and screenshot visual-regression |
| [Audio Transcription & Voice Labeling Tools](src/audio-voice/) | 11 | transcription targets an OpenAI-compatible Whisper API by default: `http://127.0.0.1:9002` |

### 🧰 Workbench — everyday utilities

The always-useful desk drawer.

| Category | Tools | Description |
|----------|------:|-------------|
| [System & Process Tools](src/system-process/) | 5 | CPU/memory/disk diagnostics, process monitoring, service and startup management |
| [File & Data Tools](src/file-data/) | 10 | Organize, dedupe, rename, sync, back up, tail, and convert files and data |
| [File Compression & Archiving](src/compression/) | 6 | Benchmark and auto-pick codecs (gzip/bzip2/xz/zstd), create + inspect archives, losslessly recompress to reclaim space (with round-trip verification), and audit a tree for compression + duplicate savings |
| [Local Environment Tools](src/local-environment/) | 10 | Local dev-service dashboards, repo health, Obsidian notes, nginx/vault/docker helpers |
| [HLS / Live Stream Capture](src/streaming/) | 2 | Capture any `.m3u8` stream to file in original quality (ffmpeg stream-copy), or transcribe it to text without saving the video (audio-only → Whisper API); require ffmpeg |

## Security & authorized use

This toolkit is for legitimate system administration, development, security assessment, and research on systems you **own or are explicitly authorized to manage**. It contains no tools for cracking passwords, bypassing authentication, or unauthorized access. The **network recon** and **network tunneling** categories are dual-use diagnostic/ops tools — use them only against infrastructure you own or have written permission to assess, and note the per-tool authorized-use notices.

## Conventions

- **Standalone:** each script runs on its own; no cross-imports between tools.
- **Config-driven:** targets, credentials, and paths come from flags or environment variables. Secrets are read only from the environment, never passed on the command line.
- **`--dry-run`** on anything that mutates state; **`--help`** everywhere.
- **Local-first:** AI/data tools default to local services (Ollama, SearXNG, a local vector DB, etc.) and degrade gracefully when a service is unreachable.