ADVERSA

LLM red-team harness — OWASP LLM Top 10 + MITRE ATLAS attack packs

AI Security & Governance — securing LLMs, agents, and the MCP supply chain.

pip install cognis-adversa
adversa scan .            # → prioritized findings in seconds

🔎 Example output

Real, reproducible output from the tool — runs offline:

$ adversa-emit --version
adversa 2.0.0

$ adversa-emit --help
usage: adversa [-h] [--version] {catalog,scan,probe,refs} ...

LLM red-team probe runner (OWASP LLM Top-10 + MITRE ATLAS).

positional arguments:
  {catalog,scan,probe,refs}
    catalog             list the probe catalog
    scan                run probes against a target
    probe               show detail for one probe
    refs                show OWASP + ATLAS reference tables

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit

$ adversa-emit catalog
ADVERSA probe catalog (12 probes)
==============================================================================
ID                       OWASP   ATLAS        SEV       NAME
------------------------------------------------------------------------------
pi.direct_override       LLM01   AML.TA0004   high      Direct instruction override
pi.indirect_payload      LLM01   AML.TA0006   critical  Indirect prompt injection via retrieved content
pi.encoded_smuggling     LLM01   AML.TA0009   medium    Encoded payload smuggling
leak.system_prompt       LLM07   AML.TA0011   high      System prompt extraction
leak.credentials         LLM02   AML.TA0010   critical  Sensitive credential disclosure
harm.dangerous_instructions LLM09   AML.TA0005   high      Dangerous-capability elicitation
harm.roleplay_jailbreak  LLM01   AML.TA0009   high      Persona/roleplay jailbreak (DAN-style)
output.xss_injection     LLM05   AML.TA0006   high      Improper output handling (XSS payload)
agency.tool_abuse        LLM06   AML.TA0006   high      Excessive agency / unsafe tool invocation
misinfo.confident_falsehood LLM09   AML.TA0014   medium    Misinformation / fabricated authority
consumption.amplification LLM10   AML.TA0014   low       Unbounded consumption (resource amplification)
poison.training_data     LLM04   AML.TA0003   medium    Data poisoning acknowledgement

Blocks above are real adversa output — reproduce them from a clone.

Related MCP server: MEOK MCP Hardening MCP

Usage — step by step

adversa is an LLM red-team probe runner mapping the OWASP LLM Top-10 + MITRE ATLAS onto runnable probes.

1. Install (Python 3.10+):

   pip install -e .            # or: pipx install adversa

2. Browse the bundled probe catalog (filter by OWASP/ATLAS/severity):

   adversa catalog --owasp LLM01 --min-severity high

3. Scan a target — the bundled secure/vulnerable references, a captured-response transcript (offline, no live endpoint), or your own module:callable of signature target(prompt) -> str:

   adversa scan vulnerable
   adversa scan transcript:demos/01-healthcare-chatbot/transcript.json
   adversa scan mypkg.mymodel:generate --owasp LLM01

4. Read the output as a table, JSON, or SARIF 2.1.0 (for GitHub code-scanning), or inspect one probe's prompts + grader + remediation:

   adversa scan vulnerable --format json | jq '.results[] | select(.passed==false)'
   adversa scan vulnerable --format sarif > adversa.sarif
   adversa probe pi.direct_override
   adversa refs        # OWASP LLM Top-10 + ATLAS tactic tables

5. Gate CI — scan exits 1 when findings are present, 0 when clean, 2 on usage error:

   - run: pip install -e . && adversa scan mypkg.mymodel:generate   # non-zero fails the job

Contents

• Why adversa? · Features · Quick start · Example · Demos · Architecture · AI stack · How it compares · Integrations · Install anywhere · Related · Contributing

Why adversa?

LLM red-team harness — OWASP LLM Top 10 + MITRE ATLAS attack packs — without standing up heavyweight infrastructure.

adversa is single-purpose, scriptable, and self-hostable: point it at a target, get prioritized results in the format your workflow already speaks (table · JSON · SARIF), gate CI on it, and let agents drive it over MCP.

Features

• ✅ 12-probe catalog mapped to OWASP LLM Top-10 (2025) + MITRE ATLAS tactics

• ✅ Severity ranking + filtering (--owasp, --atlas, --min-severity, --probe)

• ✅ Five graders (must-refuse, must-not-leak, must-not-contain, must-contain, injection-resisted)

• ✅ Transcript replay target — red-team captured responses offline, no live endpoint

• ✅ Bundled secure / vulnerable reference targets + module:callable for your own model

• ✅ Output as table · JSON · SARIF 2.1.0 (GitHub code-scanning ready)

• ✅ CI gate via exit codes (0 clean · 1 findings · 2 usage)

• ✅ 8 real-use-case demos with run commands + remediation guidance

• ✅ Runs on Linux/macOS/Windows · Docker · devcontainer

• ✅ Ports in Python, JavaScript, Go, and Rust (ports/)

Quick start

pip install cognis-adversa
adversa --version
adversa scan .                       # scan current project
adversa scan . --format json         # machine-readable
adversa scan . --fail-on high        # CI gate (non-zero exit)

Example

$ adversa scan .
  [HIGH    ] ADV-001  example finding             (./src/app.py)
  [MEDIUM  ] ADV-002  another signal              (./config.yaml)

  2 findings · risk score 5 · 38ms

Demos — real scenarios you can run now

Each demos/<NN-name>/ holds a realistic input (a captured-response transcript.json in ADVERSA's real format, or a module:callable target) plus a SCENARIO.md explaining where the data came from, the exact command, what to expect, and how to act on the findings.

adversa scan transcript:demos/01-healthcare-chatbot/transcript.json   # 4 findings, exit 1
adversa scan transcript:demos/02-post-hardening-clean/transcript.json # 0 findings, exit 0

The transcript shape is either a probe-id map ({"leak.system_prompt": "<reply>"}) or a list of {"probe_id": "...", "response": "..."} pairs — capture your model's replies once, then grade them offline as often as you like.

Architecture

flowchart LR
  IN[sources] --> P[adversa<br/>curate + validate]
  P --> OUT[query / analysis]

Use it from any AI stack

adversa is interoperable with every popular way of using AI:

• MCP server — adversa mcp (Claude Desktop, Cursor, Cognis.Studio, uncensored-fleet)

• OpenAI-compatible / JSON — pipe adversa scan . --format json into any agent or LLM

• LangChain · CrewAI · AutoGen · LlamaIndex — wrap the CLI/JSON as a tool in one line

• CI / scripts — exit codes + SARIF for non-AI pipelines

How it compares

Built in the spirit of leondz/garak, re-framed the Cognis way. Missing a credit? Open a PR.

Integrations

Pipes into your stack: SARIF for code-scanning, JSON for anything, an MCP server (adversa mcp) for AI agents, and a webhook forwarder for SIEM/Slack/Jira. See docs/INTEGRATIONS.md.

Install — every way, every platform

pip install "git+https://github.com/cognis-digital/adversa.git"    # pip (works today)
pipx install "git+https://github.com/cognis-digital/adversa.git"   # isolated CLI
uv tool install "git+https://github.com/cognis-digital/adversa.git" # uv
pip install cognis-adversa                                          # PyPI (when published)
docker run --rm ghcr.io/cognis-digital/adversa:latest --help        # Docker
brew install cognis-digital/tap/adversa                             # Homebrew tap
curl -fsSL https://raw.githubusercontent.com/cognis-digital/adversa/main/install.sh | sh

Related Cognis tools

• aegis — AI Agent Permission & Access Auditor — surfaces the lethal trifecta of credentials + injection + reach

• promptmirror — Prompt-injection & indirect-injection scanner for any LLM context input

• ledgermind — Local LLM cost & token forensics proxy with anomaly detection

• guardpost — Runtime agent firewall — PII redaction, rate limits, policy enforcement

• hallumark — LLM hallucination & grounding auditor for RAG systems

• aicard — Auto-generated NIST AI RMF / EU AI Act Annex IV model & system cards

Explore the suite → 🗂️ all 170+ tools · ⭐ awesome-cognis · 🔗 cognis-sources · 🤖 uncensored-fleet · 🧠 engram

Contributing

PRs, new rules, and demo scenarios are welcome under the collaboration-pull model — see CONTRIBUTING.md and SECURITY.md.

⭐ If adversa saved you time, star it — it genuinely helps others find it.

Interoperability

{} composes with the 300+ tool Cognis suite — JSON in/out and a shared OpenAI-compatible /v1 backbone. See INTEROP.md for the suite map, composition patterns, and reference stacks.

License

Source-available under the Cognis Open Collaboration License (COCL) v1.0 — free for personal, internal-evaluation, research, and educational use; commercial / production use requires a license (licensing@cognis.digital). See LICENSE.