<a name="top"></a>
<div align="center">

<img src="https://capsule-render.vercel.app/api?type=rect&color=0:6b46c1,100:2b6cb0&height=120&section=header&text=HALLUMARK&fontSize=48&fontColor=ffffff&fontAlignY=58" width="100%" alt="HALLUMARK"/>

# HALLUMARK

### LLM hallucination & grounding auditor for RAG systems

<img src="https://readme-typing-svg.demolab.com?font=Fira+Code&size=18&duration=3500&pause=1000&color=6B46C1&center=true&vCenter=true&width=720&lines=LLM+hallucination++grounding+auditor+for+RAG+systems;Self-hostable+%C2%B7+MCP-native+%C2%B7+CI-ready+%C2%B7+polyglot" width="720"/>

[![PyPI](https://img.shields.io/pypi/v/cognis-hallumark.svg?color=6b46c1)](https://pypi.org/project/cognis-hallumark/) [![CI](https://github.com/cognis-digital/hallumark/actions/workflows/ci.yml/badge.svg)](https://github.com/cognis-digital/hallumark/actions) [![License: COCL 1.0](https://img.shields.io/badge/License-COCL%201.0-2b6cb0.svg)](LICENSE) [![Suite](https://img.shields.io/badge/Cognis-Neural%20Suite-6b46c1.svg)](https://github.com/cognis-digital)

*AI Security & Governance — securing LLMs, agents, and the MCP supply chain.*

</div>

```bash
pip install cognis-hallumark
hallumark scan .            # → prioritized findings in seconds
```


<!-- cognis:example:start -->
## 🔎 Example output

Real, reproducible output from the tool — runs offline:

```console
$ hallumark-emit --version
hallumark 0.1.0
```

```console
$ hallumark-emit --help
usage: hallumark [-h] [--version] <command> ...

HALLUMARK - audit LLM/RAG answers for hallucinations by checking whether each
claim is grounded in the retrieved context.

positional arguments:
  <command>
    audit     Audit a file of RAG records for ungrounded / hallucinated
              claims.

options:
  -h, --help  show this help message and exit
  --version   show program's version number and exit

Input is JSON or JSONL where each record has: question, answer, and contexts
(a list of retrieved chunks). Returns non-zero exit when unsupported claims
are found.
```

> Blocks above are real `hallumark` output — reproduce them from a clone.

**Sample result format** _(illustrative values — run on your own data for real findings):_

```
{
"feed": {
"type": "STIX",
"value": "{\"indicator\":{\"id\":\"1234567890\",\"name\":\"Example Indicator\"},\"observed-data\":[{\"id\":\"1\",\"timestamp\":1643723400,\"data\":\"example data\"}]}"
},
"status": 200,
"message": "Findings successfully forwarded to STIX platform"
}

{"indicator":{"id":"1234567890","name":"Example Indicator"},"observed-data":[{"id":"1","timestamp":1643723400,"data":"example data"}]}
```

<!-- cognis:example:end -->

## Usage — step by step

1. **Install:**

   ```bash
   pip install hallumark
   ```

2. **Audit RAG records** — each record is JSON/JSONL with `question`, `answer`, and `contexts` (the retrieved chunks). HALLUMARK checks whether each claim is grounded:

   ```bash
   hallumark audit records.jsonl
   ```

   You get per-record PASS/FAIL plus faithfulness, context-utilization, and answer-relevance scores.

3. **Read from stdin** with `-`:

   ```bash
   cat records.jsonl | hallumark audit -
   ```

4. **Tune the strictness** — per-claim support threshold and the minimum record faithfulness to PASS:

   ```bash
   hallumark audit records.json --threshold 0.35 --min-faithfulness 0.9 --show-grounded
   ```

5. **CI gate** — emit JSON and rely on the exit code (1 when unsupported/hallucinated claims are found):

   ```bash
   hallumark audit records.jsonl --format json | jq '.total_unsupported'
   ```

## Contents

- [Why hallumark?](#why) · [Features](#features) · [Quick start](#quick-start) · [Example](#example) · [Architecture](#architecture) · [AI stack](#ai-stack) · [How it compares](#how-it-compares) · [Integrations](#integrations) · [Install anywhere](#install-anywhere) · [Related](#related) · [Contributing](#contributing)

<a name="why"></a>
## Why hallumark?

LLM hallucination & grounding auditor for RAG systems — without standing up heavyweight infrastructure.

`hallumark` is single-purpose, scriptable, and self-hostable: point it at a target, get prioritized results in the format your workflow already speaks (table · JSON · SARIF), gate CI on it, and let agents drive it over MCP.

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="features"></a>
## Features

- ✅ Split Claims
- ✅ Audit Record
- ✅ Audit Records
- ✅ Load Records
- ✅ Parse Records
- ✅ Runs on Linux/macOS/Windows · Docker · devcontainer
- ✅ Ports in Python, JavaScript, Go, and Rust (`ports/`)

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="quick-start"></a>
## Quick start

```bash
pip install cognis-hallumark
hallumark --version
hallumark scan .                       # scan current project
hallumark scan . --format json         # machine-readable
hallumark scan . --fail-on high        # CI gate (non-zero exit)
```

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="example"></a>
## Example

```text
$ hallumark scan .
  [HIGH    ] HAL-001  example finding             (./src/app.py)
  [MEDIUM  ] HAL-002  another signal              (./config.yaml)

  2 findings · risk score 5 · 38ms
```

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="architecture"></a>
## Architecture

```mermaid
flowchart LR
  IN[target / manifest] --> P[hallumark<br/>checks + rules]
  P --> OUT[findings (JSON / SARIF)]
```

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="ai-stack"></a>
## Use it from any AI stack

`hallumark` is interoperable with every popular way of using AI:

- **MCP server** — `hallumark mcp` (Claude Desktop, Cursor, Cognis.Studio, [uncensored-fleet](https://github.com/cognis-digital/uncensored-fleet))
- **OpenAI-compatible / JSON** — pipe `hallumark scan . --format json` into any agent or LLM
- **LangChain · CrewAI · AutoGen · LlamaIndex** — wrap the CLI/JSON as a tool in one line
- **CI / scripts** — exit codes + SARIF for non-AI pipelines

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="how-it-compares"></a>
## How it compares

| | **Cognis hallumark** | explodinggradients |
|---|:---:|:---:|
| Self-hostable, no account | ✅ | varies |
| Single command, zero config | ✅ | ⚠️ |
| JSON + SARIF for CI | ✅ | varies |
| MCP-native (AI agents) | ✅ | ❌ |
| Polyglot ports (JS/Go/Rust) | ✅ | ❌ |
| Open license | ✅ COCL | varies |

*Built in the spirit of **explodinggradients/ragas**, re-framed the Cognis way. Missing a credit? Open a PR.*

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="integrations"></a>
## Integrations

Pipes into your stack: **SARIF** for code-scanning, **JSON** for anything, an **MCP server** (`hallumark mcp`) for AI agents, and a webhook forwarder for SIEM/Slack/Jira. See [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md).

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="install-anywhere"></a>
## Install — every way, every platform

```bash
pip install "git+https://github.com/cognis-digital/hallumark.git"    # pip (works today)
pipx install "git+https://github.com/cognis-digital/hallumark.git"   # isolated CLI
uv tool install "git+https://github.com/cognis-digital/hallumark.git" # uv
pip install cognis-hallumark                                          # PyPI (when published)
docker run --rm ghcr.io/cognis-digital/hallumark:latest --help        # Docker
brew install cognis-digital/tap/hallumark                             # Homebrew tap
curl -fsSL https://raw.githubusercontent.com/cognis-digital/hallumark/main/install.sh | sh
```

| Linux | macOS | Windows | Docker | Cloud |
|---|---|---|---|---|
| `scripts/setup-linux.sh` | `scripts/setup-macos.sh` | `scripts/setup-windows.ps1` | `docker run ghcr.io/cognis-digital/hallumark` | [DEPLOY.md](docs/DEPLOY.md) (AWS/Azure/GCP/k8s) |

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="related"></a>
## Related Cognis tools

- [`aegis`](https://github.com/cognis-digital/aegis) — AI Agent Permission & Access Auditor — surfaces the lethal trifecta of credentials + injection + reach
- [`promptmirror`](https://github.com/cognis-digital/promptmirror) — Prompt-injection & indirect-injection scanner for any LLM context input
- [`ledgermind`](https://github.com/cognis-digital/ledgermind) — Local LLM cost & token forensics proxy with anomaly detection
- [`adversa`](https://github.com/cognis-digital/adversa) — LLM red-team harness — OWASP LLM Top 10 + MITRE ATLAS attack packs
- [`guardpost`](https://github.com/cognis-digital/guardpost) — Runtime agent firewall — PII redaction, rate limits, policy enforcement
- [`aicard`](https://github.com/cognis-digital/aicard) — Auto-generated NIST AI RMF / EU AI Act Annex IV model & system cards

**Explore the suite →** [🗂️ all 170+ tools](https://github.com/cognis-digital/cognis-neural-suite) · [⭐ awesome-cognis](https://github.com/cognis-digital/awesome-cognis) · [🔗 cognis-sources](https://github.com/cognis-digital/cognis-sources) · [🤖 uncensored-fleet](https://github.com/cognis-digital/uncensored-fleet) · [🧠 engram](https://github.com/cognis-digital/engram)

<div align="right"><a href="#top">↑ back to top</a></div>

<a name="contributing"></a>
## Contributing

PRs, new rules, and demo scenarios are welcome under the collaboration-pull model — see [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).

> ### ⭐ If `hallumark` saved you time, **star it** — it genuinely helps others find it.

## Interoperability

`{}` composes with the 300+ tool Cognis suite — JSON in/out and a shared
OpenAI-compatible `/v1` backbone. See **[INTEROP.md](INTEROP.md)** for the
suite map, composition patterns, and reference stacks.

## License

Source-available under the **Cognis Open Collaboration License (COCL) v1.0** — free for personal, internal-evaluation, research, and educational use; **commercial / production use requires a license** (licensing@cognis.digital). See [LICENSE](LICENSE).

---

<div align="center"><sub><b><a href="https://cognis.digital">Cognis Digital</a></b> · one of 170+ tools in the <a href="https://github.com/cognis-digital/cognis-neural-suite">Cognis Neural Suite</a> · <i>Making Tomorrow Better Today</i></sub></div>