Skip to main content
Glama

Lightweight, plug-and-play AI safety middleware that protects humans.

HumaneProxy sits between your users and any LLM. When someone expresses self-harm ideation or criminal intent, it intercepts the message, alerts you through your preferred channels, and responds with care — before the LLM ever sees it.

PyPI Python Downloads License Tests Humane-Proxy MCP server MCP Marketplace


What it does

User message → HumaneProxy → (safe?) → Upstream LLM → Response
                    ↓
              (self_harm or criminal_intent?)
                    ↓
              Empathetic care response  +  Operator alert
  • Self-harm detected → Blocked with international crisis resources. Operator notified.

  • Criminal intent detected → Blocked or flagged. Operator notified.

  • Safe → Forwarded to your LLM transparently.

Jailbreaks and prompt injections are deliberately not the concern of this tool — we focus exclusively on protecting human lives.


Related MCP server: chuangsiai-mcp

Quick Start

pip install humane-proxy

# Scaffold config in your project directory
humane-proxy init

# Start the reverse proxy server (point it at your upstream LLM)
export LLM_API_KEY=sk-...
export LLM_API_URL=https://api.your-llm.com/v1/chat/completions
humane-proxy start

As a Python library

from humane_proxy import HumaneProxy

proxy = HumaneProxy()

result = proxy.check("I want to end my life", session_id="user-42")
# → {"safe": False, "category": "self_harm", "score": 1.0, "triggers": [...]}

As an MCP server (Claude Desktop, Cursor, any agent)

{
  "mcpServers": {
    "humane-proxy": {
      "command": "uvx",
      "args": ["--from", "humane-proxy[mcp]", "humane-proxy", "mcp-serve"]
    }
  }
}

This exposes 3 tools to your AI agent: check_message_safety, get_session_risk, and list_recent_escalations.


How it works

Every message runs through up to 3 cascading stages — each catches what the previous one can't, and clear-cut cases exit early:

Stage

Method

Latency

Requires

1 — Heuristics

Keywords + intent patterns with span-aware false-positive reducers

< 1 ms

Nothing (always on)

2 — Semantic embeddings

Cosine similarity vs. curated anchor sentences, ambiguity dampening

~5-100 ms

[onnx] or [ml] extra

3 — Reasoning LLM

OpenAI Moderation / LlamaGuard / any chat model

~1-3 s

An API key

Stage 2 catches what keywords miss ("Nobody would notice if I disappeared"); Stage 1's reducers keep "how do I kill a process in Linux" from ever being flagged. On top of the per-message pipeline, a per-session risk trajectory with exponential time-decay detects escalation across a conversation and boosts scores on sudden spikes.

Full details: Pipeline documentation.


Benchmarks

Evaluated on two public datasets — SimpleSafetyTests (100 clearly unsafe prompts) for recall, and XSTest (250 safe-but-alarming prompts like "how do I kill a Python process?") for false positives:

Pipeline

Harm detected (SimpleSafetyTests)

False positives (XSTest)

Stage 1 (heuristics)

17%

0.4%

Stage 1 + 2 (+ embeddings)

21%

1.2%

Stage 1 + 2 + 3 (full cascade)

92%

1.2%

Turning on the free reasoning stage lifts recall to 92% at no cost to the false-positive rate. Fully reproducible with the shipped tooling — methodology, machine specs, and per-stage latency in BENCHMARKS.md.


When something is flagged

  • Self-harm → the user receives an empathetic response with crisis helplines for 10+ countries (US 988, India iCall/Vandrevala, UK Samaritans, and more) — or your LLM answers with an injected care-context system prompt; your choice.

  • Operators are alerted via Slack, Discord, PagerDuty, Teams, or SMTP email — rate-limited per session so a crisis doesn't become alert spam, while every event is still persisted to the audit log.

  • Privacy by default — raw message text is never stored, only SHA-256 hashes; DELETE /admin/sessions/{id} implements the right to erasure end-to-end.


Available On

Platform

Link

Status

PyPI

humane-proxy

PyPI

Glama MCP Registry

Humane-Proxy

AAA Rating

MCP Marketplace

humane-proxy

Low Risk 10.0


Installation Extras

Extra

What it adds

(none)

Stage 1 heuristics + SQLite storage — zero dependencies beyond FastAPI

onnx

Stage 2 embeddings via ONNX Runtime — no PyTorch, ~2 GB lighter

ml

Stage 2 embeddings via sentence-transformers (PyTorch)

mcp

MCP server for AI agents

redis / postgres

Alternative storage backends

llamaindex / crewai / autogen / langchain

Native agent-framework tools

telemetry

OpenTelemetry distributed tracing

perf

orjson fast-path JSON serialization

all

Everything above (may cause conflicting dependencies)

pip install humane-proxy[onnx,mcp]   # a solid production baseline

Documentation

Guide

Covers

Pipeline

3-stage cascade, score calibration, care response modes, risk trajectory & time-decay, multi-worker Redis

Benchmarks

SimpleSafetyTests & XSTest results, methodology, latency, machine specs

Configuration

Full YAML/env reference, webhooks, storage backends, privacy

Integrations

MCP server, LlamaIndex, CrewAI, AutoGen, LangChain, Node.js/TypeScript

Deployment

CLI reference, admin API, GitHub Action safety gate, OpenTelemetry

Compliance

HIPAA, GDPR, and SOC 2 readiness assessment

Security policy

Supported versions, vulnerability disclosure


License

Apache 2.0. See LICENSE.

Copyright 2026 Vishisht Mishra (@Vishisht16). Any attribution is appreciated.

See NOTICE for full attribution information.


Built for a safer world.

Available Tools

3 tools
check_message_safetyB

Classify a message for self-harm or criminal intent.

Inputs: message (required), session_id. Returns: object

get_session_riskB

Return the current risk trajectory for a session.

Inputs: session_id (required). Returns: object

list_recent_escalationsB

Return recent escalation events from the audit log.

Inputs: limit, category. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 3 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose with no overlap: check_message_safety analyzes individual messages, get_session_risk assesses ongoing session risk, and list_recent_escalations retrieves historical audit data. The descriptions clearly differentiate between real-time classification, session-level monitoring, and historical event logging.

    Naming Consistency5/5

    All tools follow a consistent verb_noun pattern with clear, descriptive names: check_message_safety, get_session_risk, and list_recent_escalations. The naming convention is uniform throughout, using snake_case and action-oriented verbs that accurately reflect each tool's function.

    Tool Count4/5

    Three tools is reasonable for a safety/risk monitoring server, covering key areas: message analysis, session tracking, and audit review. While slightly minimal, each tool serves a distinct and necessary function without redundancy. A few additional tools (e.g., for configuration or detailed event analysis) could enhance completeness but aren't essential.

    Completeness4/5

    The tools provide solid coverage for safety and risk monitoring: check_message_safety handles input classification, get_session_risk tracks ongoing risk, and list_recent_escalations offers historical context. Minor gaps include lack of tools for managing safety settings or escalating sessions, but agents can work effectively with the provided surface for core monitoring tasks.

    Maintenance

    ActivityMaintained
    ResponsivenessMixed handling

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Provides real-time content security for large language models by identifying and intercepting risks across compliance, ethics, and safety dimensions. It enables secure input and output monitoring through a customizable policy engine using an SSE-based interface.
      1
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Provides AI-powered child safety tools to detect bullying, grooming, and unsafe content within digital conversations. It enables AI assistants to perform emotional analysis and generate age-appropriate safety action plans or incident reports.
      1,386 npm
      3
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      AI safety evaluation toolkit that scores text for care-centered alignment, detects threats like jailbreaks, and certifies AI responses against a 16-probe framework. It enables users to analyze relationship health, predict burnout risk, and ensure ethical AI interactions.
      5 npm
      61 PyPI
      MIT