mcp-audit
Enables exporting audit data (calls, costs, metrics) in JSONL or CSV format for ingestion into Datadog for monitoring and analysis.
Enables exporting audit data in JSONL or CSV format for visualization and dashboarding in Grafana.
Enables exporting audit data in JSONL or CSV format for log analysis and observability in Splunk.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-auditshow me the audit summary"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-audit
mcp-audit is a Model Context Protocol server that gives AI agents observability over their own tool calls. Every time your agent invokes a tool — whether it's an MCP tool, a custom function, or an external API — mcp-audit records who called what, when, how long it took, what it cost, and whether it succeeded.
Why?
AI agents are increasingly autonomous, calling dozens of tools per session. But agents are blind to their own behavior:
❌ No way to see which tools an agent called or how often
❌ No cost tracking — agents can silently rack up API bills
❌ No latency visibility — one slow tool tanks the whole session
❌ No error correlation — which tool is failing 30% of the time?
❌ No alerting — you find out about runaway costs after the fact
mcp-audit fixes this. It's an MCP server that agents use to audit themselves.
Related MCP server: production-grade-mcp-agentic-system
Features
📊 Call Recording — log every tool invocation with timing, cost, tokens, and result
🔍 Flexible Querying — filter by session, agent, tool, server, status, cost, tags
📈 Aggregate Analytics — error rate, p50/p95/p99 latency, total cost, top tools
💰 Cost Breakdown — see exactly which tools are consuming budget, grouped by tool/server/session
🚨 Alert Rules — set thresholds (e.g. "alert if error_rate > 50%" or "alert if total_cost > $10")
📝 Trace Events — fine-grained structured logging within calls (sub-steps, HTTP requests, DB queries)
📊 Tool Health Dashboard — per-tool metrics at a glance (error rate, p95 latency, cost)
💾 SQLite Persistence — durable storage that survives restarts (drop-in replacement for memory store)
🎯 Auto-Instrumentation —
@audit_calldecorator for zero-code tracing of any Python function📤 Data Export — JSONL & CSV export for feeding data to Grafana, Datadog, Splunk, ELK
🔭 OpenTelemetry (OTLP) — Export traces and metrics in OTLP format for any OTel-compatible backend
📈 Prometheus Exposition — Native Prometheus text format for direct scraping (no OTel collector needed)
🏷️ Agent Reports — comprehensive per-agent performance summaries
🪝 Context Manager — Python
withblock for automatic call tracing
Quick Start
Install
pip install mcp-auditUse as a Python library
from mcp_audit import AuditEngine, traced_call
engine = AuditEngine()
session = engine.start_session(agent_id="my-agent")
# Wrap any function call with automatic tracing
with traced_call(engine, session_id=session.id, tool_name="web_search") as tc:
tc.set_cost(0.003)
tc.set_tokens(input_tokens=500, output_tokens=200)
result = search("best MCP servers")
tc.set_result(result)
# Query analytics
stats = engine.get_stats(session_id=session.id)
print(f"Total cost: ${stats['total_cost_usd']}")
print(f"P95 latency: {stats['p95_latency_ms']}ms")
print(f"Error rate: {stats['error_rate']}%")Use as an MCP server
Add to your MCP client config (Claude Desktop, Cursor, etc.):
{
"mcpServers": {
"mcp-audit": {
"command": "mcp-audit",
"args": ["stdio"]
}
}
}This runs mcp-audit as a real MCP server over stdio transport. Your agent can now call audit tools like record_call, get_stats, create_alert_rule, and evaluate_alerts directly through the MCP protocol.
Note: Use
mcp-audit stdio(notmcp-audit serve). Theservecommand prints configuration JSON for reference;stdioruns the actual MCP stdio transport.
Use as a Python library with FastMCP
For programmatic integration:
from mcp_audit import create_fastmcp_server
# Get a FastMCP instance with all 17 tools registered
server = create_fastmcp_server()
server.run(transport="stdio")MCP Tools (23)
Tool | Description |
| Start a new audit session for an agent |
| End a session and compute final aggregates |
| Get session details with aggregate metrics |
| List sessions with optional agent/active filters |
| Record a completed tool call (primary ingestion) |
| Look up a specific tool call by ID |
| Search calls with flexible filters |
| Log a structured trace event (sub-step) |
| Query trace events with filters |
| Aggregate statistics (error rate, percentiles, cost) |
| Comprehensive per-agent performance report |
| Cost analysis grouped by tool/server/session |
| Set threshold-based alert rules |
| List configured alert rules |
| Remove an alert rule |
| Check which alert rules are currently breached |
| Per-tool health metrics (error rate, p95, cost) |
| Get the N most recent tool calls |
| Export calls to JSONL or CSV file |
| Export traces in OpenTelemetry (OTLP) format |
| Export metrics in OpenTelemetry (OTLP) format |
| Export metrics in Prometheus text exposition format |
| High-level dashboard summary |
SQLite Persistence
For production use, persist audit data across restarts with the SQLite backend:
from mcp_audit import AuditEngine
from mcp_audit.sqlite_store import SQLiteStore
store = SQLiteStore("audit.db") # persists to disk
engine = AuditEngine(store=store)
# All calls, sessions, events, and rules now survive process restarts
session = engine.start_session(agent_id="prod-agent")SQLiteStore is a drop-in replacement for the default MemoryStore — same interface, durable storage. Uses indexed columns for efficient querying on session_id, agent_id, tool_name, status, and timestamps.
Auto-Instrumentation with @audit_call
Skip manual tracing — decorate any function and it's automatically audited:
from mcp_audit import AuditEngine
from mcp_audit.decorator import audit_call, bind_session
engine = AuditEngine()
session = engine.start_session(agent_id="my-agent")
# Option 1: explicit engine + session
@audit_call(engine, session_id=session.id, cost_fn=lambda *_: 0.001)
def search(query: str) -> list:
return [{"title": "result"}]
# Option 2: bind once, decorate everywhere
ctx = bind_session(engine, session.id)
@audit_call() # uses bound engine + session
def fetch(url: str) -> dict:
return requests.get(url).json()
@audit_call(tool_name="llm_complete", cost_fn=compute_cost)
def complete(prompt: str) -> str:
return llm.generate(prompt)
ctx.reset() # unbind when doneEvery call is automatically recorded with timing, status, and errors. Exceptions are recorded as errors and re-raised.
Data Export
Export audit data for external tools:
from mcp_audit.export import export_calls_jsonl, export_calls_csv
# JSONL for log shippers (Datadog, Splunk, ELK)
export_calls_jsonl(engine, "audit.jsonl", session_id=sid)
# CSV for spreadsheet analysis
export_calls_csv(engine, "costs.csv", agent_id="prod-agent")
# Or export to a string
from mcp_audit.export import export_to_string
text = export_to_string(engine, fmt="jsonl", limit=100)Or call the export_calls MCP tool directly from your agent.
OpenTelemetry (OTLP) Export
Export traces and metrics in OpenTelemetry format for ingestion by OTel Collectors, Jaeger, Grafana Tempo, Datadog, and more:
from mcp_audit.otlp import export_otlp_http, export_otlp_jsonl
from mcp_audit.metrics import export_otlp_metrics_http, build_metrics
# Export traces to a local OTel collector
export_otlp_http(engine, endpoint="http://localhost:4318/v1/traces")
# Export metrics (counters, histograms, gauges) to OTel collector
export_otlp_metrics_http(engine, endpoint="http://localhost:4318/v1/metrics")
# Or build OTLP metric dicts for inspection
metrics = build_metrics(engine)Or call the export_otlp / export_otlp_metrics MCP tools from your agent.
Prometheus Export
Export metrics in Prometheus text exposition format for direct scraping by Prometheus, Grafana Agent, VictoriaMetrics, or Datadog Agent — no OTel Collector required:
from mcp_audit.prometheus import build_prometheus_exposition, export_prometheus_file
# Build exposition text (serve on /metrics endpoint)
text = build_prometheus_exposition(engine)
# Write to file for node_exporter textfile collector
export_prometheus_file(engine, "/var/lib/node_exporter/mcp_audit.prom")Serve as a Prometheus endpoint (Flask example):
from flask import Flask
from mcp_audit.prometheus import build_prometheus_exposition, PROMETHEUS_CONTENT_TYPE
app = Flask(__name__)
@app.route("/metrics")
def metrics():
text = build_prometheus_exposition(engine)
return text, 200, {"Content-Type": PROMETHEUS_CONTENT_TYPE}Push to a Pushgateway:
from mcp_audit.prometheus import export_prometheus_http
export_prometheus_http(engine, endpoint="http://localhost:9091", job_name="mcp-audit")Metric families produced:
Metric | Type | Description |
| Counter | Total tool calls by tool |
| Counter | Error calls by tool |
| Histogram | Call latency distribution by tool |
| Histogram | Per-call cost distribution by tool |
| Counter | Total tokens (input + output) by tool |
| Counter | Input tokens by tool |
| Counter | Output tokens by tool |
| Gauge | Session count (scope: all/active) |
| Gauge | Overall error rate (%) |
| Gauge | Total cost (USD) |
| Gauge | Average call duration (ms) |
| Gauge | Average cost per call (USD) |
Or call the export_prometheus MCP tool from your agent.
Alert Rules
Set up automatic monitoring:
engine.create_rule(
name="high_error_rate",
metric="error_rate", # error_rate | p95_latency | cost_per_call | total_cost | call_volume
operator=">", # > | >= | < | <= | ==
threshold=50.0, # threshold value
window=100, # evaluate last N calls
)
engine.create_rule(
name="budget_exceeded",
metric="total_cost",
operator=">=",
threshold=10.0,
)
# Check if any rules are breached
alerts = engine.evaluate_rules()Architecture
┌─────────────────────────────────────────────────────┐
│ AI Agent (Claude, etc.) │
│ │
│ ┌──────────┐ ┌──────────┐ ┌───────────────────┐ │
│ │ MCP Tool │ │ MCP Tool │ │ mcp-audit │ │
│ │ Server A │ │ Server B │ │ (this server) │ │
│ └────┬─────┘ └────┬─────┘ └────────┬──────────┘ │
│ │ │ │ │
│ └──────┬───────┘ │ │
│ │ Agent calls record_call │ │
│ └──────────────────────────┘ │
└─────────────────────────────────────────────────────┘The agent calls tools on other MCP servers, then calls record_call on mcp-audit to log what happened. Alternatively, wrap tool calls programmatically using the traced_call context manager.
Development
git clone https://github.com/nyx-builds/mcp-audit.git
cd mcp-audit
uv venv .venv
VIRTUAL_ENV=$(pwd)/.venv uv pip install -e ".[dev]"
.venv/bin/python -m pytest -qLicense
MIT
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- Flicense-qualityCmaintenanceAn observability tool for agentic AI pipelines that intercepts MCP and Python tool calls to provide real-time metrics, session replays, and alerts via a local dashboard. It enables centralized monitoring of multiple MCP servers through multiplexer and proxy modes without requiring changes to existing agent code.Last updated3
- Alicense-qualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.Last updated52MIT
- Alicense-qualityAmaintenanceA local-first control plane for AI agent tools, providing policy enforcement, spend caps, rate limiting, and audit trails for MCP servers.Last updated1Apache 2.0
- Alicense-qualityDmaintenanceA MCP server for tracking AI usage metrics and structured logs across applications. Monitor model calls, analyze usage patterns, track costs, and debug AI interactions.Last updated1MIT
Related MCP Connectors
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
See, price, and control every tool call your AI agents make: policy checks, cost, and audit tools.
Control plane for autonomous software labor. Agents claim objectives over MCP with audit trail.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/nyx-builds/mcp-audit'
If you have feedback or need assistance with the MCP directory API, please join our Discord server