Skip to main content
Glama
vivekanjana76

azure-compliance-mcp

azure-compliance-mcp

CI License: MIT Python 3.12+ FastMCP 3.x

Honest Azure compliance for LLM agents — ask infra-health questions in plain English and get answers that distinguish a real fail from a not_evaluable, and never fake a pass.

An MCP server, built with FastMCP 3.x, that exposes read-only Azure resource-compliance data so an agent can answer questions like "which storage accounts allow TLS 1.0?" or "which VMs have no disk encryption?" — each finding tagged with where the verdict came from and whether it could be evaluated at all.

Demo

Why honest compliance?

Most tooling collapses "compliant", "non-compliant", and "couldn't tell" into a single pass/fail bit — so a control it can't actually measure silently shows green. This server refuses to guess:

  • fail — positive evidence a protection is absent (e.g. encryptionAtHost = false). Actionable; shows up in the default view.

  • not_evaluable — the signal genuinely isn't in the data (e.g. no guest-config policy assigned). Surfaced explicitly, never as a pass.

  • source — every finding records the signal that decided its verdict: arg | azure_policy | graph | defender.

Related MCP server: Purple AI MCP Server

Quickstart

Requires uv and Python 3.12+. Runs out of the box — mock mode needs zero Azure setup.

uv sync                          # install dependencies
uv run python scripts/demo.py    # fastest look — runs the tools against mock data
uv run server.py                 # mock data, stdio transport — works immediately

Point it at your own tenant when you're ready:

uv run server.py --mode live     # real Azure Resource Graph (e.g. after `az login`)
uv run server.py --transport http  # remote: Streamable HTTP
uv run fastmcp dev inspector server.py  # explore the tools interactively

Live mode authenticates with DefaultAzureCredential against your own tenant and is strictly read-only.

Architecture

Tools depend only on a Provider protocol, never on a concrete data source — so the same tool code runs against synthetic mock data or live Azure Resource Graph (ARG), selected at startup with --mode.

flowchart LR
    Agent["LLM agent / MCP client"] -->|stdio · HTTP| Server["FastMCP server\n(5 read-only tools)"]
    Server --> Protocol["Provider protocol\n(read-only)"]
    Protocol -->|--mode mock| Mock["MockProvider\nseeded ARG-shaped data"]
    Protocol -->|--mode live| Live["LiveProvider\nDefaultAzureCredential"]
    Live -->|injection-safe KQL| ARG[("Azure Resource Graph\nresources · policyresources ·\npatchassessmentresources · authorizationresources")]
    Live -->|getByIds| Graph[("Microsoft Graph\nprincipal existence")]

Controls (check_compliance evaluates five):

control

source

signal

required_tags

arg

resources.tags (env / owner / costCenter)

tls_min_1_2

arg

resources.properties.minimumTlsVersion

public_network_access

arg

resources.properties.publicNetworkAccess

disk_encryption

arg

securityProfile.encryptionAtHost — host-level only; OFF ⇒ fail, ADE not assessed

guest_config_extension

azure_policy

policyresources guest-config states; no data ⇒ not_evaluable

Tools (5 of a 5–6 budget):

tool

source

honest behavior

check_compliance

arg / azure_policy

the five controls above; unmeasurable signal ⇒ not_evaluable, never a fake pass

query_resources

arg

read-only projection with filters pushed into KQL

get_patch_status

arg

Update Manager assessments from patchassessmentresources; deallocated VM or no assessment data ⇒ not_evaluable with pendingUpdateCount: null (unknown ≠ 0), never a fake "current"

find_orphaned_rbac

graph

assignments from ARG authorizationresources, principal existence resolved via Microsoft Graph; orphaned only on a positive directory not-found — unresolvable (no Directory.Read.All) ⇒ not_evaluable, never orphaned on a guess

Transports: stdio (default, local) and Streamable HTTP (remote, intended behind OAuth 2.1). In stdio mode stdout is reserved for the protocol — logging goes to stderr only.

See SPEC.md for the full tool contracts.

Design decisions & tradeoffs

  • Mock provider is the default. The repo runs with zero Azure account, credentials, or network — so reviewers, CI, and new contributors get meaningful output immediately. The mock dataset is shaped exactly like ARG rows, so one mapping serves both modes.

  • Read-only by construction. No tool can create, update, or delete a resource. The live path only ever issues ARG queries; there is no mutation surface to misuse.

  • Control-based, not Azure-Policy-based. Compliance is evaluated from resource configuration (the ARG row), so it works even when no Azure Policy is assigned — except where the signal genuinely lives elsewhere (guest-config), which routes to policyresources and is honest about it.

  • The not_evaluable + source honesty model. A third status plus a provenance field means the agent can tell "this is broken" from "I can't see this from here" — the core reason to trust the output.

  • KQL pushdown for scale. Live filters (query_resources) are pushed into the ARG query (where + take) rather than fetched-then-filtered, so large tenants stay cheap. An opt-in contract test asserts the pushdown matches the reference Python filter, so both modes stay provably consistent.

  • Injection-safe escaping. ARG has no bind parameters, so every user-supplied value is encoded as an escaped KQL string literal — never concatenated raw. (ARG is read-only regardless, but the discipline is enforced and unit-tested.)

Status & roadmap

v0.3.0check_compliance, query_resources, get_patch_status, and find_orphaned_rbac shipped (mock + live), honest not_evaluable/source model across all of them, KQL pushdown, CI on every PR.

Next:

  • summarize_health — rolled-up infra-health summary

  • evals — task-level evaluation of agent answers over the mock dataset

Development

uv run pytest               # tests (live tests are opt-in: RUN_LIVE_TESTS=1)
uv run ruff check .         # lint
uv run ruff format --check . # format

Evals (opt-in)

Tests prove the tools behave; the eval suite (SPEC §5) checks the property the server exists for: given a natural-language question, does an agent call the right tool, with the right arguments, and answer correctly — including saying "that can't be determined" when the honest answer is not_evaluable.

# needs ANTHROPIC_API_KEY (env or the gitignored .env); calls the API — costs money
uv run --group evals python evals/run.py                     # full suite (~20 cases)
uv run --group evals python evals/run.py --cases rbac        # filter by case id
uv run --group evals python evals/run.py --agent-model <id>  # evaluate another agent model

# alternate provider for cost-free smoke runs (needs OPENROUTER_API_KEY);
# a non-Anthropic judge is OFF-PIN vs SPEC §5.4 — the report flags it
uv run --group evals python evals/run.py --provider openrouter \
    --agent-model <openrouter-id> --judge-model <openrouter-id>

Runs against mock mode only (deterministic, no Azure credentials). Results land in evals/results/<timestamp>-<model>.json plus a rendered summary; they're gitignored except deliberately committed baseline-* files. Structured assertions are exact-match; open-ended prose is scored by a pinned LLM judge whose ~85–92% human-agreement caveat is printed in every report — judge scores are signal, not ground truth. Deliberately not in CI (SPEC §5.6).

Security

  • All Azure tools are read-only — nothing in this server modifies Azure resources.

  • Secrets, tenant IDs, and .env are gitignored and must never be committed; live mode uses only your local DefaultAzureCredential.

  • In stdio mode, logging goes to stderr only (stdout is reserved for the protocol).

License

MIT

Available Tools

3 tools
check_complianceA

Evaluate named security/governance controls and report where each verdict came from.

Honest about provenance: every finding carries a source (arg | azure_policy | defender), and controls whose signal is not in the available data report not_evaluable rather than guessing. Most controls read the Azure Resource Graph resources row directly (no policy assignment required); guest_config_extension reads guest-config compliance from the ARG policyresources table and is not_evaluable when no policy is assigned. disk_encryption is partial — it confirms encryption-at-host from ARG but cannot see disk-level/ADE encryption, so a missing host-encryption signal is not_evaluable, not fail.

ParametersJSON Schema
NameRequiredDescriptionDefault
scopeNoRestrict to resources whose ARG resource ID contains this string (matches a subscription id, resource group, or resource name).
controlNoWhich control to check, or None to check every applicable control. Controls: guest_config_extension, tls_min_1_2, required_tags (env/owner/costCenter), disk_encryption, public_network_access.
resource_typeNoRestrict to a single ARG type, e.g. "microsoft.compute/virtualmachines".
status_filterNoReturn only "fail" rows (default), only "pass" rows, only "not_evaluable" rows, or "all". The default ("fail") answers "what's actively non-compliant?" and excludes "not_evaluable".fail

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description bears full responsibility for behavioral transparency. It fully discloses traits: honest provenance, 'not_evaluable' for missing data, partial coverage of 'disk_encryption', and specific details about 'guest_config_extension'. It sets accurate expectations about limits and data sources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is fairly concise for the complexity, front-loading the main purpose and then efficiently explaining key behavioral notes. Every sentence adds value, though it could be slightly sharper (e.g., avoid repeating 'not_evaluable' multiple times).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 parameters, 5 controls, output schema), the description covers essential behavioral aspects and edge cases (e.g., partial controls, provenance). It doesn't detail the output schema fields or all response formats, but those are handled by the output schema. The description is sufficiently complete for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds minimal value beyond the schema for parameters; it focuses on general behavior rather than enriching parameter meanings. The provenance detail pertains to output, not input semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Evaluate named security/governance controls and report where each verdict came from.' It uses a specific verb ('evaluate') and resource ('controls'), and the mention of provenance distinguishes it from typical compliance tools. Siblings are different (patch status, query resources), and this description uniquely defines its scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for compliance checks and provides details on control behaviors, but it does not explicitly state when to use this tool over siblings like 'query_resources' or 'get_patch_status'. No direct guidance on when not to use it or alternatives is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_patch_statusA

Report per-VM patch/update posture from Azure Resource Graph patch assessments.

Honest about provenance: every finding carries source="arg" — verdicts come from the ARG patchassessmentresources table (Azure Update Manager assessment data), never inferred from resource properties alone. A VM whose posture cannot be trusted is not_evaluable — a deallocated/stopped VM (Azure cannot assess a stopped VM) or a VM with no assessment data — with pendingUpdateCount null (unknown, not zero), never a fake "current".

ParametersJSON Schema
NameRequiredDescriptionDefault
scopeNoRestrict to VMs whose ARG resource ID contains this string (matches a subscription id, resource group, or resource name).
severityNoWhich update classifications count: "critical" (critical only), "security" (critical + security), or "all" (default).all
status_filterNoReturn only "pending" VMs (default), only "current", only "not_evaluable", or "all". The default answers "which VMs have pending updates?" and excludes "not_evaluable".pending

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully handles transparency. It explains the data provenance (source='arg' from patchassessmentresources table), defines the not_evaluable state for deallocated/stopped VMs, and clarifies that pendingUpdateCount is null (unknown) rather than zero. This goes well beyond a typical description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear opening sentence, followed by important caveats. It is concise without extraneous information, though it could be slightly more compact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, the description does not need to explain return values. It covers edge cases (not_evaluable), provenance, and parameter defaults. The tool is fully described for its intended use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds some context (e.g., default status_filter returns pending, excludes not_evaluable), but the schema already provides detailed descriptions for each parameter. No significant additional meaning is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Report per-VM patch/update posture from Azure Resource Graph patch assessments.' It provides a specific verb (report) and resource (per-VM patch/update posture), and distinguishes from siblings like check_compliance and query_resources by focusing on patch assessment data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for checking patch posture but does not explicitly state when to use this tool over its siblings (check_compliance, query_resources). No when-not-to-use or alternative guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_resourcesA

Look up resources with structured, read-only filters.

All filters are combined with AND. Returns an ARG-shaped projection.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of rows to return (default 100; must be >= 1).
locationNoAzure region to match exactly, e.g. "eastus".
tag_filtersNoTags that must all be present with the given values.
name_containsNoCase-insensitive substring the resource name must contain.
resource_typeNoARG type to match exactly, e.g. "microsoft.storage/storageaccounts".

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It reveals read-only behavior and ARG output shape, but does not disclose permissions, rate limits, or other behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three clear, non-redundant sentences: purpose, filter behavior, output format. Front-loaded with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema and no required parameters, the description covers essential behavior concisely. Minor gap: no mention of pagination or order, but limit parameter addresses row count.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are already documented. The description adds the AND combination behavior, which is valuable but does not elaborate on parameter meaning beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verb 'look up' and resource 'resources', along with 'structured, read-only filters', clearly distinguishing it from sibling tools (check_compliance, get_patch_status) which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states filters are combined with AND, providing usage context, but lacks explicit guidance on when to use versus alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedcheck_compliance
    • First observedget_patch_status
    • First observedquery_resources

TDQS

A4/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a distinct purpose: check_compliance evaluates specific security controls, get_patch_status reports VM patch posture, and query_resources performs generic read-only queries. No overlap in functionality.

Naming Consistency5/5

All tool names use lowercase with underscores and follow a clear verb_noun pattern (check_compliance, get_patch_status, query_resources), providing a predictable and consistent naming scheme.

Tool Count4/5

With only 3 tools, the server is tightly focused on Azure compliance and resource queries. While the count is somewhat minimal, each tool is well-justified and covers the core use cases without being overloaded.

Completeness4/5

The tool set covers compliance evaluation, patch status, and generic resource queries, which together address a broad range of compliance workflows. The generic query tool compensates for potential gaps, though some specialized compliance checks might be missing.

Maintenance

ActivityStale
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Enables natural language exploration of Azure environments by generating and executing KQL queries against Azure Resource Graph. Supports multi-tenant configurations, subscription scoping, and provides direct access to Azure resource information through conversational interactions.
    8
    2
    Apache 2.0
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables read-only access to SentinelOne's platform through MCP, allowing security investigations, threat hunting, and asset inventory queries via natural language.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables natural-language access to Oracle Cloud Infrastructure resources, allowing users to query compute instances, OKE clusters, networking, and Object Storage via MCP tools.
    MIT
  • F
    license
    A
    quality
    D
    maintenance
    Integrates Azure Cost Management APIs via MCP, enabling cost queries, forecasts, budgets, and Advisor recommendations through natural language in compatible clients.
    9
    -