Skip to main content
Glama
shigechika

entraadm-mcp

by shigechika

entraadm-mcp

English | 日本語

MCP server for Microsoft Entra ID sign-in and audit-log triage, plus the two Azure-side questions a morning patrol asks with the same credential: are our Azure VMs backed up, and is Microsoft 365 itself degraded right now. Read-only, except for one opt-in tool that signs a compromised account out everywhere.

Why this instead of the official Microsoft MCP Server for Enterprise

Microsoft ships an official MCP Server for Enterprise for Entra ID data. It is a good fit for an interactive admin at a keyboard, and is not a fit for an unattended triage bot:

  • Delegated auth only. The official server does not support app-only (client credentials) auth, so it cannot run headless behind a service account. entraadm-mcp is built for that case: app-only in production, with a delegated (az login) fallback for local development.

  • A general-purpose Graph query tool, not a fixed tool set. The official server exposes one tool that lets the model construct arbitrary GET/schema-discovery calls against Microsoft Graph. That is flexible for a human, and awkward to put behind an allow-list for an automated triage profile. entraadm-mcp exposes fifteen fixed, read-only tools instead (plus one opt-in write, below).

  • No AADSTS translation. Sign-in failures come back as raw error codes; triage still needs a lookup table. entraadm-mcp annotates every sign-in failure with what the code actually means.

  • No cross-request aggregation. Microsoft Graph itself cannot filter sign-ins on status/errorCode server-side, and has no built-in password-spray view. signin_failure_stats aggregates client-side and flags IPs with failed sign-ins against many distinct users — the pattern Entra's per-account smart lockout does not catch on its own.

Related MCP server: Microsoft Graph MCP Server

Tools

Tool

What it answers

health_check

Is Graph reachable, and can this credential read sign-in logs?

get_user

Is this account enabled, synced from on-prem, and what are its licenses?

signin_logs

Why did this user's sign-in fail (or succeed), with the AADSTS code translated?

signin_failure_stats

Tenant-wide failure aggregation: top error codes, users, apps, source IPs, and password-spray suspects

signin_success_stats

Tenant-wide success aggregation by source IP: IPs shared by several accounts, legacy-auth (SMTP/IMAP) successes — the view that finds a breach

compromise_suspects

Which accounts were probably taken over: an outside IP that sprayed many accounts and got into some (spray_breaches), successes from outside the home countries (foreign_successes, high when over a legacy protocol), and Microsoft's own risky users when the tenant has Entra ID P2

signin_by_ip

Every sign-in from one source IP: who got in from it, who was tried, and when

directory_audits

Who changed what in the directory (block/unblock, attribute edits), and when?

get_user_auth_methods

Is MFA actually registered for this account?

list_subscriptions

Which Azure subscriptions can this credential see? (ARM)

vm_backup_status

Which Azure VMs are backed up by Azure Backup, did the last backup succeed, and is any recovery point stale? (ARM)

vm_backup_policies

What schedule and retention do the Azure VM backup policies have? (ARM)

vm_activity

Did someone restart this Azure VM, or did Azure (host failure, maintenance)? Activity log and Resource Health for a time window (ARM)

m365_service_health

Is a Microsoft 365 service degraded, and which incidents and advisories are open?

daily_brief

One-call summary combining m365_service_health, signin_failure_stats, directory_audits and compromise_suspects (counts and top 5)

Every tool above is read-only. One state-changing tool exists and is off by default:

Tool

What it does

revoke_sign_in_sessions

Signs one account out everywhere (Graph revokeSignInSessions: refresh tokens and session cookies are invalidated). For a confirmed compromise. Registered only when ENTRAADM_ENABLE_REVOKE_SESSIONS=1

It looks the account up by exact userPrincipalName first and refuses a guest. Every result has the same keys, including revoked and not_covered: it does not disable the account, reset the password or remove MFA methods, and access tokens already issued keep working until they expire (typically about an hour). revoked is true when Graph confirmed it, false when nothing was done (a refusal, a deadline before the request was sent, a 4xx from Graph), and null when the outcome is unknown (a response lost to a network error, a 5xx after the retries, a deadline after the request was sent, an unreadable success answer); calling again is then safe, revocation is idempotent. Leave the switch unset on any deployment that is not behind its own approval step (a shared gateway, for instance) -- the tool is then absent from tools/list. Unblocking an account and resetting a password stay out of scope.

The four Azure Resource Manager tools live in an Entra-named server on purpose: they use the same app registration and the same azure-identity credential object (a second token scope, https://management.azure.com/.default), run in the same deployment and are covered by the same health_check. A separate server for these tools would double the operational surface. They are governance reads, not resource management; a subscription the credential has no Reader role on simply does not appear in list_subscriptions.

Auth model

Two auth modes, selected by which environment variables are set:

Mode

When

Env vars

app-only

All three set

ENTRAADM_TENANT_ID, ENTRAADM_CLIENT_ID, ENTRAADM_CLIENT_SECRET

azure-cli

None set

(uses the current az login session)

Setting one or two of the three app-only variables is a configuration error and the server refuses to start, rather than silently falling back to a different auth mode than intended.

Required Graph permissions

Tool(s)

Permission

Notes

get_user (base fields)

User.Read.All

signin_logs, signin_failure_stats, signin_success_stats, compromise_suspects, signin_by_ip, directory_audits, get_user's sign_in_activity field

AuditLog.Read.All (app-only) or the Reports Reader directory role (delegated)

get_user_auth_methods

UserAuthenticationMethod.Read.All

App-only only; not available under delegated (az login) auth in a typical tenant role assignment

m365_service_health, daily_brief's service_health section

ServiceHealth.Read.All

Under delegated auth the signed-in user also needs an Entra role that may read service health (for example Service Support Administrator or Global Reader)

compromise_suspects' microsoft_risky_users, the identity_protection probe (health_check, daily_brief)

IdentityRiskyUser.Read.All

Optional. Identity Protection needs Entra ID P2: without it Graph answers 403 "not licensed" and the probe reports not_licensed; nothing else changes

revoke_sign_in_sessions (opt-in)

User.RevokeSessions.All (app-only) or the User Administrator / Helpdesk Administrator directory role (delegated; Privileged Authentication Administrator for an administrator account)

Only when ENTRAADM_ENABLE_REVOKE_SESSIONS=1. A write permission: grant it only to a deployment that needs the tool. The account lookup before it needs User.Read.All. Without the permission the tool answers missing_permission for the auth mode in use and nothing changes

Required Azure RBAC

Tool(s)

Role

Notes

list_subscriptions

none

Lists whatever the credential can see; an empty list means no role has been granted anywhere yet

vm_backup_status, vm_backup_policies, vm_activity

Reader on each subscription

Reader is */read and covers every endpoint used. Backup Reader alone is not enough (it has no Microsoft.Compute/virtualMachines/read). Granted to the service principal (app-only) or to the signed-in user (azure-cli)

Azure RBAC is optional: health_check reports ARM reachability and the number of visible subscriptions as an informational arm probe that never changes status, so a deployment used only for Entra triage stays healthy.

A missing permission never crashes a tool. It degrades that tool (or that one field) to {"error": "...", "missing_permission": "..."} with a human-readable explanation of what role or permission is needed, so health_check and every other tool stay usable even before full permissions are granted.

Identity Protection (Entra ID P2). compromise_suspects adds Microsoft's own risk verdicts when the tenant has them: a cheap riskyUsers?$top=1 probe classifies each tenant as licensed, not_licensed (403 "not licensed": no Entra ID P2), no_permission (other 403: no IdentityRiskyUser.Read.All) or error; licensed and not_licensed are cached for an hour (a 200 whose rows say riskLevel: hidden also means no P2). The endpoint itself is probed: /subscribedSkus can be refused and a SKU list would still have to be mapped to P2. health_check reports it per tenant without changing status, and daily_brief's summary per tenant, so the day P2 appears is noticed. Licensed: microsoft_risky_users lists open (atRisk / confirmedCompromised) high-risk users and, separately, medium, marking those also in spray_breaches / foreign_successes. counts are then distinct accounts over every finding, before the lists are cut: high is any account high here or in Microsoft's high list, medium any medium here or in Microsoft's medium list that is not high (counts.microsoft_high / microsoft_medium say how many Microsoft lists). Otherwise the section is {available: false, reason} and nothing else changes -- never read it as "no risky users".

Setup

uv tool install entraadm-mcp
# or
pip install entraadm-mcp

Configuration

Set the three app-only variables for production/unattended use:

export ENTRAADM_TENANT_ID=00000000-0000-0000-0000-000000000000
export ENTRAADM_CLIENT_ID=00000000-0000-0000-0000-000000000000
export ENTRAADM_CLIENT_SECRET=your-client-secret

Or leave all three unset and run az login first for local development.

Optional:

# Default page cap for the log-scanning tools (1-50, default 50).
export ENTRAADM_MAX_PAGES_DEFAULT=50
# Wall-clock budget per tool call in seconds (default 45; 0 disables). A hosted MCP
# client cuts a call off at about 60 s, so a scan stops at the budget and returns
# what it has with capped=true.
export ENTRAADM_DEADLINE=45
# Your own address ranges (NAT egress, offices, labs). Spray suspects inside them are
# listed separately as internal_spray_suspects, labelled with the site.
export ENTRAADM_SITES_INI=/path/to/sites.ini
# The countries your users sign in from (assigned ISO 3166-1 alpha-2 codes). compromise_suspects
# reports successes from elsewhere; unset or with an invalid code, that list is skipped.
export ENTRAADM_HOME_COUNTRIES=JP
# Register revoke_sign_in_sessions, the one state-changing tool (exactly "1"; anything
# else leaves it off). Also needs the User.RevokeSessions.All permission.
export ENTRAADM_ENABLE_REVOKE_SESSIONS=1
# Serve several tenants from one process (see "Several tenants" below).
export ENTRAADM_CONFIG=/path/to/tenants.ini

ENTRAADM_SITES_INI uses the same format as keycloak-mcp's KEYCLOAK_SITES_INI, so one file can serve both (see sites.ini.example). Lookups are longest-prefix. A file that is set but cannot be read makes health_check report degraded, and every IP is then treated as external. An invalid CIDR is reported the same way, but the valid ranges are still used.

Several tenants

One process can serve several Entra tenants, the way gwsadm-mcp serves several Workspace domains. Point ENTRAADM_CONFIG at an INI file with one [tenant.<name>] section per tenant:

[tenant.staff]
tenant_id = 00000000-0000-0000-0000-000000000001
domains = example.edu
default = true

[tenant.students]
tenant_id = 00000000-0000-0000-0000-000000000002
domains = students.example.edu
; optional: an app registration of its own instead of the shared one
; client_id = 00000000-0000-0000-0000-000000000003
; client_secret_env = ENTRAADM_STUDENTS_CLIENT_SECRET

Key

Required

Meaning

tenant_id

yes

The tenant's id (GUID) or a domain name of it (with a dot, e.g. contoso.onmicrosoft.com); common, organizations and consumers are refused

domains

yes

Comma-separated UPN domains routed to this tenant (exact match, any case)

default

on one section when there are several

The tenant used when a call names none; a single section is the default anyway

client_id + client_secret_env

no, both or neither

An app of its own; client_secret_env names the env var holding its secret. Never put a secret in the file

A section without client_id reuses the env app (ENTRAADM_CLIENT_ID / ENTRAADM_CLIENT_SECRET), with the token requested from that section's tenant; with no env app it uses az login in that tenant. ENTRAADM_TENANT_ID is not used when ENTRAADM_CONFIG is set. Unset ENTRAADM_CONFIG keeps the single-tenant setup above unchanged.

To share one app across tenants, make the app registration multi-tenant (Supported account types: accounts in any organizational directory), add the same Graph application permissions, then have an administrator of each other tenant grant admin consent:

https://login.microsoftonline.com/<tenant-id>/adminconsent?client_id=<app-id>

The server refuses to start, and --check fails, when the file is unreadable, has a section that is not [tenant.*] or an unknown key, misses tenant_id or domains, has a tenant_id that is neither a GUID nor a dotted domain, gives one domain to two tenants, or has no default (or two) among several tenants. --check probes every tenant.

Routing:

  • A tool that takes a UPN (get_user, get_user_auth_methods, signin_logs, revoke_sign_in_sessions, and directory_audits when user is given) works in the tenant whose domains holds the UPN's domain (exact match). A domain no tenant claims is a refusal, not an exception: tenant_configured: false, found: null (no lookup ran) and the configured_domains. For revoke_sign_in_sessions: revoked: false, found: null, nothing is sent. List <tenant>.onmicrosoft.com in domains if such UPNs are looked up often.

  • get_user, get_user_auth_methods and signin_logs also take tenant, which wins over the UPN's domain: for a *.onmicrosoft.com or #EXT# UPN, or a B2B guest whose sign-ins are logged in the host tenant. revoke_sign_in_sessions has no such override on purpose: a write is only ever aimed at the tenant that owns the UPN's domain.

  • A tenant-wide tool (signin_failure_stats, signin_success_stats, compromise_suspects, signin_by_ip, directory_audits, m365_service_health) takes an optional tenant; default the default tenant. directory_audits with both tenant and a user whose domain another tenant claims is refused (it would search the wrong log and find nothing); a user whose domain no tenant claims (an external or partner admin, a guest) is searched in the given or default tenant, and the result says user_domain_claimed: false.

  • tenant is a section name, one of its domains, or its tenant id. Without a tenant file it may be default or the env tenant id; anything else is refused (tenant_configured: false).

  • health_check and daily_brief cover every tenant. Their top-level keys stay the default tenant's, as before; tenant names it and tenants holds the per-tenant results. The call budget (ENTRAADM_DEADLINE) is split evenly among the tenants (default first, each taking a share of what is left), so a slow tenant cannot stall the others; a tenant that does not answer is an error in its own entry, and makes health_check's status at most degraded. With ENTRAADM_DEADLINE=0 there is no budget: every tenant is scanned in full, one after another, which can outlast a client's ~60 s cutoff.

  • The Azure Resource Manager tools (list_subscriptions, vm_backup_status, vm_backup_policies, vm_activity) serve the default tenant only.

  • With a tenant file, every result names its tenant (the ARM tools: the default tenant; a refusal that picked none: null). Env-only results are unchanged, except that revoke_sign_in_sessions always carries tenant, tenant_configured and configured_domains (its keys never vary).

Usage

Claude Code (plugin)

/plugin marketplace add shigechika/entraadm-mcp
/plugin install entraadm-mcp@entraadm-mcp

Claude Code (manual)

Add to .mcp.json:

{
  "mcpServers": {
    "entraadm-mcp": {
      "type": "stdio",
      "command": "uvx",
      "args": ["entraadm-mcp"],
      "env": {
        "ENTRAADM_TENANT_ID": "${ENTRAADM_TENANT_ID:-}",
        "ENTRAADM_CLIENT_ID": "${ENTRAADM_CLIENT_ID:-}",
        "ENTRAADM_CLIENT_SECRET": "${ENTRAADM_CLIENT_SECRET:-}"
      }
    }
  }
}

Direct execution

entraadm-mcp

CLI options

Option

Effect

--version

Print the version and exit

--check

Resolve auth, probe Graph reachability and sign-in log access, print a report (plus an informational ARM line), exit 0 (or 1 on config error)

Notes

  • Coverage contract. Every result that walks a paged Graph collection carries a capped boolean when its window was not fully scanned — a partial scan is never reported as if it were exhaustive.

  • found: false is not an error. get_user and get_user_auth_methods answer a nonexistent account with {"found": false, ...}, not an error key — a typo'd userPrincipalName should never look like this server being broken.

  • Retention. Entra ID P1 retains sign-in and directory audit logs for 30 days. A window beyond that returns an empty result, not an error.

Development

uv sync --dev
uv run pytest -v
uv run ruff check .
uv run ruff format --check .

Live smoke test

uv run python scripts/smoke_test.py

Read-only (revoke_sign_in_sessions is skipped by name even when enabled), no payloads printed (tool names/statuses/row counts only), and bounded (small explicit windows/page caps) — nothing here writes to the tenant or scans more than a day of logs.

Releasing

This repository uses release-please driven by Conventional Commits. Merge a feat:/fix: PR to main, and release-please opens (or updates) a release PR; merging that PR tags a release and triggers the publish pipeline (PyPI, MCP Registry).

License

MIT

Available Tools

15 tools
compromise_suspectsA

Accounts that were probably taken over, from one walk of the sign-in log.

The success-side counterpart of signin_failure_stats' spray suspects. Two findings, both from EXTERNAL source IPs (outside the address ranges in ENTRAADM_SITES_INI; without that file every IP counts as external):

  • spray_breaches (severity high): an IP with credential failures (wrong password, unknown, locked or disabled account -- not MFA prompts, session interrupts or Conditional Access blocks, which follow a correct password) for min_spray_users or more distinct accounts in the window AND a successful sign-in as at least one account. Without internal ranges (ENTRAADM_SITES_INI unset or unreadable, internal_ranges_known: false) these are medium with a note: the tenant's own NAT egress can look like a spray source. One row per account and IP: failed_distinct_users from that IP, first/last success, client apps, countries, legacy_auth. The strongest signal: a spray that found a working password.

  • foreign_successes: successes from a country outside ENTRAADM_HOME_COUNTRIES (assigned ISO 3166-1 alpha-2 codes, e.g. "JP"; one invalid code disables this list, see home_countries.error), grouped per account with countries, the most used IPs (10, with counts), client apps, first/last, and home_success_in_window (the account also signed in from a home country or an internal IP in the window; null when not seen in a capped scan, since the unread part may hold it). high when a legacy protocol was used (password only, no MFA) or the account is also a high spray_breaches entry; medium otherwise. Unset home countries: home_countries.configured is false and this list stays empty -- nothing is guessed. A success with no country is never called foreign.

  • microsoft_risky_users: Microsoft's own verdicts from Identity Protection, which needs Entra ID P2 and the optional IdentityRiskyUser.Read.All permission. A cheap probe (cached per tenant) decides: without P2 the section is {"available": false, "reason": "not_licensed" | "hidden" | "no_permission" | "error"} and nothing else changes -- never read it as "no risky users". With P2: open (atRisk / confirmedCompromised) high and, separately, medium risky users (50 each, *_capped), each marked when also in spray_breaches / foreign_successes.

counts are distinct accounts over every finding (case-insensitive, counted before the lists are cut): high is any account high in this tool's own lists or in Microsoft's high list; medium is any account medium here or in Microsoft's medium list that is not high. With P2, microsoft_high / microsoft_medium say how many accounts Microsoft lists. Lists are capped at 50 (*_capped). how_to_read says what to act on. Like signin_failure_stats this walks the interactive sign-in log for the window (every legacy-protocol authentication is interactive) under ENTRAADM_DEADLINE; capped=true means the window was not fully read and the findings are a lower bound.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role; IdentityRiskyUser.Read.All, optional, for microsoft_risky_users).

Args: hours: How far back to look, clamped to [1, 720] (30 days). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant). min_spray_users: Distinct failed accounts that make an IP a spray source (default 5, clamped to >= 2). max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT).

NameRequiredDescriptionDefault
hoursNo
tenantNo
max_pagesNo
min_spray_usersNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: it states read-only access, required permissions, optional Identity Protection permission, behavior when P2 is absent, how ENTRAADM_SITES_INI and ENTRAADM_HOME_COUNTRIES affect judgments, capping behavior, lower-bound semantics under deadline, and the crucial warning not to read unavailable Microsoft risk data as 'no risky users'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose and the main findings, and its bullet structure makes a complex tool readable. It is still very long and dense, with some repetition and parenthetical detail, but much of that detail substitutes for a missing output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex read-only tool with no output schema and no annotations, the description is complete: it explains return sections, severity rules, count semantics, caps, environment dependencies, permissions, and the meaning of partial scans. An agent has enough information to call and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must fully annotate parameters. Its Args section defines all four: hours clamped to [1, 720], tenant as a section name or domain, min_spray_users default/clamp, and max_pages defaulting to an environment value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise purpose: finding accounts probably taken over from one walk of the sign-in log. It explicitly positions itself as the success-side counterpart of signin_failure_stats' spray suspects, so an agent can distinguish it from sibling tools without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly names signin_failure_stats as the contrasting failure-side tool and explains what findings this tool returns, but it does not explicitly state when to choose it over signin_success_stats, signin_by_ip, or daily_brief. The usage context is clear but lacks explicit exclusions or decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

daily_briefA

One-call morning-patrol summary: M365 service health, sign-in failures, spray suspects, admin actions.

Combines m365_service_health, signin_failure_stats, directory_audits and compromise_suspects (its counts and the top 5 of each list; summary.compromise_suspects has the high/medium counts) into one result with a compact summary on top. The failure stats and the compromise suspects come from one walk of the sign-in log. The shape matches this fleet's other daily_brief tools. A permission failure in one section degrades only that section's contribution to summary -- the other sections still return in full. Azure VM backup is deliberately not a section here: its ARM fan-out is the one scan that could exhaust the shared budget, so call vm_backup_status separately.

With a tenant file (ENTRAADM_CONFIG) it covers every tenant: the top-level keys stay the DEFAULT tenant's brief, exactly as in a single-tenant deployment, plus tenant (the default's name) and tenants ({name: the same brief} for each other tenant). The call budget (ENTRAADM_DEADLINE) is split: default first, each tenant gets an even share of what is left, so a busy tenant cannot starve the next one (a tenant that finishes early leaves its rest to the others). With ENTRAADM_DEADLINE=0 there is no budget and every tenant is scanned in full, one after another, which can outlast a client's ~60 s cutoff. Env-only results carry neither key.

Runs both sections synchronously in one tool call, unlike the sibling gwsadm-mcp's job+poll daily_brief. If this proves too slow for a tenant's sign-in volume against the client's tool-call timeout, port that job+poll pattern here (tracked in this repo's CLAUDE.md Roadmap).

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget passed to both sections (default: ENTRAADM_MAX_PAGES_DEFAULT). samples: Reserved for a future drill-down sample size; currently unused.

NameRequiredDescriptionDefault
hoursNo
samplesNo
max_pagesNo

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it discloses per-section graceful degradation on permission failure, the shared call-budget splitting across tenants, that results are synchronous (unlike the job+poll sibling) with a timeout risk, and what shape the return payload takes. These are non-obvious operational traits an agent could not infer from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the one-sentence summary, and nearly every paragraph carries operational value. It is somewhat long and includes repo-internal roadmap/CLAUDE.md references that don't help an agent decide or invoke, which keeps it short of a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-param aggregator with no output schema and no annotations, the description is thorough: it describes the return shape (compact summary plus top-5 lists), the multi-tenant top-level/tenant keys, budget allocation, and failure modes. An agent has everything needed to call it correctly and interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does for all three params: hours is clamped to [1,720], max_pages is a page budget passed to both sections with a named default, and samples is flagged as currently unused. The only softness is that max_pages' default is given as an env-var name rather than a resolved value, but the semantics are still clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource framing ('One-call morning-patrol summary') and enumerates exactly which data it aggregates (M365 service health, sign-in failures, spray suspects, admin actions). It explicitly names the four sibling tools it combines, so an agent can distinguish it from calling those tools individually.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly steers away from the obvious alternative: it says Azure VM backup is deliberately excluded and that the agent should call vm_backup_status separately, with a stated reason (ARM fan-out exhausting the shared budget). It also explains the multi-tenant case and the deadline=0 tradeoff, giving clear when-to-use and when-not-to-use context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

directory_auditsA

Who did what to the directory, and when -- the operator-side counterpart to signin_logs.

Every admin action against a user object (block/unblock, password reset, role assignment, attribute edits) appears here, naming the actor (initiated_by) and the affected object(s) (target_resources). This is the record a manual "unblock and reset" intervention -- like the one that closed the 2026-08-21 case this server exists to shorten -- leaves behind; it is how a later triage can tell "already handled by a human" from "still open".

user, when given, matches audits where that account is either the initiator or a target resource. Graph's directoryAudits endpoint only supports server-side $filter on the initiator (initiatedBy/user/userPrincipalName), not on targetResources, so this fetches the full time window and matches both sides client-side -- a window with many unrelated admin actions can need a larger max_pages budget than signin_logs/signin_failure_stats to find one specific user's audits; capped=true warns when that budget ran out before the window was fully scanned.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Entra ID retains directory audit logs for 30 days, same as sign-in logs.

Time-bounded: the scan stops after ENTRAADM_DEADLINE seconds (default 45) and returns what it has with capped=true. Over several days the log is dominated by device-registration noise ("Update device"), so pass category (for example UserManagement, RoleManagement, GroupManagement, ApplicationManagement) to have Graph filter server-side; that keeps a multi-day window inside the budget.

Args: category: Only this Graph audit category (letters only; default: all). user: Restrict to audits naming this userPrincipalName as actor or target (default: all). hours: How far back to look, clamped to [1, 720] (30 days). top: Maximum records to return, clamped to [1, 500]. max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant).

NameRequiredDescriptionDefault
topNo
userNo
hoursNo
tenantNo
categoryNo
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present the description carries the full burden and does so thoroughly: read-only, required permission (AuditLog.Read.All or Reports Reader role), 30-day retention, a hard scan deadline (ENTRAADM_DEADLINE, default 45s) that returns partial results with `capped=true`, and the important caveat that Graph only server-filters on the initiator so `user` matching is client-side and can silently miss records if the page budget runs out. This is exactly the behavioral context an agent needs before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and the key operational caveats are front-loaded, and the Args block is dense and useful. Some narrative framing ('like the one that closed the 2026-08-21 case this server exists to shorten') is atmospheric rather than operational and slightly inflates the length of an otherwise tight definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter, zero-annotation, no-output-schema tool, the description covers everything material: what it returns implicitly (audit records with actor and targets), how to know results are truncated, permission and auth prerequisites, retention limits, and the performance trade-off of client-side user matching. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% — the schema only has titles and defaults — so the description must compensate, and the Args block does: it defines clamp ranges for `hours` [1,720] and `top` [1,500], explains that `user` matches either actor or target side, that `category` is letters-only and server-side, and that `tenant` accepts a config section name or a domain. That is substantive meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource framing ('Who did what to the directory, and when') and immediately positions itself as 'the operator-side counterpart to signin_logs.' It enumerates the exact event types captured (block/unblock, password reset, role assignment, attribute edits) and the key fields (initiated_by, target_resources), so an agent can distinguish it from every sibling without reading the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete selection conditions: pass `category` for multi-day windows to avoid device-registration noise, raise `max_pages` when filtering by `user` because target-side matching is client-side, and use the sibling tools for sign-in rather than admin activity. It never explicitly states a 'do not use this when...' exclusion, which is the only gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userA

One account's identity/lifecycle state -- the first thing to check on any triage report.

account_enabled=false means the account itself is the whole story; stop there. A stale last_password_change_date_time alongside a fresh "wrong password" complaint (AADSTS50126 in signin_logs) is the most common on-the-ground pattern: the password changed or expired somewhere, and a cached credential on one device is now stale. on_premises_sync_enabled=true means this account is synced from an on-premises directory (Entra Connect) -- Entra is a downstream copy of its password via Password Hash Sync, not the source of truth. licenses names are resolved from the tenant's SKU catalog (/subscribedSkus, page budget from ENTRAADM_MAX_PAGES_DEFAULT); licenses_capped: true appears only when that scan was cut short before resolving one of this account's own licenses -- when present, one or more licenses entries is a raw skuId rather than a friendly name.

sign_in_activity needs an additional Graph read (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role) beyond what the rest of this tool needs. If that permission is missing, every other field above still returns and sign_in_activity alone degrades to {"error": ..., "missing_permission": "AuditLog.Read.All"}.

A nonexistent account is a normal answer, not a tool failure: the result is {"found": false, "user_principal_name": upn} rather than an error key, so a typo'd UPN in a triage report cannot be mistaken for this tool being broken.

Read-only (User.Read.All application permission, or an equivalent delegated read). Requires an exact userPrincipalName, not a display name or partial match.

Args: upn: The account's userPrincipalName, e.g. "user@example.edu". tenant: Search this tenant instead of the one claiming the UPN's domain (section name, domain or tenant id): for a .onmicrosoft.com or #EXT# UPN, or a B2B guest's sign-ins in the host tenant.

NameRequiredDescriptionDefault
upnYes
tenantNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it declares read-only access, required permissions, partial degradation of sign_in_activity, license-capping behavior, on-premises sync semantics, and how nonexistent accounts return found=false rather than an error.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well front-loaded and organized around the key triage decision points. Some details, such as the SKU page-budget mechanism, are verbose, though they remain relevant because no output schema exists to explain those return fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations, no output schema, and zero schema description coverage, the description supplies the behavioral, permission, and return-shape context an agent needs. It covers the main failure mode, degraded fields, and the exact-match requirement for the primary identifier.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description fully documents both parameters beyond the schema titles. It explains that upn must be an exact userPrincipalName, and it describes tenant's purpose with concrete cases such as onmicrosoft.com, #EXT#, and B2B guest sign-ins.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and scope: an account's identity/lifecycle state, and positions it as the first check in triage. It distinguishes itself from signin_logs by explicitly referencing that sibling for the AADSTS50126 sign-in pattern.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear primary use case (first thing to check on triage) and a stopping condition when account_enabled=false, plus a common pattern tying it to signin_logs. However, it does not explicitly name when other sibling tools such as get_user_auth_methods would be preferable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_auth_methodsA

Registered authentication methods for one account -- is MFA actually set up?

mfa_registered answers "would this account survive a password-spray hit": True iff at least one non-password method is registered (Authenticator app, phone, FIDO2 security key, Windows Hello, a temporary access pass, software OATH token, or a platform credential/passkey). password itself is excluded from that count -- every account has one, so its presence alone says nothing about MFA coverage.

A nonexistent account is a normal answer, not a tool failure: the result is {"found": false, "user_principal_name": upn} rather than an error key, matching get_user's contract.

Read-only (UserAuthenticationMethod.Read.All application permission). This endpoint is app-only only: it is not exposed to delegated (azure-cli) auth under this tenant's current role assignment, so it degrades to a permission error under azure-cli auth even when other tools work.

Args: upn: The account's userPrincipalName. tenant: Search this tenant instead of the one claiming the UPN's domain (section name, domain or tenant id): for a .onmicrosoft.com or #EXT# UPN, or a B2B guest's sign-ins in the host tenant.

NameRequiredDescriptionDefault
upnYes
tenantNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it declares read-only status, the exact permission (UserAuthenticationMethod.Read.All), and a real operational constraint — app-only auth only, degrading to a permission error under azure-cli. It also pre-empts a common failure mode by stating that a nonexistent account returns {found: false} rather than an error, matching get_user's contract.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key claim (MFA registered or not) is front-loaded in the first two lines, followed by the precise mfa_registered definition, the not-found contract, and the auth caveat. The password-exclusion paragraph is slightly belabored but earns its place by preventing a misread of the count.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations exist, and the description covers the negative case's return shape, the permission model, and the auth-mode limitation. It does not sketch the shape of a successful result (the list of registered methods), which leaves a small gap for a tool whose whole purpose is enumerating them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must document both parameters. The tenant argument gets rich treatment — what it overrides (the domain-claiming tenant), accepted forms (section name, domain, tenant id), and the scenarios that require it. The upn argument is only glossed as 'the account's userPrincipalName', which is adequate but adds little.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Registered authentication methods for one account') and immediately frames the practical question it answers ('is MFA actually set up?'). This is clearly distinguishable from sibling get_user, which returns account attributes rather than auth method registrations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The MFA-coverage framing plus the 'password is excluded from the count' note tells an agent exactly what question this tool answers, and the tenant argument is scoped with concrete cases (onmicrosoft.com, #EXT#, B2B guest sign-ins in host tenant). It stops short of naming an alternative tool or an explicit when-not-to-use condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

health_checkA

Fleet-standard health probe: service/version/status plus two independent Graph probes.

graph confirms Microsoft Graph is reachable at all (GET /users with $top=1 -- needs only User.Read.All, the minimum permission every deployment of this server needs anyway). signin_probe additionally confirms the current credential can read sign-in logs -- the permission every other tool here except get_user depends on. Both probes always run, independently of each other: a tenant that has AuditLog.Read.All but not (yet) the baseline User.Read.All would otherwise have this report "Graph unreachable" -- a fabricated diagnosis, since Graph plainly is reachable if the other probe succeeds. status is derived from the two outcomes: healthy when both succeed, degraded when exactly one does (Graph is reachable but some permission is missing), error only when neither does.

Read-only. Always returns the same keys regardless of outcome (detail is null on success, a translated message on failure), so a caller never has to branch on which keys are present.

sites reports the optional ENTRAADM_SITES_INI table (configured, ranges, error). A configured file that fails to load is an error here and leaves status no better than degraded: without it every internal IP would be reported as an external spray suspect.

With a tenant file (ENTRAADM_CONFIG) every tenant is probed. The top-level keys stay the DEFAULT tenant's, as in a single-tenant deployment, plus tenant (the default's name) and tenants: {name: {default, domains, auth_mode, status, graph, signin_probe}} for every tenant, the default included. Each tenant's probes run under an even share of the call budget (ENTRAADM_DEADLINE), so a tenant that does not answer is an error there instead of stalling the call. Another tenant that is not healthy makes the top-level status at most degraded (the default tenant alone can make it error). arm is the default tenant's only. Env-only results carry neither key.

identity_protection is the Identity Protection licence probe (licensed / not_licensed / no_permission / error, cached per tenant for an hour; see compromise_suspects), per tenant in tenants too. Informational: not_licensed (no Entra ID P2) and the rest never change status.

home_countries reports ENTRAADM_HOME_COUNTRIES (configured, countries, error), which compromise_suspects needs for its foreign-success finding. Informational: unset or invalid never changes status.

arm is a third, informational probe (Azure Resource Manager reachable, and how many subscriptions this credential can see). It never moves status: Azure RBAC is optional for this server, and a deployment used only for Entra triage must not sit at degraded forever because nobody granted it a Reader role. It runs last under its own short budget (_ARM_PROBE_SECONDS) so an unresponsive ARM cannot push this whole call past the client's tool-call timeout. The budget bounds the request, not the first token acquisition (azure-identity is synchronous; in azure-cli mode it shells out to az), which is the same limit every tool here has.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it states 'Read-only', explains how 'status' is derived from the two probe outcomes, documents the per-tenant call-budget/deadline trade-off, the one-hour Identity Protection cache, and that a configured-but-broken sites file caps status at degraded. It even explains why the ARM probe runs last under its own budget (tool-call timeout avoidance) — behavior an agent cannot infer from schema or annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with a clear one-line summary and organized by return key, but at roughly 450 words for a zero-parameter probe it is bloated with design rationale ('a fabricated diagnosis, since Graph plainly *is* reachable', 'azure-identity is synchronous') that does not help an agent decide or call correctly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations exist, so the description must document returns — and it covers status semantics, per-tenant nesting, sites, identity_protection, home_countries and arm in detail. Gaps remain: the contents of the top-level 'service' and 'version' keys, and the shape of 'detail' messages, are never described.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so the baseline of 4 applies; nothing in the schema needs elaboration beyond the environment variables (ENTRAADM_CONFIG, ENTRAADM_SITES_INI, ENTRAADM_DEADLINE, ENTRAADM_HOME_COUNTRIES) the description helpfully names as implicit inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the specific verb and output ('Fleet-standard health probe: service/version/status plus two independent Graph probes'), so an agent knows exactly what it returns. It does not, however, distinguish itself from the sibling 'm365_service_health', which an agent could easily confuse with this server-side probe.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the probe's purpose (confirming Graph reachability, baseline permissions, optional licence/config availability) suggests calling it when diagnosing configuration or permission failures, but there is no explicit 'use this when' statement and no exclusion against m365_service_health or daily_brief.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_subscriptionsA

The Azure subscriptions this credential can see, as ARM lists them.

Azure Resource Manager in the DEFAULT tenant only, also when several tenants are configured (ENTRAADM_CONFIG).

Read-only (GET /subscriptions; needs no RBAC beyond being able to see a subscription at all). The discovery anchor for vm_backup_status and vm_backup_policies: both accept a subscription id or display name from this list. An empty list with auth working means no Reader role has been granted anywhere yet. capped marks a listing cut short by the page budget.

NameRequiredDescriptionDefault
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it declares read-only semantics, the underlying call (GET /subscriptions), the RBAC prerequisite (nothing beyond visibility), the tenant boundary (DEFAULT tenant only even with multiple tenants configured), and the meaning of the `capped` signal when results are truncated. That is unusually rich behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core statement and scope constraint come first, followed by a tight sequence of safety, routing, diagnostics, and the `capped` note — every element earns its place. The parenthetical ``ENTRAADM_CONFIG`` aside adds a bit of noise for an agent that cannot set env vars mid-call.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No annotations and no output schema, so the description must cover behavior and returns; it does cover read-only status, tenant scope, truncation, and the empty-result case. What it omits is the shape of a returned subscription (id vs display name fields), which matters because downstream tools accept either form.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one optional parameter exists and the schema documents none of it (0% coverage), so the description must compensate. It does partially by explaining the "page budget" concept behind `capped`, which an agent can connect to `max_pages`, but it never states what `max_pages` does, its default of null, or what value to pass to avoid truncation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb/resource ("The Azure subscriptions this credential can see, as ARM lists them") and scopes it to the default tenant. No sibling in the list enumerates subscriptions, and the description explicitly names the two tools this result feeds, so an agent can place it precisely.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames the tool as "the discovery anchor for vm_backup_status and vm_backup_policies," telling the agent to call this first when those tools need a subscription id or display name. It also supplies interpretation guidance (an empty list with working auth means no Reader role was granted) and the tenant scope caveat, which is exactly the when/when-not context needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

m365_service_healthA

Microsoft 365 service status per service, and the open incidents and advisories.

Two Graph reads (ServiceHealth.Read.All): healthOverviews (one row per service with its current status) and the unresolved serviceAnnouncement/issues, newest first. A sign-in failure spike reads very differently while Entra or Exchange has an open incident, which is why daily_brief carries this section too. Each section degrades on its own ({"error", "missing_permission"}); summary is computed from whatever came back. capped means either collection was cut short by the page budget, so the counts are lower bounds. The message center (messages) needs a different permission and is not read.

Args: max_pages: Page budget for each collection (default: ENTRAADM_MAX_PAGES_DEFAULT). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant).

NameRequiredDescriptionDefault
tenantNo
max_pagesNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does: two Graph reads, required ServiceHealth.Read.All permission, ordering (newest first), independent per-section degradation with an {"error","missing_permission"} shape, summary computed from partial results, and capped as a lower-bound flag. It even discloses that message center is deliberately not read.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the one-line purpose, then progressively richer detail; every clause (degradation, capped semantics, excluded messages endpoint) adds needed context. Backtick-heavy prose makes it slightly denser than necessary but nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations, yet the description covers return structure (per-service rows, issues collection, summary, error sections), partial-failure semantics, and count caveats. An agent has everything it needs to call this and interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does for both parameters: max_pages as a per-collection page budget with its default constant, and tenant as a config section name or one of its domains. The default value referencing an opaque ENTRAADM_MAX_PAGES_DEFAULT symbol is the only minor gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific resource (Microsoft 365 service status / open incidents and advisories) and a clear scope. It is unmistakable next to read-only log and audit siblings such as signin_logs or directory_audits, and it explicitly ties its role to daily_brief.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a concrete decision context — a sign-in failure spike reads differently when Entra or Exchange has an open incident — which tells the agent when this matters. It does not state exclusions or an explicit 'use X instead of Y' routing, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_by_ipA

Every sign-in from one source IP: who got in from it, who was tried, and when.

The follow-up to a spray_suspects or shared_ips hit: Graph can filter sign-ins on ipAddress server-side, so this is one cheap query, not a log walk. users summarises the IP per account (successes, failures, first/last seen, up to 50 accounts) over every row fetched; events lists the newest top entries that match result ("all" by default, or "success" / "failure"), each carrying the account name and the same AADSTS annotation as signin_logs. capped=true means the page budget or the deadline ran out before the window was fully read; events_truncated=true means more matching rows were read than top returns (the users summary still counts them).

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role).

Args: ip: The source IPv4 or IPv6 address, exactly as the sign-in log shows it. hours: How far back to look, clamped to [1, 720] (30 days). result: "all" (default), "success", or "failure" -- which events to list. top: Maximum events to return, clamped to [1, 500]. max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant).

NameRequiredDescriptionDefault
ipYes
topNo
hoursNo
resultNoall
tenantNo
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden and does: it declares read-only behavior, names the required permission (AuditLog.Read.All) and the azure-cli alternative (Reports Reader role), and explains capped/events_truncated semantics including that the users summary still counts truncated rows.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded and organized into prose plus an Args block, with no filler sentences. The return-shape paragraph is dense and slightly long, but each clause (users, events, capped, events_truncated) conveys distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must explain returns, and it does in detail: per-account summary with success/failure/first/last seen and the 50-account cap, newest top events with AADSTS annotation, plus the two truncation flags. Nothing an agent needs to call or interpret this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does — every parameter is documented with meaning, defaults, and clamping ranges (hours in [1,720], top in [1,500], result enum all/success/failure, tenant as config section or domain, max_pages budget).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+scope: every sign-in from one source IP, with who succeeded, who was tried, and when. It also distinguishes itself from adjacent tools (spray_suspects, shared_ips, signin_logs) that an agent might otherwise pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames itself as the follow-up to a spray_suspects or shared_ips hit and notes it is a single server-side filtered query, not a log walk — strong 'when to use' guidance. It does not explicitly say when to prefer the sibling signin_logs instead, leaving one routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_failure_statsA

Tenant-wide sign-in failure aggregation -- the Entra ID counterpart to the RADIUS failure patrol.

Time-bounded (ENTRAADM_DEADLINE, default 45 s): on a wide window or a busy day the scan stops early and capped=true marks the counts as a lower bound; narrow hours for a full count.

Aggregates failed sign-ins across the whole tenant into four views: top AADSTS error codes (with the same meaning annotations as signin_logs), top failing users, top applications, and top source IPs. spray_suspects flags any IP with failed sign-ins against 5 or more distinct users -- Entra's smart lockout is per-account, so a low-and-slow password spray from one IP across many accounts does not trip it the way a brute force against one account does; this is the observation a per-account view cannot make on its own. This mirrors the KeyCloak-side spray detection this fleet already relies on; neither the official Microsoft MCP Server for Enterprise nor Graph itself offers this aggregation.

When ENTRAADM_SITES_INI names the tenant's own address ranges, IPs inside them get site (and site_name) in top_ips, and suspects inside them move from spray_suspects to internal_spray_suspects: a NAT egress or a room of shared workstations fails for many users every day, and would otherwise bury an outside source. Internal suspects are kept, not dropped, because an internal host can be compromised too. sites reports whether the file loaded; an invalid CIDR is reported in sites.error while the valid ranges are still used, and a file that cannot be read leaves every IP in spray_suspects.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Graph cannot filter sign-ins on status/errorCode server-side, so this walks up to max_pages of the full sign-in log for the window and aggregates client-side -- capped=true means the page budget ran out before the window was fully scanned, so the counts below are a sample of the window, not a census of it.

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant).

NameRequiredDescriptionDefault
hoursNo
tenantNo
max_pagesNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: it discloses the read-only permission model (AuditLog.Read.All or Reports Reader), the ENTRAADM_DEADLINE cap and `capped=true` lower-bound semantics, the page-budget sampling behavior, and the fact that Graph cannot filter on status/errorCode server-side. Failure modes for the sites file (invalid CIDR still uses valid ranges, unreadable file demotes all IPs to `spray_suspects`) are also spelled out.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the time-bound caveat are front-loaded, but the text is roughly 400 words for a three-parameter tool. The `capped=true` lower-bound/sample explanation appears twice, and the paragraph about RADIUS/KeyCloak parity and the official Microsoft MCP Server is vendor positioning rather than agent-relevant instruction.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, yet the description fully describes the returned views (`top_ips`, `spray_suspects`, `internal_spray_suspects`, `sites`/`sites.error`) and what the spray flag means. Combined with permissions, deadline behavior and degradation paths, an agent has everything needed to call and interpret it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and does: `hours` is documented with its [1, 720] clamp, `max_pages` with its environment-variable default, and `tenant` with the accepted forms ([tenant.<name>] section or domain) and the default tenant fallback. Every parameter gains meaning not present in the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource — tenant-wide aggregation of failed sign-ins — and enumerates the four output views (AADSTS codes, failing users, apps, source IPs). Explicitly distinguishes itself from sibling `signin_logs` (which it references for error-code meanings) and from per-account/per-IP views, so an agent can route correctly without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete operational guidance — narrow `hours` when the scan caps and you need a census rather than a sample, and the tool is positioned as the failure/spray counterpart to the success statistics and per-IP siblings. It lacks an explicit 'do not use this when…' exclusion (e.g. use `signin_logs` for individual event detail), which keeps it from a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_logsA

One user's recent sign-in events, AADSTS-annotated.

The most direct answer to "why can't this person log in": each entry's error_code_meaning translates the raw AADSTS code (e.g. 50126 -> "invalid credentials (wrong password)") so triage rarely needs a second lookup. result filters client-side after the Graph fetch (Graph cannot filter sign-ins on status/errorCode server-side): "failure" (the default) keeps only failed attempts, "success" keeps only clean ones, "all" keeps everything.

Because the filter is client-side, this walks pages until it has collected top matching entries or exhausts max_pages -- a mostly- successful user can otherwise mean paging through hundreds of rows to find a handful of failures. capped=true means the page budget ran out (or top was reached) before the whole window was scanned; a low match count alongside capped=true is evidence of "no more found within the budget", not "no more exist".

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Entra ID P1 retains sign-in logs for 30 days; hours beyond that returns an empty result, not an error.

Args: user: The account's userPrincipalName. hours: How far back to look, clamped to [1, 720] (30 days). result: "failure" (default), "success", or "all". top: Maximum matching entries to return, clamped to [1, 500]. max_pages: Page budget for the client-side filter walk (default: ENTRAADM_MAX_PAGES_DEFAULT). tenant: Search this tenant instead of the one claiming the UPN's domain (section name, domain or tenant id): for a .onmicrosoft.com or #EXT# UPN, or a B2B guest's sign-ins in the host tenant.

NameRequiredDescriptionDefault
topNo
userYes
hoursNo
resultNofailure
tenantNo
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: read-only, required AuditLog.Read.All permission (or Reports Reader role for azure-cli), the 30-day P1 retention limit and the fact that out-of-range `hours` returns empty rather than erroring. It also discloses the non-obvious client-side filtering/paging behavior and the precise meaning of `capped=true` so a low match count isn't misread as 'no failures exist'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the diagnostic payoff before any mechanics, and every paragraph adds information an agent needs. Minor redundancy: the client-side filtering rationale is restated twice ('filters client-side after the Graph fetch' then 'Because the filter is client-side'), costing a sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description names the returned signals (`error_code_meaning`, `result`, `capped`) and explains each. Combined with permission requirements, retention limits and paging semantics, an agent has everything needed to call this correctly and interpret the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the Args section is the only documentation and it covers all six parameters with real meaning: `user` is a UPN, `hours` clamps to [1,720], `result` has three values with client-side filter semantics, `top` clamps to [1,500], `max_pages` defaults to an env var, and `tenant` explains the onmicrosoft.com/#EXT#/B2B-guest cases. This substantially exceeds what the bare schema conveys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource+scope: 'One user's recent sign-in events, AADSTS-annotated', and frames the diagnostic question it answers ('why can't this person log in'). The explicit 'One user's' scoping separates it from the aggregate siblings signin_failure_stats / signin_success_stats / signin_by_ip without needing to name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'The most direct answer to "why can't this person log in"' gives clear situational context for when to reach for this tool, and the result/top semantics imply the exploratory-vs-targeted use cases. It stops short of naming when NOT to use it (e.g. fleet-wide failure counts belong in signin_failure_stats), so no explicit exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_success_statsA

Tenant-wide successful sign-in aggregation by source IP -- the view that finds a breach.

signin_failure_stats shows who is being attacked; this shows whether anyone got in. The breach signature is one source IP signing in successfully as several different accounts, most often over a legacy protocol (clientAppUsed such as "Authenticated SMTP" or "IMAP4", which carry no MFA). shared_ips lists the IPs with successes for min_distinct_users or more distinct accounts, most-shared first (up to 50 IPs, shared_ips_capped when more qualified; account names up to 25 per IP), with the client apps and the countries seen. legacy_auth_users lists the accounts that succeeded over a legacy protocol at all, with how many IPs and countries they came from.

A campus NAT, a VDI farm or a shared proxy also puts many accounts behind one IP, so a shared IP is a lead, not a verdict: the caller excludes its own egress ranges and reads the client apps and countries before calling anything a breach. Graph cannot filter sign-ins on status/errorCode server-side, so like signin_failure_stats this walks the sign-in log for the window and aggregates client-side. The walk covers interactive sign-ins only (Graph's default listing): every legacy-protocol authentication is logged as interactive, so none is missed, but non-interactive token refreshes are not counted; capped=true means the page budget or the deadline (ENTRAADM_DEADLINE, default 45 s) ran out first and the counts are a lower bound -- narrow hours for a full count.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role).

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT). min_distinct_users: Distinct accounts an IP needs to appear in shared_ips (default 2, clamped to >= 2). tenant: Tenant to read: a [tenant.] section name or one of its domains (default: the default tenant).

NameRequiredDescriptionDefault
hoursNo
tenantNo
max_pagesNo
min_distinct_usersNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does so thoroughly: read-only access, required permissions, client-side aggregation, interactive-only sign-ins, capped results as lower bounds, and deadline/page-budget behavior are all disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but front-loaded and appropriately sized for a complex aggregation tool. Each paragraph adds operational meaning, and the Args section compensates for the schema's lack of descriptions rather than repeating structured data.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, but the description explains the returned views, caps, account limits, and caveats thoroughly enough for an agent to interpret results. It also covers permissions and aggregation limitations that are critical for correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema description coverage is 0%, the Args section fully explains each parameter: hours is clamped to [1, 720], max_pages uses an environment default, min_distinct_users defaults to 2 and is clamped to >= 2, and tenant accepts a tenant section name or domain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: tenant-wide successful sign-in aggregation by source IP. It explicitly distinguishes this tool from its sibling signin_failure_stats, making the intended view clear without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names the alternative and the condition that selects it: signin_failure_stats shows who is attacked, while this tool shows whether anyone got in. It also warns that shared IPs from NAT, VDI, or proxies are leads rather than verdicts, which is explicit usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vm_activityA

What happened to an Azure VM in a time window: operations people ran, and events Azure raised.

Azure Resource Manager in the DEFAULT tenant only, also when several tenants are configured (ENTRAADM_CONFIG).

Answers "did someone restart this VM, or did Azure?" from the Azure activity log of the VM's subscription. Read-only; needs the Reader role on the subscription. Two reads: the VM's resource group (server-side filter, so a busy subscription cannot crowd the VM's own events out of the page budget) feeds health and vm_operations; the whole subscription feeds the rest.

  • health: Resource Health events of the VM (cause PlatformInitiated = Azure, e.g. Redeploying due to host failure; UserInitiated = an operation on the VM, or Reboot initiated from inside the machine for a restart from the guest OS).

  • vm_operations: Administrative events on the VM and its sub-resources (restart, deallocate, redeploy, extension changes), with caller (a user principal name or an application id). One operation usually logs several events (Started, Accepted, Succeeded) sharing a correlation_id.

  • other_operations: Administrative events on any other resource of the subscription (count of events, operations = distinct correlation ids, items = the newest 50). Zero here and in vm_operations means nobody operated the subscription through Azure Resource Manager in the window; it says nothing about actions inside the guest OS.

  • service_health: Azure incidents and planned maintenance announced to the subscription.

  • other_categories: counts of the remaining categories (Alert, Policy, other resources' health events, ...).

vm is the VM name, matched case-insensitively across the subscriptions this credential can see; narrow it with subscription (id or display name from list_subscriptions) or resource_group when the name is not unique (ambiguous lists the candidates). A single match found while some subscription could not be listed completely is not taken as unique: it comes back as candidates with search_complete: false, to be confirmed by naming its subscription. Naming the subscription also keeps the search short: without it every visible subscription is listed, and the search may use at most half of the call's time budget. The window is start..end (ISO timestamps with an offset, e.g. 2026-10-21T21:00:00+09:00; seconds may be omitted), or the hours before end (default 24; end defaults to now); at most 168 hours, within the 90 days the activity log keeps. Times in the result are UTC as Azure returns them. capped marks a read cut short by the page or time budget (capped_reads says which): then an empty section is not proof that nothing happened.

NameRequiredDescriptionDefault
vmYes
endNo
hoursNo
startNo
max_pagesNo
subscriptionNo
resource_groupNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: states it is read-only, requires the Reader role, operates only in the DEFAULT tenant even when several are configured, and explains the dual-read strategy, page/time budgets, the 'capped'/'capped_reads' markers, and the critical caveat that an empty section is not proof nothing happened.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The headline answers are front-loaded and the bulleted breakdown of return sections is easy to scan. It is long and the tenant/search-completeness discussion is dense, but nearly every sentence adds operational value rather than repeating the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description documents each returned section (health causes, vm_operations with caller and correlation_id, other_operations counts and truncation to 50, service_health, other_categories) plus the truncation flags. An agent has enough to call the tool and interpret results correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

At 0% schema coverage this description must compensate, and it largely does: explains vm name matching (case-insensitive, ambiguity via 'ambiguous'/'candidates'/'search_complete'), subscription and resource_group narrowing, the start/end/hours relationship with defaults, the 168-hour/90-day limits, and ISO offset format. The one gap is max_pages, which is never named or explained despite the repeated 'page budget' references.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+scope: 'What happened to an Azure VM in a time window: operations people ran, and events Azure raised.' It is clearly distinguishable from siblings like vm_backup_status or health_check, and the 'did someone restart this VM, or did Azure?' line pins the intent down further.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete scenario ('did someone restart this VM, or did Azure?') and tells the agent to use list_subscriptions to source the subscription identifier, plus when to add subscription/resource_group. It does not explicitly say when to prefer a sibling tool instead, which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vm_backup_policiesA

Schedule and retention of the Azure VM backup policies in each Recovery Services vault.

Azure Resource Manager in the DEFAULT tenant only, also when several tenants are configured (ENTRAADM_CONFIG).

Read-only; needs the Reader role on the subscription. Per policy: policy_type (V1 = SimpleSchedulePolicy, V2 = SimpleSchedulePolicyV2 with an hourly / daily / weekly schedule), schedule (frequency, run times in UTC, weekdays, or the hourly window), retention (LongTermRetentionPolicy: daily / weekly / monthly / yearly blocks with their day selection and duration), instant_rp_days, timezone and protected_item_count (ARM's own count). A schedule or retention type this server does not know is returned as {"type": <name>, "unparsed": true}, never dropped.

subscription is an id or display name from list_subscriptions (None = every visible subscription); an unknown or ambiguous name returns found: false. Coverage flags as in vm_backup_status: skipped_vaults per subscription, skipped subscriptions when the budget ran out, capped overall.

NameRequiredDescriptionDefault
max_pagesNo
subscriptionNo

TDQS

A4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does: it declares read-only, names the required Reader role, pins tenant scope, defines failure modes (unknown/ambiguous subscription → `found: false`), and guarantees no policy type is silently dropped (`{"type": <name>, "unparsed": true}`). It also explains the truncation model (skipped_vaults per subscription, skipped subscriptions on budget exhaustion, capped overall), which is exactly the kind of behavior an agent cannot infer elsewhere.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads purpose, then scope/permissions, then the per-field contract, then parameter and truncation semantics — a sensible reading order. It is dense and somewhat long, and the enumerated field glosses (V1/V2, retention blocks) are heavy, but each earns its place as the only documentation of the return shape given there is no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description must define the return payload, and it does at the field level including the unparsed-type fallback and ARM's own protected_item_count. The only real gap is `max_pages`, whose paging behavior is gestured at ('budget', 'capped') rather than stated, which matters for an agent deciding how many calls to make.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does for `subscription`: accepted forms, the None default's meaning, and the unknown/ambiguous resolution behavior. `max_pages` is never named directly — its effect is only inferable from the 'budget ran out' / `capped` language, leaving one of two parameters under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the resource precisely (Azure VM backup policies per Recovery Services vault) and what it exposes (schedule and retention), and it implicitly distinguishes itself from the sibling vm_backup_status by deferring to it for coverage-flag semantics. It never uses an explicit retrieval verb, but 'Read-only' plus the per-policy field list makes the operation unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives solid operating context: DEFAULT tenant only even with multiple tenants in ENTRAADM_CONFIG, read-only, Reader role on the subscription, and how to pass `subscription` (id or display name from list_subscriptions, None = all). What's missing is an explicit statement of when to reach for this tool versus vm_backup_status or vm_activity — the routing is only implied by the shared coverage-flag vocabulary.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vm_backup_statusA

Which Azure VMs are backed up by Azure Backup, and whether the last backup succeeded.

Azure Resource Manager in the DEFAULT tenant only, also when several tenants are configured (ENTRAADM_CONFIG).

Joins every VM of a subscription (Microsoft.Compute/virtualMachines) with the Azure-VM protected items of its Recovery Services vaults, on the VM's ARM id (case-insensitively; ARM casing differs between the two services). Read-only; needs the Reader role on the subscription.

Per VM:

  • protected: true when a protected item exists with protectionState Protected or IRPending (first backup not done yet, note says so); false when there is none, or it is ProtectionStopped / ProtectionPaused / ProtectionError (data may be retained, backups are not running -- the state is shown); null (unknown) when no item was found but the vault scan was incomplete (capped, a skipped vault, an unreadable or unreached subscription -- in the global join, any subscription's) -- a partial scan never produces a false "unprotected".

  • backup_ok: true for lastBackupStatus Completed / CompletedWithWarnings, false for Failed, null before any backup ran or while one is running (InProgress, Warning, an unknown value). An item whose protectionState is missing is protected: null with a note. When two vaults hold an item for one VM, the one still backing up wins (duplicate_items counts them).

  • stale: no recovery point (lastRecoveryPoint, the proof of a successful backup) within stale_hours. The default 36 h suits daily policies; raise it for weekly ones (policy_name is in the row).

subscription is an id or display name from list_subscriptions; None scans every visible subscription and joins globally, so a VM backed up to a vault in another subscription is matched. With one subscription named, only its vaults are scanned (coverage says so). An unknown or ambiguous name returns found: false.

orphans are protected items whose VM no longer exists (backup data retained for a deleted VM), reported only when that VM's subscription was listed completely. One subscription's permission error is reported on its own entry; the others still return. When the wall-clock budget runs out, the remaining subscriptions are listed under skipped and capped is set.

NameRequiredDescriptionDefault
max_pagesNo
stale_hoursNo
subscriptionNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden and does so: read-only, requires the Reader role on the subscription, DEFAULT tenant even when multiple are configured, and a wall-clock budget that can leave subscriptions in 'skipped' with 'capped' set. It also documents failure semantics that would otherwise cause silent misinterpretation — a partial scan never yields a false 'unprotected', and a permission error on one subscription is reported per-entry while others still return.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence and the remaining text is dense rather than padded — most of it is per-field return semantics that are necessary because no output schema exists. It is nonetheless long for a 3-parameter tool, with deeply nested parentheticals that slow scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description needs to explain both behavior and return shape, and it does: protected/backup_ok/stale semantics, duplicate_items and orphan handling, and partial-coverage signaling. The only meaningful omission is what max_pages controls and how it interacts with the 'capped' outcome.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It fully specifies 'subscription' (id or display name from list_subscriptions; None scans all and joins globally) and 'stale_hours' (default 36h suits daily policies, raise for weekly ones, policy_name is in the row). 'max_pages' is left completely undefined, which is a real gap even though the wall-clock/capped behavior is described.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource statement ('Which Azure VMs are backed up by Azure Backup, and whether the last backup succeeded') and immediately scopes it to the default tenant. This is clearly distinguishable from the sibling vm_backup_policies (which would concern policy definitions) without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating conditions: already-backed-up-by-Azure-Backup, DEFAULT tenant only, and points to list_subscriptions as the source of valid subscription values. It explains the None-vs-named-subscription semantics and the unknown/ambiguous-name behavior, but never states when an agent should prefer vm_backup_policies or vm_activity instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 9 tool updatesv1.8.0
    • Addedcompromise_suspects
    • Changeddirectory_audits1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedget_user1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedget_user_auth_methods1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedm365_service_health1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedsignin_by_ip1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedsignin_failure_stats1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedsignin_logs1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
    • Changedsignin_success_stats1 field changed
      • addedInput schema / properties / tenant
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Tenant"
        +}
  2. 1 tool updatev1.4.0
    • Addedvm_activity
  3. 4 tool updatesv1.3.0
    • Addedlist_subscriptions
    • Addedm365_service_health
    • Addedvm_backup_policies
    • Addedvm_backup_status
  4. 3 tool updatesv1.1.0
    • Changeddirectory_audits1 field changed
      • addedInput schema / properties / category
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Category"
        +}
    • Addedsignin_by_ip
    • Addedsignin_success_stats
  5. 7 tool updatesv0.1.0
    • First observeddaily_brief
    • First observeddirectory_audits
    • First observedget_user
    • First observedget_user_auth_methods
    • First observedhealth_check
    • First observedsignin_failure_stats
    • First observedsignin_logs

TDQS

A4.3/5.0

Scored across 15 tools

Disambiguation4/5

The tools mostly target clearly separate resources or analyses: one user, one IP, tenant-wide failure stats, tenant-wide success stats, compromise findings, directory audits, Azure VM backup, and M365 health. The main overlap is that daily_brief aggregates several other tools, and signin_failure_stats vs compromise_suspects both involve spray analysis, but their descriptions clearly distinguish attack-side versus breach-side views.

Naming Consistency4/5

All names use snake_case consistently and most are readable domain/resource phrases. There is a minor convention split between verb-first retrieval tools like get_user and list_subscriptions and noun-phrase report tools like signin_logs and vm_backup_status, but the pattern remains predictable.

Tool Count5/5

Fifteen tools is within the well-scoped range and each tool covers a distinct part of the Entra triage, M365 health, or Azure VM monitoring surface. The supporting discovery tool and daily summary tool both have clear reasons to exist.

Completeness4/5

For a read-only triage server, coverage is strong: user identity, auth methods, sign-in and audit logs, failure/success/compromise analytics, M365 health, and Azure VM backup/activity are all present. Minor gaps remain around broad user search or listing and remediation actions, but these are consistent with the stated read-only scope.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • -
    license
    Not graded
    quality
    Not graded
    maintenance
    Provides secure access to Microsoft Entra ID (Azure AD) resources including users, devices, and applications through Microsoft Graph API. Enables querying organizational data with comprehensive audit logging to Azure Blob Storage.
    -