Skip to main content
Glama
shigechika

entraadm-mcp

by shigechika

entraadm-mcp

Английский | 日本語

MCP-сервер для разбора журналов входа и аудита Microsoft Entra ID. Только чтение.

Почему это вместо официального Microsoft MCP Server for Enterprise

Microsoft поставляет официальный MCP Server for Enterprise для данных Entra ID. Он хорошо подходит для интерактивного администратора за клавиатурой и не подходит для автоматического бота разбора журналов:

  • Только делегированная аутентификация. Официальный сервер не поддерживает аутентификацию только для приложения (client credentials), поэтому он не может работать без головы за служебной учётной записью. entraadm-mcp создан для этого случая: только приложение в продакшене, с делегированным (az login) запасным вариантом для локальной разработки.

  • Универсальный инструмент запросов к Graph, а не фиксированный набор инструментов. Официальный сервер предоставляет один инструмент, который позволяет модели выполнять произвольные вызовы GET/обнаружения схемы к Microsoft Graph. Это гибко для человека, но неудобно для размещения за списком разрешений для автоматического профиля разбора. entraadm-mcp вместо этого предоставляет семь фиксированных инструментов только для чтения.

  • Нет перевода AADSTS. Сбои входа возвращаются в виде необработанных кодов ошибок; для разбора всё ещё нужна справочная таблица. entraadm-mcp аннотирует каждый сбой входа с указанием фактического значения кода.

  • Нет агрегации между запросами. Сам Microsoft Graph не может фильтровать входы по status/errorCode на стороне сервера и не имеет встроенного представления для перебора паролей. signin_failure_stats агрегирует на стороне клиента и помечает IP-адреса с неудачными входами против многих разных пользователей — это тот паттерн, который умная блокировка Entra для каждой учётной записи сама по себе не ловит.

Related MCP server: Microsoft Graph MCP Server

Инструменты

Инструмент

Что он отвечает

health_check

Доступен ли Graph, и может ли эта учётная запись читать журналы входа?

get_user

Включена ли эта учётная запись, синхронизирована ли она из локальной среды, и какие у неё лицензии?

signin_logs

Почему вход этого пользователя не удался (или удался), с переводом кода AADSTS?

signin_failure_stats

Агрегация сбоев по всему тенанту: основные коды ошибок, пользователи, приложения, исходные IP-адреса и подозреваемые в переборе паролей

directory_audits

Кто что изменил в каталоге (блокировка/разблокировка, изменения атрибутов) и когда?

get_user_auth_methods

Действительно ли MFA зарегистрирован для этой учётной записи?

daily_brief

Сводка за один вызов, объединяющая signin_failure_stats и directory_audits

Каждый инструмент доступен только для чтения. Операции записи (разблокировка учётной записи, сброс пароля, отзыв сеанса) выходят за рамки этого сервера.

Модель аутентификации

Два режима аутентификации, выбираемые в зависимости от того, какие переменные окружения установлены:

Режим

Когда

Переменные окружения

app-only

Все три установлены

ENTRAADM_TENANT_ID, ENTRAADM_CLIENT_ID, ENTRAADM_CLIENT_SECRET

azure-cli

Ни одна не установлена

(использует текущий сеанс az login)

Установка одной или двух из трёх переменных app-only является ошибкой конфигурации, и сервер отказывается запускаться, а не молча переключается на другой режим аутентификации, чем предполагалось.

Требуемые разрешения Graph

Инструмент(ы)

Разрешение

Примечания

get_user (базовые поля)

User.Read.All

signin_logs, signin_failure_stats, directory_audits, поле sign_in_activity из get_user

AuditLog.Read.All (только для приложения) или роль каталога Reports Reader (делегированная)

get_user_auth_methods

UserAuthenticationMethod.Read.All

Только для приложения; недоступно при делегированной (az login) аутентификации в типичном назначении ролей тенанта

Отсутствующее разрешение никогда не приводит к сбою инструмента. Оно понижает этот инструмент (или это одно поле) до {"error": "...", "missing_permission": "..."} с понятным объяснением того, какая роль или разрешение необходимы, поэтому health_check и все остальные инструменты остаются работоспособными даже до предоставления полных разрешений.

Настройка

uv tool install entraadm-mcp
# or
pip install entraadm-mcp

Конфигурация

Установите три переменные app-only для продакшена/неавтоматического использования:

export ENTRAADM_TENANT_ID=00000000-0000-0000-0000-000000000000
export ENTRAADM_CLIENT_ID=00000000-0000-0000-0000-000000000000
export ENTRAADM_CLIENT_SECRET=your-client-secret

Или оставьте все три неустановленными и сначала выполните az login для локальной разработки.

Необязательно:

# Default page cap for the log-scanning tools (1-50, default 5).
export ENTRAADM_MAX_PAGES_DEFAULT=5

Использование

Claude Code (плагин)

/plugin marketplace add shigechika/entraadm-mcp
/plugin install entraadm-mcp@entraadm-mcp

Claude Code (вручную)

Добавьте в .mcp.json:

{
  "mcpServers": {
    "entraadm-mcp": {
      "type": "stdio",
      "command": "uvx",
      "args": ["entraadm-mcp"],
      "env": {
        "ENTRAADM_TENANT_ID": "${ENTRAADM_TENANT_ID:-}",
        "ENTRAADM_CLIENT_ID": "${ENTRAADM_CLIENT_ID:-}",
        "ENTRAADM_CLIENT_SECRET": "${ENTRAADM_CLIENT_SECRET:-}"
      }
    }
  }
}

Прямое выполнение

entraadm-mcp

Параметры CLI

Параметр

Эффект

--version

Вывести версию и выйти

--check

Разрешить аутентификацию, проверить доступность Graph и доступ к журналам входа, вывести отчёт, выйти с кодом 0 (или 1 при ошибке конфигурации)

Примечания

  • Контракт покрытия. Каждый результат, который проходит по разбитой на страницы коллекции Graph, содержит логическое значение capped, когда его окно не было полностью просканировано — частичное сканирование никогда не выдаётся за исчерпывающее.

  • found: false — это не ошибка. get_user и get_user_auth_methods отвечают на несуществующую учётную запись с {"found": false, ...}, а не с ключом error — опечатка в userPrincipalName никогда не должна выглядеть как сбой этого сервера.

  • Хранение. Entra ID P1 хранит журналы входа и аудита каталога в течение 30 дней. Окно за пределами этого срока возвращает пустой результат, а не ошибку.

Разработка

uv sync --dev
uv run pytest -v
uv run ruff check .
uv run ruff format --check .

Живой смоук-тест

uv run python scripts/smoke_test.py

Только чтение, без вывода полезных данных (только имена инструментов/статусы/количество строк) и ограничено (небольшие явные окна/лимиты страниц) — здесь ничего не записывается в тенант и не сканируется более чем за день журналов.

Релиз

Этот репозиторий использует release-please на основе Conventional Commits. Объедините PR с feat:/fix: в main, и release-please откроет (или обновит) PR релиза; объединение этого PR создаёт тег релиза и запускает конвейер публикации (PyPI, MCP Registry).

Лицензия

MIT

Available Tools

13 tools
daily_briefA

One-call morning-patrol summary: M365 service health, sign-in failures, spray suspects, admin actions.

Combines m365_service_health, signin_failure_stats and directory_audits into one result with a compact summary on top, matching the shape of this fleet's other daily_brief tools. A permission failure in one section degrades only that section's contribution to summary -- the other sections still return in full. Azure VM backup is deliberately not a section here: its ARM fan-out is the one scan that could exhaust the shared budget, so call vm_backup_status separately.

Runs both sections synchronously in one tool call, unlike the sibling gwsadm-mcp's job+poll daily_brief. If this proves too slow for a tenant's sign-in volume against the client's tool-call timeout, port that job+poll pattern here (tracked in this repo's CLAUDE.md Roadmap).

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget passed to both sections (default: ENTRAADM_MAX_PAGES_DEFAULT). samples: Reserved for a future drill-down sample size; currently unused.

ParametersJSON Schema
NameRequiredDescriptionDefault
hoursNo
samplesNo
max_pagesNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and largely meets it: it discloses that sections run synchronously in one call, that a permission failure degrades only that section's summary contribution while others still return in full, and that the page budget is shared across sections. It does not state permission scopes or explicitly declare the operation read-only, leaving a small gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose before the correct usage routing and the arg list, and every paragraph addresses a decision the caller makes. It carries some low-value internal detail ('tracked in this repo's CLAUDE.md Roadmap') that an agent cannot act on, but the surrounding sentences about slowness and timeouts are genuinely actionable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-param, annotation-free read tool with no output schema, the description covers invocation, cost, failure modes, and exclusions thoroughly. The one residual gap is that the return shape is only characterized as a 'compact summary' matching other fleet daily_brief tools, so result structure must be discovered empirically.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does: hours is documented with its clamp range [1, 720] and meaning; max_pages is described as a budget passed to both sections with its default source; samples is flagged as reserved and currently unused, which directly prevents an agent from misusing it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States exactly what the tool returns: 'One-call morning-patrol summary: M365 service health, sign-in failures, spray suspects, admin actions.' It then names the three sibling tools it aggregates (m365_service_health, signin_failure_stats, directory_audits) and explicitly excludes vm_backup_status, so an agent can distinguish it from every sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use (morning patrol, one-call aggregation), when-not-to-use-and-why (backup excluded because ARM fan-out would exhaust the shared budget; call vm_backup_status separately), and names the alternative implementation pattern (gwsadm-mcp job+poll) with the condition under which it would be preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

directory_auditsA

Who did what to the directory, and when -- the operator-side counterpart to signin_logs.

Every admin action against a user object (block/unblock, password reset, role assignment, attribute edits) appears here, naming the actor (initiated_by) and the affected object(s) (target_resources). This is the record a manual "unblock and reset" intervention -- like the one that closed the 2026-08-21 case this server exists to shorten -- leaves behind; it is how a later triage can tell "already handled by a human" from "still open".

user, when given, matches audits where that account is either the initiator or a target resource. Graph's directoryAudits endpoint only supports server-side $filter on the initiator (initiatedBy/user/userPrincipalName), not on targetResources, so this fetches the full time window and matches both sides client-side -- a window with many unrelated admin actions can need a larger max_pages budget than signin_logs/signin_failure_stats to find one specific user's audits; capped=true warns when that budget ran out before the window was fully scanned.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Entra ID retains directory audit logs for 30 days, same as sign-in logs.

Time-bounded: the scan stops after ENTRAADM_DEADLINE seconds (default 45) and returns what it has with capped=true. Over several days the log is dominated by device-registration noise ("Update device"), so pass category (for example UserManagement, RoleManagement, GroupManagement, ApplicationManagement) to have Graph filter server-side; that keeps a multi-day window inside the budget.

Args: category: Only this Graph audit category (letters only; default: all). user: Restrict to audits naming this userPrincipalName as actor or target (default: all). hours: How far back to look, clamped to [1, 720] (30 days). top: Maximum records to return, clamped to [1, 500]. max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT).

ParametersJSON Schema
NameRequiredDescriptionDefault
topNo
userNo
hoursNo
categoryNo
max_pagesNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the full behavioral burden and does so: read-only, required AuditLog.Read.All application permission or Reports Reader role for azure-cli auth, 30-day Entra ID retention, a 45s ENTRAADM_DEADLINE scan cap returning partial results with capped=true, and the important client-side vs. server-side filter distinction (only initiatedBy supports $filter). This is well beyond what any structured field supplies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core 'who did what to the directory, and when' framing, and the Args block is dense and useful. It is longer than strictly necessary, though -- the aside about the 2026-08-21 case this server exists to shorten adds narrative color rather than invocation-relevant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, no annotations, and a scan-heavy tool with a page budget and timeout, so the description has to be self-sufficient -- and it is. It covers return-relevant fields (initiated_by, target_resources, capped), permissions, retention window, and the operational tradeoff of narrowing by category, leaving no gap an agent would need filled before calling it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate fully and it does: every one of the 5 args is documented with meaning plus constraints (hours clamped to [1,720], top clamped to [1,500], category letters-only, max_pages default reference, user matches either actor or target). The user param's dual actor/target semantics in particular is only knowable from the prose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific resource (directory audits) and states exactly what it captures: admin actions against user objects (block/unblock, password reset, role assignment, attribute edits), with actor and target fields. It explicitly positions itself as the 'operator-side counterpart to signin_logs,' so an agent can separate it from the sibling sign-in tools without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing guidance: use this for admin/intervention history rather than sign-in events, and pass category (e.g. UserManagement) to keep multi-day windows within budget. It also names the sibling tools it shares a page budget model with and states the condition under which a larger max_pages is needed (many unrelated admin actions in-window).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userA

One account's identity/lifecycle state -- the first thing to check on any triage report.

account_enabled=false means the account itself is the whole story; stop there. A stale last_password_change_date_time alongside a fresh "wrong password" complaint (AADSTS50126 in signin_logs) is the most common on-the-ground pattern: the password changed or expired somewhere, and a cached credential on one device is now stale. on_premises_sync_enabled=true means this account is synced from an on-premises directory (Entra Connect) -- Entra is a downstream copy of its password via Password Hash Sync, not the source of truth. licenses names are resolved from the tenant's SKU catalog (/subscribedSkus, page budget from ENTRAADM_MAX_PAGES_DEFAULT); licenses_capped: true appears only when that scan was cut short before resolving one of this account's own licenses -- when present, one or more licenses entries is a raw skuId rather than a friendly name.

sign_in_activity needs an additional Graph read (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role) beyond what the rest of this tool needs. If that permission is missing, every other field above still returns and sign_in_activity alone degrades to {"error": ..., "missing_permission": "AuditLog.Read.All"}.

A nonexistent account is a normal answer, not a tool failure: the result is {"found": false, "user_principal_name": upn} rather than an error key, so a typo'd UPN in a triage report cannot be mistaken for this tool being broken.

Read-only (User.Read.All application permission, or an equivalent delegated read). Requires an exact userPrincipalName, not a display name or partial match.

Args: upn: The account's userPrincipalName, e.g. "user@example.edu".

ParametersJSON Schema
NameRequiredDescriptionDefault
upnYes

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and excels. It discloses the meaning of key fields (account_enabled, on_premises_sync_enabled, licenses_capped), the graceful degradation of sign_in_activity without the required permission, and that a nonexistent account returns {'found': false} rather than an error. It also states the read-only nature and exact UPN requirement. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Though long, the description is densely packed with prerequisite triage knowledge—field semantics, permission nuances, and edge cases—and every sentence adds value. It is structured with clear paragraphs and code-formatting for fields, making it scannable. The front-loaded purpose statement ensures the agent quickly knows what the tool does.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description covers the essential return fields (account_enabled, last_password_change_date_time, on_premises_sync_enabled, licenses, licenses_capped, sign_in_activity), their meanings, and expected edge behavior. It also describes permission requirements and the graceful degradation path. Nothing an agent needs to call it and interpret results is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage for the only parameter, upn, is 0%, but the description fully compensates: 'Args: upn: The account's userPrincipalName, e.g. "user@example.edu".' It also stresses the need for an exact match, not display name. This gives the agent everything needed to format the argument correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'One account's identity/lifecycle state -- the first thing to check on any triage report.' This is a specific verb+resource (get identity/lifecycle state) and explicitly positions it as the initial step, distinguishing it from sibling tools that focus on sign-in logs, auth methods, or audits. The purpose is unmistakable and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context: 'first thing to check' and instructs to 'stop there' if account_enabled=false. It also explains a common pattern for stale passwords. However, it never explicitly names alternative tools or states when not to use this tool (e.g., 'for auth methods, use get_user_auth_methods'). The order is implied but not contrasted with siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_auth_methodsA

Registered authentication methods for one account -- is MFA actually set up?

mfa_registered answers "would this account survive a password-spray hit": True iff at least one non-password method is registered (Authenticator app, phone, FIDO2 security key, Windows Hello, a temporary access pass, software OATH token, or a platform credential/passkey). password itself is excluded from that count -- every account has one, so its presence alone says nothing about MFA coverage.

A nonexistent account is a normal answer, not a tool failure: the result is {"found": false, "user_principal_name": upn} rather than an error key, matching get_user's contract.

Read-only (UserAuthenticationMethod.Read.All application permission). This endpoint is app-only only: it is not exposed to delegated (azure-cli) auth under this tenant's current role assignment, so it degrades to a permission error under azure-cli auth even when other tools work.

Args: upn: The account's userPrincipalName.

ParametersJSON Schema
NameRequiredDescriptionDefault
upnYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states the read-only permission, the app-only limitation that leads to permission errors under azure-cli, the behavior for nonexistent accounts (returns a specific structure instead of an error), and the precise semantics of mfa_registered. It does not detail the full response structure for existing accounts, which is a minor gap, but the key behaviors are well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear opening statement, a code-formatted explanation of the key field, a note on nonexistent accounts, and a security/permission caveat. It is informative but not bloated; each paragraph serves a distinct purpose. It could be slightly more concise, but the structure aids readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description fully covers the MFA-checking purpose and the nonexistent-account case, but it does not specify the full response format for an existing account beyond the mfa_registered boolean. Since there is no output schema, an agent may not know if the response includes a list of methods, their types, or other fields. This is a notable gap for a tool that returns a data structure, though it may be sufficient for the primary use case of checking MFA.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides only the parameter name 'upn' with no description (0% coverage). The description compensates by stating 'Args: upn: The account's userPrincipalName.' This adds the meaning and domain context, making it clear what value to provide. While it doesn't elaborate on format or constraints, the parameter is simple and the explanation is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear, specific purpose: retrieving registered authentication methods for an account and determining whether MFA is actually set up. It explains the key field (mfa_registered) and distinguishes itself from sibling tools that deal with sign-in logs or audits. The verb and resource are unambiguous, and the contrast with other tools is implicit but effective.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use this tool (to check MFA coverage) and includes a practical caveat about app-only auth under azure-cli. It references get_user's contract for nonexistent accounts, offering a form of alternative comparison. However, it does not explicitly name siblings like signin_logs or signin_failure_stats as alternatives, leaving some usage inference to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

health_checkA

Fleet-standard health probe: service/version/status plus two independent Graph probes.

graph confirms Microsoft Graph is reachable at all (GET /users with $top=1 -- needs only User.Read.All, the minimum permission every deployment of this server needs anyway). signin_probe additionally confirms the current credential can read sign-in logs -- the permission every other tool here except get_user depends on. Both probes always run, independently of each other: a tenant that has AuditLog.Read.All but not (yet) the baseline User.Read.All would otherwise have this report "Graph unreachable" -- a fabricated diagnosis, since Graph plainly is reachable if the other probe succeeds. status is derived from the two outcomes: healthy when both succeed, degraded when exactly one does (Graph is reachable but some permission is missing), error only when neither does.

Read-only. Always returns the same keys regardless of outcome (detail is null on success, a translated message on failure), so a caller never has to branch on which keys are present.

sites reports the optional ENTRAADM_SITES_INI table (configured, ranges, error). A configured file that fails to load is an error here and leaves status no better than degraded: without it every internal IP would be reported as an external spray suspect.

arm is a third, informational probe (Azure Resource Manager reachable, and how many subscriptions this credential can see). It never moves status: Azure RBAC is optional for this server, and a deployment used only for Entra triage must not sit at degraded forever because nobody granted it a Reader role. It runs last under its own short budget (_ARM_PROBE_SECONDS) so an unresponsive ARM cannot push this whole call past the client's tool-call timeout. The budget bounds the request, not the first token acquisition (azure-identity is synchronous; in azure-cli mode it shells out to az), which is the same limit every tool here has.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does so thoroughly: it declares read-only, enumerates the probes and required permissions, states both probes always run independently, defines the healthy/degraded/error derivation, guarantees a stable key set with detail null on success, and explains the ARM probe's isolation and timeout budget. This is unusually complete disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core is well front-loaded, but the definition runs five dense paragraphs for a zero-argument probe. Passages about `_ARM_PROBE_SECONDS`, azure-identity's synchronous token acquisition, and azure-cli shelling out are implementation trivia that do not help an agent select or invoke the tool, so not every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must (and does) explain the return shape: the service/version/status keys, the sites and arm sub-objects, the three status values, and the guarantee that detail is null on success. Nothing an agent needs to call and interpret this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline of 4 applies; there are no arguments whose meaning needs supplementing. The description instead spends its effort on return-key semantics, which belongs to contextual completeness rather than parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+scope: 'Fleet-standard health probe: service/version/status plus two independent Graph probes.' It is unmistakably about this server's own readiness, which separates it from the sibling m365_service_health (tenant service status) and from the data tools it gates. An agent knows what this returns without opening anything else.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides strong interpretative context — what each probe proves about permissions, and why a missing baseline permission would fabricate a 'Graph unreachable' diagnosis — which implicitly tells the agent when this tool is the right diagnostic. However, it never states when to call it versus m365_service_health or daily_brief, and offers no explicit exclusions, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_subscriptionsA

The Azure subscriptions this credential can see, as ARM lists them.

Read-only (GET /subscriptions; needs no RBAC beyond being able to see a subscription at all). The discovery anchor for vm_backup_status and vm_backup_policies: both accept a subscription id or display name from this list. An empty list with auth working means no Reader role has been granted anywhere yet. capped marks a listing cut short by the page budget.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_pagesNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does so: it declares the operation read-only, states the RBAC requirement (no role beyond being able to see a subscription), explains what an empty result implies, and defines the `capped` truncation flag. This is exactly the beyond-schema context annotations would otherwise supply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in one clean sentence, followed by progressively finer behavioral detail in short blocks. The parenthetical ARM/RBAC aside and backtick formatting add usable precision, though the phrasing is slightly literary ('as ARM lists them') for an agent-facing description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-annotation, single-parameter, no-output-schema tool this covers the essentials: read-only nature, auth/RBAC expectations, empty-result meaning, and truncation signal. It stops short of describing the returned item shape beyond noting id and display name are usable as inputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the single `max_pages` parameter is entirely undocumented in the schema. The description alludes to a 'page budget' and a `capped` truncation flag, which lets an agent infer that paging is bounded, but it never names the parameter, its default (null), or what value would avoid truncation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the exact resource (Azure subscriptions), the scope (those this credential can see), and the underlying operation (`GET /subscriptions`, i.e. an ARM list). It further distinguishes itself from siblings by identifying itself as the discovery anchor for `vm_backup_status` and `vm_backup_policies`, so an agent can place it without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the when-to-use path: this is the source of subscription ids or display names that the two backup sibling tools accept. It also gives a diagnostic trigger — an empty list while `auth` works means no Reader role has been granted anywhere. Both an entry point and an alternative-selection rule are present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

m365_service_healthA

Microsoft 365 service status per service, and the open incidents and advisories.

Two Graph reads (ServiceHealth.Read.All): healthOverviews (one row per service with its current status) and the unresolved serviceAnnouncement/issues, newest first. A sign-in failure spike reads very differently while Entra or Exchange has an open incident, which is why daily_brief carries this section too. Each section degrades on its own ({"error", "missing_permission"}); summary is computed from whatever came back. capped means either collection was cut short by the page budget, so the counts are lower bounds. The message center (messages) needs a different permission and is not read.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_pagesNo

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it names the required Graph permission, discloses per-section degradation with a concrete error shape ({"error", "missing_permission"}), explains that summary is derived from whatever returned, defines 'capped' as lower-bound counts due to the page budget, and notes the message center is out of scope.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first sentence, with operational detail following. The heavy use of backticked fragments and parentheticals makes it dense, but nearly every clause adds information rather than restating structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read tool with no annotations and no output schema, it adequately describes the return shape (one row per service, unresolved issues newest-first) and failure modes. It could go further on how degraded sections appear alongside computed summaries, but the essentials are present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% for the single max_pages parameter, so the description must compensate. It introduces the 'page budget' concept and the 'capped' consequence, which implies what max_pages controls, but it never names or defines the parameter's units or default behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource and scope: M365 service status per service plus open incidents and advisories, and names the two Graph collections it reads (healthOverviews and serviceAnnouncement/issues). It also draws a boundary against a sibling (daily_brief carries this section) and against the message center, which it explicitly does not read.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear diagnostic context for when this matters ('a sign-in failure spike reads very differently while Entra or Exchange has an open incident') and links to the daily_brief consumer. It stops short of explicit when-to-use/when-not-to-use routing against the other sibling tools, so no full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_by_ipA

Every sign-in from one source IP: who got in from it, who was tried, and when.

The follow-up to a spray_suspects or shared_ips hit: Graph can filter sign-ins on ipAddress server-side, so this is one cheap query, not a log walk. users summarises the IP per account (successes, failures, first/last seen, up to 50 accounts) over every row fetched; events lists the newest top entries that match result ("all" by default, or "success" / "failure"), each carrying the account name and the same AADSTS annotation as signin_logs. capped=true means the page budget or the deadline ran out before the window was fully read; events_truncated=true means more matching rows were read than top returns (the users summary still counts them).

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role).

Args: ip: The source IPv4 or IPv6 address, exactly as the sign-in log shows it. hours: How far back to look, clamped to [1, 720] (30 days). result: "all" (default), "success", or "failure" -- which events to list. top: Maximum events to return, clamped to [1, 500]. max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT).

ParametersJSON Schema
NameRequiredDescriptionDefault
ipYes
topNo
hoursNo
resultNoall
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: it declares read-only status, states the exact permission/role required (AuditLog.Read.All application permission or Reports Reader role), and explains operational signals (capped, events_truncated) beyond anything in structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and behavior before the Args block. The prose is dense and slightly wordy in places (e.g., the elaborated users/events description), but each sentence conveys usable information rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description still explains return structure (users summary with up to 50 accounts, events with AADSTS annotations, capped/events_truncated flags), permissions, and all parameter semantics. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it documents all five parameters: ip format, hours clamp [1,720], result enum values, top clamp [1,500], and max_pages default. This is exactly the compensation required when the schema is silent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb+resource+scope: 'Every sign-in from one source IP: who got in from it, who was tried, and when.' It also implicitly separates itself from signin_logs by framing the results as IP-scoped, and names upstream triggers (spray_suspects, shared_ips).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly positions itself as 'the follow-up to a spray_suspects or shared_ips hit' and explains that Graph filters on ipAddress server-side, making it a cheap query rather than a log walk. It does not explicitly name signin_logs as the alternative to use for non-IP-scoped queries, so the when-not is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_failure_statsA

Tenant-wide sign-in failure aggregation -- the Entra ID counterpart to the RADIUS failure patrol.

Time-bounded (ENTRAADM_DEADLINE, default 45 s): on a wide window or a busy day the scan stops early and capped=true marks the counts as a lower bound; narrow hours for a full count.

Aggregates failed sign-ins across the whole tenant into four views: top AADSTS error codes (with the same meaning annotations as signin_logs), top failing users, top applications, and top source IPs. spray_suspects flags any IP with failed sign-ins against 5 or more distinct users -- Entra's smart lockout is per-account, so a low-and-slow password spray from one IP across many accounts does not trip it the way a brute force against one account does; this is the observation a per-account view cannot make on its own. This mirrors the KeyCloak-side spray detection this fleet already relies on; neither the official Microsoft MCP Server for Enterprise nor Graph itself offers this aggregation.

When ENTRAADM_SITES_INI names the tenant's own address ranges, IPs inside them get site (and site_name) in top_ips, and suspects inside them move from spray_suspects to internal_spray_suspects: a NAT egress or a room of shared workstations fails for many users every day, and would otherwise bury an outside source. Internal suspects are kept, not dropped, because an internal host can be compromised too. sites reports whether the file loaded; an invalid CIDR is reported in sites.error while the valid ranges are still used, and a file that cannot be read leaves every IP in spray_suspects.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Graph cannot filter sign-ins on status/errorCode server-side, so this walks up to max_pages of the full sign-in log for the window and aggregates client-side -- capped=true means the page budget ran out before the window was fully scanned, so the counts below are a sample of the window, not a census of it.

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT).

ParametersJSON Schema
NameRequiredDescriptionDefault
hoursNo
max_pagesNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With zero annotations, the description carries the full behavioral burden and does so thoroughly: read-only status, required AuditLog.Read.All permission or Reports Reader role, the ENTRAADM_DEADLINE time budget, the meaning of capped=true, client-side aggregation due to Graph's lack of server-side status filtering, and site/CIDR error handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and time-bound caveat are front-loaded, and most detail is load-bearing for a tool with no annotations. Some sentences stray into marketing/backstory (the KeyCloak fleet comparison and 'neither the official Microsoft MCP Server... nor Graph itself offers this'), which could be trimmed without loss.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description fully specifies the returned views, the capped/lower-bound semantics, permission requirements, and site-file failure modes. An agent has everything needed to call and interpret the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does: 'hours' is documented with its [1, 720] clamp and 'max_pages' with its ENTRAADM_MAX_PAGES_DEFAULT default and its role in triggering capped counts. This adds meaning the bare schema (titles and defaults only) does not provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Tenant-wide sign-in failure aggregation') and enumerates the four output views (AADSTS error codes, failing users, applications, source IPs). It clearly distinguishes itself from siblings by naming its counterpart ('RADIUS failure patrol') and contrasting with per-account views like signin_logs and signin_by_ip.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage context: narrow 'hours' for a full count, capped counts are a lower bound, and it is the aggregation a per-account view cannot produce. However, it never explicitly routes the agent between signin_failure_stats, signin_logs, and signin_success_stats, leaving that inference to the reader.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_logsA

One user's recent sign-in events, AADSTS-annotated.

The most direct answer to "why can't this person log in": each entry's error_code_meaning translates the raw AADSTS code (e.g. 50126 -> "invalid credentials (wrong password)") so triage rarely needs a second lookup. result filters client-side after the Graph fetch (Graph cannot filter sign-ins on status/errorCode server-side): "failure" (the default) keeps only failed attempts, "success" keeps only clean ones, "all" keeps everything.

Because the filter is client-side, this walks pages until it has collected top matching entries or exhausts max_pages -- a mostly- successful user can otherwise mean paging through hundreds of rows to find a handful of failures. capped=true means the page budget ran out (or top was reached) before the whole window was scanned; a low match count alongside capped=true is evidence of "no more found within the budget", not "no more exist".

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role). Entra ID P1 retains sign-in logs for 30 days; hours beyond that returns an empty result, not an error.

Args: user: The account's userPrincipalName. hours: How far back to look, clamped to [1, 720] (30 days). result: "failure" (default), "success", or "all". top: Maximum matching entries to return, clamped to [1, 500]. max_pages: Page budget for the client-side filter walk (default: ENTRAADM_MAX_PAGES_DEFAULT).

ParametersJSON Schema
NameRequiredDescriptionDefault
topNo
userYes
hoursNo
resultNofailure
max_pagesNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and excels. It discloses read-only permissions (AuditLog.Read.All or Reports Reader), retention limits (30 days), clamping behavior, client-side filtering, page-walking mechanics, and the meaning of capped=true. It even clarifies that low match counts with capped=true indicate a budget limit, not absence of data. This is a model of transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured, front-loading the core purpose, then providing necessary nuance about filtering and paging, then permissions, then a clean parameter list. Every sentence contributes to correct usage; there is no filler or redundancy. It is long but appropriately so for a tool with complex client-side behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (client-side filtering, pagination, permission requirements, retention window, edge cases) and the absence of an output schema, the description is remarkably complete. It covers all scenarios an agent needs to invoke the tool correctly and interpret results, including the capped flag and empty results. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description compensates thoroughly. It explains each parameter in detail: user's userPrincipalName, hours clamped to [1,720], result options with default 'failure', top clamped to [1,500], and max_pages with default. It also explains the functional meaning of result (client-side filter) and the cap semantics, providing far more than the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool returns 'one user's recent sign-in events' and frames it as 'the most direct answer to why can't this person log in'. It identifies the AADSTS annotation as a key value-add, distinguishing it from sibling tools like signin_failure_stats that likely provide aggregated statistics. The purpose is specific and immediately actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage context: it is positioned as the go-to for individual login troubleshooting, and it explains important behavioral caveats like client-side filtering that require adjusting expectations. It implies when not to use it (e.g., for stats, use signin_failure_stats) and details what happens when the page budget is exhausted (capped=true). Alternatives are not explicitly named but clearly indicated by context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signin_success_statsA

Tenant-wide successful sign-in aggregation by source IP -- the view that finds a breach.

signin_failure_stats shows who is being attacked; this shows whether anyone got in. The breach signature is one source IP signing in successfully as several different accounts, most often over a legacy protocol (clientAppUsed such as "Authenticated SMTP" or "IMAP4", which carry no MFA). shared_ips lists the IPs with successes for min_distinct_users or more distinct accounts, most-shared first (up to 50 IPs, shared_ips_capped when more qualified; account names up to 25 per IP), with the client apps and the countries seen. legacy_auth_users lists the accounts that succeeded over a legacy protocol at all, with how many IPs and countries they came from.

A campus NAT, a VDI farm or a shared proxy also puts many accounts behind one IP, so a shared IP is a lead, not a verdict: the caller excludes its own egress ranges and reads the client apps and countries before calling anything a breach. Graph cannot filter sign-ins on status/errorCode server-side, so like signin_failure_stats this walks the sign-in log for the window and aggregates client-side. The walk covers interactive sign-ins only (Graph's default listing): every legacy-protocol authentication is logged as interactive, so none is missed, but non-interactive token refreshes are not counted; capped=true means the page budget or the deadline (ENTRAADM_DEADLINE, default 45 s) ran out first and the counts are a lower bound -- narrow hours for a full count.

Read-only (AuditLog.Read.All application permission, or -- for azure-cli auth -- the Reports Reader directory role).

Args: hours: How far back to look, clamped to [1, 720] (30 days). max_pages: Page budget (default: ENTRAADM_MAX_PAGES_DEFAULT). min_distinct_users: Distinct accounts an IP needs to appear in shared_ips (default 2, clamped to >= 2).

ParametersJSON Schema
NameRequiredDescriptionDefault
hoursNo
max_pagesNo
min_distinct_usersNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does so: read-only, required permission (AuditLog.Read.All / Reports Reader role), client-side aggregation because Graph cannot filter on status/errorCode server-side, interactive-sign-ins-only scope with the reasoning that legacy auth is logged as interactive, and lower-bound semantics when the page budget or ENTRAADM_DEADLINE expires. This is exactly the context annotations would otherwise supply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the failure/success contrast are front-loaded, and the dense middle does real work. It is longer than strictly necessary — the breach-signature narrative and the NAT/VDI caveat could be tightened — but almost every sentence carries operational information rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description must explain return values directly, and it does: shared_ips (up to 50, ordering, per-IP account/client-app/country detail, shared_ips_capped) and legacy_auth_users with IP/country counts. Combined with the permission, scope, and cap caveats, an agent has everything needed to call and interpret this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and it does for all three params: hours clamped to [1,720], max_pages defaulting to ENTRAADM_MAX_PAGES_DEFAULT (page budget), and min_distinct_users defaulting to 2 with a clamp of >=2. No parameter is left to inference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb+resource+scope (tenant-wide successful sign-in aggregation by source IP) and explicitly contrasts itself with the sibling signin_failure_stats ('shows who is being attacked; this shows whether anyone got in'). An agent can distinguish it from every sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing to signin_failure_stats, plus genuine interpretation guidance: shared IPs are 'a lead, not a verdict' because NAT/VDI/proxy produce the same signature, and it tells the caller to exclude its own egress ranges before concluding a breach. It also names the condition that invalidates results (capped=true) and the remedy (narrow hours).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vm_backup_policiesA

Schedule and retention of the Azure VM backup policies in each Recovery Services vault.

Read-only; needs the Reader role on the subscription. Per policy: policy_type (V1 = SimpleSchedulePolicy, V2 = SimpleSchedulePolicyV2 with an hourly / daily / weekly schedule), schedule (frequency, run times in UTC, weekdays, or the hourly window), retention (LongTermRetentionPolicy: daily / weekly / monthly / yearly blocks with their day selection and duration), instant_rp_days, timezone and protected_item_count (ARM's own count). A schedule or retention type this server does not know is returned as {"type": <name>, "unparsed": true}, never dropped.

subscription is an id or display name from list_subscriptions (None = every visible subscription); an unknown or ambiguous name returns found: false. Coverage flags as in vm_backup_status: skipped_vaults per subscription, skipped subscriptions when the budget ran out, capped overall.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_pagesNo
subscriptionNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it declares read-only access, the Reader role prerequisite, the unknown-type fallback ({"type": <name>, "unparsed": true}, never dropped), the found: false behavior for unknown/ambiguous subscription names, and the skipped/capped coverage semantics. It stops short of describing pagination behavior or result volume.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and safety profile are front-loaded in the first two lines, and the remaining dense blocks each document a distinct behavior (output shape, fallback, parameter semantics, coverage flags) with no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must describe returns, and it does so thoroughly: per-policy fields, the unparsed fallback, and the coverage flags. The only real gap is the unexplained max_pages control, which matters for a paginated multi-subscription read.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It fully explains subscription (id or display name from list_subscriptions, None = all, ambiguous name yields found: false), but max_pages is never mentioned except obliquely through the 'budget ran out' coverage flag, leaving one of two parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific resource (Azure VM backup policies in each Recovery Services vault) and the exact payload (schedule and retention), so an agent knows precisely what it retrieves. It lacks an explicit verb (List/Get) and its differentiation from siblings is only implicit via a reference to vm_backup_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the read-only nature and the required Reader role, and clarifies that subscription comes from list_subscriptions with None meaning all visible subscriptions. However, it never explicitly says when to prefer this tool over vm_backup_status or the other backup/log siblings, leaving routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vm_backup_statusA

Which Azure VMs are backed up by Azure Backup, and whether the last backup succeeded.

Joins every VM of a subscription (Microsoft.Compute/virtualMachines) with the Azure-VM protected items of its Recovery Services vaults, on the VM's ARM id (case-insensitively; ARM casing differs between the two services). Read-only; needs the Reader role on the subscription.

Per VM:

  • protected: true when a protected item exists with protectionState Protected or IRPending (first backup not done yet, note says so); false when there is none, or it is ProtectionStopped / ProtectionPaused / ProtectionError (data may be retained, backups are not running -- the state is shown); null (unknown) when no item was found but the vault scan was incomplete (capped, a skipped vault, an unreadable or unreached subscription -- in the global join, any subscription's) -- a partial scan never produces a false "unprotected".

  • backup_ok: true for lastBackupStatus Completed / CompletedWithWarnings, false for Failed, null before any backup ran or while one is running (InProgress, Warning, an unknown value). An item whose protectionState is missing is protected: null with a note. When two vaults hold an item for one VM, the one still backing up wins (duplicate_items counts them).

  • stale: no recovery point (lastRecoveryPoint, the proof of a successful backup) within stale_hours. The default 36 h suits daily policies; raise it for weekly ones (policy_name is in the row).

subscription is an id or display name from list_subscriptions; None scans every visible subscription and joins globally, so a VM backed up to a vault in another subscription is matched. With one subscription named, only its vaults are scanned (coverage says so). An unknown or ambiguous name returns found: false.

orphans are protected items whose VM no longer exists (backup data retained for a deleted VM), reported only when that VM's subscription was listed completely. One subscription's permission error is reported on its own entry; the others still return. When the wall-clock budget runs out, the remaining subscriptions are listed under skipped and capped is set.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_pagesNo
stale_hoursNo
subscriptionNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so richly: read-only, requires the Reader role on the subscription, explains that a partial scan never yields a false 'unprotected', how duplicate vault items are resolved, how orphans are only reported on fully-listed subscriptions, and how permission errors and the wall-clock budget (capped/skipped) surface.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded one-line summary followed by scannable bullet groups for each per-VM field. It is long, but for a join with this much conditional output semantics, nearly every sentence adds needed precision, so only minor trimming is possible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, yet the description effectively documents the return contract (protected, backup_ok, stale, orphans, skipped, capped, found). For a complex cross-service join with no annotations, an agent has everything needed to call and interpret it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It thoroughly explains subscription (id or display name, None scans all) and stale_hours (default 36 for daily policies, raise for weekly). max_pages is only obliquely implied through the capped/skipped discussion and is never named, leaving one of three parameters under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line gives a specific verb and resource ('Which Azure VMs are backed up by Azure Backup, and whether the last backup succeeded') and the body names the exact Azure resource types it joins. It is plainly distinguishable from the sibling vm_backup_policies, which handles policies rather than per-VM status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives useful usage context – pass a subscription id/name from list_subscriptions, pass None to scan everything, raise stale_hours from 36 for weekly policies. However, it never explicitly states when to choose this over vm_backup_policies or the other siblings, leaving the routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv1.3.0
    • Addedlist_subscriptions
    • Addedm365_service_health
    • Addedvm_backup_policies
    • Addedvm_backup_status
  2. 3 tool updatesv1.1.0
    • Changeddirectory_audits1 field changed
      • addedInput schema / properties / category
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "default": null,
        +  "title": "Category"
        +}
    • Addedsignin_by_ip
    • Addedsignin_success_stats
  3. 7 tool updatesv0.1.0
    • First observeddaily_brief
    • First observeddirectory_audits
    • First observedget_user
    • First observedget_user_auth_methods
    • First observedhealth_check
    • First observedsignin_failure_stats
    • First observedsignin_logs

TDQS

A4.3/5.0

Scored across 13 tools

Disambiguation4/5

The four sign-in tools (signin_logs, signin_by_ip, signin_failure_stats, signin_success_stats) share a data source but are clearly scoped by user vs IP vs failures vs successes, and get_user vs get_user_auth_methods are cleanly separated. daily_brief deliberately overlaps by composing m365_service_health, signin_failure_stats, and directory_audits, but is explicitly framed as a one-call aggregator rather than a peer.

Naming Consistency4/5

Names are uniformly snake_case and descriptive nouns/actions, with a consistent 'signin_' family and 'vm_' family. The only minor inconsistency is verb prefixes: get_user/get_user_auth_methods and list_subscriptions use verbs while signin_logs, directory_audits, m365_service_health, and daily_brief are bare noun phrases.

Tool Count5/5

Thirteen tools is well within the sweet spot for a focused triage/diagnostic server, and each tool covers a distinct query or aggregation. No obvious filler or redundant tool bloats the set.

Completeness4/5

The surface covers the core triage lifecycle: identity/lifecycle state, MFA registration, per-user and per-IP sign-in detail, tenant-wide failure and success aggregation, admin audit history, M365 service health, and Azure VM backup status/policies. Gaps exist for role/group membership lookup, device compliance, and Identity Protection risk detections, but these are minor for a read-only triage-oriented server and the aggregate tools let agents work around most of them.

Maintenance

ActivityMaintained
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • -
    license
    Not graded
    quality
    Not graded
    maintenance
    Provides secure access to Microsoft Entra ID (Azure AD) resources including users, devices, and applications through Microsoft Graph API. Enables querying organizational data with comprehensive audit logging to Azure Blob Storage.
    -