cipp-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.7.3
Disambiguation4/5
Most tools have distinct resources (users, groups, mailboxes, policies, etc.), but a few pairs like list_standards vs list_standard_templates and get_tenant_details vs get_tenant_drift vs get_tenant_alignment could cause misselection without careful reading.
Naming Consistency4/5
All tools share the cipp_ prefix and largely follow a verb_noun pattern (list, create, get, set). Minor deviations include cipp_bec_check (noun_check), cipp_add_scheduled_item vs create_*, and get vs list for tenant details.
Tool Count2/5
With 43 tools, this exceeds the 25+ threshold for 'too many.' While CIPP is a broad platform, the set includes many highly specific list-only operations that inflate the count.
Completeness3/5
The tool set covers tenant, user, group, mailbox, security, and compliance domains extensively for reading, but many resources are list-only with no update/delete (e.g., groups, conditional access policies, mailboxes). User deletion exists only via offboarding, leaving notable gaps for full lifecycle management.
Average 3.5/5 across 43 of 43 tools scored. Lowest: 2.1/5.
See the Tool Scores section below for per-tool breakdowns.
- 5 of 6 community issues answered or closed in the last 6 months
- 32 commits in the last 12 weeks
- Last stable release on August 13, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
Add a LICENSE file by following GitHub's guide. Once GitHub recognizes the license, the system will automatically detect it within a few hours.
If the license does not appear after some time, you can manually trigger a new scan using the MCP server admin interface.
MCP servers without a LICENSE cannot be installed.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as authentication requirements, rate limits, pagination behavior, or error handling. The simple verb 'List' gives no insight into side effects or data volume.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with only 5 words. It avoids verbosity, but it might be too terse, omitting necessary context. Still, it is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has no annotations, no output schema, and multiple sibling tools, the description should provide more context about return values, filtering behavior, and scope. The current description is insufficient for an agent to confidently use this tool without additional information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are fully described in the input schema with clear descriptions. The tool description adds no additional information about parameters beyond what the schema already provides. Schema coverage is 100%, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'List mailboxes in a tenant' which is a clear verb+resource. However, it does not specify the scope of mailboxes (e.g., all types) nor differentiate from the sibling tool cipp_list_mailbox_permissions. The schema provides the type enum, but the description itself lacks this nuance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives such as cipp_list_users or cipp_list_mailbox_permissions. The description does not mention any prerequisites or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as side effects, permissions required, or idempotency. The minimal description fails to compensate for missing annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short but lacks valuable content. It is under-specified rather than concise, missing important context about the tool's behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, nested objects) and lack of output schema, the description is far too brief. It does not explain return values, error conditions, or behavior when parameters are omitted, leaving significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond the schema, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool creates a new scheduled task, but does not differentiate from possibly related tools like cipp_list_scheduled_items. However, as the only creation tool among scheduled item tools, the purpose is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, prerequisites, or scenarios to avoid. The description provides no context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It fails to disclose any behavioral traits such as read-only nature, pagination, rate limits, required permissions, or whether results are sorted or limited. The description is insufficient for the agent to understand runtime behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at one sentence, but it lacks structure and essential detail. The brevity comes at the cost of completeness; a better balance would include at least one additional sentence on behavior or output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 3 parameters, no output schema, and no annotations, the description does not cover return format, pagination, or how to interpret log entries. For a tool that lists audit logs, the output is critical but unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no additional meaning beyond the parameter names already documented in the schema (days, type, tenantFilter). It does not clarify format, defaults, or usage tips.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists audit log entries for a tenant, specifying the resource (audit logs) and scope (tenant). It implicitly differentiates from siblings like cipp_list_logs which may refer to other logs, and cipp_list_tenants which lists tenants, not logs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as cipp_list_logs or other audit-related tools. No exclusions or prerequisites are mentioned, leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description should disclose behavioral traits. It only says 'Get detailed information' but does not specify it is read-only, what data is returned, or any side effects. The description fails to compensate for the absence of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is appropriately concise for a simple tool, though it could include more detail without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description provides minimal context. It lacks specifics on what 'detailed information' includes, which could be inferred from sibling tools but is not explicit. Adequate but not enriched.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (parameter has description). The description adds no extra meaning beyond the schema; it reiterates the tool's purpose without enriching parameter understanding. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('detailed information about a specific tenant'). It implies a focused retrieval compared to sibling tools like cipp_list_tenants, but does not explicitly differentiate or specify the scope of 'detailed information'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like cipp_list_tenants for listing or other detail tools. There is no mention of prerequisites or typical workflow steps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only says 'Get', implying a read operation without elaboration. It does not address whether results are paginated, filtered, or if any access requirements exist, leaving a significant transparency gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence that names the action and target. It is efficient and front-loaded, though it could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema and no annotations, so the description should clarify what the returned 'results' contain, how they are structured, or any limitations. It currently says only 'results', which is vague for a tool that clearly returns a report or list.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage for the single 'tenantFilter' parameter, including a description and the 'allTenants' option. The tool description adds no additional parameter meaning, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves 'Best Practice Analyser results' for a tenant, identifying a specific resource and scope. It distinguishes from siblings like cipp_list_standards or cipp_get_tenant_alignment by naming a unique resource, though the verb 'Get' is generic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as cipp_list_standards or cipp_get_tenant_drift. There are no contextual cues about prerequisites or typical use cases, leaving the agent to infer suitability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It fails to disclose behavioral traits such as pagination, filtering, permissions, or return format. The single sentence provides minimal insight beyond the tool's existence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence. It efficiently conveys the core purpose without redundancy, though additional structure could improve scannability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and annotations, the description is incomplete. It lacks details on pagination, result ordering, group types returned, or any limitations, which are important for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters ('search' and 'tenantFilter'). The description does not add additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List groups in a tenant' is clear and specific, using a verb and resource. It distinguishes itself from sibling 'cipp_list_user_groups' which lists groups for a specific user.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'cipp_list_user_groups' or 'cipp_create_group'. The description does not mention any specific scenarios or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It only states 'list CIPP application logs' without disclosing behavioral traits such as authentication requirements, scope (e.g., tenant-wide?), rate limits, or whether listing mutates state. The default behavior of retrieving 'recent logs' is implied by the parameter but not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no waste. However, it may be overly concise at the expense of helpful detail. Still, it earns a high score for conciseness because every word is functional.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should explain the return format or contents of the logs. It does not. The tool is simple (one optional param), but the lack of behavioral detail and result description makes it less complete. Siblings like 'cipp_list_audit_logs' might have richer descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents the 'dateFilter' parameter with its format and usage. The description adds no extra semantic information beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'CIPP application logs'. However, the sibling 'cipp_list_audit_logs' suggests a potential distinction between application logs and audit logs, which is not clarified. The purpose is clear but lacks differentiation from a closely related tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'cipp_list_audit_logs'. There are no prerequisites, exclusions, or context about appropriate usage scenarios. The agent receives no help in deciding which log tool to invoke.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose behavior beyond listing. No mention of authentication, rate limits, or what 'scheduled tasks' entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks any additional structure or emphasis. It is not verbose but also not exceptional.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and many sibling list tools, description does not explain what 'scheduled tasks' refers to or the return format, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; schema coverage is 100%. Description adds no param info, which is acceptable as there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List' and resource 'scheduled tasks in CIPP'. It is unambiguous but does not distinguish from other list tools like cipp_list_logs or cipp_list_tenants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., cipp_list_alert_queue). No prerequisites or use cases mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description omits behavioral traits like read-only nature, required permissions, or side effects. It does not disclose that the operation is safe or idempotent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. It is efficiently front-loaded with the core action, though it could benefit from slightly more detail without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description does not clarify what the tool returns (e.g., group names, IDs, or both). For a list operation, this lack of return information reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are already well-documented in the input schema (`userId` and `tenantFilter`). The description adds no additional meaning beyond the schema, achieving the baseline for 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List groups a user is a member of' clearly states the action (list) and resource (user groups). It differentiates from sibling `cipp_list_groups` which lists all groups, though it could explicitly distinguish them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives like `cipp_list_groups`. The description does not provide context such as prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It only says 'List users', implying a read operation, but does not state safety, pagination, rate limits, or required permissions. The minimal description leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, front-loading the purpose. While efficient, it could include slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description fails to provide context on return values, default behavior (e.g., lists all users), or pagination. For a list tool, this is a significant omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with clear descriptions for each parameter. The tool description adds no additional meaning beyond the schema, resulting in a baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List users in a tenant' clearly states a specific verb and resource, and is distinct from sibling tools like cipp_list_tenants or user action tools. However, it lacks nuance about the optional search capability.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to choose listing all users versus searching. There is no mention of prerequisites or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It only says 'Trigger', implying a mutation, but does not disclose whether the check is synchronous, what happens during execution, or any side effects. This is insufficient for an agent to understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded. Every word is necessary and there is no redundancy. Given the low complexity, this is optimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks important context such as what the check does, expected output, and any requirements. With no output schema and no annotations, the agent is left without enough information to use the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (the single parameter has a clear description). The tool description does not add any extra parameter semantics beyond what is in the schema. Baseline 3 is appropriate as the schema already provides meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states it triggers a standards compliance check, which clearly identifies the action and resource. It distinguishes from sibling tools like cipp_list_standards (listing) by indicating a triggered action. However, it could be more specific about what the check involves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as cipp_list_standards or cipp_list_standard_templates. The description does not mention prerequisites or context, leaving the agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavior. It does not specify output format, pagination, rate limits, or side effects. 'List users and their MFA status' is vague; agent may not know if it returns all users, a subset, or status types.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, brief but lacking useful structure or contextual details. Adequately concise but not optimally informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no output schema, description does not explain return structure (e.g., user names, MFA status values). Insufficient for an agent to fully understand behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter description already explaining 'tenantFilter' usage. Tool description adds no extra meaning; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List users and their MFA status in a tenant', which is a specific verb-resource pair. It distinguishes from sibling tools like cipp_list_users (users only) and cipp_reset_mfa (modify MFA).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Does not mention scenarios (e.g., 'Use to check MFA compliance') or exclude cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the action without disclosing any behavioral traits such as read-only nature, pagination, permissions needed, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence with no redundant words. It is highly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, the description is minimally adequate. However, it lacks behavioral details and usage context, which are gaps given the absence of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100% and the single parameter 'tenantFilter' is well-documented in the schema. The tool description adds no extra meaning beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb ('List') and resource ('Conditional Access policies') with scope ('for a tenant'). It is unambiguous but does not explicitly differentiate from sibling list tools like cipp_list_tenants or cipp_list_users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. With many sibling list tools, context on when to choose this one would be helpful but is absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as required permissions, side effects, or actions taken during the check.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description does not provide enough context about what the check entails, expected results, or side effects, making it incomplete for a security tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides full descriptions for both parameters (userId and tenantFilter), so baseline 3 is appropriate as the tool description adds no extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Run a Business Email Compromise check on a user' uses a specific verb and resource, and clearly distinguishes from sibling tools focused on user management or tenant details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only says 'Check', implying a read operation, but fails to disclose if DNS queries are made, permissions required, or whether results are cached. This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, terse sentence with no unnecessary words. It efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema, the description should explain what the response contains (e.g., health status for each check). It fails to do so, leaving the agent without critical information about how to use the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100% and the parameter description already includes domain/ID scope and 'allTenants'. The tool description adds no additional semantic information beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Check'), the resource ('domain health'), and the specific technologies checked ('DMARC, DKIM, SPF'), along with the scope ('for a tenant'). This distinguishes it from sibling tools like cipp_list_tenants or cipp_get_tenant_details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no when-to-use or when-not-to-use guidance, nor does it mention alternative tools. Without such guidance, an agent cannot differentiate this from other checking tools like cipp_list_standards.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not disclose default behavior (e.g., that some tenants may be filtered out) or what 'allTenants=true' changes. The parameter description in the schema adds some context, but the main description lacks transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no wasted words. However, it is overly minimal and could include more context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input schema and absence of output schema, the description is mostly adequate but fails to explain the default filtering behavior implied by the parameter, leaving a gap for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for the 'allTenants' parameter. The description adds no additional meaning beyond what the schema already provides, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'List' and the resource 'all tenants managed in CIPP'. It distinguishes from sibling tools like cipp_get_tenant_details which focuses on a single tenant.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., cipp_get_tenant_details) and no exclusions or prerequisites. The parameter hint about 'default filter' is only in the schema, not the main description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not disclose any behavioral traits such as read-only nature, performance considerations, rate limits, or error handling. The simple statement 'list all' lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is front-loaded and immediately conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple listing with no parameters and no output schema, the description is adequate but minimal. It does not explain return format, pagination, or behavior across tenants, which would be helpful for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters in the input schema, and schema coverage is 100%, so baseline is 3. The description adds no parameter information, but none is needed. The lack of parameters is clear from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'CSP licenses', and scope 'across tenants'. It distinguishes from sibling tools like cipp_list_tenants and cipp_list_users by specifying a different resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, no when-not-to-use conditions, and no context about expected usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It implies a read-only operation through the word 'List' but does not disclose additional behavioral traits such as permission requirements, return format, pagination, or what 'usage' entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the verb and resource. Every word is informative and there is no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool, the description is minimally sufficient to understand its purpose, but it lacks information about the returned data format or scope of 'usage'. With no output schema and no annotations, a bit more context would be helpful for full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully documents the single parameter 'tenantFilter' with explicit description, achieving 100% schema coverage. The tool description adds no extra parameter semantics, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb 'List' and resource 'license assignments and usage' scoped to a tenant. It distinguishes from sibling tools like cipp_list_csp_licenses by specifying tenant-level license data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as cipp_list_csp_licenses or cipp_list_tenants. The description merely states what it does without explaining suitable contexts or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It only states 'list permissions', offering no details on side effects, required permissions, rate limits, or return format. This is insufficient for an agent to understand the tool's behavior beyond a basic read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that immediately conveys the action and target resource. No wasted words, and the structure is front-loaded with the verb.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and simple parameters, the description is minimally adequate. However, it omits any mention of return values, error conditions, or pagination. For a read operation, this might be sufficient, but more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage; both parameters (upn, tenantFilter) have clear descriptions. The tool's description adds no further semantic value beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'permissions on a specific mailbox'. It is distinct from sibling tools like cipp_list_mailboxes or cipp_list_users which list entire mailboxes or users, not permissions on a specific mailbox.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not indicate when to use this tool versus alternatives, nor does it mention any prerequisites or limitations. Relies solely on the tool name and description implying its purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description gives no behavioral details beyond the basic operation. It does not disclose permissions, rate limits, pagination, or the read-only nature of the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence that front-loads the purpose. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with one parameter, the description is adequate but lacks details about output format, scope of results, or relation to sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter 'tenantFilter' is fully described in the schema. The description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'compliance standards configured for a tenant', distinguishing it from sibling tools like cipp_list_tenants or cipp_run_standards_check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like cipp_run_standards_check, nor any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the basic action without mentioning traits like pagination, error handling, rate limits, or the effect of invalid inputs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, lean sentence with no superfluous words. It is front-loaded and efficiently communicates the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and annotations, the description is minimally adequate for a simple list operation. However, it omits details about the return format, pagination, or any additional filtering, leaving room for ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters have clear descriptions (userId and tenantFilter). The tool description does not add extra meaning beyond the schema, but schema already provides adequate detail, giving a baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'devices enrolled by a user'. It uniquely identifies the tool among siblings (no other device listing tool) and clearly states the scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any when-to-use or when-not-to-use guidance. It lacks context on prerequisites or alternatives, leaving the agent with no decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only says 'list' without details on return format, pagination, or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, but could include more context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description does not explain what roles are returned, making it minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, schema coverage 100%; description adds no value beyond the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'List' and resource 'GDAP roles', and distinguishes from sibling 'cipp_list_gdap_invites' which lists invites.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use or alternatives, but for a simple list tool the context is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only states 'List' implying read-only, but lacks details on alert scope, limits, or safety. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, clear sentence with no extraneous words; perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameter-less list tool, the description is minimally adequate but omits return format or any behavioral context. Adequate given low complexity but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Tool has zero parameters; per calibration baseline of 4, the description appropriately adds no param info since none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' with resource 'queued alerts' and scope 'across all tenants', clearly differentiating it from sibling tools that focus on tenants, users, or groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like cipp_list_audit_logs or cipp_list_scheduled_items; no when-not or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only states the action without any disclosure of side effects, permissions required, or behavior (e.g., whether it is read-only). The agent must infer safety, increasing risk of misuse.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence that front-loads the purpose. No extraneous words, appropriately sized for a simple list tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description is minimally complete for a list operation with no parameters, it lacks details about the return format or common use cases. Given no output schema, some additional context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (schema coverage 100%), so the description adds no param info. A score of 4 is baseline for zero-parameter tools, as no further semantic clarification is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'List' and resource 'pending GDAP relationship invites', distinguishing it from sibling tools like cipp_list_tenants or cipp_list_gdap_roles.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. However, it is implied that this tool is for listing pending invites, which is a distinct use case among sibling list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds context beyond annotations (e.g., high-impact, security/mail use cases, reversibility). However, it contradicts the idempotentHint=true annotation, as creating a group is inherently non-idempotent (duplicate groups may be created). This reduces transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (two sentences plus a warning), front-loaded with the caution icon, and every sentence conveys essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, and the description does not explain the return value (e.g., group ID, confirmation). The idempotency contradiction remains unresolved. Overall, it is adequate for a simple creation tool but lacks full context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description ties the tool's use cases (security, mail) to parameters like securityEnabled and mailEnabled, but does not elaborate on parameter semantics beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new group in the tenant, with specific uses (security policy assignments, mail distribution), distinguishing it from sibling tools like cipp_list_groups (listing) and cipp_create_user (user creation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises confirming with the user before invoking and notes reversibility, but does not specify when not to use it (e.g., if an equivalent group already exists) or explicitly compare to alternative creation methods among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true, readOnlyHint=false, idempotentHint=true), the description adds that the operation is reversible by editing again, may grant/revoke roles or license eligibility, and requires user confirmation. This provides useful behavioral context not captured in structured data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. It front-loads a warning emoji and 'HIGH-IMPACT', then states the purpose and key behaviors. No wasted words, every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, high-impact, destructive) and no output schema, the description covers major behavioral aspects but lacks details on return values or immediate effects. It does not specify how to revert changes or what happens to existing settings not mentioned. An output schema would help, but the description provides adequate context for an AI agent to use the tool cautiously.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the schema already documents all parameters. The tool description does not add specific parameter details beyond what the schema provides, but it does give context that the tool can modify roles and licenses, which aligns with parameters like licenses and removeLicenses. Baseline 3 applies due to high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'Edits an existing user's properties' with specific examples like directory attributes, usage location, and license/role modifications. It distinguishes from sibling tools like cipp_create_user or cipp_disable_user by focusing on editing existing users.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description warns that it is high-impact and advises confirming with the user before invoking, but provides no explicit guidance on when to use this tool versus alternatives like cipp_create_user or cipp_disable_user. It does not specify prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It correctly indicates the action (get) but lacks disclosure about idempotency or read-only nature. Acceptable for a simple tool but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, straightforward sentence with no wasted words. It is optimally concise for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no output schema, and performs a simple read operation, the brief description is complete enough for an agent to understand its purpose and effect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100% (empty). The description adds nothing beyond the schema, but a baseline of 4 is appropriate for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets version information, with a specific verb and resource. Despite the simplicity, it is distinguishable from sibling tools, none of which are version-related.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use the tool versus alternatives, nor any context about prerequisites or typical use cases. For a simple read-only tool, it would benefit from stating it is safe to call.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true and destructiveHint=false, aligning with the read-only list operation. Description adds no further behavioral context beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, concise sentence with no extraneous information; front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with no parameters and clear annotations, the description is fully sufficient; no gaps identified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100%. Description is not required to add parameter detail, meeting the baseline for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List', resource 'CIPP Standards Templates', and scope 'across the partner tenant', making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings like cipp_list_standards or cipp_list_tenants; agent must infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond the destructiveHint annotation by explicitly stating the password invalidation and reversibility. However, does not disclose potential side effects like session termination or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise, with only two sentences. The high-impact warning is front-loaded, and every word adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core purpose and important caveats, but omits mention of return values (e.g., whether the new password is returned) and required permissions, which would be helpful given the high-impact nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema already provides 100% description coverage for parameters. The tool description does not add significant new meaning beyond stating the operation, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly identifies the tool's action: resetting a user's password and invalidating the current one. It distinguishes well from sibling tools like cipp_disable_user or cipp_revoke_sessions by focusing on password reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a clear when-not-to-use directive ('Confirm with the user before invoking'), but lacks explicit guidance on when to prefer this tool over alternatives or prerequisites, such as permissions or user existence.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as destructive and non-read-only. The description adds value by explaining that it sends automated messages to senders and is reversible, providing practical context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the warning and state the core action. No wasted words; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (13 params, no output schema), the description is minimal. It covers the essential behavioral impact but lacks details on parameter usage (e.g., scheduling vs immediate). Parameter descriptions compensate somewhat, but completeness is moderate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the parameter descriptions, which are already detailed. No additional parameter context provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool configures the out-of-office/auto-reply for a mailbox, which is a specific verb and resource. It distinguishes from siblings like set_email_forwarding by focusing on auto-reply functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description warns of high impact and suggests confirming with the user, but does not explicitly state when to use this tool versus alternatives (e.g., other mailbox or user management tools). Implies caution, not specific use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description states 'report' which aligns with readOnlyHint=true and destructiveHint=false, but adds no additional behavioral context such as permissions, rate limits, or response details. Annotations already cover the non-destructive nature, so description adds minimal value here.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the key action 'Report standards drift'. No filler or redundant information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately explains the tool's purpose and usage of the only parameter. Lacks details about the output format (e.g., list of drift items), but given the tool's simplicity and absence of output schema, the description is reasonably complete. Could clarify what constitutes 'drift' more explicitly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description_coverage is 100% (single parameter tenantFilter has a description in schema). The description repeats the schema's guidance about omitting for all tenants, adding no new meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'report' and resource 'standards drift' with clear reference to tenant's assigned Standards Template. Distinguishes from sibling tools like cipp_list_standards and cipp_run_standards_check by focusing on deviation detection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on omitting the tenantFilter parameter to report drift across all tenants. However, does not directly contrast with siblings like cipp_run_standards_check or cipp_get_tenant_alignment, leaving some ambiguity about when to choose this tool over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of disclosing behavior. The word 'List' implies a read-only operation, but it does not mention permissions, response format, pagination, or potential side effects. For a simple list tool, this is adequate but not thorough, so a mid-range score is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly worded sentence with no filler. It states exactly what the tool does and includes a helpful parenthetical clarification, achieving high information density in minimal space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simple nature (one parameter, no output schema, no nested objects), the description is adequately complete. It defines the resource and the required scope, and the schema handles parameter details. Slight lack of explicit return value or read-only confirmation is minor for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers 100% of the parameter meaning, including allowed values and special keyword 'allTenants'. The tool description adds no additional parameter semantics beyond what the schema provides, so the baseline score of 3 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), the resource ('named locations'), and adds clarification ('trusted IPs') to remove ambiguity. It is distinct from sibling tools by specifying a unique resource type, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description correctly indicates the context ('for a tenant') and the scope parameter, making it clear when to use this tool (when needing named locations). It does not explicitly exclude alternatives, but the specific resource name differentiates it sufficiently from sibling list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true, but the description adds valuable context: the action is reversible by re-enabling MFA, and it requires user confirmation. This goes beyond the annotation flags, though idempotentHint=true is already clear from the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two sentences with a prominent warning prefix. Every word adds value, and the structure is front-loaded with the most critical information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description covers the key behavioral aspects (impact, reversibility, prerequisite confirmation). It could mention expected response or next steps, but the essentials are sufficient for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters are fully described in the input schema (100% coverage). The description adds no new parameter-specific details, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool resets all MFA methods for a user, specifying the exact action and resource. It distinguishes from sibling tools like cipp_revoke_sessions by focusing on MFA reset specifically.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a strong caution ('HIGH-IMPACT') and advises confirming with the user, but does not explicitly state when to use this tool over alternatives like cipp_reset_password or cipp_revoke_sessions. Usage context is implied but not direct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds context beyond annotations: notes high-impact, directory presence, credentials, license/role eligibility, and reversibility. Annotations already indicate destructive and write, so description complements without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences with critical warning symbol and impact label; no wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers core functionality, impact, and reversibility for a creation tool, but lacks any mention of output or expected return, which is acceptable given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are well-documented. Description adds no new parameter details beyond schema, meeting baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Creates a new user account in the tenant' with specific verb and resource, and distinguishes from siblings like edit/disable by emphasizing high-impact and reversibility.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to 'Confirm with the user before invoking' and notes reversibility, but does not directly compare to alternative tools for similar actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as destructive and not read-only. The description adds valuable behavioral context: it is reversible ('Reversible by the user signing in again'), and it is high-impact. It explains that the user will be forced to re-authenticate, which goes beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences plus a warning symbol. It front-loads the high-impact nature ('⚠ HIGH-IMPACT') and then succinctly explains the action, reversibility, and requirement to confirm. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simple purpose and the fact that the schema fully describes parameters, the description covers the key contextual aspects: impact, reversibility, and need for confirmation. It does not mention what happens if the user has no active sessions, but for the typical use case, this is complete enough. No output schema exists, but the return value is not critical here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage for both parameters, describing userId as Azure AD object ID or UPN, and tenantFilter as tenant domain or ID with 'allTenants' option. The description adds no additional parameter semantics beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Revokes all active sessions for a user, forcing them to re-authenticate.' It specifies the verb (revoke) and resource (sessions), and distinguishes itself from sibling tools like reset password or disable user by focusing specifically on session revocation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a strong usage guideline: 'Confirm with the user before invoking.' This tells the agent when to use the tool (only after confirmation). However, it does not explicitly mention when not to use it or provide alternatives like reset MFA or password reset for similar outcomes, which would have elevated the score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true, readOnlyHint=false), the description reveals the tool is high-impact, silently redirects mail, is a common exfiltration vector, and is reversible. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four concise sentences, front-loaded with the warning icon, and every sentence adds value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation tool, the description covers impact, reversibility, and appropriate caution. Missing details about return values or success/error behavior, but acceptable given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and adequately documents all parameters. The description adds no additional semantics to individual parameters, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'set' and resource 'email forwarding', and explains the action of redirecting incoming mail to another address. It distinguishes from siblings like cipp_set_out_of_office by focusing on forwarding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a caution about data exfiltration and instructs to confirm with the user, but does not provide explicit guidance on when not to use or alternative tools for similar tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds behavioral context: the output is an alignment percentage used for remediation decisions. This adds value beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. Purpose is front-loaded, usage guidance follows. Every word serves a function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input schema (1 optional parameter), no output schema, and annotations covering safety, the description is fully complete. It explains what the tool reports and why it matters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the single parameter well-documented. Description's mention of 'Omit tenantFilter to report on all tenants' essentially repeats the schema description, adding no new semantic meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Report each tenant's alignment percentage against its assigned Standards Templates' and identifies it as the key signal for deciding which standards to promote. This specific verb+resource combination distinguishes it from siblings like cipp_get_tenant_drift or cipp_list_standards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it's the key signal for deciding safe standards to promote, providing clear context. Also notes optional tenantFilter behavior. Lacks explicit exclusions or alternative tool mentions, but context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full behavioral burden. It clearly states the tool checks connectivity, implying no side effects or destructive actions. This is sufficient for a read-only ping tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It conveys the tool's purpose efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no output schema, and a simple function (connectivity check), the description is complete. It provides all necessary context for an agent to understand and use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100%. The description adds no parameter info, which is acceptable since no parameters exist. Baseline of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Check CIPP API connectivity' uses a specific verb (Check) and resource (CIPP API connectivity), clearly indicating a health check operation that distinguishes itself from sibling CRUD tools like create/edit/delete actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives is provided. However, the simple nature of a connectivity check implies it should be used to verify API access before other operations, but this is not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description goes beyond annotations by warning of high impact and tenant modification on next standards run, while annotations only indicate non-readonly and non-destructive. No contradiction. Provides critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence with key warning, action, and instruction, all front-loaded. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description covers purpose, impact, and user confirmation, but lacks details on return value or error states. Still fairly complete for a mutation tool with side effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already provides 100% coverage for the single parameter 'template' with a description that includes the requirement for 'tenantFilter'. Description adds minimal extra meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Creates or updates' a 'CIPP Standards Template' (upsert by GUID), distinguishing it clearly from sibling tools that list, delete, or check standards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to 'Confirm with the user before invoking' and warns about high impact, but does not specify when not to use or mention alternative tools for read-only operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses critical behaviors beyond annotations: it queues a job (not immediate), returns on queue success not completion, is irreversible, and requires opt-in actions. This complements the destructiveHint and readOnlyHint annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loading the destructive warning, and uses two effective sentences to convey purpose and critical caveats without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers key aspects: purpose, destructive nature, opt-in requirement, async behavior, and verification step. It lacks details on expected return structure but that is acceptable given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter well-described. The tool description does not add additional parameter information beyond the schema, warranting the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool queues CIPP's offboarding job for a user, using a specific verb and resource. The destructive warning distinguishes it from sibling tools like cipp_disable_user or cipp_reset_password.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides important usage guidance: every action is opt-in with at least one required, confirm with user before invoking, and the job is only queued so outcome must be verified. However, it does not explicitly compare to alternative tools or give 'when not to use' scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint: true), the description adds critical context: 'Tenants assigned to it lose the standards it enforced.' This explains the downstream consequences of deletion, which is not deducible from annotations alone. The warning emoji and 'HIGH-IMPACT' label further enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences and a warning emoji. The critical information (high-impact, permanent deletion, consequence) is front-loaded. Every sentence serves a purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter delete tool with no output schema, the description fully covers the essential context: what it does, the irreversible nature, and the effect on tenants. No additional information is needed for an agent to correctly invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully describes the only parameter (templateId: 'The GUID of the Standards Template to delete.') with 100% coverage. The description adds no additional semantic value beyond stating 'by ID'; thus, it meets the baseline of 3 for a tool with complete schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action: 'Permanently deletes a CIPP Standards Template by ID.' It clearly distinguishes from sibling tools like cipp_list_standard_templates and cipp_create_standard_template by focusing on deletion and its irreversible impact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear context-specific guideline: 'Confirm with the user before invoking.' This advises the agent on when to use the tool (only after user confirmation). While it doesn't explicitly list alternatives or when not to use, the high-impact warning and sibling tool names imply when other tools are appropriate (e.g., list or create).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations: it notes reversibility ('Reversible by re-enabling the account') and warns 'HIGH-IMPACT,' which is not captured in the destructiveHint or other annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the warning. No unnecessary words. Efficient structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 2 required parameters, no output schema, and simple functionality. The description covers the core action, reversibility, and a usage caveat, which is complete for this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline 3. The description does not add extra meaning to the parameters beyond what the schema provides. The schema itself includes clear descriptions for userId and tenantFilter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Disables a user account, blocking sign-in.' The verb 'disable' and resource 'user account' are specific. It distinguishes from sibling tools like cipp_list_users and cipp_offboard_user.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises 'Confirm with the user before invoking,' which provides a clear usage guideline. However, it does not explicitly mention when not to use this tool or compare to alternatives like cipp_offboard_user for permanent removal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses key behaviors: cross-tenant fan-out with error handling (403 returns as error row instead of failing), default exclusion of Microsoft-built-in apps, and the ability to include them. This is detailed and honest about how the tool operates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph but well-structured, front-loading the core purpose and then adding usage details. Every sentence adds value without redundancy. It is appropriately sized for the information conveyed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description lists the returned fields (appId, displayName, publisher, etc.). It covers purpose, usage, parameters, and special behavior (cross-tenant fan-out, error handling, default exclusions). No critical gaps remain for this tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds meaning beyond the schema: for tenantFilter, it clarifies 'Tenant domain name or ID' and the special value 'allTenants'; for includeBuiltIn, it explains the default (false) and what gets included. This enriches understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists enterprise applications (service principals) in a tenant, including third-party SaaS apps integrated via OAuth. It specifies the returned fields (appId, displayName, etc.). There are no sibling tools with overlapping functionality, so it is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (listing enterprise apps) and how to use it with tenantFilter='allTenants' for cross-tenant fan-out. It also notes the default exclusion of Microsoft-built-in apps and the includeBuiltIn parameter. However, it does not explicitly mention when not to use it or provide alternative tools, but given the context, no direct alternative exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![cipp-mcp MCP server](https://glama.ai/mcp/servers/wyre-technology/cipp-mcp/badges/card.svg)](https://glama.ai/mcp/servers/wyre-technology/cipp-mcp)

Score Badge

Copy to your README.md:

[![cipp-mcp MCP server](https://glama.ai/mcp/servers/wyre-technology/cipp-mcp/badges/score.svg)](https://glama.ai/mcp/servers/wyre-technology/cipp-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/wyre-technology/cipp-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server