Skip to main content
Glama
thebylito

keycloak-mcp-server

by thebylito

Keycloak Model Context Protocol Server

A comprehensive Model Context Protocol (MCP) server for Keycloak administration, providing 80+ tools to manage users, realms, clients, roles, groups, sessions, events, organizations, protocol mappers, user attributes, client scopes, and identity providers directly from AI assistants like Claude Desktop or Cursor AI.

🚀 Features

👤 User Management

  • ✅ Create, update, and delete users

  • ✅ List, search, and get user details

  • ✅ Reset user passwords

  • ✅ Logout user sessions

  • ✅ Manage user roles and groups

  • NEW: User attributes management (critical for organization data)

🏛️ Realm Management

  • ✅ List, create, update, and delete realms

  • ✅ Get detailed realm settings and configurations

  • ✅ Manage realm-level security policies

🔧 Client Management

  • ✅ Register, update, and delete clients/applications

  • ✅ List all clients in realms

  • ✅ Configure client settings and redirect URIs

  • NEW: Protocol mappers management (critical for JWT claims)

🎭 Role Management

  • ✅ Create, update, and delete roles (realm and client-level)

  • ✅ Assign and remove roles from users and groups

  • ✅ List all roles and user role assignments

  • NEW: Composite roles and role hierarchies

  • NEW: Advanced role operations by ID

  • NEW: Find users with specific roles

👥 Group Management

  • ✅ Create, update, and delete user groups

  • ✅ Add and remove users from groups

  • ✅ Manage hierarchical group structures

  • NEW: Group attributes management

  • NEW: Child groups and subgroup management

  • NEW: Group member listing and management

🏢 Organization ManagementNEW

  • ✅ Create, update, and delete organizations

  • ✅ Add and remove organization members

  • ✅ List organizations and members

  • ✅ Organization attributes management

🔗 Identity Provider ManagementNEW

  • ✅ Create, update, and delete identity providers (SSO)

  • ✅ Identity provider mapper management

  • ✅ SAML and OIDC provider configuration

  • ✅ External user attribute mapping

🎯 Client Scopes ManagementNEW

  • ✅ Create, update, and delete client scopes

  • ✅ Protocol mappers for client scopes

  • ✅ Token scope management

📊 Session & Event Management

  • ✅ List active user sessions

  • ✅ Monitor authentication and admin events

  • ✅ Clear event logs and manage session lifecycles

🛡️ Advanced Features

  • Bulletproof authentication with fresh client instances

  • Comprehensive error handling with detailed logging

  • Cross-platform support (Windows, macOS, Linux)

  • Production-ready with TypeScript and robust architecture

  • Organization JWT Claims - Solve organization visibility in tokens

  • 80+ Tools - Complete Keycloak administration coverage

Related MCP server: keycloak-source-mcp

📋 Prerequisites

  • Node.js 18 or higher

  • Running Keycloak instance (local or remote)

  • Keycloak admin credentials with appropriate permissions

  • AI Assistant that supports MCP (Claude Desktop, Cursor AI, etc.)

📦 Installation

npm install -g keycloak-mcp-server

Using NPX (No Installation Required)

npx keycloak-mcp-server

Local Project Installation

npm install keycloak-mcp-server

Local Development

git clone https://github.com/M0-AR/keycloak-mcp-server.git
cd keycloak-mcp-server
npm install
npm run build

⚙️ Configuration

For Cursor AI

Add to your Cursor MCP configuration file (~/.cursor/mcp.json):

{
  "mcpServers": {
    "keycloak": {
      "command": "npx",
      "args": ["keycloak-mcp-server"],
      "env": {
        "KEYCLOAK_URL": "https://your-keycloak-instance.com",
        "KEYCLOAK_ADMIN": "your-admin-username",
        "KEYCLOAK_ADMIN_PASSWORD": "your-admin-password"
      }
    }
  }
}

Option 2: If Installed Globally

{
  "mcpServers": {
    "keycloak": {
      "command": "keycloak-mcp-server",
      "env": {
        "KEYCLOAK_URL": "https://your-keycloak-instance.com", 
        "KEYCLOAK_ADMIN": "your-admin-username",
        "KEYCLOAK_ADMIN_PASSWORD": "your-admin-password"
      }
    }
  }
}

For Claude Desktop

Add to your Claude Desktop configuration:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "keycloak": {
      "command": "npx",
      "args": ["keycloak-mcp-server"],
      "env": {
        "KEYCLOAK_URL": "https://your-keycloak-instance.com",
        "KEYCLOAK_ADMIN": "your-admin-username",
        "KEYCLOAK_ADMIN_PASSWORD": "your-admin-password"
      }
    }
  }
}

🌍 Environment Variables

Variable

Description

Default

Required

KEYCLOAK_URL

The base URL of your Keycloak instance

http://localhost:8080

KEYCLOAK_ADMIN

Admin username

admin

KEYCLOAK_ADMIN_PASSWORD

Admin password

admin

��️ Available Tools (80+ Tools)

👤 User Management Tools

create-user

Creates a new user in a specified realm.

Create a user in "master" realm: username "john.doe", email "john@example.com", first name "John", last name "Doe"

update-user

Updates user information (email, names, enabled status).

Update user "user-id-123" in "master" realm to change email to "newemail@example.com"

delete-user

Deletes a user from a realm.

Delete user with ID "user-id-123" from "master" realm

list-users

Lists all users in a realm.

List all users in the "master" realm

search-users

Search users with filters (username, email, firstName, lastName).

Search for users with email containing "wateen.io" in "master" realm, limit 10 results

get-user

Get detailed information about a specific user.

Get details for user ID "user-id-123" in "master" realm

reset-user-password

Reset a user's password.

Reset password for user "user-id-123" in "master" realm to "newPassword123", make it temporary

logout-user

Logout all sessions for a specific user.

Logout all sessions for user "user-id-123" in "master" realm

set-user-attributesNEW

Set user attributes (critical for organization data storage).

Set organization attribute for user "user-id-123" in "master" realm: {"organization": ["wateen-corp"]}

get-user-attributesNEW

Get user attributes including unmanaged attributes.

Get all attributes for user "user-id-123" in "master" realm

🏛️ Realm Management Tools

list-realms

Lists all available realms.

Show me all available realms in Keycloak

create-realm

Creates a new realm with configurable settings.

Create a new realm called "company" with display name "Company Realm", enabled

update-realm

Updates realm settings and configurations.

Update realm "company" to change display name to "Updated Company"

delete-realm

Deletes an existing realm.

Delete the realm "test-realm"

get-realm-settings

Retrieves detailed settings of a realm.

Get detailed settings for the "master" realm

🔧 Client Management Tools

create-client

Registers a new client/application in a realm.

Create client "my-app" in "master" realm with redirect URIs ["http://localhost:3000/*"]

update-client

Updates client settings (redirect URIs, protocol mappers, etc.).

Update client "my-app" in "master" realm to add new redirect URI "https://app.example.com/*"

delete-client

Removes a client from a realm.

Delete client "old-app" from "master" realm

list-clients

Lists all clients in a realm.

List all clients in the "master" realm

create-protocol-mapperNEW

Create protocol mappers for clients (critical for JWT organization claims).

Create organization group mapper for client "my-app" in "master" realm to include "organization" claim in JWT

update-protocol-mapperNEW

Update existing protocol mappers.

Update protocol mapper "mapper-id-123" for client "my-app" in "master" realm

delete-protocol-mapperNEW

Delete protocol mappers from clients.

Delete protocol mapper "mapper-id-123" from client "my-app" in "master" realm

list-protocol-mappersNEW

List all protocol mappers for a client.

List all protocol mappers for client "my-app" in "master" realm

🎯 Client Scopes Management Tools ⭐ NEW

create-client-scope

Create a new client scope for managing token scopes.

Create client scope "organization-scope" in "master" realm for organization claims

update-client-scope

Update existing client scope.

Update client scope "scope-id-123" in "master" realm to change description

delete-client-scope

Delete a client scope.

Delete client scope "scope-id-123" from "master" realm

list-client-scopes

List all client scopes in a realm.

List all client scopes in the "master" realm

get-client-scope

Get details of a specific client scope.

Get details for client scope "scope-id-123" in "master" realm

create-client-scope-protocol-mapperNEW

Create protocol mappers for client scopes.

Create organization mapper for client scope "organization-scope" in "master" realm

update-client-scope-protocol-mapperNEW

Update protocol mappers in client scopes.

Update protocol mapper "mapper-id-123" in client scope "scope-id-456" in "master" realm

delete-client-scope-protocol-mapperNEW

Delete protocol mappers from client scopes.

Delete protocol mapper "mapper-id-123" from client scope "scope-id-456" in "master" realm

list-client-scope-protocol-mappersNEW

List protocol mappers for a client scope.

List all protocol mappers for client scope "scope-id-123" in "master" realm

🏢 Organization Management Tools ⭐ NEW

create-organization

Create a new organization.

Create organization "wateen-corp" with description "Wateen Corporation" in "master" realm

update-organization

Update existing organization.

Update organization "org-id-123" in "master" realm to change name to "Updated Corp"

delete-organization

Delete an organization.

Delete organization "org-id-123" from "master" realm

list-organizations

List all organizations in a realm.

List all organizations in "master" realm with search "wateen", limit 10

get-organization

Get details of a specific organization.

Get details for organization "org-id-123" in "master" realm

add-organization-member

Add a user to an organization.

Add user "user-id-123" to organization "org-id-456" in "master" realm

remove-organization-member

Remove a user from an organization.

Remove user "user-id-123" from organization "org-id-456" in "master" realm

list-organization-members

List all members of an organization.

List all members of organization "org-id-123" in "master" realm, limit 20

🎭 Role Management Tools

create-role

Creates roles at realm or client level.

Create a realm role "manager" with description "Manager role" in "master" realm

update-role

Modifies role attributes.

Update role "manager" in "master" realm to change description to "Updated manager role"

delete-role

Deletes roles.

Delete role "old-role" from "master" realm

list-roles

Lists all roles in a realm.

List all roles in the "master" realm

list-client-roles

Lists all roles defined on a client.

List all roles on client "my-app" in "master" realm

assign-role-to-user

Assigns a realm role to a user, or a client role when clientId is supplied.

Assign role "manager" to user "user-id-123" in "master" realm
Assign client role "editor" on client "my-app" to user "user-id-123" in "master" realm

remove-role-from-user

Removes a realm role from a user, or a client role when clientId is supplied.

Remove role "manager" from user "user-id-123" in "master" realm
Remove client role "editor" on client "my-app" from user "user-id-123" in "master" realm

get-user-roles

Gets all roles directly assigned to a user, as { realmMappings, clientMappings }.

Get all roles for user "user-id-123" in "master" realm

create-composite-roleNEW

Create composite roles (role hierarchies).

Create composite role from "parent-role-id" with child roles ["child-role-1", "child-role-2"] in "master" realm

get-composite-rolesNEW

Get composite roles for a role.

Get composite roles for role "role-id-123" in "master" realm, limit 10

delete-composite-rolesNEW

Delete composite roles from a role.

Remove composite roles ["child-role-1", "child-role-2"] from role "parent-role-id" in "master" realm

get-role-by-idNEW

Get role details by ID.

Get role details for role ID "role-id-123" in "master" realm

update-role-by-idNEW

Update role by ID.

Update role "role-id-123" in "master" realm to change name to "new-role-name"

delete-role-by-idNEW

Delete role by ID.

Delete role with ID "role-id-123" from "master" realm

find-users-with-roleNEW

Find users with a realm role, or with a client role when clientId is supplied.

Find all users with role "manager" in "master" realm, limit 20
Find all users with client role "editor" on client "my-app" in "master" realm

assign-role-to-groupNEW

Assign a realm role to a group, or a client role when clientId is supplied.

Assign role "developer" to group "group-id-123" in "master" realm
Assign client role "editor" on client "my-app" to group "group-id-123" in "master" realm

remove-role-from-groupNEW

Remove a realm role from a group, or a client role when clientId is supplied.

Remove role "developer" from group "group-id-123" in "master" realm

get-group-rolesNEW

Get roles assigned to a group, as { realmMappings, clientMappings }.

Get all roles for group "group-id-123" in "master" realm

list-available-group-rolesNEW

List roles a group could still be granted — realm roles, or a client's roles when clientId is supplied.

List available roles for group "group-id-123" in "master" realm

list-composite-group-rolesNEW

List composite roles for a group.

List composite roles for group "group-id-123" in "master" realm

👥 Group Management Tools

create-group

Creates user groups.

Create a group called "developers" in "master" realm

update-group

Updates group attributes.

Update group "group-id-123" in "master" realm to change name to "senior-developers"

delete-group

Deletes groups.

Delete group "group-id-123" from "master" realm

list-groups

Lists all groups in a realm.

List all groups in the "master" realm

manage-user-groups

Adds or removes users from groups.

Add user "user-id-123" to group "group-id-456" in "master" realm

set-group-attributesNEW

Set group attributes (organization metadata).

Set organization attributes for group "group-id-123" in "master" realm: {"department": ["engineering"]}

get-group-attributesNEW

Get group attributes.

Get all attributes for group "group-id-123" in "master" realm

create-child-groupNEW

Create a child group (subgroup).

Create child group "junior-devs" under parent group "group-id-123" in "master" realm

list-sub-groupsNEW

List subgroups of a parent group.

List subgroups of parent group "group-id-123" in "master" realm, limit 10

list-group-membersNEW

List members of a group.

List all members of group "group-id-123" in "master" realm, limit 20

🔗 Identity Provider Management Tools ⭐ NEW

create-identity-provider

Create a new identity provider for SSO integration.

Create SAML identity provider "company-saml" in "master" realm with SSO URL and certificate

update-identity-provider

Update an existing identity provider.

Update identity provider "company-saml" in "master" realm to change display name

delete-identity-provider

Delete an identity provider.

Delete identity provider "old-saml" from "master" realm

list-identity-providers

List all identity providers in a realm.

List all identity providers in "master" realm

get-identity-provider

Get details of a specific identity provider.

Get details for identity provider "company-saml" in "master" realm

create-identity-provider-mapper

Create a mapper for identity provider (external user mapping).

Create user attribute mapper for identity provider "company-saml" in "master" realm

update-identity-provider-mapper

Update an identity provider mapper.

Update mapper "mapper-id-123" for identity provider "company-saml" in "master" realm

📊 Session & Event Management Tools

list-sessions

Lists all active sessions in a realm.

List all active sessions in "master" realm

get-user-sessions

Lists active sessions for a specific user.

Get active sessions for user "user-id-123" in "master" realm

list-events

Retrieves authentication and admin events.

List last 10 events in "master" realm

clear-events

Clears event logs.

Clear all events in "master" realm

🧪 Testing & Development

Testing with MCP Inspector

npx @modelcontextprotocol/inspector npx keycloak-mcp-server

Visit http://localhost:6274 to test all 80+ tools interactively.

Local Development

npm run watch    # Auto-rebuild on changes
npm run dev     # Test server directly

Stress Testing

The server has been stress-tested with 80+ consecutive operations without authentication failures, demonstrating production-level reliability.

🔧 Architecture

Bulletproof Authentication System

  • Fresh Client Instances: Creates new KcAdminClient for every request

  • Retry Logic: Exponential backoff with 2 attempts maximum

  • Connection Management: 15-second timeout with proper cleanup

  • Error Handling: Comprehensive error messages for all scenarios

TypeScript Implementation

  • Type Safety: Full TypeScript coverage with proper interfaces

  • Error Handling: Detailed error messages and logging

  • Modular Design: Clean separation of concerns

📈 Production Ready

This package has been extensively tested and validated:

  • 80+ consecutive operations without authentication failures

  • Cross-realm operations working seamlessly

  • Parallel tool execution supported

  • Complex search queries with multiple filters

  • Error recovery and detailed logging

  • TypeScript compilation with zero errors

  • Complete Keycloak API coverage with organization management

🎯 JWT Organization Problem Solved

This package specifically addresses the common JWT organization problem:

  • User Attributes: Store organization data in user attributes

  • Protocol Mappers: Create mappers to include organization in JWT tokens

  • Client Scopes: Manage token scopes for organization claims

  • Organizations: Full organization lifecycle management

  • Group Attributes: Store organization metadata in groups

Example workflow:

  1. Create organization using create-organization

  2. Set user organization attribute using set-user-attributes

  3. Create protocol mapper using create-protocol-mapper to include organization in JWT

  4. Add user to organization using add-organization-member

🔒 Security Best Practices

  • Use environment variables for credentials

  • Enable HTTPS for production Keycloak instances

  • Use strong admin passwords

  • Regularly rotate credentials

  • Monitor admin events and sessions

🤝 Contributing

  1. Fork the repository

  2. Create a feature branch: git checkout -b feature/amazing-feature

  3. Commit changes: git commit -m 'Add amazing feature'

  4. Push to branch: git push origin feature/amazing-feature

  5. Open a Pull Request

📄 License

MIT License - see LICENSE file for details.

🆘 Support

📊 Package Stats

  • 80+ Tools: Complete Keycloak administration coverage

  • Production Ready: Extensively tested and validated

  • TypeScript: Full type safety and modern development experience

  • Cross-Platform: Windows, macOS, and Linux support

  • Zero Dependencies Issues: Robust dependency management

  • Organization Management: Solve JWT organization visibility problems

  • Advanced Features: Protocol mappers, client scopes, identity providers


Made with ❤️ for the Keycloak and AI community

Available Tools

86 tools
add-organization-memberC

Add a user to an organization

ParametersJSON Schema
NameRequiredDescriptionDefault
orgIdYesOrganization ID
realmYesRealm name
userIdYesUser ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must carry the full burden of behavioral disclosure. It does not mention side effects, permission requirements, idempotency, error behavior, or what happens if the user is already a member. The description only states the action, providing minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundant words. It is front-loaded and direct. However, it is slightly under-specified for the dimensions it covers, but concise formatting is not the issue.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description lacks essential context such as whether the user must already exist, if the operation is idempotent, or what the response indicates. The simple schema does not make up for the missing behavioral and usage details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (orgId, realm, userId each have descriptions), so the baseline is 3. The description adds no additional meaning beyond the schema, such as the order of parameters or relationships between them. It is adequate but does not compensate further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'add' and the resource 'a user to an organization', which matches the tool name. It distinguishes from sibling tools like remove-organization-member and list-organization-members. However, it lacks any additional scope or context (e.g., required role or realm behavior) that would make it a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., user must exist), exclusions, or recommended usage scenarios. The description simply states the action without contextual cues.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

assign-role-to-groupA

Assign a realm role, or a client role when clientId is given, to a group

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID
clientIdNoClient ID to resolve roleName as a client role. Omit for a realm role
roleNameYesRole name

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description only states the assignment action without disclosing behavioral traits such as idempotency, permission requirements, or response behavior. The clientId vs realm role distinction is parameter semantics, not behavioral transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One clear, front-loaded sentence that fully conveys the core purpose and the key conditional. No filler or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple assignment operation, the description covers the core purpose with 4 parameters and no output schema. However, it omits behavioral details like idempotency or expected response status, which an agent might need to know to confirm success.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description reiterates the clientId/realm role distinction but adds no new meaning beyond the schema; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (assign), the resource (a role to a group), and distinguishes between realm and client roles via the clientId condition. It also differentiates from sibling tools like assign-role-to-user by specifying the target is a group.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly explains when to use clientId (for client roles) vs omit it (for realm roles), providing clear context for parameter usage. It does not explicitly mention alternatives or exclusions, but the target audience (a group vs a user) is implicitly clear from the tool name and sibling set.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

assign-role-to-userA

Assign a realm role, or a client role when clientId is given, to a user

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID
clientIdNoClient ID to resolve roleName as a client role. Omit for a realm role
roleNameYesRole name

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden of behavioral disclosure. It does add context about the dual realm/client role behavior, but it does not mention permissions, idempotency, what happens if the role already exists, or the response format. This is a clear but basic mutation tool; more transparency would be valuable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of about 15 words, front-loaded with the action and resource. It is optimally concise with no filler or redundant detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 4-parameter assignment tool with no output schema, the description covers the essential behavior and the one conditional parameter (clientId). It could mention response details or error conditions, but given the low complexity, it is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so per the rubric the baseline is 3. The description adds no new parameter semantics beyond what the schema already provides, and the clientId explanation mirrors the schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Assign a realm role, or a client role when clientId is given, to a user'. It uses a specific verb ('assign') and identifies the resource (role) and recipient (user). It also distinguishes between realm and client roles, which differentiates it from sibling tools like remove-role-from-user or assign-role-to-group.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context by specifying that clientId is used for client roles, otherwise a realm role is assigned. However, it does not explicitly mention alternatives or when not to use this tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clear-eventsC

Clear event logs

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavioral traits. 'Clear' implies a destructive action, but the description fails to state whether the action is irreversible, the exact scope (all events vs. filtered), permission requirements, or side effects. For a destructive operation, this is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at three words, with no filler or unnecessary detail. It is front-loaded and easily parsed. However, the brevity contributes to under-specification, but conciseness as a dimension is well-handled.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema and annotations, the description is the only source of context. It fails to explain what 'clear' operationally entails—whether it deletes all events, is permanent, or is scoped to the realm. This is inadequate for a destructive tool, even with low parameter complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for the single required parameter 'realm', described as 'Realm name'. The description adds no additional meaning beyond the schema, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Clear event logs' uses a specific verb ('Clear') and resource ('event logs'), making the primary purpose unambiguous. It distinguishes from sibling 'list-events' and other read operations, though it lacks an explicit scope (e.g., 'in the realm') beyond what the schema implies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as 'list-events' or other deletion tools. There is no mention of when it is appropriate, what prerequisites exist, or when another tool would be preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-child-groupC

Create a child group (subgroup)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesGroup name
realmYesRealm name
attributesNoGroup attributes
parentGroupIdYesParent group ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral disclosure. It only says 'Create a child group' without mentioning permissions, parent group validation, idempotency, or side effects. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no wasted words. It clearly states the action and resource without any redundant elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, no annotations, and a one-line description, the tool lacks essential context for an agent: no return format, no side effects, no edge cases (e.g., duplicate names, invalid parent group), and no guidance on handling the nested attributes parameter. This makes it incomplete for a reliable call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with descriptive parameter names and summaries. The tool description adds no extra meaning beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action (create) and the resource (child group/subgroup), and the qualifier 'child' distinguishes it from the sibling 'create-group'. It could be more explicit about the parentGroupId requirement, but the core purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given on when to use this tool vs. alternatives like create-group. The word 'child' implies subgroup creation, but there is no mention of prerequisites, exclusions, or conditions that would help an agent choose this tool over others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-clientB

Register a new client/application in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoClient name
realmYesRealm name
enabledNoEnabled status
clientIdYesClient ID
descriptionNoClient description
publicClientNoPublic client
redirectUrisNoRedirect URIs

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for disclosing behavioral traits. It only states the action ('Register') without mentioning permissions, idempotency, side effects, or error behavior. For a mutation tool, this lack of detail leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the essential purpose without extraneous words. It is concise and well-structured, making it easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite a rich schema (7 params), there is no output schema and no annotations. The description only covers the basic creation action, omitting details like prerequisites, idempotency, return values, or potential errors. For a create operation with moderate complexity, this is inadequate for full contextual completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides descriptions for all 7 parameters, giving 100% coverage. The tool description adds no additional parameter-level context beyond 'client/application in a realm', so it neither compensates for nor detracts from the schema's already sufficient parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Register' and identifies the resource as 'client/application in a realm', clearly distinguishing it from siblings like update-client, delete-client, and list-clients. It unambiguously states the tool's function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for creating clients, but it does not explicitly state when to use it vs. alternatives such as update-client or list-clients. No exclusions or alternative tool references are provided, so guidance is only implied by the verb 'Register'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-client-scopeB

Create a new client scope (CRITICAL for managing token scopes)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesClient scope name
realmYesRealm name
protocolNoProtocol (e.g., openid-connect)
attributesNoClient scope attributes
descriptionNoClient scope description

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the full burden of disclosing behavior. It only states that it creates a client scope and hints at its role in token scopes, but does not mention side effects, authorization requirements, idempotency, or error behavior (e.g., duplicate name). This is a significant gap for a create operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence and front-loaded with the core action. The parenthetical 'CRITICAL for managing token scopes' adds a bit of context but could be seen as slightly dramatic; still, it is concise and no words are wasted. It earns a 4 for efficiency.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with 5 parameters (2 required), no output schema, and no annotations, the description is under-specified. It gives no information about return values, prerequisites (e.g., realm must exist), or special constraints. An agent would need to infer too much about how to correctly invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (all 5 parameters have descriptions). The description text adds no additional meaning beyond the schema, so the baseline of 3 applies. It does not clarify required fields, uniqueness constraints, or format expectations beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action: 'Create a new client scope'. The verb 'create' and resource 'client scope' are specific, and it distinguishes itself from sibling tools like update-client-scope and delete-client-scope by focusing on creation. The parenthetical adds relevant context about token scopes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It does not mention that this is for new scopes, that update-client-scope should be used for modifications, or any prerequisites such as an existing realm. The sibling list exists but the description does not reference it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-client-scope-protocol-mapperC

Create a protocol mapper for a client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProtocol mapper name
realmYesRealm name
configNoProtocol mapper configuration
protocolYesProtocol (e.g., openid-connect)
clientScopeIdYesClient scope ID
protocolMapperYesProtocol mapper type

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of disclosing side effects. It only states 'create' without revealing whether the operation is idempotent, whether it overwrites existing mappers, required permissions, or any consequences. This is a minimal disclosure for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, which is concise. However, it does not earn its place by adding valuable information beyond the tool name; it simply rephrases it. The structure is not problematic, but the content is so minimal that it borders on under-specification rather than effective conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 6 parameters, no output schema, and no annotations, the description is severely incomplete. It does not explain the protocol mapper concept, the purpose of the 'config' nested object, or any return value or errors. An agent would struggle to use this tool correctly without additional context from documentation or examples.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all parameters, each with a short description. The tool description adds no additional meaning beyond the schema, so the baseline of 3 applies. The 'config' object is mentioned in the schema but not elaborated, and the description does not compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a protocol mapper for a client scope' restates the tool name in sentence form, providing little additional clarity. It does specify the resource (client scope protocol mapper) but does not distinguish it from the similar create-protocol-mapper tool or explain what a protocol mapper is. The verb and resource are clear, but the description is essentially a tautology of the tool name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like create-protocol-mapper (for clients) or update-client-scope-protocol-mapper. The description does not mention any conditions or exclusions. Without context, the agent cannot determine the appropriate scenario for this tool compared to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-composite-roleC

Create composite roles (role hierarchies)

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
rolesYesArray of roles to compose
roleIdYesRole ID

TDQS

C2.7/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It merely states the action and a synonym for composite; it does not mention permissions, side effects, idempotency, prerequisites, or what happens to existing roles. There is no added context beyond the tool's name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words, making it concise and front-loaded. However, it is arguably under-specified, so it loses a point for not balancing brevity with necessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex tool creating role hierarchies, the description and schema provide minimal context. There is no explanation of the relationship between roleId and roles, whether roles must already exist, or what the resulting composite role structure looks like. No output schema or annotations compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds no parameter-specific meaning; 'role hierarchies' is a general concept but does not clarify how 'roleId' and 'roles' interact.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Create composite roles', and adds a clarifying parenthetical 'role hierarchies' that distinguishes it from simple role creation. However, it doesn't explicitly differentiate from sibling tools like 'create-role' or point to alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as 'create-role' or 'update-role'. The phrase 'role hierarchies' implies a use case but it is not explicit about when to choose this over related role tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-groupB

Create user groups

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesGroup name
realmYesRealm name
parentIdNoParent group ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description offers no additional behavioral context. It doesn't disclose permission requirements, the significance of 'parentId' (e.g., whether it creates a subgroup), or any side effects beyond the creation itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence that immediately conveys the purpose without unnecessary elaboration. It earns its place by being clear and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a simple create operation and the schema covers parameters fully. However, the description doesn't explain the optional 'parentId' relationship to sibling tool 'create-child-group', nor what the tool returns, making it only minimally complete for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for all 3 parameters, covering 100% of them. The description adds no extra semantics or usage tips beyond what the schema states, which is acceptable given the high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create user groups' clearly states the action (create) and the resource (user groups), making the tool's function obvious. However, it doesn't differentiate from the sibling tool 'create-child-group', leaving ambiguity about whether this handles top-level groups only.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'create-child-group' or 'create-role'. The description lacks any exclusions or contextual hints for choosing this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-identity-providerC

Create a new identity provider for SSO integration

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name
configNoIdentity provider configuration
enabledNoEnabled status
linkOnlyNoLink only
providerIdYesProvider ID (e.g., saml, oidc)
storeTokenNoStore token
trustEmailNoTrust email
displayNameNoDisplay name
addReadTokenRoleOnCreateNoAdd read token role on create
firstBrokerLoginFlowAliasNoFirst broker login flow alias

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden but only says 'create' with no detail on side effects, prerequisites, default values, error handling, or idempotency. It does not mention whether a realm must exist or what happens on duplicate aliases, leaving significant behavioral ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word adds context, making it appropriately concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 11 parameters, no annotations, and no output schema, this one-sentence description is inadequate. It omits return format, required preconditions, and usage scenarios, leaving the agent without enough context to use the tool correctly in a complex operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 11 parameters, so the schema does the heavy lifting. The description adds general SSO context but does not enrich parameter meanings beyond what is already in the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (create) and resource (identity provider) with an SSO integration scope. However, it does not distinguish from the sibling tool import-identity-provider-from-url, which also creates an identity provider but via URL import.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like import-identity-provider-from-url or when to prefer updating an existing provider. There are no mentions of prerequisites, exclusions, or contextual triggers, leaving the agent without selection support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-identity-provider-mapperC

Create a mapper for identity provider (external user mapping)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesMapper name
aliasYesIdentity provider alias
realmYesRealm name
configNoMapper configuration
identityProviderMapperYesMapper type

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility for behavioral disclosure. It only states a creation action, but does not reveal side effects, required permissions, what happens if the mapper exists, whether the operation is idempotent, or what the 'config' object should contain. This is insufficient for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that immediately conveys the core purpose. It is appropriately sized, though it could be expanded with a second sentence to offer practical guidance without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description lacks essential context for a create operation. It does not reference the list-identity-provider-mapper-types sibling for valid mapper types, does not explain the config object structure, and provides no information about the response or error conditions. With no output schema and no annotations, the description is too thin to fully support an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema documents all parameters adequately. The description adds only a small semantic hint ('external user mapping') but does not elaborate on the 'config' object or the possible values for 'identityProviderMapper'. Baseline 3 is appropriate because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb ('Create') and specific resource ('mapper for identity provider'), with a parenthetical explaining it's for external user mapping. It clearly distinguishes from sibling tools like update-identity-provider-mapper and delete-identity-provider-mapper, though it could be slightly more explicit that it creates a new mapper.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. There is no mention of the relationship to list-identity-provider-mapper-types, which would be useful for selecting the correct mapper type, nor any prerequisites like needing an existing identity provider alias.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-organizationB

Create a new organization (CRITICAL for organization management)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesOrganization name
realmYesRealm name
enabledNoOrganization enabled status
attributesNoOrganization attributes
descriptionNoOrganization description

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits, but it only states the basic action. It does not mention side effects, return values, required parent resources (e.g., an existing realm), idempotency, or permission requirements. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise at one sentence, though the 'CRITICAL for organization management' phrase is not essential. It remains front-loaded and does not contain wasteful content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters, nested objects, and no output schema, the description is under-specified. It does not explain success responses, error conditions, or preconditions, leaving the agent without adequate context for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of parameters with individual descriptions. The tool description adds no extra meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a new organization' with a specific verb and resource, distinguishing it from sibling operations like update, delete, or list. The parenthetical emphasizing criticality adds urgency but does not obscure purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is provided for when to use this tool over alternatives. Usage is implied by the name and the create/update/delete pattern among sibling organization tools, but no prerequisites, exclusions, or conditions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-protocol-mapperA

Create a protocol mapper for a client (CRITICAL for JWT organization claims)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProtocol mapper name
realmYesRealm name
configNoProtocol mapper configuration
clientIdYesClient ID
protocolYesProtocol (e.g., openid-connect)
protocolMapperYesProtocol mapper type (e.g., oidc-group-membership-mapper)

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states the action and a vague 'CRITICAL' note, but does not disclose side effects (e.g., token impact), permissions required, or whether existing mappers are overwritten. This is insufficient for a creation operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with a parenthetical emphasis note. It is concise and front-loaded with the essential action, earning its place without clutter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (6 parameters, nested config object, no output schema), the description is incomplete. It does not hint at return values, the expected structure of 'config', or any prerequisites for the protocol mapper. The 'CRITICAL' note is vague and does not compensate for these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (each parameter has a description), so the baseline is 3. The description does not add extra semantic detail beyond the schema; config is only named as 'Protocol mapper configuration' without further elaboration, but the schema itself covers the basic meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (create) and resource (protocol mapper for a client), using a specific verb and object. It distinguishes from related sibling tools like create-client-scope-protocol-mapper by specifying 'for a client'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for a client' provides clear context that this is the client-level mapper creation tool, distinguishing it from client-scope mappers. The 'CRITICAL for JWT organization claims' hint gives a use case, though no explicit alternatives or exclusions are listed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-realmB

Create a new realm with configurable settings

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
enabledNoEnabled status
displayNameNoDisplay name

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only says 'create' and does not disclose side effects, uniqueness constraints, permission requirements, or error behavior if the realm already exists. This is insufficient for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, short sentence with no redundant words. It is concise, though it sacrifices some specificity for brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description does not explain return values, failure conditions, or prerequisites for creating a realm. For a create operation, this is a significant gap beyond the bare action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents all three parameters with descriptions (100% coverage), so the description adds little beyond labeling them as 'configurable settings'. The baseline of 3 applies because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create') and resource ('realm'), and 'new' distinguishes it from update-realm/delete-realm. However, 'configurable settings' is vague and does not specify which settings, though the core purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the word 'new' – use when creating a realm. There is no explicit mention of when not to use or alternatives, but sibling names like update-realm and delete-realm provide context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-roleA

Create roles at realm or client level

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdNoClient ID for client roles
roleNameYesRole name
descriptionNoRole description

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so description carries the full burden. It mentions realm/client level scope but does not disclose permissions, duplicate behavior, idempotency, or return results. For a mutation tool, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no fluff, directly states purpose. Perfectly concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple create tool with well-documented schema, but lacks return value details and side-effect information (e.g., duplicate roles, permissions). Given no output schema and no annotations, more would be expected for full completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% description coverage, with all parameters meaningfully described (realm, clientId, roleName, description). The description adds no new semantic value beyond restating 'realm or client level', which is already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it creates roles, with explicit scope at realm or client level. The verb 'Create' and resource 'roles' are specific, and the realm/client distinction distinguishes it from role management tools like update-role or delete-role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implied usage for creating simple roles, but no explicit guidance on when to choose this over create-composite-role or how to scope realm vs client. Sibling tools include create-composite-role, yet no differentiation is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create-userB

Create a new user in a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
emailYesEmail address
realmYesRealm name
lastNameYesLast name
usernameYesUsername
firstNameYesFirst name

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden for behavioral disclosure. It only states the action and scope, but doesn't mention required permissions, potential conflicts (e.g., duplicate username), or what happens on error. This is insufficient for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no filler words, making it easy to parse quickly. It avoids redundancy and is appropriately front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is too terse for a 5-parameter create operation with no annotations or output schema. It lacks critical context like whether realm must exist, whether duplicate usernames are rejected, or what the response contains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers all 5 parameters with short descriptions, so the baseline is 3. The description adds no additional parameter semantics—it merely mentions 'specific realm' which maps directly to the realm parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Create') and resource ('user') plus a scope ('in a specific realm'), clearly distinguishing it from siblings like update-user, delete-user, and list-users. It leaves no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for creating a user within a realm but doesn't explicitly state when to use it instead of related tools, nor does it mention prerequisites like the realm needing to exist. The context is clear but not fully developed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-clientB

Remove a client from a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdYesClient ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action but does not communicate that the deletion is permanent, destructive, or any potential side effects on associated data. This is a significant gap for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It directly conveys the core purpose without fluff, earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool is a simple delete operation with 2 parameters and no output schema, the description is minimally adequate but lacks critical behavioral context. With no annotations, it should disclose the destructive nature, reversibility, or error handling, but it does not. This leaves the agent under-informed for a potentially impactful operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters (realm and clientId) are already self-explanatory. The description 'from a realm' hints at the realm parameter but adds no new meaning beyond what the schema provides. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Remove a client from a realm' uses a specific verb ('Remove') and resource ('client'), clearly distinguishing it from sibling tools that delete other entities like users or roles. It fully states the action and scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. The context signals show many sibling delete tools, but no comparison or explicit usage context is offered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-client-scopeC

Delete a client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientScopeIdYesClient scope ID

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only says 'Delete', implying destructiveness, but does not mention if the deletion is permanent, cascades to related entities, or requires special permissions. No side effects or error behavior are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words, but it is under-specified. While concise, it lacks substantive details needed to understand the tool's behavior, making it minimally acceptable in structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with two well-documented parameters, the description is adequate but not complete. It provides no context about side effects, prerequisites, or response expectations, and with no annotations or output schema, the agent is missing important information for a destructive operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents both parameters (realm, clientScopeId) with 100% coverage. The description adds no additional semantic content about the parameters, so it meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a client scope' is a direct restatement of the tool name, providing no additional nuance or distinction beyond the verb+resource. It does not add detail about what a client scope is or how this deletion differs from deleting other resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No information is provided about when to use this tool, prerequisites, or alternative approaches. The description simply states the action without any context about use cases, permanence, or safer alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-client-scope-protocol-mapperB

Delete a protocol mapper from a client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
mapperIdYesProtocol mapper ID
clientScopeIdYesClient scope ID

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the action 'delete' without mentioning permanence, required permissions, or potential side effects. This is insufficient for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence: 'Delete a protocol mapper from a client scope'. It is front-loaded, contains no unnecessary words, and is appropriately sized for the simple operation it describes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a delete tool with no annotations and no output schema, the description is too minimal. It fails to provide crucial context about irreversibility, prerequisites (e.g., the mapper must exist), or any safety warnings. The simple operation still needs more behavioral context than one sentence.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with each of the three parameters (realm, clientScopeId, mapperId) having a brief explanation. The description adds no additional parameter semantics beyond what the schema already provides, aligning with the baseline for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is specific: 'Delete a protocol mapper from a client scope' clearly states the verb (delete) and resource (protocol mapper) with the scope (client scope), distinguishing it from sibling tools like delete-protocol-mapper or delete-client-scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by specifying the exact resource and scope, but it does not explicitly contrast with alternatives such as delete-protocol-mapper or explain when this tool should be preferred. No when-not-to-use guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-composite-rolesC

Delete composite roles from a role

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
rolesYesArray of roles to remove
roleIdYesRole ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must disclose behavioral traits. It only says 'Delete composite roles from a role' and does not clarify whether this destroys the composite role entities or just their associations, nor does it mention permissions, reversibility, or side effects. The ambiguity around what is actually deleted is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no filler words. Every word contributes to the core meaning, making it highly efficient and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema and no annotations, and the description is extremely brief. It lacks essential context such as prerequisites (e.g., the role must exist), how the 'roles' array should be structured, and what the tool returns. The presence of many sibling role-management tools increases the need for clearer contextual guidance, which is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (all parameters have descriptions), so the baseline is 3. The description does not elaborate on parameter meaning beyond the schema. The schema's 'Array of roles to remove' is ambiguous, but the description's 'from a role' adds a small clarifier that the removal is scoped to a role. No additional details about role IDs or names are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete composite roles from a role' clearly identifies the action (delete) and the resource (composite roles), and the phrase 'from a role' differentiates it from deleting the role itself. However, it does not explicitly distinguish it from sibling tools like 'delete-role' or 'delete-role-by-id', which could cause confusion in edge cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention that this is specifically for removing composite role assignments from a role, nor does it advise using 'delete-role' for full role deletion. Sibling tools exist, but no exclusions or recommended contexts are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-groupC

Delete groups

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID

TDQS

C2.1/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must disclose behavioral traits. It only says 'Delete groups' with no mention of side effects, required permissions, irreversibility, or impact on related entities like group members, sub-groups, or role mappings. This is a severe gap for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short, but it is under-specified rather than appropriately concise. While there is no fluff, the lack of necessary context prevents it from earning full marks for structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has two parameters and no output schema, but the description does not explain deletion behavior such as cascade effects, authorization requirements, or error conditions. The description is inadequate for an agent to fully understand the tool's impact.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters realm and groupId are already documented. The description adds no additional meaning beyond the schema, which is the baseline expectation for full coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete groups' is essentially a restatement of the tool name 'delete-group', adding no new information. It does specify a verb and resource, but fails to differentiate from sibling tools beyond what the name already implies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, no prerequisites, and no context about typical use cases. It simply states the action without any situational framing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-identity-providerC

Delete an identity provider

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name

TDQS

C2.7/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Delete an identity provider' but does not mention any side effects, prerequisites, reversibility, or impact on associated mappers. This is a significant gap for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, extremely brief and front-loaded. However, it repeats the tool name without adding value, so it doesn't fully 'earn its place,' though it remains concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with two parameters and no output schema, the description might be minimally sufficient, but because it is a delete operation with no mention of consequences or related resources (e.g., mappers), it feels incomplete. The tool would benefit from noting impact or prerequisite steps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for both parameters ('Identity provider alias' and 'Realm name'). The description adds no extra meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('an identity provider'), making the purpose unambiguous. It does not explicitly differentiate from sibling tools like 'delete-identity-provider-mapper', but the identity provider vs. mapper distinction is implicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool over alternatives, or any context about when deletion is appropriate. It simply restates the operation without offering usage scenarios or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-identity-provider-mapperC

Delete an identity provider mapper

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name
mapperIdYesMapper ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility. It only states the deletion action without disclosing consequences, reversibility, required permissions, or side effects—a significant gap for a destructive tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with zero waste. However, it is essentially a restatement of the tool name and adds little value beyond the name itself, making it efficient but minimal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple deletion tool with no annotations and no output schema, the description lacks behavioral context such as idempotency, side effects, prerequisites, or post-conditions. This makes it incomplete for an agent to fully understand the impact.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all three parameters (realm, alias, mapperId) with descriptions, providing 100% coverage. The description adds no extra parameter meaning, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete an identity provider mapper' with a specific verb and resource, making the purpose unambiguous. It distinguishes from sibling tools like update/get/list by the delete action, though it lacks additional scoping details.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. There is no mention of prerequisites, alternatives, or conditions under which deletion should occur.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-organizationB

Delete an organization

ParametersJSON Schema
NameRequiredDescriptionDefault
orgIdYesOrganization ID
realmYesRealm name

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations available, the description carries the full burden of behavioral disclosure. It only says 'Delete an organization' without indicating irreversibility, cascading effects on members or associated data, or permission requirements. The destructive nature is implied but not elaborated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no redundancy or fluff. It is concise, though minimal; the lack of elaboration is more a completeness issue than a conciseness issue.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive operation with no annotations and no output schema, the description should provide more context about consequences, side effects, or scope of deletion. The current description is adequate only for a trivial understanding of the tool's action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters (orgId and realm) already have clear descriptions. The tool description adds no further semantic detail, but the schema adequately documents the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (delete) and resource (organization), which directly indicates the tool's function. It distinguishes from sibling tools that target other resources like realms, users, or clients.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It merely states the action without mentioning prerequisites, exclusions, or situational context that would help an agent decide to invoke this vs. other delete tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-protocol-mapperA

Delete a protocol mapper from a client

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdYesClient ID
mapperIdYesProtocol mapper ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of disclosing behavioral traits. It clearly states the destructive action and what is deleted, but lacks details on reversibility or required permissions. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no wasted words, effectively front-loading the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete operation with three straightforward parameters, the description is mostly complete. The only gap is the lack of explicit differentiation from similar siblings, but the client-level context provides sufficient clarity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes all three parameters with complete coverage (100%), and the description adds no additional parameter meaning beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Delete' and clearly identifies the resource as 'a protocol mapper from a client', effectively differentiating it from sibling tools like delete-client-scope-protocol-mapper.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'from a client' provides clear context that this tool is for client-level protocol mappers, implying when to use it. However, it does not explicitly mention alternatives or exclusions, such as not using it for client-scope protocol mappers.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-realmB

Delete an existing realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden. It fails to disclose that deletion is destructive, irreversible, may cascade to related resources, or require special permissions. It adds no behavioral information beyond the tool's name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler or redundancies. It is efficiently concise for a simple delete operation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive operation with no annotations and no output schema, the description lacks essential context about irreversibility, preconditions, or side effects. It is too sparse to fully inform an agent about the tool's implications.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with 'realm' described as 'Realm name'. The description adds no additional parameter context, but the schema already fully documents the only param, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Delete an existing realm' with a specific verb and resource, directly distinguishing it from sibling deleters like delete-user and delete-client. The purpose is unambiguous and complete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool vs alternatives, prerequisites, or consequences. The bare statement 'Delete an existing realm' gives no context for decision-making beyond the obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-roleC

Delete roles

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdNoClient ID for client roles
roleNameYesRole name

TDQS

C2/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it reveals nothing beyond the name. It does not mention side effects (e.g., cascading removal from composites), reversibility, or permissions, and the plural 'roles' is misleading given a singular roleName.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is under-specified, consisting of a two-word phrase that restates the tool name. While short, it lacks any structural detail that would help an agent understand scope or usage, making it minimally viable at best.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a delete tool with an optional clientId and no output schema or annotations, the description is incomplete. It fails to clarify the distinction between deleting realm roles and client roles, or what happens to dependent entities, leaving significant gaps for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds no meaning beyond the schema fields; the optional clientId and its role in targeting client roles remain unexplained in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete roles' is essentially a restatement of the tool name 'delete-role' with no added specificity, failing to distinguish it from sibling tools like delete-role-by-id or delete-composite-roles. It does not mention whether it handles realm roles, client roles, or both.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidance is provided. The description gives no indication of when to use this tool versus other delete tools, nor does it explain the optional clientId parameter for client roles or any prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-role-by-idB

Delete role by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
roleIdYesRole ID

TDQS

B3/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states the action without explaining cascading effects, reversibility, required permissions, or return behavior. This is a significant gap for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise phrase with no unnecessary words. It is front-loaded and appropriately sized for a simple delete operation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive operation with no output schema and no annotations, the description is too minimal. It fails to mention consequences, necessary permissions, or whether associations with users/groups are removed, which are important for correct tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters, with descriptions for both 'realm' and 'roleId'. The description adds no additional semantic meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete role by ID' uses a clear verb 'delete' and specifies the resource (role) and identifier (by ID). It clearly distinguishes from sibling tools such as 'delete-role' (likely by name) and 'delete-composite-roles' (composite roles).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'delete-role' or 'delete-composite-roles'. There is no mention of context, exclusions, or preferred scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete-userA

Delete a user from a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'Delete a user', which is inherently destructive, but does not mention permanence, side effects (e.g., removing user sessions or cascading deletions), permission prerequisites, or reversibility. This is a significant gap for a mutation tool with no annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that conveys the essential action and scope without any superfluous words. It is well-suited for quick parsing by an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (two required string parameters, no output schema), the description is minimally adequate. However, it lacks behavioral context such as whether the deletion is permanent, what happens to related entities, or any prerequisites, which would be helpful for complete understanding. It is not incomplete enough to warrant a 2, but not rich enough for a 4.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameters with descriptions ('Realm name' and 'User ID'). The tool description does not add any additional semantic meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('a user from a specific realm'), making the tool's purpose distinct from sibling tools like update-user, get-user, or delete-realm. The addition of 'from a specific realm' provides contextual scoping that differentiates it from other deletion tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for deleting a user in a realm, but it does not explicitly state when to use this tool versus alternatives (e.g., update-user to modify or logout-user to end sessions). No exclusions or alternative recommendations are provided, so the guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find-users-with-roleA

Find users with a realm role, or with a client role when clientId is given

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
clientIdNoClient ID to resolve roleName as a client role. Omit for a realm role
roleNameYesRole name

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses the dual behavior (realm vs client role) which is useful, but does not state anything about read-only safety, pagination behavior, or error conditions. Given the tool is explicitly a 'find' operation, it is implicitly read-only, but this is not confirmed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence that conveys the purpose and key condition. No fluff or redundant wording, making it highly efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no output schema, the description covers the core functionality but omits expected return details (e.g., user fields, pagination format). However, the parameters themselves are fully described in the schema, and the description clarifies the primary invocation pattern. It is minimally adequate but leaves uncertainty about the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add meaning beyond the schema's clientId description, which already explains the distinction between realm and client roles. The parameters are adequately documented individually.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states a specific verb ('Find') and resource ('users') with a clear scope ('with a realm role, or with a client role when clientId is given'). This distinguishes it from broader tools like list-users or search-users by focusing on role-based lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a conditional usage rule: use clientId for client roles, omit for realm roles. This gives clear context for when to use the tool, though it does not explicitly mention alternative tools or exclusions. The indication 'when clientId is given' is a practical guideline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-client-scopeB

Get details of a specific client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientScopeIdYesClient scope ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. 'Get details' implies a read-only operation but does not explicitly state non-mutating behavior, required permissions, or what 'details' entails. It lacks context about return format or error semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is perfectly concise and front-loaded with the verb and resource. Every word contributes to the meaning, with no unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple getter with only two parameters and no output schema, the description is minimally adequate. However, it lacks usage guidance and behavioral context, which would be helpful even for a straightforward read operation, leaving some gaps for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for both parameters (realm and clientScopeId) with clear descriptions. The description adds no extra meaning beyond the schema, so the baseline of 3 applies; it neither compensates nor detracts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and the resource ('client scope'), specifying it's for a 'specific' one, which distinguishes it from list, create, update, and delete operations. It could be more explicit that it fetches by ID, but the schema clarifies the identifier parameter.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like list-client-scopes. It doesn't state prerequisites, exclusions, or typical use cases, leaving the agent to infer context from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-composite-rolesC

Get composite roles for a role

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
roleIdYesRole ID
searchNoSearch term

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must carry the full burden of behavioral disclosure. It only restates the action (get) and provides no information about pagination, search behavior, authorization requirements, or whether results are filtered by realm or role scope.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence without redundancy. It front-loads the verb and resource, but its brevity sacrifices necessary detail for completeness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema, no annotations, and only a one-sentence description. It fails to explain return format, pagination behavior, search semantics, or how this differs from related role tools. This is highly inadequate for a tool with 5 parameters and no structured output metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for all five parameters, so the individual parameter descriptions already define their meanings. The tool description adds no extra semantic value beyond what is already in the schema, warranting the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the verb 'Get' and the resource 'composite roles' scoped 'for a role'. It is distinguishable from sibling tools like 'delete-composite-roles' and 'create-composite-role', though it could be more explicit about what 'composite' means in this context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as 'get-role-by-id' or 'list-composite-group-roles'. There is no mention of prerequisites, typical scenarios, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-group-attributesC

Get group attributes

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID

TDQS

C2.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Get', implying a read operation, but fails to mention return format, authentication needs, side effects, or error behavior. This is minimal and inadequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short but under-specified. It merely rephrases the tool name without adding any new information, so the sentence fails to earn its place as a useful description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and no annotations, the description should explain what attributes are returned or any relevant context. It does not, leaving agents to rely solely on the sparse schema descriptions. This is incomplete for a complete understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for both parameters (realm and groupId), so the baseline is 3. The description adds no parameter-specific meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get group attributes' is a direct restatement of the tool name 'get-group-attributes'. It identifies the action and resource but adds no scope, detail, or differentiation from sibling tools like get-user-attributes or get-group-roles, making it a tautology.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as set-group-attributes, get-user-attributes, or get-group-roles. The description lacks any context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-group-rolesA

Get roles assigned to a group, returned as { realmMappings, clientMappings }

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It adds a useful detail by specifying the return shape, but it does not mention permissions, whether the roles returned are direct or inherited/composite, or any error conditions. The description adds some transparency but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the action ('Get roles assigned to a group') and includes the return structure. There is no redundant or extraneous wording, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read tool with two well-described parameters and no output schema, the description is largely complete. It explains the purpose and the response shape. However, it could be more complete by clarifying whether the roles are direct assignments only or include composite/inherited roles, and by mentioning any required permissions. Still, it is adequate for most use cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully describes both parameters ('realm' and 'groupId') with 100% coverage. The description does not add any semantic meaning beyond the schema; it merely restates the concept of getting roles for a group. Since the schema covers the parameters, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Get' and clearly identifies the resource as 'roles assigned to a group'. It also specifies the return format as '{ realmMappings, clientMappings }', which distinguishes it from sibling tools like 'get-user-roles' or 'list-available-group-roles'. This is a clear and specific purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes it clear what the tool does, but it doesn't explicitly state when to use it over alternatives like 'list-available-group-roles' or 'list-composite-group-roles'. The usage context is implied through the verb and resource, but there is no explicit exclusion or comparison, so the guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-identity-providerA

Get details of a specific identity provider

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'Get details' and does not mention response format, error handling (e.g., not found), or any prerequisites, leaving the agent without critical behavioral context beyond the obvious read-only nature of 'get'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no unnecessary words or repetition. It efficiently conveys the core purpose without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get operation with two schema-documented parameters, the description is minimally adequate. However, the absence of an output schema and any detail on return values or error behavior leaves gaps in what the agent can expect from invoking the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% coverage for both parameters with descriptions ('Identity provider alias' and 'Realm name'). The description adds no additional parameter semantics beyond what the schema already contains, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'details of a specific identity provider', which distinguishes it from sibling list, create, update, and delete tools. The specificity of 'details' and 'specific' makes the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use guidance or alternative tools are mentioned. However, the required parameters (realm and alias) imply this is for targeted lookups, making the usage context reasonably clear by inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-identity-provider-mapperA

Get details of a specific identity provider mapper

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name
mapperIdYesMapper ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. 'Get' implies a read-only operation with no side effects, which is minimal behavioral disclosure. It does not mention error handling, permissions, or return format.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence ('Get details of a specific identity provider mapper') that conveys the essential purpose without waste. It is appropriately sized for a simple getter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple getter with fully documented schema parameters, the description is sufficient. It doesn't explain what 'details' includes, but no output schema exists and the tool is straightforward. Slightly more context about the returned data would improve completeness, but it's not critical.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each parameter (realm, alias, mapperId) having a clear description. The tool description adds no parameter-specific information, so it relies on the schema, earning the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Get' and resource 'details of a specific identity provider mapper', clearly distinguishing from sibling list/create/update/delete mapper tools. The word 'specific' differentiates it from list-identity-provider-mappers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit usage guidance or alternatives are given. Usage is implied by the name and description: use when you need details of one specific mapper. It does not mention when not to use it or point to list/get alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-organizationA

Get details of a specific organization

ParametersJSON Schema
NameRequiredDescriptionDefault
orgIdYesOrganization ID
realmYesRealm name

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states 'Get details' without disclosing behavior such as read-only nature, output format, required permissions, or error handling. This is a significant gap for a tool with no structured transparency signals.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundant information. It is front-loaded and every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter get tool with no output schema, the description is adequate: the purpose is clear and distinguishes from siblings. However, it lacks explicit usage guidance and behavioral details, which would make it more complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers both parameters with descriptions (orgId and realm), so coverage is 100%. The description adds no additional parameter meaning beyond what the schema already provides, which aligns with the baseline score of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Get' and the resource 'organization' with the qualifier 'specific', clearly indicating a single-entity retrieval. This distinguishes it from siblings like 'list-organizations' (list all) and create/update/delete operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving details of one organization, but does not explicitly state when to use it versus alternatives like 'list-organizations' or provide contextual guidance. The verb 'get' vs 'list' gives an implied usage, but no clear when-to-use or exclusions are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-realm-settingsA

Retrieve detailed settings of a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose side effects and safety. The verb 'Retrieve' implies a read-only operation, which is transparent about non-destructive behavior. However, it does not mention error handling, permissions, or response format, leaving some ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (one parameter, no output schema), the description is adequate but minimal. It says 'detailed settings' but does not enumerate what those settings include or hint at return values. For a simple get tool, this is sufficient but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides full coverage for the single parameter 'realm' with description 'Realm name'. The tool description does not add any extra parameter semantics beyond what the schema states, so it meets the baseline but provides no additional value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Retrieve' and identifies the resource as 'detailed settings of a realm', which clearly distinguishes it from sibling tools like list-realms (listing all realms) and update-realm (modifying settings). It is concise and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for fetching a specific realm's settings but does not explicitly state when to use it versus alternatives like list-realms or update-realm. There is no mention of exclusions or conditions, so it relies on the tool name and context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-role-by-idC

Get role details by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
roleIdYesRole ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the full burden of behavioral disclosure. It only says 'Get role details' which implies read-only, but does not mention required permissions, realm-dependence, error handling, or response format. This is insufficient for a tool with no structured metadata.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no redundant wording. It is front-loaded and communicates the essential purpose without unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of annotations and output schema, the description is too minimal to be complete. It fails to explain how realm and roleId interact, what 'role details' includes, or any return behavior, making it hard for an agent to invoke correctly without additional assumptions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully describes both parameters ('realm' and 'roleId') with 100% coverage. The description adds no extra meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and resource ('role details') with identifier ('by ID'), clearly conveying the tool's function. It distinguishes from list-roles or update-role-by-id, though it omits the realm scoping which is a key part of the operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like list-roles or get-composite-roles. No exclusions or context are provided, leaving the agent without decision support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-userA

Get user details by ID from a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It indicates a read operation ('Get'), which is non-destructive, but it does not disclose behavior such as possible errors, return format, or whether the operation is read-only. The description is not misleading, but it adds minimal behavioral context beyond what the tool name implies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that immediately states the verb, resource, and parameters. It is front-loaded and contains no filler or redundant phrasing. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple getter tool with two parameters and no output schema, the description is mostly complete. It specifies the lookup criteria (realm, userId) and the general purpose. However, it could be slightly more explicit about what 'user details' includes (e.g., profile attributes, roles, etc.), though the sibling context and schema partially compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: the schema already explains 'realm' as 'Realm name' and 'userId' as 'User ID'. The description's 'from a specific realm' slightly reinforces the realm's role, but it does not add meaning beyond the schema. Baseline is 3 due to high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get'), names the resource ('user details'), and adds the scoping mechanism ('by ID from a specific realm'). This clearly distinguishes it from sibling tools like list-users and search-users, which have broader or different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when you have a known user ID and realm. It does not explicitly state when not to use it or name alternatives, but the context is clear given the tool's getter nature and the sibling set. No exclusions are mentioned, but the specificity of 'by ID' and 'from a specific realm' provides sufficient guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-user-attributesA

Get user attributes including unmanaged attributes

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of disclosing behavior. The verb 'Get' implies a read-only operation, and 'including unmanaged attributes' adds useful context about the return scope. However, it does not specify permissions, error behavior, or what constitutes 'unmanaged', leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, succinct sentence that is front-loaded with the key action and includes the most important differentiator without any wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with two well-documented params and no output schema, but the description covers the essential purpose and the unique 'unmanaged attributes' scope. It lacks details on response format or permissions, but for a simple getter it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for both parameters (realm and userId), and the description adds no additional parameter-level semantics. Baseline of 3 is appropriate as the schema already documents the parameters clearly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Get') and resource ('user attributes'), and the qualifier 'including unmanaged attributes' distinguishes it from sibling tools like 'get-user' and 'get-group-attributes'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is for retrieving user attributes, especially when unmanaged attributes are needed, but it does not explicitly state when to use it over alternatives or provide exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-user-rolesA

Get all roles directly assigned to a user, returned as { realmMappings, clientMappings }

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It discloses the return structure ({ realmMappings, clientMappings }) and scopes the operation to directly assigned roles, but does not mention read-only semantics, required permissions, or error behavior. The added detail about the return shape provides some value beyond the schema, but the lack of annotation support leaves gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that immediately states the action, target, and return shape. Every word adds value, and there is no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read operation with two self-explanatory parameters and no output schema, the description is fairly complete. It specifies the return structure and the 'directly assigned' scope. It could elaborate on the exact structure of realmMappings and clientMappings, but given the tool's simplicity and the absence of an output schema, this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides full descriptions for both parameters ('realm' and 'userId'), so the baseline is 3. The description does not add additional detail about the parameters beyond what the schema already states, such as the nature of the userId (e.g., internal ID vs username). No extra semantic value is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves all roles directly assigned to a user, with the specific verb 'Get' and the resource being 'roles directly assigned to a user'. It distinguishes from sibling tools like 'list-roles' (all roles in a realm) and 'get-group-roles' (roles assigned to a group) by focusing on the user's direct assignments.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'directly assigned' implies the tool should be used when needing only direct assignments, not composite or inherited roles, but it does not explicitly name alternatives or when-not-to-use scenarios. Despite lacking explicit exclusions, the context is clear enough for an agent to infer appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get-user-sessionsC

List active sessions for a user

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It implies a read-only operation but lacks details on permissions, pagination, result format, or any side effects. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no wasted words. It effectively conveys the core purpose without unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the low complexity, the description is missing key context for an agent to fully understand the tool: it doesn't clarify how it differs from list-sessions, what permissions are needed, or what the return structure looks like (no output schema). It feels under-specified for a tool that could be confused with a sibling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and both parameters (realm, userId) are adequately described in the schema. The description adds no additional param context, but the schema already does the heavy lifting, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists active sessions for a user, using a specific verb and resource. It is unambiguous but does not explicitly differentiate from the sibling list-sessions tool, so it misses the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives like list-sessions. The description only states what it does, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

import-identity-provider-from-urlB

Import identity provider configuration from URL

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
fromUrlYesURL to import from
providerIdYesProvider ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description must cover behavioral traits. It merely says 'Import' without clarifying whether this creates, updates, or upserts an identity provider, what happens if the provider already exists, or what side effects (e.g., network calls, overwrites) are involved. This is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word is relevant and the core action is immediately clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and a moderate-complexity import operation, the description is too sparse. It omits success/failure behavior, prerequisites, and how it relates to other identity provider management tools, leaving the agent without essential context for reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with all three parameters (realm, providerId, fromUrl) having basic descriptions. The description adds nothing beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Import identity provider configuration from URL' — a specific verb (Import), resource (identity provider configuration), and source (URL). This clearly distinguishes the tool from siblings like create-identity-provider and update-identity-provider.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios where URL-based import is preferred, nor does it exclude any use cases. The agent is left to infer context from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-available-group-rolesA

List roles a group could still be granted -- realm roles, or a client's roles when clientId is given

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID
clientIdNoClient ID to list that client's available roles instead of realm roles

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It indicates the tool returns a filtered set (roles that can still be granted), which is useful, but it does not mention permissions, whether already-assigned roles are excluded, or any sorting/pagination behavior. The phrase adds transparency but is incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, well-structured sentence with no filler. It states the core purpose and the one conditional variation (clientId) efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only listing tool, the description covers what, scope, and parameter usage sufficiently. It is complete enough for correct invocation given the well-documented schema, though it could mention whether composite roles are included or excluded to be more thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema covers all parameters (100%). The description repeats the realm-vs-client distinction already present in the schema, but does not add new semantics beyond the schema descriptions. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is specific and action-oriented: 'List roles a group could still be granted' identifies the exact resource and scope. The optional 'when clientId is given' further distinguishes which role set is returned, differentiating it from sibling tools like list-roles or get-group-roles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'could still be granted' implies usage for pre-assignment discovery, and the clientId conditional clarifies how to narrow to a specific client. However, it does not explicitly name alternatives or exclusions (e.g., 'for currently assigned roles, use get-group-roles'), so clear context but no explicit when-not.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-client-rolesA

List all roles defined on a client

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdYesClient ID

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. The verb 'List' clearly indicates a read-only operation, which is useful. However, it does not disclose details such as whether composite roles are included, pagination behavior, or the response structure. For a simple list operation, the basic transparency is adequate, but richer context would improve it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the operation. Every word earns its place, with no redundancy or fluff. It is concise without sacrificing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has only two parameters and no output schema, but the description does not clarify whether 'all roles' includes only top-level roles or also composite roles, which is a relevant ambiguity given sibling tools like 'list-composite-roles'. The description is sufficient for a basic list but lacks completeness in delineating edge cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameters, each with a description ('Realm name' and 'Client ID'). The tool description adds no additional meaning beyond the schema, so the baseline score of 3 is appropriate. No extra context is provided about how the parameters interact or format requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all roles defined on a client' uses a specific verb ('List') and identifies the resource ('roles') and scope ('on a client'). It clearly distinguishes from sibling tools like 'list-roles' by indicating client-scoped roles, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'on a client' implies the tool is used for client-specific roles, and the existence of 'list-roles' suggests a realm-level alternative. However, the description does not explicitly state when to use this vs. list-roles, nor does it mention any exclusions or preconditions. Usage context is implied, not explicitly guided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-clientsB

List all clients in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'List all clients' and does not mention return format, pagination, authentication needs, or the read-only nature beyond what the name implies. This lack of detail is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler, front-loading the verb and resource. Every word contributes to the tool's purpose, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema and no annotations, so the description should provide context about return values or behavior. It simply says 'List all clients in a realm' without noting whether full client objects, summaries, or paginated results are returned. This underspecification makes it incomplete for an agent selecting the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents the single 'realm' parameter with 100% coverage. The description adds minimal context by clarifying that listing is scoped to a realm, but does not add semantics beyond the schema's 'Realm name' description. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'List' with the resource 'clients' scoped to 'realm', clearly distinguishing it from sibling tools like create-client, update-client, and delete-client. There is no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context: it lists clients within a specified realm. However, it does not explicitly state when to use this tool over alternatives or provide exclusions. Since there are no direct sibling tools for listing clients, the guidance is minimal but adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-client-scope-protocol-mappersA

List protocol mappers for a client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientScopeIdYesClient scope ID

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. The verb 'list' implies a read-only operation, which is a useful signal, but the description does not elaborate on response format, pagination, filtering, permissions, or any side effects. For a simple list operation, this is minimally adequate but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no redundant words or filler. It front-loads the action and resource, making it immediately scannable and valuable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter list operation with no output schema, the description gives the core purpose and leaves little ambiguity about the operation's intent. However, it does not describe the shape of the returned list, any default ordering, or whether there are system limits, which could be helpful. Given the low complexity, the description is mostly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for both parameters ('Realm name' and 'Client scope ID'), achieving 100% coverage. The tool description adds no additional meaning or context for these parameters, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'list' and identifies the resource as 'protocol mappers' with the scope modifier 'for a client scope', clearly distinguishing it from the sibling 'list-protocol-mappers' which likely applies to clients. This provides a precise and unambiguous purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus the similar 'list-protocol-mappers' or 'list-identity-provider-mappers' tools. The description does not mention any exclusions, prerequisites, or alternative tools, leaving the agent to infer the intended usage solely from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-client-scopesA

List all client scopes in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states the operation is a list, but does not disclose return format, pagination behavior, required permissions, or other side effects. Minimal transparency beyond the action itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Exactly one sentence, front-loaded with the verb and object, no redundant words or fluff. Perfectly concise for a simple listing tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, no nested objects), the description is largely sufficient. It misses details like return value structure or permission requirements, but for a list operation this is a minor shortfall.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully describes the only parameter (realm) with 'Realm name'. The description's phrase 'in a realm' adds minor contextual meaning but does not significantly enrich understanding beyond the schema. Baseline 3 is appropriate given 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('client scopes') with a clear scope ('in a realm'). It unambiguously distinguishes from sibling tools like list-clients or list-client-scope-protocol-mappers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use it (when listing client scopes), but provides no explicit guidance on when not to use it or how it compares to similar listing tools. No alternatives are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-composite-group-rolesB

List composite roles for a group

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'List', implying a read operation, but does not reveal permissions, response format, pagination, or whether composite roles are directly assigned or inherited. This minimal disclosure adds little beyond the tool's name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no redundant words. It is concise and front-loaded, earning its place without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is too brief for a domain-specific tool. It does not explain what 'composite roles' are, how they relate to groups, or how the response differs from similar tools. No output schema exists, so the description should provide more context, but it leaves the agent with ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with descriptions for both 'realm' and 'groupId'. The tool description adds no additional parameter semantics, but since the schema already documents parameters, the baseline of 3 applies. No examples or format details are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List composite roles for a group' uses a specific verb ('List') and resource ('composite roles for a group'), clearly distinguishing it from sibling tools like 'list-available-group-roles' or 'get-group-roles'. The scope is explicit, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternative role-listing tools. The description does not mention any context, prerequisites, or exclusions, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-eventsA

Retrieve authentication and admin events

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
typeNoEvent type filter
realmYesRealm name

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description relies on the verb 'Retrieve' to imply a read-only, non-destructive operation. It does not explicitly mention safety characteristics or other behavior such as pagination or return format, but the action is clearly a read. This is minimally transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of five words ('Retrieve authentication and admin events'), which is appropriately concise and front-loaded. It contains no fluff or redundant information, earning a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple nature of the tool and lack of output schema, the description is adequate but not complete. It does not explicitly state what is returned (e.g., a list), but the tool name 'list-events' implies a list. It covers the key purpose but leaves return details implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with all three parameters (realm, max, type) having descriptive text. The description adds no extra meaning to these parameters, so the baseline of 3 is appropriate per the calibration guidelines.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Retrieve authentication and admin events' uses a specific verb ('Retrieve') and identifies the resource (authentication and admin events). It clearly distinguishes from sibling tools like 'clear-events' (which is destructive) and 'list-sessions' (which lists sessions), making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention conditions for use, exclusions, or reference other tools like 'clear-events' for event management. No context is given beyond the basic function.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-group-membersB

List members of a group

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
groupIdYesGroup ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of disclosing behavioral traits. The verb 'List' implies a read operation, but it does not disclose pagination behavior, authorization requirements, or whether the result includes nested members. This is minimal behavioral context beyond the operation itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It fully captures the tool's core action without any unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list operation with fully documented parameters, the description is minimally adequate. However, it omits any mention of pagination, return format, or how it differs from organization-member listing, leaving some context gaps for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%; all four parameters have descriptions in the schema, so the description does not need to explain them. The tool description adds no extra parameter meaning beyond the schema, matching the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List members of a group' uses a specific verb ('List') and resource ('members of a group'), clearly distinguishing it from sibling tools like list-groups. However, it lacks additional scope or context, such as pagination or realm handling, so it is clear but not as rich as the high calibration example.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance is provided. The description does not mention when to prefer this over list-groups, list-organization-members, or manage-user-groups, nor does it state any exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-groupsA

List all groups in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. 'List' implies a read-only operation, but it does not clarify whether subgroups are included, whether pagination applies, or what happens if the realm does not exist. The core behavior is disclosed, but edge cases and scope ambiguity remain.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, succinct sentence with no filler. It conveys the essential information clearly and front-loads the purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter list tool, the description is adequate. However, ambiguity about whether 'all groups' includes subgroups and the lack of response format or error semantics make it only partially complete given the sibling tool context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents the realm parameter with a description, and schema description coverage is 100%. The description reinforces that the realm parameter defines the scope, but adds no extra format or syntax details, so it meets the baseline without exceeding it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all groups in a realm' uses a specific verb ('list') with a clear resource ('groups') and scope ('in a realm'). It distinguishes from siblings like list-sub-groups and list-group-members by targeting all groups at the realm level.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the clear purpose: use this when you need to list all groups in a realm. However, there is no explicit mention of when not to use it or alternatives, such as list-sub-groups for child groups, so guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-identity-provider-mappersB

List all mappers for an identity provider

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses that the tool lists all mappers for a specific identity provider (a scoping constraint), but does not mention permission requirements, response format, or side effects. 'List' implies read-only, but no explicit statement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, grammatically correct, front-loaded with the verb and resource. Zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter list tool, the description and schema cover the essentials. However, with no output schema and no annotations, it doesn't describe the return value or any behavioral constraints beyond the listing. It's adequate but has gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers both parameters with clear descriptions ('Identity provider alias', 'Realm name'). The description adds no additional semantic meaning beyond what the schema provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'List' with resource 'all mappers for an identity provider', clearly distinguishing from sibling tools like get-identity-provider-mapper (singular). However, it does not explicitly name alternatives, so it's not a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool vs alternatives like get-identity-provider-mapper or list-identity-provider-mapper-types. It simply states what it does without context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-identity-provider-mapper-typesB

List available mapper types for an identity provider

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states 'List available mapper types', which doesn't explicitly disclose that the operation is read-only, what the response format is, or any permission requirements. The verb 'list' implies safety, but the description adds no behavioral detail beyond the name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that is front-loaded with the primary action and resource. Every word earns its place with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with no output schema, the description is minimally adequate but lacks guidance on expected output format, pagination, or error conditions. It is sufficient for an agent to understand the basic function, but not rich enough to fully anticipate behavior in complex scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters ('alias' and 'realm') are fully described in the input schema (100% coverage). The description itself does not add any parameter-level meaning, but the schema already provides sufficient detail. Baseline 3 is appropriate when schema covers all parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'List' and identifies the resource as 'available mapper types for an identity provider', which is distinct from the sibling tool 'list-identity-provider-mappers' (which lists configured mappers). This clearly conveys the tool's scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as 'list-identity-provider-mappers' or 'create-identity-provider-mapper'. It does not mention any exclusions, prerequisites, or typical use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-identity-providersB

List all identity providers in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
searchNoSearch term

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must carry the full burden of behavioral disclosure. It only restates the tool's name without explaining pagination, search behavior, sorting, or return format, leaving the agent uncertain about how the operation behaves beyond the obvious listing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one sentence, front-loaded with the action and resource, and free of unnecessary words. It is as concise as possible while conveying the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 4 parameters, no annotations, and no output schema, the description is insufficient. It fails to explain how max, first, and search interact, what the response contains, or whether 'all' means literally all or requires pagination. This leaves significant gaps for an agent to invoke the tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers all parameters with descriptions, so the baseline is 3. The tool description adds no additional meaning; however, the phrase 'all identity providers' conflicts with the presence of max/first/search parameters, subtly implying pagination but not explaining it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List') and the resource ('identity providers') with a scope ('in a realm'), which distinguishes it from sibling tools like list-identity-provider-mappers and list-realms. The purpose is immediately understandable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage but provides no explicit guidance on when to choose this tool over alternatives. It doesn't mention exclusions or how it relates to get-identity-provider or list-identity-provider-mappers, making the context clear but not fully defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-organization-membersC

List all members of an organization

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
orgIdYesOrganization ID
realmYesRealm name
searchNoSearch term

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It states 'List all members' but does not address pagination behavior (max/first parameters), search filtering, ordering, or error scenarios, which are relevant for this type of list operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no fluff. It is front-loaded with the key action and resource, making it ideal for quick scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema and annotations, the description is too sparse. It does not explain the return format, pagination limits, or behavior of the search parameter, which would be expected for a complete tool definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for all five parameters, so the description does not need to repeat them. However, the description adds no extra meaning about how parameters interact (e.g., how search works with orgId), so it remains at the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List') and the resource ('members of an organization'), making the purpose immediately understandable. It is distinguishable from sibling tools like list-users and list-group-members, though it does not explicitly differentiate itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as list-users or list-group-members. There is no mention of exclusions or prerequisites, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-organizationsB

List all organizations in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
searchNoSearch term

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must disclose behavioral traits. It states the operation but does not clarify pagination behavior (the presence of max/first parameters suggests it may not return 'all' in one page), nor does it mention search filtering, read-only nature, or any response format. This is insufficient for a tool with no annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that communicates the core purpose without unnecessary words. It is front-loaded and easy to parse, making it an excellent example of conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately defines the basic operation but lacks important context. The word 'all' conflicts with the presence of pagination parameters, and without an output schema, the return structure is not explained. For a simple list tool, it is minimally viable but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with clear descriptions for max, first, realm, and search. The description adds no additional meaning beyond the schema, so it meets the baseline without compensating for any gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all organizations in a realm' uses a specific verb ('list'), defines the resource ('organizations'), and scopes it by realm. This clearly differentiates it from sibling tools like get-organization (single) and list-organization-members (members of an organization).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention that get-organization should be used for a single organization, or that list-organization-members is for members. No exclusions or context are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-protocol-mappersA

List protocol mappers for a client

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdYesClient ID

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It conveys a read-only listing operation scoped to a client, but does not disclose return format, pagination, or mapper types. This is minimal but not contradictory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence that front-loads the action and scope without any filler. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but with no output schema or annotations, the description does not address return values or usage context. It is adequate for a basic list operation but lacks supplementary details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes both parameters with 100% coverage, and the description adds no additional meaning for 'realm' or 'clientId'. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists protocol mappers for a client, using a specific verb and resource. 'For a client' differentiates it from siblings like list-client-scope-protocol-mappers and list-identity-provider-mappers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It does not mention related tools like list-client-scope-protocol-mappers or any exclusions, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-realmsB

List all available realms

ParametersJSON Schema
NameRequiredDescriptionDefault
random_stringYesDummy parameter for no-parameter tools

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description is minimal and does not disclose behavioral traits beyond the core action. The input schema requires a 'random_string' dummy parameter, and the description does not explain this quirk, nor does it mention permissions, pagination, or response format. With no annotations, the description fails to fully carry the transparency burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no wasted words. It is concise and front-loaded with the essential verb and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but the description is incomplete: it does not mention the dummy parameter requirement, expected response shape, or any permissions/scope details. Given there is no output schema, more context would be valuable for an agent deciding to invoke this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter with a clear description ('Dummy parameter for no-parameter tools'). The tool description adds no parameter-related information, which is acceptable since the schema already explains the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all available realms' uses a specific verb ('List') and resource ('realms'), clearly distinguishing this tool from siblings like create-realm, delete-realm, and get-realm-settings. It is direct and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (when you need to list realms) but provides no explicit guidance on when to prefer this over related tools or any exclusions. It is adequate but does not explicitly mention alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-rolesB

List all roles in a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description should disclose behavioral traits. It merely says 'list' without mentioning whether results are paginated, whether authentication is required, or what 'all roles' includes. No additional behavioral context is provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that directly states the operation. It contains no filler or redundant information, earning top marks for brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description does not explain what the list returns (e.g., role names, metadata). It also fails to differentiate from similar role-listing tools, leaving the agent without sufficient context for a complete invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers the single parameter 'realm' with description 'Realm name'. The description adds no extra meaning beyond restating 'specific realm', so it meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all roles in a specific realm' uses a specific verb and resource, clearly indicating the scope (realm-level roles). It distinguishes itself from sibling tools like list-client-roles and get-role-by-id by specifying the realm context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as list-client-roles or get-role-by-id. The description states what it does but no context about preferred scenarios, exclusions, or fallback tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-sessionsA

List all active sessions in a realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
clientIdNoClient ID filter

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It indicates a read operation ('List') and scopes to 'active' sessions, adding some behavioral context. However, it doesn't disclose pagination, result structure, or other side effects, which is a gap for a tool without annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no unnecessary words. It is front-loaded with the action and resource, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with two parameters and no output schema, the description is minimally adequate. It states the core purpose and scope but omits optional filtering (clientId) and any mention of return values, which could be expected in a complete description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The tool description adds no extra parameter details, so the baseline score of 3 applies; it neither enhances nor detracts from the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the verb 'List', the resource 'active sessions', and the scope 'in a realm'. It distinguishes from siblings like get-user-sessions by indicating all active sessions realm-wide, not per-user.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'in a realm' implies this is for realm-wide session listing, but it doesn't explicitly mention when to use this over alternatives like get-user-sessions. No exclusions or alternative tool names are provided, only implied context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-sub-groupsA

List subgroups of a parent group

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
firstNoFirst result index
realmYesRealm name
searchNoSearch term
parentGroupIdYesParent group ID

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. 'List' implies a read-only operation, but the description does not disclose pagination behavior (max/first), search semantics, or whether subgroups are direct or recursive. It is not misleading but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no wasted words. Every term ('List', 'subgroups', 'parent group') is informative and necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is minimal. It covers the core purpose but lacks operational detail such as pagination, search, and return structure. Given there is no output schema and 5 parameters, a bit more context would help, but the tool is conceptually simple.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all parameters are already described in the input schema. The description adds no additional parameter context, so it meets the baseline but does not enhance understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('subgroups of a parent group'), clearly distinguishing it from 'list-groups' which presumably lists top-level groups. The parent group relation is explicit, leaving no ambiguity about the tool's function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'list-groups' or 'create-child-group'. It only states the core function, relying on the agent to infer usage context from the parameters and sibling names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list-usersB

List users in a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full behavioral burden, but it offers no details beyond the action. It does not disclose whether results are paginated, what happens if the realm doesn't exist, or any permissions required, leaving the tool's behavior largely opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler or redundant wording. It is directly front-loaded with the action and scope, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and no output schema, but the description is still sparse. It does not mention that the tool returns an array of users or any limitations, relying on the user to infer basic list behavior. This is minimally complete for the low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with the 'realm' parameter described as 'Realm name'. The description's phrase 'in a specific realm' merely restates this, adding no extra semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (list), the resource (users), and the scope (in a specific realm). This distinguishes it from sibling tools like get-user or search-users.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention search-users for filtered results or any scenarios, so an agent cannot determine the right tool choice from the description alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

logout-userB

Logout all sessions for a specific user

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states the core action and does not disclose side effects (e.g., token invalidation), authorization requirements, or irreversibility. This is a significant gap for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no filler. It is front-loaded and directly conveys the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description lacks critical context such as return values, error scenarios, preconditions, or side effects. With no output schema and no annotations, the description is under-specified for safe invocation, especially given the destructive nature of logging out all sessions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% since both realm and userId have basic descriptions. The tool description adds no additional parameter meaning, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Logout all sessions for a specific user.' The verb 'Logout' and resource 'all sessions for a specific user' are specific and distinguish it from sibling tools like reset-user-password or update-user.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no context on when to use this tool versus alternatives, no prerequisites, and no exclusions. It is a bare statement of functionality without any selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

manage-user-groupsB

Add or remove users from groups

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
actionYesAction to perform
userIdYesUser ID
groupIdYesGroup ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description must carry the full behavioral burden. It only states that it adds/removes users, but does not disclose side effects, permission requirements, idempotency, or failure modes. This is insufficient for a state-changing operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that delivers the core purpose without any filler. It is perfectly concise and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of annotations and an output schema, the description should provide more context about invocation outcomes, errors, or conditions. The minimal text leaves gaps that could lead to incorrect usage or expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all four parameters, including an enum for 'action'. The description adds no extra semantic meaning beyond what the schema already provides, warranting the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verbs 'Add or remove' and clearly identifies the resource ('users from groups'), making its function unambiguous. It distinguishes itself from sibling tools like create-group or assign-role-to-group, which handle different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as assign-role-to-group or create-group. The description lacks any context about prerequisites, exclusions, or preferred scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove-organization-memberB

Remove a user from an organization

ParametersJSON Schema
NameRequiredDescriptionDefault
orgIdYesOrganization ID
realmYesRealm name
userIdYesUser ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states what the operation does, but does not mention side effects, idempotency, whether it fails for non-members, or any impact on user roles/permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence that is easy to parse and front-loaded with the essential action. It is appropriately sized for a simple tool, though it could be slightly more informative without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple removal operation with three required parameters and no output schema. The description is minimal and does not explain return behavior or edge cases, but for this level of complexity, a basic statement may suffice. Still, it leaves gaps about effects and error conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for all three parameters (realm, orgId, userId), giving 100% coverage. The description adds no further semantic detail beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Remove a user from an organization' with a specific verb and resource. It distinguishes from siblings like 'add-organization-member' and 'list-organization-members' by explicitly denoting the removal operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool vs alternatives, no prerequisites, no exclusion criteria. The description is purely a statement of action, leaving the agent to infer usage solely from the name and schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove-role-from-groupA

Remove a realm role, or a client role when clientId is given, from a group

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID
clientIdNoClient ID to resolve roleName as a client role. Omit for a realm role
roleNameYesRole name

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

As a mutation tool with no annotations, the description does not disclose permissions, reversibility, or failure behavior. It does clarify that removal targets a specific role type (realm or client based on clientId), but lacks deeper side-effect information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with the action, no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple removal operation, the description covers the essential behavior and the key conditional parameter. It could mention that the role assignment is removed without deleting the role itself, but the wording implies this.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description adds meaning beyond the schema by explaining when to omit clientId for realm roles and that clientId resolves roleName as a client role.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Remove') and resource ('group'), clearly states the scope (realm role vs client role via clientId), and distinguishes from sibling tools like remove-role-from-user and assign-role-to-group.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context on when to use (removing a role from a group) and how to choose between realm and client roles via clientId. No explicit alternatives are named, but sibling tool names make the distinction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove-role-from-userA

Remove a realm role, or a client role when clientId is given, from a user

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID
clientIdNoClient ID to resolve roleName as a client role. Omit for a realm role
roleNameYesRole name

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description shoulders the burden of behavioral disclosure, but it only says 'Remove' without explaining side effects, idempotency, required permissions, or error behavior when the role is not assigned. This is a significant gap for a mutating operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tightly worded sentence that gets straight to the point. No wasted words, and the conditional clause adds important nuance without bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple role-removal tool, the description covers the core action and parameter semantics via the schema, making it minimally viable. However, the lack of behavioral notes (e.g., what happens if role not found, whether removal is permanent) leaves gaps in completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides full descriptions for all four parameters, including clientId's role-resolution behavior. The description adds little beyond restating that clientId switches to client role, so it meets the baseline but doesn't elevate understanding further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Remove' and clearly identifies the resource: a role from a user. It also distinguishes between realm roles and client roles with the clientId condition, making it distinct from siblings like assign-role-to-user or remove-role-from-group.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells when to include clientId ('when clientId is given') to remove a client role versus omitting it for a realm role. This provides clear conditional usage guidance, though it doesn't explicitly mention alternatives or when not to use the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reset-user-passwordB

Reset a user's password in a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID
temporaryNoWhether password is temporary
newPasswordYesNew password

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry behavioral disclosure. It only states the action of resetting a password without mentioning side effects (e.g., invalidating sessions), required permissions, or implications of the 'temporary' flag. This is a significant gap for a sensitive mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, eight-word sentence that is front-loaded with the verb and resource. It contains no filler or redundant words and is appropriately sized for a simple tool. Every word contributes to stating the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no annotations, no output schema, and a minimal description. It lacks information about expected response, side effects, or when to use the 'temporary' parameter. For a password reset operation with four parameters, the description is insufficient for confident invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all four parameters with descriptions, achieving 100% coverage. The description adds no extra semantic value beyond restating the realm scoping. It does not clarify interactions like whether 'temporary' forces a password change on next login, so it stays at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Reset' and clearly identifies the resource ('a user's password') and scope ('in a specific realm'). It distinguishes itself from sibling tools like update-user and logout-user by focusing on password reset. The action is unambiguous, especially with the newPassword parameter in the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as update-user or set-user-attributes. It does not mention any prerequisites, exclusions, or preferred contexts. The mention of 'specific realm' is a scoping detail rather than usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search-usersC

Search users in a specific realm with filters

ParametersJSON Schema
NameRequiredDescriptionDefault
maxNoMaximum results
emailNoEmail filter
realmYesRealm name
searchNoSearch term
lastNameNoLast name filter
usernameNoUsername filter
firstNameNoFirst name filter

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations available, the description must convey behavioral traits. It only says 'Search users' which implies a read operation, but does not disclose how filters are combined, case sensitivity, partial vs exact matching, pagination, or the effect of the 'max' parameter. This is insufficient for a tool with multiple filter options.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence, front-loaded with the verb and resource. However, it is perhaps too terse, omitting valuable details that could be included without bloat. It is not verbose, but the brevity borderlines under-specification rather than efficient conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters, 1 required, no output schema, and no annotations, the one-sentence description is inadequate. It does not explain how to construct a search, whether filters are ANDed, or what constitutes a valid search term. More detail is needed to invoke the tool correctly beyond the raw schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with each parameter having a short description (e.g., 'Email filter', 'Search term'). The tool description adds no extra meaning beyond the schema, and the schema itself is minimal. Baseline of 3 is appropriate since the schema covers the parameters, but the description does not enhance understanding of how they interact.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Search users in a specific realm with filters' clearly identifies the verb (search) and resource (users), and adds scope (specific realm) and capability (filters). It is distinguishable from sibling tools like list-users or get-user, though it does not explicitly name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as list-users or get-user. It does not state whether this should be used for partial matches, filtering by criteria, or when a specific user ID is unknown. No explicit use cases or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set-group-attributesC

Set group attributes (organization metadata)

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
groupIdYesGroup ID
attributesYesGroup attributes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not state whether the operation replaces existing attributes entirely or merges with them, what permissions are required, or what the response indicates. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short and front-loaded with the verb and resource, making it concise and free of waste. However, it omits important behavioral and usage context, so the brevity is more a result of under-specification than carefully chosen completeness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has three parameters, a nested object, no output schema, and no annotations, the description is the sole source of guidance. It only covers the primary action and provides no context about side effects, return values, or how the attributes parameter is interpreted. This is insufficient for an agent to invoke the tool confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for all three parameters with basic descriptions. The tool description adds a small clarification that attributes are 'organization metadata,' but otherwise adds no new meaning to realm, groupId, or attributes. This meets the baseline expectation but does not go beyond it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Set') and the target resource ('group attributes'), with a parenthetical clarifying these are organization metadata. This distinguishes it from related tools like get-group-attributes and create-group, though it doesn't detail the exact semantics of 'set' (e.g., merge vs replace).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like update-group, create-group, or set-user-attributes. The description only states the function with no mention of prerequisites, exclusions, or scenarios where this tool is preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set-user-attributesB

Set user attributes (CRITICAL for storing organization data)

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
userIdYesUser ID
attributesYesUser attributes (key-value pairs where values are arrays)

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavior disclosure. It only says 'Set user attributes', which implies mutation but does not disclose whether attributes are merged or replaced, any permission requirements, side effects, or what happens to existing attributes. The CRITICAL note is about importance, not behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words, but the capitalized 'CRITICAL' adds emphasis without substantive information. It is concise but somewhat noisy, earning a 4 rather than a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, the description is too sparse. There is no output schema, no annotations, and no explanation of return values, error cases, or behavior on existing attributes. For a mutation tool with nested objects, this is insufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds no additional parameter-level meaning beyond the schema; it does not explain how realm or userId are obtained, nor the expected structure of the attributes values beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'set' with a clear resource 'user attributes', which distinguishes it from sibling tools like 'get-user-attributes' and 'set-group-attributes'. The parenthetical '(CRITICAL for storing organization data)' emphasizes its specific role in storing org data, further clarifying purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context by noting it is CRITICAL for storing organization data, but it does not explicitly state when to use this tool over alternatives such as 'update-user' or 'get-user-attributes'. No exclusions or alternative guidance are provided, so it earns a middle score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-clientC

Update client settings (redirect URIs, protocol mappers, etc.)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoClient name
realmYesRealm name
enabledNoEnabled status
clientIdYesClient ID
descriptionNoClient description
publicClientNoPublic client
redirectUrisNoRedirect URIs

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for disclosing behavior. It indicates a mutation ('Update') but does not specify whether it performs partial or full replacement, what happens to omitted fields, or any side effects/permissions. The mention of 'protocol mappers' is ambiguous since no such parameter exists in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, clearly front-loaded with the verb and resource. It is concise but omits critical context, so it earns high marks for structure but not for completeness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an update tool with 7 parameters and no annotations or output schema, the description is insufficient. It lacks details about the update semantics, whether fields are optional or partial, potential effects, and required permissions. The ambiguity around 'protocol mappers' further undermines completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 7 parameters, so the schema already documents them adequately. The description adds the notion of 'protocol mappers' but no corresponding parameter exists, which may confuse. It does not add meaningful semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Update' with the resource 'client settings' and provides concrete examples ('redirect URIs, protocol mappers'), making the tool's purpose clear. It does not explicitly contrast with sibling tools like update-protocol-mapper, but the client-scope is evident.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like update-protocol-mapper or update-realm. It only states what it does, leaving the agent to infer the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-client-scopeB

Update an existing client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoClient scope name
realmYesRealm name
protocolNoProtocol
attributesNoClient scope attributes
descriptionNoClient scope description
clientScopeIdYesClient scope ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must fully disclose behavior. It only says 'update', which implies mutation, but does not mention whether updates are partial or full, required permissions, idempotency, or side effects on related resources. This leaves significant behavioral ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no wasted words. It efficiently states the action and target, making it perfectly front-loaded and economical.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With six parameters, a nested object, and no output schema or annotations, the description must provide substantial context. However, it only offers a minimal verb phrase, leaving unclear how the update behaves, what is required, and what the outcome is. This is inadequate for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% description coverage for all six parameters, so the baseline is 3. The description itself adds no parameter-level detail, but it does not need to because the schema already documents each field. No extra meaning is provided beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the clear verb 'update' with the specific resource 'client scope', and the word 'existing' distinguishes it from creation or retrieval tools. This directly contrasts with sibling tools like create-client-scope, delete-client-scope, and list-client-scopes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It simply states the operation without context, such as requiring an existing client scope ID or when to prefer update-client-scope-protocol-mapper.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-client-scope-protocol-mapperB

Update a protocol mapper for a client scope

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProtocol mapper name
realmYesRealm name
configNoProtocol mapper configuration
mapperIdYesProtocol mapper ID
protocolYesProtocol
clientScopeIdYesClient scope ID
protocolMapperYesProtocol mapper type

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only says 'Update,' implying mutation, but does not explain side effects, idempotency, or required permissions. This is insufficient for a mutating operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no unnecessary words. It is well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (7 parameters, nested config object, no output schema), the description is too sparse. It does not explain what the config object contains, what protocol mapper types are valid, or what the response will be. More context is needed for an agent to use this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides complete descriptions for all 7 parameters, and the description adds no additional parameter semantics. Since schema coverage is 100%, a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (update) and the resource (protocol mapper for a client scope), distinguishing it from the generic update-protocol-mapper sibling by specifying the client scope context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like update-protocol-mapper or create-client-scope-protocol-mapper. The description only states what it does, not when it should be chosen.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-groupD

Update group attributes

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoGroup name
realmYesRealm name
groupIdYesGroup ID

TDQS

D1.9/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only says 'Update group attributes' with no information about mutation effects, partial vs. full replacement, permissions, or error handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely brief, but this is under-specification rather than conciseness. It restates the tool name almost verbatim and omits essential context that would justify its brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations, output schema, and usage guidance, the description is too minimal to be complete. It does not explain the update behavior, potential side effects, or relationship with other group operations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% description coverage for all three parameters (realm, groupId, name), so the baseline is 3. The description adds no additional meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update group attributes' identifies a clear resource and action (update a group), but 'attributes' is vague and does not specify which fields (e.g., name) are updatable. It also does not distinguish this from sibling 'set-group-attributes', which likely handles custom key-value attributes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives like 'set-group-attributes', 'create-group', or 'delete-group'. There is no mention of prerequisites or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-identity-providerB

Update an existing identity provider

ParametersJSON Schema
NameRequiredDescriptionDefault
aliasYesIdentity provider alias
realmYesRealm name
configNoIdentity provider configuration
enabledNoEnabled status
linkOnlyNoLink only
providerIdNoProvider ID
storeTokenNoStore token
trustEmailNoTrust email
displayNameNoDisplay name
addReadTokenRoleOnCreateNoAdd read token role on create
firstBrokerLoginFlowAliasNoFirst broker login flow alias

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. However, it only restates the purpose and provides no information about partial vs. full update semantics, permission requirements, side effects, or response behavior. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that directly states the tool's function with no redundant words. It is front-loaded and concise, though arguably too minimal for a complex tool, it is well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (11 parameters, no annotations, no output schema), yet the description provides no information about update semantics, return values, or parameter relationships. It is insufficient for an agent to understand the full behavior of the operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with each of the 11 parameters described. The tool description adds no additional parameter context, but the schema already provides adequate meaning, meeting the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Update' with the resource 'identity provider', clearly distinguishing it from create/delete/get/list siblings. It also specifies 'existing', implying modification of an already-created provider. This is a clear, non-tautological statement of purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives. The verb 'Update' implies usage when modifying an existing provider, but there is no mention of when not to use it or comparison to create/delete. Context is purely implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-identity-provider-mapperC

Update an identity provider mapper

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesMapper name
aliasYesIdentity provider alias
realmYesRealm name
configNoMapper configuration
mapperIdYesMapper ID
identityProviderMapperYesMapper type

TDQS

C2.4/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description is essentially a tautology of the tool name, repeating 'Update an identity provider mapper' without disclosing any behavioral traits such as side effects, required permissions, reversibility, or response behavior. Since no annotations are provided, the description carries the full burden, and it adds nothing beyond the verb already present in the name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise at one sentence, but it is under-specified and essentially restates the tool name. It does not earn its place by adding new information, and its brevity comes across as vacuous rather than efficiently structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the schema being complete, the description lacks crucial context such as the requirement to specify a realm, alias, and mapperId, and what the operation entails beyond 'update'. With no annotations and no output schema, the description is insufficient for a tool with 6 parameters, especially one that updates a configuration object. The agent is left to rely entirely on parameter names and descriptions without high-level guidance.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for all 6 parameters (name, alias, realm, config, mapperId, identityProviderMapper), achieving 100% coverage. The description adds no additional parameter details, but with full schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update an identity provider mapper' clearly identifies the action (update) and the resource (identity provider mapper), which distinguishes it from sibling tools like create-identity-provider-mapper, list-identity-provider-mappers, and delete-identity-provider-mapper. However, it lacks explicit scope or differentiating details such as the need for realm/alias/mapperId, so it is clear but not exceptionally specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, no conditions or prerequisites, and no mention of when an update is appropriate. It simply states the action with no contextual direction, leaving the agent to infer usage from the tool name and schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-organizationB

Update an existing organization

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoOrganization name
orgIdYesOrganization ID
realmYesRealm name
enabledNoOrganization enabled status
attributesNoOrganization attributes
descriptionNoOrganization description

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states 'update an existing organization' without explaining update semantics (e.g., partial vs. full replacement, handling of omitted fields, idempotency, or required permissions). This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence that is front-loaded and contains zero fluff. It is appropriately sized for a simple update operation, though it could have been more informative without sacrificing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (6 parameters, nested attributes, required realm and orgId) and lack of annotations/output schema, the description is too terse. It does not mention the required identifiers, the scope of updates, or the return behavior. While the schema covers parameter syntax, the operational context is incomplete for an agent to invoke the tool safely and correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with all six parameters described in the input schema. The description adds no parameter-specific information, but the baseline of 3 applies because the schema does the heavy lifting. No compensation is needed, but also no extra value is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update an existing organization' uses a specific verb ('update') and resource ('organization'), clearly distinguishing it from create/delete/list/get operations on organizations. It is unambiguous and action-oriented.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for modifying existing organizations, but it does not explicitly state when to use this tool versus alternatives like create-organization or update-realm. No exclusions or preconditions are mentioned, so guidance is merely implied by the tool name and description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-protocol-mapperC

Update a protocol mapper for a client

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProtocol mapper name
realmYesRealm name
configNoProtocol mapper configuration
clientIdYesClient ID
mapperIdYesProtocol mapper ID
protocolYesProtocol
protocolMapperYesProtocol mapper type

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states 'Update' which implies mutation, but does not disclose whether the operation is a full replacement, whether it is idempotent, what happens to existing configuration, or any required permissions. No side effects or response details are provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no redundant wording. It is appropriately front-loaded and contains only the essential action statement. There is no wasted text, achieving maximum conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having 7 parameters and a nested config object, the description is extremely minimal. It lacks behavioral context, expected effects, or guidance on how the tool fits into the broader client update workflow. The absence of annotations and output schema further compounds the incompleteness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for all 7 parameters (100% coverage), including 'name', 'protocol', 'protocolMapper', and 'config'. The description adds no additional meaning beyond the schema, so it meets the baseline of 3 but does not exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update a protocol mapper for a client' clearly identifies the action (update), the resource (protocol mapper), and the scope (for a client). This distinguishes it from sibling tools like update-client-scope-protocol-mapper or update-identity-provider-mapper, though it does not elaborate on what the update entails.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention that it is specifically for client-level protocol mappers, nor does it contrast with client-scope or identity-provider mapper updates. There are no usage prerequisites or scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-realmC

Update realm settings and configurations

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
enabledNoEnabled status
displayNameNoDisplay name

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only says 'Update realm settings and configurations,' which implies a write operation but does not disclose side effects, permission requirements, reversibility, or impact on existing settings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short sentence, front-loaded with the action and resource. There is no filler, but the brevity sacrifices detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations or output schema, the description is too minimal to be complete. It does not clarify whether partial updates are supported (e.g., whether omitting optional fields resets them), what response to expect, or any prerequisites. This is a simple tool but the description leaves important gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all three parameters, so the baseline is 3. The description adds no additional semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool updates realm settings and configurations, using a specific verb and resource. It clearly differentiates from sibling update tools targeting other resources (e.g., update-client, update-role) but does not enumerate which settings beyond the schema, so it is clear but not fully distinguishing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool vs alternatives like get-realm-settings or create-realm. There is no mention of prerequisites, exclusions, or preferred scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-roleC

Modify role attributes

ParametersJSON Schema
NameRequiredDescriptionDefault
realmYesRealm name
newNameNoNew role name
clientIdNoClient ID for client roles
roleNameYesCurrent role name
descriptionNoRole description

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It only states 'Modify role attributes' with no detail on permissions, reversibility, whether existing attributes are replaced or merged, or any side effects. This is minimal and comparable to the low-transparency example.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with zero wasted words. It is efficient, though it sacrifices detail; however, conciseness focuses on structure, and this is well-structured for its minimal length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with five parameters and no annotations or output schema, the description is grossly incomplete. It does not explain what attributes can be modified, the effect of omitting optional parameters, or any constraints. This is a significant gap for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters are documented in the schema. The description adds no additional parameter meaning, but per the rubric, the baseline is 3 when schema coverage is high, and the description does not need to compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Modify role attributes' clearly conveys that this tool updates a role's properties, using a specific verb and resource. However, it does not distinguish from sibling tool 'update-role-by-id', which also modifies roles, so it lacks sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'update-role-by-id' or 'delete-role'. There is no mention of prerequisites, exclusions, or suitable scenarios, leaving the usage context entirely implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-role-by-idC

Update role by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoRole name
realmYesRealm name
roleIdYesRole ID
compositeNoIs composite role
attributesNoRole attributes
clientRoleNoIs client role
containerIdNoContainer ID
descriptionNoRole description

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only says 'update', with no information about side effects, idempotency, required permissions, handling of omitted fields, or success/failure responses. For a mutating tool, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with no fluff, but it is under-specified rather than efficiently informative. It provides no more value than repeating the tool name and is too minimal to be useful for an agent, similar to the 'Process' example in calibration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 8 parameters, no annotations, no output schema, and no usage context, the description is incomplete. It lacks details about what fields are updatable, how omitted fields behave, error scenarios, or any prerequisites. The schema's full parameter documentation mitigates some gaps but cannot substitute for operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides descriptions for all 8 parameters (100% coverage), so the schema carries the parameter semantics. The description itself adds no parameter-level information, but per the guidelines, a baseline of 3 is appropriate when schema coverage is high.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'update' and the resource 'role by ID', indicating the operation's core purpose. However, it doesn't explicitly differentiate from the sibling tool 'update-role', which likely updates a role by name or other key, leaving some ambiguity for an agent selecting between them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'update-role', 'delete-role-by-id', or 'create-role'. The description does not mention prerequisites, such as the role existing or the relevance of the 'realm' and 'roleId' required parameters, nor does it suggest conditions for choosing this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update-userC

Update user information in a specific realm

ParametersJSON Schema
NameRequiredDescriptionDefault
emailNoEmail address
realmYesRealm name
userIdYesUser ID
enabledNoUser enabled status
lastNameNoLast name
usernameNoUsername
firstNameNoFirst name

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral disclosure. It only restates the action without mentioning whether updates are partial or full, whether the user must already exist, permission requirements, or side effects. This is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler or irrelevant details. It is efficient, though it sacrifices important nuance in favor of brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter mutation tool with no output schema and no annotations, a one-sentence description is inadequate. It does not explain update semantics, error conditions, or how realm and userId relate, leaving the agent to infer critical context from the schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are already documented in the schema. The description adds no additional meaning beyond the generic phrase 'user information', providing no extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Update') and resource ('user information') with a scope ('in a specific realm'), clearly distinguishing it from create/delete/get/list user tools. It could further differentiate from set-user-attributes, but the core action is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like set-user-attributes, create-user, or delete-user. There are no exclusions, prerequisites, or typical use cases mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 86 tool updatesv2.0.0
    • First observedadd-organization-member
    • First observedassign-role-to-group
    • First observedassign-role-to-user
    • First observedclear-events
    • First observedcreate-child-group
    • First observedcreate-client
    • First observedcreate-client-scope
    • First observedcreate-client-scope-protocol-mapper
    • First observedcreate-composite-role
    • First observedcreate-group
    • First observedcreate-identity-provider
    • First observedcreate-identity-provider-mapper
    • First observedcreate-organization
    • First observedcreate-protocol-mapper
    • First observedcreate-realm
    • First observedcreate-role
    • First observedcreate-user
    • First observeddelete-client
    • First observeddelete-client-scope
    • First observeddelete-client-scope-protocol-mapper
    • First observeddelete-composite-roles
    • First observeddelete-group
    • First observeddelete-identity-provider
    • First observeddelete-identity-provider-mapper
    • First observeddelete-organization
    • First observeddelete-protocol-mapper
    • First observeddelete-realm
    • First observeddelete-role
    • First observeddelete-role-by-id
    • First observeddelete-user
    • First observedfind-users-with-role
    • First observedget-client-scope
    • First observedget-composite-roles
    • First observedget-group-attributes
    • First observedget-group-roles
    • First observedget-identity-provider
    • First observedget-identity-provider-mapper
    • First observedget-organization
    • First observedget-realm-settings
    • First observedget-role-by-id
    • First observedget-user
    • First observedget-user-attributes
    • First observedget-user-roles
    • First observedget-user-sessions
    • First observedimport-identity-provider-from-url
    • First observedlist-available-group-roles
    • First observedlist-client-roles
    • First observedlist-client-scope-protocol-mappers
    • First observedlist-client-scopes
    • First observedlist-clients
    • First observedlist-composite-group-roles
    • First observedlist-events
    • First observedlist-group-members
    • First observedlist-groups
    • First observedlist-identity-provider-mapper-types
    • First observedlist-identity-provider-mappers
    • First observedlist-identity-providers
    • First observedlist-organization-members
    • First observedlist-organizations
    • First observedlist-protocol-mappers
    • First observedlist-realms
    • First observedlist-roles
    • First observedlist-sessions
    • First observedlist-sub-groups
    • First observedlist-users
    • First observedlogout-user
    • First observedmanage-user-groups
    • First observedremove-organization-member
    • First observedremove-role-from-group
    • First observedremove-role-from-user
    • First observedreset-user-password
    • First observedsearch-users
    • First observedset-group-attributes
    • First observedset-user-attributes
    • First observedupdate-client
    • First observedupdate-client-scope
    • First observedupdate-client-scope-protocol-mapper
    • First observedupdate-group
    • First observedupdate-identity-provider
    • First observedupdate-identity-provider-mapper
    • First observedupdate-organization
    • First observedupdate-protocol-mapper
    • First observedupdate-realm
    • First observedupdate-role
    • First observedupdate-role-by-id
    • First observedupdate-user

TDQS

C2.9/5.0
Disambiguation3/5

While most tools map to distinct resources, the large number of role-related tools (create-role, create-composite-role, update-role, update-role-by-id, delete-role, delete-role-by-id) and similar mapper tools create potential confusion. Some tools like manage-user-groups and find-users-with-role could also be misselected without careful reading.

Naming Consistency5/5

All tool names consistently follow a verb-noun hyphenated pattern (e.g., create-user, list-clients, delete-identity-provider-mapper). The style is uniform and predictable, making it easy to infer function from the name.

Tool Count1/5

With 86 tools, this server far exceeds the reasonable scope for a coherent toolset, even for a complex system like Keycloak. The count is in the extreme range and will overwhelm agents, increasing selection difficulty and cognitive load.

Completeness4/5

The toolset covers most core Keycloak admin operations across realms, users, clients, roles, groups, organizations, identity providers, and mappers. However, notable gaps exist such as get-client and get-group (by ID), which are standard CRUD operations agents would expect.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    A Model Context Protocol server that enables management of Keycloak users and realms through a standardized interface, providing tools for user creation, deletion, role assignment, and group management.
    9
    48
    13
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    An MCP server that enables AI assistants to navigate, search, and analyze local Keycloak source code to support developer customizations like SPIs and authenticators. It provides tools for searching classes, generating boilerplate code, detecting breaking changes between versions, and tracing dependencies.
    8
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    An MCP server for Keycloak Admin REST API, enabling user, group, event, and security management through service account authentication.
    30
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    A Model Context Protocol (MCP) server that provides a natural language interface for managing Keycloak identity and access management through its REST API.
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/thebylito/keycloak-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server