Skip to main content
Glama

MCP-сервер Portkey Admin

MCP-сервер для API администратора Portkey. Управляйте промптами, конфигурациями, аналитикой, ключами API и многим другим из любого MCP-клиента.


Быстрый старт

Вам понадобится API-ключ Portkey с соответствующими правами доступа (scopes). Получите его на своей панели управления Portkey в разделе API Keys.

Claude Code

claude mcp add -e PORTKEY_API_KEY=your_key portkey-admin -- npx -y portkey-admin-mcp

Cursor / Windsurf / VS Code

Добавьте в конфигурацию MCP (.cursor/mcp.json, .windsurf/mcp.json или .vscode/mcp.json):

{
  "mcpServers": {
    "portkey-admin": {
      "command": "npx",
      "args": ["-y", "portkey-admin-mcp"],
      "env": {
        "PORTKEY_API_KEY": "your_api_key"
      }
    }
  }
}

Запуск напрямую

PORTKEY_API_KEY=your_key npx -y portkey-admin-mcp

Чтобы открыть только ограниченный набор инструментов в stdio-клиентах, установите PORTKEY_TOOL_DOMAINS:

PORTKEY_API_KEY=your_key \
PORTKEY_TOOL_DOMAINS=prompts,analytics \
npx -y portkey-admin-mcp
git clone https://github.com/s-b-e-n-s-o-n/portkey-admin-mcp.git
cd portkey-admin-mcp
npm install && npm run build

Затем используйте эту конфигурацию:

{
  "mcpServers": {
    "portkey-admin": {
      "command": "node",
      "args": ["/path/to/portkey-admin-mcp/build/index.js"],
      "env": {
        "PORTKEY_API_KEY": "your_api_key"
      }
    }
  }
}

Related MCP server: agentforge

Что вы можете делать

Категория

Инструменты

Примеры

Промпты

14

Создание, версионирование, рендеринг, выполнение, миграция, продвижение промптов

Части промптов

7

Повторно используемые фрагменты промптов с версионированием

Метки промптов

5

Организация версий промптов (production, staging, dev)

Конфигурации

6

Маршрутизация шлюза, кэширование, повторные попытки, балансировка нагрузки

API-ключи

5

Создание и управление ключами API с ограниченными правами

Виртуальные ключи

5

Управление ключами доступа провайдеров

Коллекции

5

Группировка промптов по приложениям или проектам

Провайдеры

5

Управление конфигурациями AI-провайдеров

Интеграции

10

Интеграции провайдеров, модели, доступ к рабочим пространствам

MCP-интеграции

10

Интеграции внешних MCP-инструментов

MCP-серверы

10

Реестр MCP-серверов и их возможностей

Guardrails

5

Политики безопасности контента

Лимиты использования

7

Лимиты на затраты и потребление токенов

Лимиты частоты запросов

5

Контроль частоты запросов

Аналитика

20

Затраты, задержки, ошибки, токены, кэш, обратная связь

Логирование

8

Сбор и экспорт логов

Трассировка

2

Создание и обновление обратной связи по трассировкам

Пользователи и рабочие пространства

20

Управление пользователями, приглашения, участники рабочих пространств

Аудит

1

Доступ к логам аудита

Всего 150 инструментов. Полный список с описаниями см. в ENDPOINTS.md.


Права доступа (Scopes) API-ключа

Большинство инструментов работают с сервисным ключом уровня рабочего пространства (workspace-scoped), у которого включены права «Выбрать все» (Select All). Это охватывает промпты, конфигурации, виртуальные/API-ключи, провайдеров, guardrails, интеграции рабочих пространств, MCP-серверы, лимиты частоты/использования, логи, завершения промптов и управление пользователями рабочего пространства.

Инструменты уровня Enterprise (28)

Следующие инструменты требуют прав уровня организации, которые доступны только в планах Portkey Enterprise. В планах рабочих пространств они возвращают ошибку 403 You do not have enough permissions to execute this request. Их описания содержат суффикс Enterprise-gated. Returns 403 on non-Enterprise Portkey plans., чтобы MCP-клиенты знали об этом заранее.

Область

Инструменты

Требуемые права

Аналитика (20)

get_cost_analytics, get_request_analytics, get_token_analytics, get_latency_analytics, get_error_analytics, get_error_rate_analytics, get_cache_hit_latency, get_cache_hit_rate, get_users_analytics, get_error_stacks_analytics, get_error_status_codes_analytics, get_user_requests_analytics, get_rescued_requests_analytics, get_feedback_analytics, get_feedback_models_analytics, get_feedback_scores_analytics, get_feedback_weighted_analytics, get_analytics_group_users, get_analytics_group_models, get_analytics_group_metadata

org-level analytics.view

Аудит

list_audit_logs

audit_logs.list

Интеграции уровня организации

get_integration, list_integration_models, list_integration_workspaces

organisation_integrations.read

Пользователи уровня организации

list_all_users, get_user, get_user_stats, list_user_invites

organisation_users.list / organisation_users.read

Другие требования к правам доступа

Функция

Требуется

Завершение промптов (run_prompt_completion)

право completions.write + метаданные биллинга (app, env)

Создание сервисного API-ключа уровня организации через create_api_key

organisation_service_api_keys.create (Enterprise)

Если инструмент возвращает 403 с ошибкой Portkey AB03, это означает отсутствие прав доступа, а не неисправность эндпоинта.


HTTP-сервер (Экспериментально)

Статус: HTTP-транспорт работает, но развертывание на хостинге не полностью проверено для продакшена. Используйте stdio (npx) для надежной работы.

Сервер поддерживает Streamable HTTP для удаленного доступа:

PORTKEY_API_KEY=your_key \
MCP_HOST=127.0.0.1 \
MCP_PORT=3000 \
MCP_PUBLIC_BASE_URL=https://mcp.example.com \
MCP_AUTH_MODE=bearer \
MCP_AUTH_TOKEN=your_secret \
node build/server.js

Или через npx (пакет portkey-admin-mcp включает HTTP-бинарный файл):

PORTKEY_API_KEY=your_key MCP_AUTH_MODE=bearer MCP_AUTH_TOKEN=your_secret \
  npx -y -p portkey-admin-mcp portkey-admin-mcp-http

Для локального использования HTTP оставьте MCP_HOST по умолчанию 127.0.0.1. Устанавливайте MCP_HOST=0.0.0.0 только если вам намеренно нужно принимать соединения извне локальной машины, например, в Docker или через обратный прокси на другом интерфейсе.

Конфигурация

Переменная

По умолчанию

Описание

PORTKEY_API_KEY

(обязательно)

Ваш API-ключ Portkey

PORTKEY_TOOL_DOMAINS

Опциональный набор инструментов по умолчанию (через запятую) для stdio/HTTP, например prompts,analytics

MCP_HOST

127.0.0.1

Адрес привязки

MCP_PORT

3000

Порт

MCP_PUBLIC_BASE_URL

Публичный абсолютный базовый URL для рекламы из /auth/info и страницы статуса; рекомендуется для развертываний на хостинге

MCP_AUTH_MODE

none

none, bearer или clerk (none заблокирован для HTTP, если не переопределен явно)

MCP_AUTH_TOKEN

Секрет для bearer-авторизации

MCP_ALLOW_UNAUTHENTICATED_HTTP

Установите true только для намеренной локальной отладки HTTP без аутентификации

MCP_SESSION_MODE

stateful

stateful или stateless

MCP_MAX_SESSIONS

100

Максимальное количество одновременных сессий MCP до отклонения новых запросов инициализации

MCP_EVENT_STORE

off

off, memory или redis

MCP_REDIS_URL

URL Redis для общего хранилища событий

MCP_TLS_KEY_PATH

Ключ TLS для нативного HTTPS

MCP_TLS_CERT_PATH

Сертификат TLS для нативного HTTPS

ALLOWED_ORIGINS

Список разрешенных источников CORS

MCP_TRUST_PROXY

false

Доверять заголовкам прокси (для обратных прокси)

RATE_LIMIT_MAX_BUCKETS

10000

Максимальное количество отдельных корзин лимитов частоты в памяти

Включена экспериментальная поддержка Vercel. См. docs/VERCEL_DEPLOYMENT.md для инструкций по настройке.

Ключевые моменты:

  • Использует stateless-режим с хранилищем событий Redis

  • Требует авторизацию Clerk или bearer

  • Оставьте MCP_TLS_* не заданными (Vercel завершает HTTPS)

  • Установите MCP_PUBLIC_BASE_URL на URL вашего развертывания, чтобы рекламируемые эндпоинты MCP никогда не зависели от заголовков запроса

  • Vercel не поддерживает WebSockets — только Streamable HTTP/SSE

docker build -t portkey-admin-mcp .
docker run \
  -e PORTKEY_API_KEY=your_key \
  -e MCP_HOST=0.0.0.0 \
  -e MCP_PORT=3000 \
  -e MCP_AUTH_MODE=bearer \
  -e MCP_AUTH_TOKEN=your_secret \
  -p 3000:3000 \
  portkey-admin-mcp

Эндпоинты состояния (Health)

Путь

Цель

GET /health

Живучесть сервера

GET /ready

Готовность (включает опциональную проверку подключения к Portkey)

GET /auth/info

Метаданные конфигурации авторизации


Разработка

npm run dev           # stdio with hot reload
npm run dev:http      # HTTP with hot reload
npm test              # unit + contract tests
npm run test:e2e      # MCP protocol tests
npm run test:http     # HTTP endpoint smoke test
npm run ci            # full pipeline (lint + typecheck + test + build + e2e + verify)

npm run dev:http теперь по умолчанию требует MCP_AUTH_MODE=bearer или MCP_AUTH_MODE=clerk. Для намеренного локального тестирования без аутентификации установите MCP_ALLOW_UNAUTHENTICATED_HTTP=true.


Лицензия MIT · Вдохновлено r-huijts/portkey-admin-mcp-server

↑ Наверх

Available Tools

178 tools
add_workspace_memberA

Add an existing org user to a workspace with a role. Requires a UUID user_id; use list_all_users to find it, and invite_user first if the person is not yet in the org.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesRole in the workspace
user_idYesThe user ID to add (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID to add the member to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a mutation tool (readOnlyHint=false). Description adds precondition that user must exist in org, but does not disclose potential error cases like duplicate membership or permission requirements. Good but could be more thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: purpose and usage guidance. No redundant information, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given moderate complexity and presence of output schema, description covers key preconditions and usage steps. Could mention error handling for duplicate members, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with descriptions. Description adds value by clarifying user_id must be UUID (not email) and referencing list_all_users, which goes beyond schema details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Add an existing org user to a workspace with a role,' specifying verb, resource, and constraints. It distinguishes from sibling tools like invite_user and list_all_users by referencing them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides prerequisites: requires UUID user_id, use list_all_users to find it, and invite_user if not in org. This gives clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

archive_deploymentA

Enterprise-gated. Archive a registered Gateway deployment by UUID. Use get_deployment first to confirm the target. Portkey soft-deletes the record; this stops treating it as active but does not permanently remove its history. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesDeployment UUID to archive

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false and destructiveHint=false, so the description carries the behavioral burden and delivers: it discloses soft-delete behavior ('stops treating it as active but does not permanently remove its history') and auth requirements ('Returns 403 on non-Enterprise Portkey plans'). This is exactly the kind of context the annotations do not provide. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is dense and front-loaded, but 'Enterprise-gated' is repeated verbatim at the start and end — a redundant sentence that fails the 'every sentence earns its place' test. Minor trimming would make this concise rather than merely short.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with an output schema and annotations, the description is complete: it states the action, the prerequisite, the post-effect on the record, and access restrictions. Return values are covered by the output schema, so nothing an agent needs to invoke this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — the id parameter already has a description ('Deployment UUID to archive'). The tool description reinforces that id is a deployment UUID and implicitly sources it from get_deployment, adding marginal context beyond the schema, which matches the baseline-3 expectation for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb-resource pair — 'Archive a registered Gateway deployment by UUID' — and clearly distinguishes archival from the delete_* siblings by disclosing soft-delete semantics. Naming get_deployment as the prerequisite further anchors what this tool is for within the large sibling set.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit usage guidance: 'Use get_deployment first to confirm the target,' which tells the agent the correct precondition. The soft-delete explanation also implies when not to use it (when permanent removal is required), though no alternative tool is explicitly named as a contrasting option.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancel_log_exportA
Destructive

Cancel a pending or running log export job, unlike start_log_export which queues one or delete_integration which removes the source. This permanently stops that export, takes effect immediately, and does not roll back already-processed rows; call create_log_export and start_log_export again to retry. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to cancel

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description discloses key behaviors: the export is 'permanently stops', takes effect 'immediately', and 'does not roll back already-processed rows'. It also states the 403 response on non-Enterprise plans, which is valuable context not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the purpose. Each sentence adds distinct value: what it does, behavioral consequences/retry, and access limitations. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the destructive nature, the description covers the essential context: what it cancels, permanence, no rollback, how to retry, and access restrictions. With an output schema present, return values need not be explained. This is complete for a cancellation mutation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage with a description for export_id ('The unique ID of the log export to cancel'). The tool description adds no additional parameter semantics, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Cancel a pending or running log export job.' It also distinguishes from siblings by explicitly naming start_log_export and delete_integration, making the tool's unique role clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states when to use this tool versus alternatives: 'unlike start_log_export which queues one or delete_integration which removes the source.' It also provides the retry workflow (create_log_export and start_log_export again) and enterprise access expectations, offering clear guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_api_keyA

Create a Portkey API key for auth. Org keys grant broader access; workspace keys are scoped. WARNING: The key secret is returned ONCE in the tool result and will be visible in MCP transcripts and LLM context — store it securely immediately. Using the key grants access immediately according to its scopes, defaults, and limits. Workspace keys require workspace_id and user keys require user_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the API key
typeYesKey type: 'organisation' for org-wide access or 'workspace' for workspace-scoped
scopesYesPermission scopes for the key (e.g., ['logs.read', 'analytics.read'])
user_idNoUser ID (required for user sub-type keys)
sub_typeYesSub-type: 'user' for user-associated keys or 'service' for service accounts
expires_atNoExpiration date in ISO 8601 format
descriptionNoOptional description for the key
rate_limitsNoRequest or token rate limits, or null to clear them
alert_emailsNoEmail addresses for alerts
credit_limitNoCredit limit for usage
workspace_idNoWorkspace ID (required for workspace-type keys)
rate_limit_rpmNoRate limit in requests per minute
alert_thresholdNoAlert threshold percentage (0-100)
organisation_idNoOrganisation UUID
rotation_policyNoAutomatic API-key rotation policy, or null to disable it
default_metadataNoDefault metadata key-value pairs
default_config_idNoDefault configuration ID to use with this key
default_allow_config_overrideNoAllow callers to override the default config

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds highly valuable behavioral context beyond annotations: the key secret is returned exactly once, will be visible in MCP transcripts and LLM context, and must be stored immediately. It also discloses that access activates immediately according to scopes, defaults, and limits. This is exactly the kind of operational risk an agent needs to know.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well organized: purpose first, then scope distinction, then the security-critical warning, then required parameter relationships. Every sentence earns its place and there is no redundant schema repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 18-parameter mutation tool with an output schema, the description covers purpose, key type selection, required IDs, immediate activation, and the one-time secret exposure behavior. This is sufficient for an agent to use the tool correctly without needing the schema to explain operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all 18 parameters with full coverage, so the baseline is 3. The description reinforces which IDs are required for which key types, but this information is also present in the schema. It adds little genuinely new parameter-specific meaning, though it does help synthesize the key type relationships.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create a Portkey API key') with a clear purpose ('for auth') and distinguishes org keys from workspace keys by scope. However, it does not explicitly contrast itself with the sibling create_virtual_key, so sibling differentiation is incomplete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides useful conditional guidance: org keys for broader access, workspace keys for scoped access, and it flags the required companion parameters (workspace_id for workspace keys, user_id for user keys). It does not name alternatives or state when this tool should be avoided in favor of a sibling like create_virtual_key.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_collectionA

Create a new prompt collection for organizing prompts by app. Use this when you need a new namespace before create_prompt; returns the collection id and slug, and does not move any prompts.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesCollection name (e.g., 'hourlink', 'apizone', 'research-pilot')
workspace_idNoWorkspace ID to create collection in

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, destructiveHint=false) indicate write operation that is not destructive. Description adds that it returns collection id and slug and does not move prompts, providing useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is concise, one sentence with two clauses. Front-loaded with main purpose. Could be slightly more structured but is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given simple tool with output schema, description covers key aspects: purpose, usage context, return values. No missing critical details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so schema already describes both parameters. Description adds marginal value: it relates 'name' to namespace concept, but doesn't add semantic detail beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Create' and resource 'prompt collection' with explicit purpose 'organizing prompts by app'. Distinguishes from sibling create_prompt by noting it's a prerequisite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use this when you need a new namespace before create_prompt', providing clear context. Does not explicitly state when not to use, but context implies it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_configA

Create a config that defines routing, cache, retry, and targets for requests; use update_config to modify an existing one and list_config_versions for history. At least one setting is required, new configs become active immediately once referenced by a key or prompt, and the call returns the new id and version_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the new configuration
targetsNoArray of target providers with virtual keys
cache_modeNoCache mode: 'simple' or 'semantic'
workspace_idNoWorkspace ID to create config in
cache_max_ageNoCache max age in seconds
strategy_modeNoRouting strategy: 'loadbalance' or 'fallback'
retry_attemptsNoNumber of retry attempts (1-5)
retry_on_status_codesNoHTTP status codes to retry on (e.g., [429, 500, 502, 503])

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses important behavioral traits beyond annotations: 'At least one setting is required, new configs become active immediately once referenced by a key or prompt, and the call returns the new id and version_id.' This adds activation semantics and return value details, giving the agent clear expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, and every clause earns its place. The structure is efficient with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 8-parameter schema with full descriptions and an output schema (so return values are already documented), the description covers all necessary contextual aspects: what it does, when to use it, key behavioral rules, and activation implications. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds a non-obvious constraint ('At least one setting is required') not enforced by the schema (which only requires name), and mentions return fields (id, version_id), providing extra semantic context beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Create a config that defines routing, cache, retry, and targets for requests.' It uses a specific verb (create) and resource (config), and distinguishes itself from siblings by referencing update_config and list_config_versions as alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides usage guidance: 'use update_config to modify an existing one and list_config_versions for history.' This tells the agent when to use create_config versus related tools, and also implies it is for new configs only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_feedbackA

Create feedback for a trace or request. Writes a new feedback record linked by trace_id, returns the created feedback IDs and status, and takes effect immediately; use update_feedback when correcting an existing record.

ParametersJSON Schema
NameRequiredDescriptionDefault
valueYesFeedback value/rating. Common patterns: 1 for positive (thumbs up), 0 for negative (thumbs down), or use a scale like 1-5.
weightNoOptional weighting factor for the feedback. Use to give more importance to certain feedback.
metadataNoOptional custom metadata for categorization and analysis (e.g., feedback_source, category, user_segment).
trace_idYesThe trace ID to associate the feedback with. This links feedback to a specific request/generation.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate write operation (readOnlyHint false) and non-destructive (destructiveHint false). The description adds behavioral context: 'takes effect immediately' and 'returns the created feedback IDs and status', which go beyond basic annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that communicates the core action, return value, and alternative. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity (4 parameters, one nested object) and presence of an output schema, the description covers purpose, usage, and behavioral impact. It could mention that weight/metadata are optional, but schema already does so. Almost complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (all parameters have descriptions). The description does not add new semantic meaning beyond the schema; it only reinforces the trace_id link already documented. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('create feedback'), resource ('for a trace or request'), and the linking mechanism ('by trace_id'). It explicitly distinguishes itself from the sibling tool 'update_feedback' by mentioning its use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'use update_feedback when correcting an existing record', which is a clear when-to-use vs when-not-to-use instruction. No other usage guidelines needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_guardrailA

Create a guardrail with checks and actions for request filtering. Create it first, then reference it from configs; the new version becomes the policy anchor for downstream use.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the guardrail
checksYesArray of checks to apply (at least one required)
actionsYesActions to take when guardrail checks pass or fail
workspace_idNoWorkspace ID to create the guardrail in
organisation_idNoOrganisation ID (required if workspace_id not provided)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already show this is a write, non-idempotent, non-destructive operation. The description adds lifecycle context: the created guardrail becomes the policy anchor for downstream configs, which is useful beyond the structured fields. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two focused sentences state the action and the key lifecycle constraint without filler. The wording is compact and front-loaded, though 'policy anchor for downstream use' is slightly abstract.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with a rich schema, output schema, and annotations, the description covers the essential workflow: create first, then reference from configs. Additional behavioral details are not necessary because the schema documents parameters thoroughly and annotations cover safety.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters and their nested fields. The description only names checks and actions at a high level and does not add semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Create') and resource ('guardrail'), and clarifies its purpose as request filtering via checks and actions. This distinguishes it from the many sibling tools like get_guardrail, update_guardrail, and delete_guardrail.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly instructs to create the guardrail before referencing it from configs, which communicates the correct call order. It does not explicitly contrast with update_guardrail, but the create-first guidance is clear context for when to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_integrationA

Create an AI-provider integration that becomes the source for workspace providers. ai_provider_id identifies the backend; provider-specific fields configure Azure, Bedrock, Vertex, or custom hosts. For workspace-scoped integrations, create_default_provider controls automatic provider creation. key is write-only, but secret_mappings can resolve it or configuration fields from Secret References at runtime. pricing_adjustments apply negotiated discounts or markups to cost accounting. Use update_integration_models and update_integration_workspaces after creation; returns the new integration id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoAPI key for the provider (if required)
nameYesHuman-readable name for the integration
slugNoURL-friendly identifier (auto-generated from name if not provided)
aws_regionNoAWS region (for AWS Bedrock)
api_versionNoAPI version (for Azure OpenAI)
custom_hostNoCustom base URL for the provider
descriptionNoOptional description of the integration
workspace_idNoWorkspace ID for workspace-scoped integrations
resource_nameNoResource name (for Azure OpenAI)
vertex_regionNoGCP region (for Vertex AI)
ai_provider_idYesID of the AI provider (e.g., 'openai', 'anthropic', 'azure-openai', 'aws-bedrock', 'vertex-ai')
deployment_nameNoDeployment name (for Azure OpenAI)
secret_mappingsNoUnique runtime Secret Reference mappings; mapping key allows the key field to be omitted
aws_access_key_idNoAWS access key ID (for AWS Bedrock)
vertex_project_idNoGCP project ID (for Vertex AI)
pricing_adjustmentsNoNegotiated discount or markup multipliers for cost accounting
aws_secret_access_keyNoAWS secret access key (for AWS Bedrock)
default_provider_slugNoCustom slug for the automatically created workspace provider
create_default_providerNoCreate a provider automatically for a workspace-scoped integration; defaults to true

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds significant behavioral context beyond annotations: key is write-only, secret_mappings can resolve keys or configuration fields at runtime, and pricing_adjustments affect cost accounting. It also discloses that the integration becomes the source for workspace providers and returns the new integration id and slug. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise yet comprehensive, with each sentence serving a distinct purpose: purpose, key field explanations, behavior controls, security notes, and follow-up steps. It is front-loaded with the core action and avoids redundancy, making it effective for a tool with 19 parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex creation tool, the description covers essential aspects: purpose, key parameters, conditional behavior, follow-up actions, and return value. The output schema further defines the return. It lacks explicit prerequisites (e.g., existence of ai_provider_id) but overall provides sufficient context for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While the schema already covers all 19 parameters (100% coverage), the description adds valuable semantic grouping: ai_provider_id identifies the backend, provider-specific fields configure cloud providers, and create_default_provider controls automatic provider creation. It also explains the relationship between key and secret_mappings, going beyond individual parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's primary function: 'Create an AI-provider integration that becomes the source for workspace providers.' It distinguishes this from sibling tools like create_provider and create_mcp_integration by specifying the integration's role. The mention of provider-specific fields for Azure, Bedrock, Vertex, and custom hosts further clarifies its scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use the tool, including context for workspace-scoped integrations via create_default_provider. It also directs users to use update_integration_models and update_integration_workspaces after creation. However, it does not explicitly state when not to use this tool or mention alternatives like create_provider for different scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_log_exportA

Create a log export definition with filters and requested fields. This only sets up the export and does not start processing; call start_log_export next, then use get_log_export or download_log_export to inspect or retrieve the finished result. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
ai_modelNoFilter by specific AI model names
cost_maxNoMaximum cost filter
cost_minNoMinimum cost filter
time_maxYesMaximum time filter in date format (e.g., '2024-01-31' or ISO 8601)
time_minYesMinimum time filter in date format (e.g., '2024-01-01' or ISO 8601)
descriptionNoHuman-readable description for the export job
workspace_idNoWorkspace ID for the export
total_units_maxNoMaximum total units (tokens) filter
total_units_minNoMinimum total units (tokens) filter
requested_fieldsYesFields to include in export: id, trace_id, created_at, request, response, is_success, ai_org, ai_model, req_units, res_units, total_units, request_url, cost, cost_currency, response_time, response_status_code, mode, config, prompt_slug, metadata

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses a key behavioral trait—that creation does not start processing—and the expected follow-up workflow. Also notes the 403 error on non-Enterprise plans. While annotations indicate readOnlyHint=false, the description adds meaningful context about the asynchronous workflow and access restriction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each with a distinct purpose: action, workflow steps, and access caveat. Information is front-loaded and no redundant wording appears.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex tool with 10 parameters and an output schema, the description covers the critical workflow (create→start→retrieve) and enterprise restriction. Return values are already documented by the output schema, so no additional explanation is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% coverage with detailed descriptions for all 10 parameters, including enums for requested_fields. The description only adds generic 'filters and requested fields', so it does not need to elaborate further. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a log export definition with filters and requested fields' with a specific verb and resource. It further distinguishes the tool from siblings by explicitly noting that it 'only sets up the export and does not start processing', preventing confusion with start_log_export or download_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit workflow guidance: 'call start_log_export next, then use get_log_export or download_log_export'. Also mentions Enterprise-gating and 403 behavior, which helps decide when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_mcp_integrationA

Create a Portkey integration for an external MCP server URL. For headers auth, provide custom_headers or a Secret Reference mapping targeting configurations.custom_headers; secret_mappings resolve protected values at runtime without storing them in the tool call. Organisation admin keys normally need workspace_id. After creation, create_mcp_server and configure capabilities/access; returns the integration id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesURL endpoint of the MCP server to integrate
nameYesDisplay name for the MCP integration
slugNoCustom slug. Auto-generated if omitted
auth_typeYesAuthentication type. Current examples are 'none', 'headers' (custom headers), and 'oauth_auto' (OAuth); Portkey may accept additional values.
transportYesMCP transport protocol: 'http' (streamable HTTP) or 'sse' (server-sent events)
descriptionNoDescription of the MCP integration
workspace_idNoWorkspace ID — required when using organization admin API keys
configurationsNoAdditional documented or forward-compatible configuration fields. For headers auth, configurations.custom_headers is a string-to-string header map.
custom_headersNoCustom headers for authentication (e.g. { "Authorization": "Bearer xxx" }). Sent via configurations.custom_headers
secret_mappingsNoRuntime Secret Reference mappings; every configurations.<field> target must be unique

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry the basic safety profile (readOnlyHint=false, destructiveHint=false), so the bar is lower. The description adds real behavioral value: secret_mappings resolve protected values at runtime without storing them in the tool call, the tool does not configure capabilities (that belongs to create_mcp_server), and it returns id and slug. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences and roughly 55 words with the purpose front-loaded and each sentence carrying distinct information. The second sentence is dense (semicolon linking the headers-auth choice with runtime secret resolution), and the last sentence mixes next-step routing with the return-value note, making the tail slightly cluttered. Nothing is wasted, but it is not as clean as it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 10-parameter tool with nested secret_mappings objects and an open-world auth_type, the description covers the main decision points: what it creates, how headers auth works, the org admin key requirement, and the mandatory follow-on tool. The output schema exists, lowering the burden. Gaps remain — the oauth_auto path is not addressed in prose and the boundary with create_integration is only implicit — but the essential call path is fully navigable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by tying together the parameter interplay: custom_headers is sent via configurations.custom_headers and can be replaced by a secret mapping targeting that same field, secret_mappings resolve at runtime, and workspace_id becomes relevant specifically for org admin keys. It does not walk through every parameter, but the treacherous auth-related ones are covered.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Create a Portkey integration for an external MCP server URL' — and the MCP qualifier distinguishes it from the generic create_integration sibling. It also explicitly names create_mcp_server as the separate follow-on tool, so an agent can tell this tool apart from the rest of the MCP family immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides conditional guidance for headers auth (custom_headers vs a Secret Reference mapping targeting configurations.custom_headers), flags the workspace_id prerequisite for org admin keys, and routes the agent to create_mcp_server as the required next step. This is clear, actionable context that names a sibling, though it does not explicitly state when-not scenarios such as 'for non-MCP integrations use create_integration'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_mcp_serverA

Create an MCP server under an existing integration. Registers the server and returns the new id and slug; use list_mcp_integrations first to find the parent integration, then capabilities or access tools to configure it.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the MCP server
slugNoCustom slug. Auto-generated if omitted
descriptionNoDescription of the MCP server
mcp_integration_idYesID or slug of the MCP integration this server belongs to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a write operation (readOnlyHint=false) and non-destructive (destructiveHint=false). The description adds that it registers the server and returns id/slug, but does not elaborate on potential side effects or additional behavior beyond creation. With minimal annotation context, the description provides some added value but not extensive behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two focused sentences. The first sentence states the core purpose, and the second provides contextual workflow advice. While efficient, the second sentence could be slightly more compact, but no unnecessary information is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (stated in context), the description appropriately omits detailed return value documentation. It mentions the returned id and slug. The description also advises on a necessary prerequisite step. For a creation tool with good parameter documentation, this is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 4 parameters. The description mentions the required parameters (name, mcp_integration_id) and notes slug and description are optional, but does not add new constraints or semantics beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates an MCP server under an existing integration. It specifies the resource ('MCP server') and action ('Create'), and distinguishes from sibling tools by mentioning the prerequisite of using list_mcp_integrations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: use list_mcp_integrations first to find the parent integration, then use this tool to create the server, and later use capabilities or access tools for configuration. This clearly indicates when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_promptA

Create a new prompt template and initial version. Use this for first-time setup; use migrate_prompt for idempotent CI/CD flows. Accepts plain text or structured chat messages, creates a new version immediately, and returns the prompt id, slug, and version id. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the prompt
modelNoModel identifier (e.g., 'gpt-4', 'claude-3-opus'). Required unless ai_model_id or finetune_id is provided
toolsNoTool definitions for tool use
stringNoLegacy prompt template string. Use plain text for single-message prompts, or a JSON-encoded messages array string for multi-message chat prompts.
dry_runNoWhen true, validate without creating
messagesNoStructured chat template alias. Serialized to the legacy string format before creation.
functionsNoFunction definitions for function calling
parametersYesDefault values for template variables
ai_model_idNoAI model ID (alternative to model). Required unless model or finetune_id is provided
finetune_idNoFine-tune ID (alternative to model). Required unless model or ai_model_id is provided
tool_choiceNoTool choice strategy
virtual_keyYesVirtual key slug for model access
collection_idYesCollection ID to organize the prompt in (use list_collections to find)
is_raw_templateNoRender the stored template as a raw structural template
template_metadataNoCustom metadata (app, env, source_file, etc.)
version_descriptionNoDescription for this prompt version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already declaring readOnlyHint=false and idempotentHint=false, the description adds useful behavior: it creates a new version immediately and returns prompt id, slug, and version id. It does not contradict the annotations. It could add a bit more about side effects or validation behavior, but the key mutation semantics are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four compact sentences, each earning its place: purpose, usage context, behavior, and parameter guidance. No filler or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 16 parameters, nested objects, and an output schema, this description provides the critical orientation: what it creates, when to use it instead of migrate_prompt, how to handle message formats, and what it returns. It doesn't exhaustively explain every field, but the schema already does that. Minor missing context like dry_run behavior is covered by the schema, so the description is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds meaningful clarification beyond the schema by explaining the relationship between plain text, structured messages, and the JSON-encoded string format, specifically preferring messages for multi-message chat prompts. This helps an agent choose between the two overlapping parameter options.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create a new prompt template and initial version') and clearly distinguishes it from migrate_prompt and update_prompt. It also names the resource and the immediate effect, so an agent can identify what this tool does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use this for first-time setup and points to migrate_prompt for idempotent CI/CD flows. It also gives guidance on choosing between messages and string for multi-message prompts, which is practical invocation guidance beyond the schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_prompt_labelA

Create a prompt label for tagging prompt versions such as production, staging, or experiment. Requires either organisation_id or workspace_id to set scope, returns the new label id, and does not assign it to any versions yet.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the label
color_codeNoHex format, e.g. '#FF5733'. Optional — omit for default color.
descriptionNoDescription of the label
workspace_idNoWorkspace ID to create the label in
organisation_idNoOrganisation ID to create the label in

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate it is not read-only (write operation), not destructive, and open world. The description adds that it returns the new label id and does not assign to any versions. This provides useful behavioral context beyond annotations, but could mention if there are any side effects on existing labels or other entities.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the purpose, and every word adds value. No redundant or vague phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists, the description does not need to detail return structure but mentions the id. It covers scope requirement, lack of assignment, and optional color. For a creation tool with 5 parameters, this is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with descriptions. The description adds that either organisation_id or workspace_id is required (though schema marks only name as required), which is a critical constraint. It also clarifies that color_code is optional and defaults to a default color. This adds meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a prompt label' and gives concrete examples like 'production, staging, or experiment'. It distinguishes the tool from siblings by noting that it does not assign the label to any versions yet, which sets it apart from assignment or update actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies that either organisation_id or workspace_id is required to set scope, providing clear guidance on parameter usage. It also notes that the label is not assigned to versions, implying a subsequent step. However, it does not explicitly state when to use this tool versus alternatives like update_prompt_label.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_prompt_partialA

Create a reusable prompt partial for inclusion with {{> partial_name}}. Use this for shared snippets or macros; returns the partial id, slug, and version id, and the new version stays inactive until published.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the partial
stringYesThe partial content/template string
workspace_idNoWorkspace ID to create partial in (required for org-level API keys)
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide basic hints (readOnlyHint=false), and the description adds behavioral context: returns specific IDs (partial id, slug, version id) and notes the version remains inactive until published. This goes beyond annotations without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. The verb and resource are front-loaded, and the key return values and state are succinctly covered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the existence of an output schema, the description appropriately focuses on key return fields and version state. It omits details like workspace_id requirements for org-level keys, but overall it is sufficiently complete for a creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with descriptions for all 4 parameters. The description does not add extra meaning beyond the schema, so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create') and the resource ('reusable prompt partial'), and explains its purpose for shared snippets/macros. It distinguishes itself from siblings like 'create_prompt' by emphasizing reusability and the inactive status until publication.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case ('for shared snippets or macros') and mentions that the version stays inactive until published, which guides when to use. However, it does not explicitly state when not to use this tool or mention alternatives like 'publish_partial' for activation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_providerA

Create a workspace provider backed by an org integration. The provider inherits the integration key, but its limits and expiration are enforced independently for that workspace. Returns the new provider id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the provider
noteNoOptional note or description for the provider
slugNoCustom slug for the provider. Auto-generated with random suffix if omitted
expires_atNoExpiration date in ISO 8601 format
credit_limitNoCredit limit for usage
workspace_idNoWorkspace ID - required when using organization admin API keys
integration_idYesIntegration slug for the provider (e.g., 'openai', 'anthropic', 'azure-openai')
periodic_resetNoPeriod for resetting usage limits: 'monthly' or 'weekly'. Defaults to 'monthly'.
alert_thresholdNoAlert threshold percentage (0-100)
rate_limit_unitNoMust be provided together with rate_limit_value. Values: 'rpm' (requests/min), 'rph' (requests/hour), or 'rpd' (requests/day).
rate_limit_valueNoMust be provided together with rate_limit_unit.
usage_limit_typeNoType of usage limit: 'cost' (monetary) or 'tokens' (token count). Defaults to 'cost'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a write operation (readOnlyHint=false) that is not destructive (destructiveHint=false). The description adds behavioral detail that the provider inherits the integration key but enforces limits and expiration independently per workspace, providing nuance beyond annotations. It could mention error conditions like duplicate names.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences efficiently cover purpose, behavioral context, and return value with no wasted words. Very concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 12 parameters and the presence of an output schema, the description adequately covers creation and the return of id and slug. It could mention the prerequisite of an existing integration, but overall it is complete for a creation tool with annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing a baseline of 3. The description adds context about how integration_id and workspace_id interact (inherits key but independent limits), which adds value beyond individual parameter descriptions. No redundancy.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Create' and the resource 'workspace provider backed by an org integration', distinguishing it from sibling tools like create_integration and update_provider. It also mentions the return value.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a provider tied to an integration is needed, but does not explicitly state when to use or not use this tool versus siblings like create_integration or create_virtual_key. No exclusions or alternatives are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_rate_limitA

Create a request or token rate-limit policy. Conditions and grouping must be non-empty; target llm uses model/provider keys, while target mcp_tools uses MCP server and tool keys.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoOptional policy display name
typeYesCounter type
unitYesCounter reset interval
valueYesMaximum requests or tokens
targetNoLLM or MCP-tool enforcement target
group_byYesCounter grouping dimensions
conditionsYesPolicy match conditions
workspace_idNoWorkspace UUID scope
organisation_idNoOrganisation UUID scope

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish a write, non-idempotent, non-destructive profile, and the description agrees with that. It adds value beyond the annotations by disclosing a validation constraint (non-empty conditions/grouping will otherwise fail) and the target-dependent key scoping behavior. It does not disclose duplicate-policy behavior, but idempotentHint=false already signals that risk.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences totaling roughly 35 words: the core purpose comes first, and the two critical constraints follow immediately. No filler, no repetition of schema content, and every clause carries meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter create tool with an output schema present, the two constraints most likely to cause a failed call — non-empty conditions/grouping and target-appropriate keys — are both captured. The only minor gap is that the description never guides when to choose 'requests' versus 'tokens', though the schema's type and value descriptions cover their meaning. Nothing essential for a correct first call is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description earns a 4 by adding the cross-parameter insight that the choice of target (llm vs mcp_tools) dictates which keys are valid in conditions and group_by — a relationship no individual schema field description states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb ('Create') and resource ('request or token rate-limit policy'), which cleanly differentiates this from siblings update_rate_limit, delete_rate_limit, get_rate_limit, and the similarly-named create_usage_limit. The requests-vs-tokens distinction is front-loaded so an agent immediately understands it creates counter-based rate-limit policies, not spend caps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides operational context — conditions and grouping must be non-empty, and target llm versus target mcp_tools determines which keys apply — but it never names an alternative or a when-not condition. An agent must infer when to prefer create_rate_limit over create_usage_limit or update_rate_limit from the create_ verb and resource name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_scim_workspace_mappingA

Map one identity-provider SCIM group to a Portkey workspace role so current and future group members receive access automatically. Provide exactly one of scim_group_id or scim_group_name; a name can pre-create the Portkey SCIM group before the IdP provisions it. Use list_scim_groups to discover existing groups and list_workspaces for the workspace ID. This changes access provisioning and is distinct from add_workspace_member, which grants one user directly. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesWorkspace role automatically granted to group members
workspace_idYesPortkey workspace ID that the SCIM group should access
scim_group_idNoExisting identity-provider SCIM group ID; provide this or scim_group_name, but not both
scim_group_nameNoExisting SCIM group display name; provide this or scim_group_id, but not both

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a mutating operation (readOnlyHint=false), but the description adds side-effect context: it provisions access for current and future members, can pre-create the group via scim_group_name, and may return 403 on non-Enterprise plans. This goes well beyond the structured fields and provides essential behavioral expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each adding distinctive information: core purpose, parameter constraint/pre-creation, discovery tool references, side-effect/enterprise-403 warning. No redundant filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex SCIM mapping operation with an output schema present, the description covers purpose, prerequisites, parameter selection rule, side effects, and error condition. It leaves return-value documentation to the output schema, which is acceptable. No significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all four parameters with descriptions covering the one-of constraint. The description reinforces the exactly-one rule and adds the pre-creation behavior for scim_group_name, which is not in the schema. Given 100% schema coverage, this extra nuance elevates the score above baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+object+effect: 'Map one identity-provider SCIM group to a Portkey workspace role so current and future group members receive access automatically.' It clearly distinguishes from sibling `add_workspace_member` by stating it grants access via group rather than individually.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names prerequisite discovery tools (list_scim_groups, list_workspaces) and compares to the alternative add_workspace_member, stating when this tool is appropriate ('distinct from... grants one user directly'). Also notes Enterprise-gated plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_secret_referenceA

Create a reference to a secret stored in AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. Authentication credentials are sent to Portkey and exposed to this MCP transcript; use short-lived or workload identity modes when possible. Returns the new UUID and slug, never the resolved external secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesHuman-readable name
slugNoOptional stable slug containing only letters, numbers, underscores, or hyphens; Portkey generates one when omitted
tagsNoOptional string key-value tags for filtering and organization
secret_keyNoOptional key within a structured external secret; null uses the whole secret value
auth_configYesChoose exactly one documented authentication shape matching manager_type. Credentials are sent to Portkey and visible in the MCP transcript; prefer workload identity or short-lived credentials when available.
descriptionNoOptional description; use null to leave it empty
secret_pathYesPath of the secret in the external secret manager
manager_typeYesExternal secret manager: aws_sm for AWS Secrets Manager, azure_kv for Azure Key Vault, or hashicorp_vault for HashiCorp Vault; must match the auth_config field family
organisation_idNoOrganisation UUID; required when the MCP server is not authenticated with a Portkey API key
allowed_workspacesNoWorkspace UUIDs or slugs allowed to use this reference; omit when allow_all_workspaces is true
allow_all_workspacesNoWhether every workspace may use this reference; defaults to true and cannot be true with allowed_workspaces

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that authentication credentials are sent to Portkey and exposed in the transcript, and clarifies that the return value is a UUID and slug, not the actual secret. This adds value beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with no filler; front-loaded with purpose and followed by important behavioral notes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (11 parameters, auth patterns) and the existence of an output schema, the description covers the essentials: creation, security implications, and return type. Could mention the relationship between manager_type and auth_config, but schema descriptions compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions; the tool description does not add new parameter-level details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it creates a reference to external secrets from three named providers and explicitly distinguishes itself from other tools by mentioning it never returns the resolved secret.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides security guidance to prefer short-lived or workload identity modes, but does not explicitly compare against sibling tools like update_secret_reference or delete_secret_reference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_usage_limitA

Create a cumulative cost or token usage-limit policy with non-empty conditions and grouping. A periodic reset can be weekly, monthly, or omitted for a cumulative lifetime limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoOptional policy display name
typeYesBudget type
group_byYesUsage grouping dimensions
conditionsYesPolicy match conditions
credit_limitYesMaximum cumulative usage
workspace_idNoWorkspace UUID scope
periodic_resetNoOptional reset schedule
alert_thresholdNoOptional alert threshold
organisation_idNoOrganisation UUID scope

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a write operation and non-destructive; the description adds value by clarifying that an omitted periodic_reset results in a cumulative lifetime limit. It also communicates the validation constraint that conditions and grouping must be non-empty, which is useful beyond the schema alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short, front-loaded sentences capture the purpose, key constraints, and reset behavior with no filler. Every clause adds distinct information that an agent needs for correct tool selection and invocation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input schema, output schema, and annotations, the description covers the essential decision-making context. It could be slightly more complete by explicitly contrasting with create_rate_limit, but the current wording is sufficient for an agent to understand the tool's role.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaningful parameter semantics by explaining that periodic_reset can be weekly, monthly, or null for a lifetime cumulative limit, and by associating 'cumulative cost or token' with the type parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Create'), resource ('cumulative cost or token usage-limit policy'), and key constraints ('with non-empty conditions and grouping'). This distinguishes it from related siblings like create_rate_limit and update_usage_limit by emphasizing cumulative usage budgets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use this tool: when creating a cumulative cost/token policy rather than a rate-based limit. It does not explicitly name alternatives or state exclusions, but the 'cumulative' qualifier and reset semantics provide clear contextual guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_virtual_keyA

Store a provider API key as a virtual key. The raw key is encrypted and only returned at creation time, so save the returned slug and use it in prompts/configs. Optional usage and rate limits apply immediately, and the tool returns the new slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoProvider API key; omit when secret_mappings supplies key
nameYesDisplay name for the virtual key
noteNoOptional note or description for this key
providerYesProvider slug (e.g., 'openai', 'anthropic', 'azure-openai', 'google')
expires_atNoExpiration in ISO 8601 format
api_versionNoAPI version (for Azure OpenAI)
credit_limitNoMaximum usage cost threshold
workspace_idNoWorkspace ID to create the key in
resource_nameNoResource name (for Azure OpenAI)
rate_limit_rpmNoRequests per minute limit
alert_thresholdNoPercentage of credit_limit at which to send alert emails (0-100)
deployment_nameNoDeployment name (for Azure OpenAI)
secret_mappingsNoSecret Reference mappings for key or model_config fields
deployment_configurationsNoAzure deployment configurations with API versions and aliases

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal a non-read-only, non-idempotent write, but the description adds genuinely critical behavior beyond them: the raw key is encrypted, it is returned exactly once at creation time, usage/rate limits take effect immediately, and the tool returns a slug. An agent that misses the one-time-return fact could make an irreversible reference mistake later.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, all information-bearing, with purpose front-loaded. The only minor flaw is a slight redundancy: 'only returned at creation time' and 'returns the new slug' both convey that the slug is the output, so the third sentence's fresh content is just the immediate-application-of-limits point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a 14-parameter tool and an existing output schema, the description correctly focuses on the one operational gotcha an agent must not miss (one-time raw-key return and slug retention) while leaving parameter details to the fully covered schema. It is complete enough to invoke the tool correctly; Azure-specific conditional parameters are already documented per-property in the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema carries the parameter-documentation burden and the baseline is 3. The description adds a small amount of semantic glue ('usage and rate limits' maps to credit_limit and rate_limit_rpm, and 'returns the new slug' hints at the output), but it does not compensate for or extend the schema's parameter docs in any substantial way.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource pairing ('Store a provider API key as a virtual key') and immediately establishes the defining traits that separate this from plain API-key creation: encryption, slug-based reference, and use in prompts/configs. Against a sibling set containing both create_api_key and create_virtual_key, this clearly identifies the niche.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear usage context: the tool is for persisting a provider key in a referenceable form, and it instructs the agent to save the returned slug and use it in prompts/configs. It stops short of explicitly naming alternatives or stating when not to use it (e.g., versus create_api_key), so it earns a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_workspaceA

Create a workspace to isolate resources, API keys, and team members. If slug is omitted it is auto-generated from the name; returns the new workspace id, name, and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the workspace
slugNoURL-friendly slug (auto-generated if not provided)
usersNoExisting user IDs to add
metadataNoCustom metadata key-value pairs
is_defaultNoSet as default workspace (1 = yes, 0 = no)
descriptionNoDescription of the workspace
rate_limitsNoInitial workspace rate-limit settings
usage_limitsNoInitial workspace usage-limit settings

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already indicating a non-read-only, non-idempotent mutation, the description adds useful behavioral details: slug auto-generation when omitted and the returned fields (id, name, slug). This is meaningful context beyond the structured annotation data, though it does not detail every side effect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences front-load the action and purpose, then add the key default behavior and return value. Every sentence earns its place; no filler or redundant expansion.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (8 parameters, nested rate/usage-limit objects), but the schema and output schema cover those details thoroughly. The description supplies the important high-level context, isolation purpose, and creation behavior, making it adequate without needing to enumerate nested settings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% property description coverage, so the baseline is 3. The description repeats the slug auto-generation rule already present in the schema and does not add meaning for the other seven parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'Create a workspace to isolate resources, API keys, and team members.' This clearly differentiates the tool from the many sibling create_* tools that create API keys, providers, virtual keys, prompts, or other entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear usage context by explaining the purpose ('to isolate resources, API keys, and team members'), which implies when workspace creation is appropriate. It does not explicitly name alternatives or exclusions, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_api_keyA
Destructive

Delete an API key by UUID. This cannot be undone, revokes access immediately, and can break active sessions using the key. Returns success after revocation.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavioral details beyond annotations: states it cannot be undone, revokes immediately, can break sessions, and returns success after revocation. Annotations only provide destructiveHint=true.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, all information relevant and front-loaded. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the effect and response, but does not mention prerequisites like permissions. For a simple delete, it is still fairly complete given annotations and output schema exist.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of the single parameter with description. Description adds no extra param details beyond mentioning UUID, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states specific action 'Delete an API key by UUID', clearly distinguishing from siblings like create_api_key and update_api_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly mentions permanence, immediate revocation, and potential impact on active sessions. Could be improved by contrasting with alternatives, but the destructive context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_collectionA
Destructive

Delete a prompt collection by ID. This cannot be undone; prompts stay in the workspace but lose their collection grouping, so reassign them first if organization matters.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idYesCollection ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate destructiveHint=true. The description adds context about irreversibility and the effect on prompts (lose grouping), going beyond annotations. It does not contradict any annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: first states the purpose, second adds behavioral context. Every sentence adds value, and it is front-loaded. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core action and its effects. It is complete for a simple delete operation with one parameter. Could mention the output schema, but not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for collection_id. The tool description does not add additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete a prompt collection by ID', specifying the verb and resource. It distinguishes itself from siblings like delete_prompt by focusing on collections and their effect on prompt grouping.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on when to use (delete collection) and warns about consequences ('cannot be undone'). It suggests reassigning prompts first if organization matters, but does not explicitly list alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_configA
Destructive

Delete a config by slug. This is permanent, removes all versions, and breaks anything still pointing at that slug; check list_config_versions first.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesConfiguration slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description adds key details: permanent deletion, removal of all versions, and breaking anything pointing to the slug. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with two concise clauses, front-loaded with the action. Every word adds value; no unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with an output schema (exists elsewhere), the description covers the action, permanence, and a crucial precaution. It is fully adequate for an agent to use without ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter 'slug' with description 'Configuration slug to delete'. The description does not add additional parameter details, meeting the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Delete a config by slug.' It explicitly names the resource and action, and the sibling tools (create_config, update_config, get_config, list_configs) make it distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description warns of permanence and breaks dependencies, and suggests checking list_config_versions first. It provides clear when-to-use/not-use context, but lacks explicit mention of alternatives like update_config for deprecation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_guardrailA
Destructive

Delete a guardrail by id or slug. This is irreversible and removes the check from any configs that reference it, so review dependent configs first.

ParametersJSON Schema
NameRequiredDescriptionDefault
guardrail_idYesThe guardrail UUID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond the annotations (destructiveHint=true) by detailing the irreversible nature and the cascading effect on configs. This adds valuable behavioral context about consequences, complementing the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences. The first sentence states the purpose, and the second provides a critical warning. No unnecessary words, earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple nature of a delete operation with one required parameter and an existing output schema, the description covers all necessary context: what it does, how to identify the guardrail, and the irreversible consequences. Complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'guardrail_id' is already well-described in the schema as 'The guardrail UUID or slug to delete'. The description does not add additional meaning beyond what the schema provides, meeting the baseline for 100% coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and the resource 'guardrail', specifying identification by 'id or slug'. It distinguishes itself from sibling tools like 'create_guardrail' or 'update_guardrail'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: it warns that the action is irreversible and removes checks from dependent configs, advising to review those first. While it doesn't explicitly list when not to use, the warning implies careful consideration. It offers clear context for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_integrationA
Destructive

Delete an integration by slug. This is irreversible and stops the org-level connection, which will break dependent virtual keys, providers, and workspace access.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already set destructiveHint=true and readOnlyHint=false. The description adds value by specifying irreversibility and cascading effects on virtual keys, providers, and workspace access.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the primary action, and every sentence adds critical information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter delete operation, the description covers purpose, consequences, and parameter. It is nearly complete, though the output schema is not described.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the description adds no extra meaning beyond the schema's explanation of the 'slug' parameter. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete an integration by slug,' providing a specific verb and resource. It distinguishes this tool from siblings like delete_api_key or delete_provider, which target different entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that deletion is irreversible and breaks dependent entities, giving clear context for when to use the tool. However, it does not explicitly mention when not to use it or suggest alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_integration_modelA
Destructive

Delete a custom model from an integration. Built-in models should be disabled instead, because deletion only applies to custom entries. Returns success after the custom model is removed.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
model_slugYesThe slug of the model to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description explains that deletion only applies to custom entries and that success is returned after removal. This adds behavioral context not fully captured by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words, front-loaded with action and key constraint. Highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with full schema coverage and an output schema, the description covers purpose, usage, behavioral traits, and return behavior completely. Annotations and schema handle the rest.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description does not add additional meaning to the parameters beyond their names and definitions in the schema. Baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete a custom model from an integration') and distinguishes between custom and built-in models, implying that built-ins should be handled differently. This differentiates it from sibling tools like 'update_integration_models'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use (custom models) and when not to (built-in models should be disabled instead), providing clear usage guidelines. It does not name the alternative tool but implies it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_mcp_integrationA
Destructive

Delete an MCP integration and all servers beneath it. This is irreversible, removes connected access immediately, and should only be used after confirming nothing depends on the integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavioral context beyond annotations: irreversible, removes access immediately, cascades to servers. Annotations only mark destructiveHint=true; the description fills in the details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Action and implications front-loaded, making it easy to quickly understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive delete operation, the description covers irreversibility, immediate impact, cascade, and a usage precaution. Output schema exists, so return values need not be described. Comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides a description for the single parameter 'id' (the integration ID or slug). The tool description does not add further parameter information, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Delete an MCP integration and all servers beneath it', using a specific verb and resource. It distinguishes from sibling delete operations like delete_mcp_server by noting the cascade effect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: after confirming nothing depends on the integration, and warns of irreversibility and immediate access removal. No alternative tools are named, but the caution is sufficient for an irreversible delete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_mcp_serverA
Destructive

Delete an MCP server instance. This is irreversible, removes connected users' access immediately, and should be used only after confirming no workflows depend on the server.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true, but the description adds critical behavioral context: the operation is irreversible and immediately removes connected users' access. This goes beyond the annotation's binary hint. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, providing all necessary information without redundancy. The first sentence states the action, and subsequent sentences add essential warnings. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is an output schema, the description does not need to describe return values. It covers the irreversible nature, immediate user impact, and the caution about workflows. It is sufficient for the agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The parameter 'id' is fully described in the schema ('The MCP server ID or slug to delete'). The description does not add any additional semantic meaning beyond what the schema already provides. With 100% schema coverage, a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('MCP server instance'), making the purpose immediately clear. Among many sibling delete tools, the tool name 'delete_mcp_server' combined with the description sufficiently differentiates it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes an important usage condition: 'should be used only after confirming no workflows depend on the server.' This advises the agent on when it is appropriate to invoke the tool. However, it does not explicitly mention alternatives or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_promptA
Destructive

Delete a prompt and all its versions by id. This cannot be undone, immediately breaks callers using the slug, and should only be used after checking list_prompt_versions or confirming you do not need an audit trail.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark destructiveHint=true and readOnlyHint=false. The description adds specific behavioral context: 'This cannot be undone, immediately breaks callers using the slug,' which goes beyond annotations. However, it does not detail any rate limits or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no unnecessary words. It front-loads the core action ('Delete a prompt...'), then adds critical caveats. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive one-parameter tool with an output schema, the description covers what is deleted, irreversibility, impact on callers, and prerequisite checks. It is complete given the tool's simplicity and the presence of annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for the single parameter 'prompt_id' with a clear description ('Prompt ID or slug to delete'). The tool description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt and all its versions by id.' It uses a specific verb ('Delete') and resource ('prompt and all its versions'), distinguishing it from sibling tools like 'delete_prompt_label' or 'delete_prompt_partial'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'should only be used after checking list_prompt_versions or confirming you do not need an audit trail.' It warns about irreversibility and impact on callers ('immediately breaks callers using the slug'), helping decide when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_prompt_labelA
Destructive

Delete a prompt label by ID. This cannot be undone; versions carrying the label lose it, and any workflow resolving by that label will need a replacement.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond annotations by detailing the irreversible nature and downstream effects on versions and workflows. Annotations only flag destructiveHint=true, but the description explains what gets destroyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states the action, second explains key consequences. Every word is necessary, no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has one parameter and an output schema (though not shown, it exists). The description covers the essential behavioral impact, making it complete for this simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage and only one parameter, the schema already describes the parameter as 'Label ID to delete.' The description does not add additional semantics, so baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt label by ID.' It uses a specific verb and resource, distinguishing it from sibling tools like update_prompt_label or create_prompt_label.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit consequences: 'cannot be undone', 'versions carrying the label lose it', and 'any workflow resolving by that label will need a replacement.' This helps the agent decide when to use it cautiously, though it does not explicitly name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_prompt_partialA
Destructive

Delete a prompt partial by ID. This cannot be undone, and prompts that reference it with {{> name}} will fail to render until you replace the reference.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description adds that deletion is irreversible and affects dependent prompts. This clarifies the side-effect and non-idempotent nature beyond the structured annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states the action, second adds the critical consequence. No unnecessary words, well front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter tool and existence of an output schema (per context signals), the description adequately covers the action and side effects. It is complete for the complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter with 100% schema coverage. The schema already describes it as 'Prompt partial ID or slug to delete.' The description adds no extra details beyond 'by ID,' so baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt partial by ID.' It specifies the resource and identifier, distinguishing it from siblings like 'update_prompt_partial' and 'create_prompt_partial.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the consequence (permanent deletion, prompt render failures) but does not explicitly state when to use this vs. alternatives like updating or disabling. The warning about reference replacement provides practical guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_providerA
Destructive

Delete a workspace provider by slug. This is irreversible and will break prompts, configs, and virtual keys that reference it; use delete_integration for the org source instead. Returns success after the provider is removed.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the provider to delete
workspace_idNoWorkspace ID - required when using organization admin keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true. The description goes further by specifying that deletion is irreversible and will break prompts, configs, and virtual keys, adding valuable behavioral details beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with action, then consequences and alternative. Every word is necessary and there is no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With destructiveHint=true and clear description of consequences, the tool is well-specified. However, it lacks details about permissions or rate limits, which would make it fully complete. Output schema existence is implied but not detailed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage for both parameters (slug, workspace_id). The description does not add extra meaning beyond what the schema already provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and the resource 'workspace provider by slug'. It also distinguishes itself from the sibling tool 'delete_integration' for the org source, eliminating ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use (delete a workspace provider) and when not to (use delete_integration for org source). Also warns about irreversibility and impacts on other entities, providing clear context for decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_rate_limitA
Destructive

Delete a rate-limit policy by id. Portkey removes the active throttling policy immediately, so inspect dependent callers and the full policy before deleting it.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesRate-limit policy UUID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, and the description adds valuable context beyond that: removal is immediate and impacts active throttling, so dependent callers should be checked. This meaningfully augments the structured metadata without contradicting it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler. The core action is front-loaded, and the second sentence adds only high-value safety context. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a single fully documented parameter, an output schema, and destructive annotations. The description adds the missing behavioral warning about immediate policy removal and dependent callers, making it complete for safe invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the id parameter already documented as 'Rate-limit policy UUID to delete.' The description only restates 'by id' and does not add new semantic detail beyond what the schema provides, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Delete a rate-limit policy by id.' This clearly separates it from sibling delete tools like delete_usage_limit, and the additional context about removing the active throttling policy reinforces what the operation does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear practical guidance: before deleting, inspect dependent callers and the full policy. It does not explicitly name alternatives such as get_rate_limit or update_rate_limit, but the warning about immediate removal provides sufficient decision context for when this destructive action is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_scim_workspace_mappingA
DestructiveIdempotent

Delete a SCIM group-to-workspace mapping by mapping_id and stop future group updates from affecting that workspace. Existing provisioned members remain in the workspace and must be managed separately. Inspect list_scim_workspace_mappings first; this does not delete the identity-provider group or the Portkey workspace. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
mapping_idYesSCIM workspace mapping ID from list_scim_workspace_mappings

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint, idempotentHint), the description adds critical behavior: existing provisioned members remain, Enterprise-gating, and 403 error on non-Enterprise plans. This gives the agent important context to set expectations and handle errors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tightly written in three sentences, each providing unique, necessary information. It is front-loaded with the primary action, followed by consequential details, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, rich annotations, and presence of an output schema, the description covers all relevant context: purpose, side effects, prerequisites, exclusions, gating, and error behavior. It is fully self-sufficient for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value by specifying that mapping_id comes from list_scim_workspace_mappings. This reinforces the parameter's origin, though it could include more details about required format or validation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Delete') and resource ('SCIM group-to-workspace mapping'), clearly distinguishing it from sibling tools like create_scim_workspace_mapping and list_scim_workspace_mappings. It also states the key side effect (stopping future group updates), making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to 'Inspect list_scim_workspace_mappings first', providing a clear prerequisite and source for the mapping_id. It also clarifies exclusions (does not delete the identity-provider group or the Portkey workspace), helping the agent avoid misuse.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_secret_referenceA
DestructiveIdempotent

Permanently delete a Secret Reference. This is irreversible and integrations or virtual keys using it can fail to resolve credentials immediately; confirm dependencies before deletion.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and idempotentHint=true; description adds detail about irreversible nature and immediate impact on dependent integrations/virtual keys, providing context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with action and permanence, then dependency warning. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive operation with full annotations and an output schema, description comprehensively covers consequences and prerequisites.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameter with description 'Secret Reference UUID or slug'. Description adds no further parameter meaning beyond what schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Permanently delete a Secret Reference' with specific verb and resource. Distinguishes from sibling delete tools by specifying the entity type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly warns to confirm dependencies before deletion and describes consequences, but does not state when to avoid using or suggest alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_usage_limitA
Destructive

Archive a cumulative usage-limit policy by id. The policy stops enforcing its budget but its historical record remains available through archived-status reads.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesUsage-limit policy UUID to archive

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructive behavior, and the description adds useful nuance: the policy stops enforcing its budget but remains available through archived-status reads. This clarifies that the operation is an archival action with retained history, going beyond the raw destructiveHint annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, compact sentence that front-loads the primary action and then explains the consequential behavior. Every word contributes meaning, with no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive operation with an output schema and clear annotations, the description covers what the tool does, what happens to the policy, and what remains accessible afterward. Nothing necessary for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the only parameter 'id' is already well-described as 'Usage-limit policy UUID to archive'. The tool description repeats 'by id' but does not add additional semantic detail beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb, 'Archive', and a specific resource, 'cumulative usage-limit policy by id', making the tool's action clear. It also distinguishes itself from related destructive tools like delete_rate_limit by clarifying that this is an archive operation, not a hard deletion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use the tool: when you want to stop a usage-limit policy from enforcing its budget while retaining its historical record. It does not explicitly state when not to use it or name alternative tools, but the retention behavior provides clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_userA
Destructive

Delete a user from the org by id. This is permanent, removes org and workspace memberships, revokes API keys, and ends active sessions; use delete_user_invite for pending invites instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (destructiveHint=true), the description details the permanent nature of deletion and its consequences: removes memberships, revokes keys, ends sessions. This provides comprehensive behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first delivers the core purpose and effects, the second provides an alternative. It is concise and front-loaded with essential information, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter, the description covers purpose, behavioral effects, and an alternative. The output schema exists but does not require additional description. The tool is fully specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single parameter 'user_id' with a description. The tool description does not add additional semantics beyond the schema. Since schema coverage is 100%, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'delete', the resource 'user', and the method 'by id'. It differentiates from the sibling tool 'delete_user_invite' by specifying that tool is for pending invites. The scope of deletion is thoroughly explained.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool (to delete a user by ID) and provides a direct alternative for a specific case ('use delete_user_invite for pending invites instead'). This is clear guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_user_inviteA
Destructive

Delete a pending invite and revoke its invite link. This does not affect existing users; use delete_user for full user removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include destructiveHint=true, so the safety profile is already indicated. The description adds that the invite link is revoked and that existing users are unaffected, providing extra context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no redundant information. Every phrase adds value, and the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and an existing output schema, the description covers purpose, scope, and exclusions thoroughly, leaving no ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with one parameter (invite_id) already documented. The description does not elaborate on parameter meaning beyond schema, but the schema itself is clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Delete a pending invite and revoke its invite link,' specifying the action and resource. Distinguishes from 'delete_user' by noting it does not affect existing users, helping the agent differentiate among sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides when to use (delete pending invite) and when not to (does not affect existing users), and directs to 'delete_user' for full removal, offering a clear alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_virtual_keyA
Destructive

Delete a virtual key by slug. This is irreversible and will break prompts and configs that reference the slug, so confirm no active dependencies first. Returns success after removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the virtual key to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds valuable context beyond annotations (destructiveHint=true), explaining irreversible nature and secondary effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, front-loaded with purpose, no extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Simple tool with one parameter, description covers purpose, side effects, and return value adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with adequate description for the single parameter; description adds no additional meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool deletes a virtual key by slug, distinguishing it from sibling delete tools for other resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly warns about irreversibility and impact on prompts/configs, advising to confirm no active dependencies first.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_workspaceA
Destructive

Delete a workspace by id. This is permanent and removes the workspace, its members, configs, API keys, and resources.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesThe workspace ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint: true), the description provides concrete details: 'permanent and removes the workspace, its members, configs, API keys, and resources.' This fully discloses the consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two sentences: the first states the action, the second the consequences. No unnecessary words, and critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and an output schema (not shown but present), the description fully covers the effect and scope. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage (workspace_id described as 'The workspace ID to delete'), the tool description adds no new meaning beyond 'by id', which is redundant. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and resource 'workspace by id', making the purpose unambiguous. It distinguishes from other delete tools by specifying 'workspace'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool should be used when permanent deletion is intended, but does not explicitly state when to use it over alternatives (e.g., no mention of archiving or deactivation). It also lacks guidance on prerequisites or caution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

disconnect_mcp_server_connectionA
DestructiveIdempotent

Disconnect one user's active session from a Portkey-managed MCP server. This immediately ends that connection but does not revoke future access; use update_mcp_server_user_access when access itself should be removed. Provide user_id with service keys (user keys derive it), and workspace_id with organisation admin keys. Inspect list_mcp_server_connections first when the target is uncertain.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesMCP server ID or slug containing the active connection
user_idNoUser UUID to disconnect; required with service keys and derived from user keys when omitted
workspace_idNoWorkspace ID or slug; required with an organisation admin API key

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral context beyond annotations: it states the action 'immediately ends that connection but does not revoke future access,' clarifying the scope of destructiveness. It also explains key derivation for user keys, supplementing the destructiveHint and idempotentHint annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: core action, scope/alternative, and parameter guidance plus a best-practice tip. No redundancy, well front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with three parameters (one required) and an output schema, the description fully covers purpose, exclusions, auth requirements, and a pre-check suggestion. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds valuable context: 'Provide user_id with service keys (user keys derive it), and workspace_id with organisation admin keys.' This explains the conditional necessity of optional parameters, going beyond the static schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource: 'Disconnect one user's active session from a Portkey-managed MCP server.' It clearly distinguishes from the sibling tool update_mcp_server_user_access, which is for removing access rather than just ending an active session.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is given: 'use update_mcp_server_user_access when access itself should be removed' and 'Inspect list_mcp_server_connections first when the target is uncertain.' It also specifies key-type-dependent parameter usage, making it clear when to provide user_id vs workspace_id.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

download_log_exportA
Read-onlyIdempotent

Get a signed URL for downloading a completed log export. The export must already be finished; use get_log_export to confirm readiness and start_log_export if it has not run yet. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to download

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds context beyond these: the export must be completed, and non-Enterprise plans receive a 403. This is valuable behavioral information not present in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, and followed by necessary conditions and alternative tool guidance. Every sentence contributes value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with an output schema, the description covers prerequisites, alternative tools, error conditions (403), and access restrictions, making it fully complete for selecting and invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters with a clear description for export_id. The tool description adds no additional parameter-level semantics, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Get a signed URL for downloading a completed log export.' It distinguishes from sibling tools by specifying the export must be completed and referencing related tools like get_log_export and start_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance on when to use: the export must already be finished; directs the user to get_log_export to confirm readiness and start_log_export if it has not run yet. Also notes the Enterprise-gating prerequisite and potential 403 error.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_metadataA
Read-onlyIdempotent

Get a paginated metadata breakdown with total_groups, group_count, and a metadata_groups array grouped by the required metadata_key. Use this for custom breakdowns like per-environment or per-feature analysis; pass metadata_key in addition to the time window. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoZero-based page number; the first page is 0
metadata_keyYesThe metadata key to group by (e.g., 'env', 'app', 'client_id')
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds genuinely useful context beyond annotations: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans,' disclosing plan entitlement and a concrete error status an agent will encounter. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences with zero filler, and the highest-value information is front-loaded: purpose and return shape first, then usage scenario, then the enterprise restriction. Every clause earns its place, including the two short clauses covering the 403 behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 32 parameters, 3 required fields, and an output schema, the description covers the essentials: what it returns, its intended use case, the required parameter combination, and the plan-level restriction. The output schema handles return-value details, so nothing critical is missing; the only small gap is not naming the exact time-window parameter keys, though the schema covers them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented, warranting the baseline 3. The description adds minimal parameter meaning beyond the schema, only highlighting that metadata_key is required alongside the time window; it does not elaborate on formats or constraints that the schema doesn't already provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get'), a clear resource ('paginated metadata breakdown'), and the distinctive grouping mechanism ('grouped by the required metadata_key'). It even names the return fields (total_groups, group_count, metadata_groups array), making it unambiguous versus the many sibling analytics tools and the get_analytics_group_users/models/providers variants.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use it: 'Use this for custom breakdowns like per-environment or per-feature analysis', and instructs the caller to 'pass metadata_key in addition to the time window'. It does not explicitly name alternatives or state when not to use it, but the 'custom breakdowns' framing plus the self-descriptive sibling names make the routing nearly inference-free.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_modelsA
Read-onlyIdempotent

Get a paginated per-model breakdown with total_groups, group_count, and a models array containing request count, cost, and token usage. Use this to compare model cost, popularity, and efficiency; use get_token_analytics or get_cost_analytics for time-series trends instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoZero-based page number; the first page is 0
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the bar is lower. The description adds genuine value beyond annotations by disclosing pagination behavior and the enterprise entitlement constraint ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'), which is critical operational context. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: the first states what it returns, the second provides when-to-use routing, and the third discloses the enterprise gating. The core purpose is front-loaded; the alternative routing and gating note follow in logical order with zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Yes. With an output schema present, return values don't need explaining; with annotations covering idempotence and read-only safety, that burden is lifted; the description covers purpose, sibling routing, pagination, and the 403 failure mode. The two required time-range parameters are fully documented in the schema. Nothing an agent needs to call this correctly is missing despite the 31-parameter complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline of 3 applies. The description's mention of 'request count, cost, and token usage' in the output subtly echoes the filter dimensions (cost_min/max, token min/max params), but it doesn't add parameter-level meaning beyond what the schema already documents. The schema carries the heavy lifting here, and that's acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Get'), resource ('paginated per-model breakdown'), and the exact output shape (total_groups, group_count, models array with request count, cost, token usage). It also names the grouping dimension (per-model), which immediately distinguishes it from sibling group analytics tools like get_analytics_group_users, get_analytics_group_providers, and get_analytics_group_metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the use case ('compare model cost, popularity, and efficiency') and names the two sibling alternatives with the exact condition that should route the agent elsewhere ('use get_token_analytics or get_cost_analytics for time-series trends instead'). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_providersA
Read-onlyIdempotent

Enterprise-gated. Get provider-grouped analytics for one workspace and time range with selectable metrics, ordering, pagination, and optional total count. Use this when comparing provider traffic or reliability; requested metric fields are preserved in each provider row. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
columnsNoProvider metrics to include in each grouped row
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
order_byNoMetric or field used to order groups
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoNumber of provider groups per page
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoZero-based result page
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
include_totalNoRequest the total provider-group count
order_by_typeNoSort direction accepted by Portkey, such as asc or desc
workspace_slugYesWorkspace slug to analyze
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the operation safe (readOnlyHint, idempotentHint, non-destructive), so the description only needs to add context. It adds the Enterprise gating policy and 403 failure mode, plus the guarantee that 'requested metric fields are preserved in each provider row'—useful behavioral details beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, but 'Enterprise-gated' appears twice verbatim, wasting a sentence slot. The second sentence mixes use-case rationale with a behavioral note, which is functional but could have been tightened; the repetition drags the efficiency down.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a 35-parameter schema and an output schema, the description doesn't need to explain return values. It covers the core use case, enterprise restriction, and preserved fields. It doesn't mention that workspace_slug and time bounds are required, but the schema's required list already carries that burden, so the description is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%; every parameter already has a meaningful description with constraints and legacy/structured aliases. The main description adds no parameter-level detail, but at this coverage level the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource pair: 'Get provider-grouped analytics for one workspace and time range with selectable metrics, ordering, pagination, and optional total count.' This precisely identifies what the tool does and contrasts with siblings like get_analytics_group_users and get_analytics_group_models via the 'provider-grouped' qualifier.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent when to use it: 'Use this when comparing provider traffic or reliability.' It also gives an exclusion constraint (Enterprise-gated, 403 on non-Enterprise plans). It does not name sibling alternatives explicitly, so it stops short of full when-not-to-use guidance, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_usersA
Read-onlyIdempotent

Get a paginated per-user breakdown with total_groups, group_count, and a users array containing request count, cost, and token usage. Use this for billing, audits, or top-consumer analysis; use get_users_analytics for aggregate active and new user trends. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoZero-based page number; the first page is 0
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive behavior. The description adds valuable context beyond annotations by disclosing the paginated nature, the enterprise gating, and the 403 response on non-Enterprise plans. This is exactly the kind of behavioral detail that helps an agent set expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences: purpose and output shape first, usage guidance and alternative second, enterprise constraint last. Every sentence earns its place and no information is repeated from structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, detailed parameter descriptions, and annotations covering side effects, the description supplies exactly the missing selection context: purpose, use cases, the closest sibling alternative, and the enterprise-gating failure mode. Nothing critical is missing for an agent to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Every parameter already has a description in the schema, so the baseline is 3. The tool description does not add parameter-level meaning beyond what the schema provides, but it doesn't need to given 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: 'Get a paginated per-user breakdown', and details the response contents (total_groups, group_count, users array with cost and token usage). It also differentiates from get_users_analytics, making sibling selection straightforward.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description is explicit about when to use the tool: 'Use this for billing, audits, or top-consumer analysis', and directly names the alternative for aggregate trends: 'use get_users_analytics for aggregate active and new user trends'. This leaves little to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_api_keyA
Read-onlyIdempotent

Fetch one API key by UUID without revealing the secret. Use this to inspect scopes, defaults, limits, expiration, and reset state before changing access.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description adds the behavioral trait of not revealing the secret, which is beyond annotations. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main purpose, followed by usage guidance. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, what it inspects, and when to use it. With output schema present, return values are not required. Context is complete for a safe read operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a description for the 'id' parameter. The description does not add additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Fetch one API key by UUID' with a specific verb and resource, and distinguishes from siblings like list_api_keys by specifying 'one API key' and 'without revealing the secret'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to inspect scopes, defaults, limits, expiration, and reset state before changing access,' providing clear context for when to use it, though it does not explicitly mention when not to use it or list alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cache_hit_latencyA
Read-onlyIdempotent

Get cache-hit-only latency time-series data with summary.total_latency, summary.avg_latency, and per-bucket total/avg latency. Use this to evaluate cached-response speed; use get_latency_analytics for all requests. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior, so the description's job is lighter. It adds meaningful context beyond annotations by disclosing the Enterprise licensing gate and the 403 error behavior, plus clarifying that the data is cache-hit-only and includes per-bucket time-series breakdown. This is useful behavioral information not present in the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences: the first states what the tool returns, the second gives usage guidance and the alternative, and the third discloses access restrictions. Every sentence earns its place, and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the large sibling workspace, the description fully identifies this tool's niche among many analytics tools and provides the essential usage and failure conditions. The schema covers all 29 parameters and an output schema exists, so the description doesn't need to explain return structure. Minor gaps like pagination or bucket interval specification are not described, but they are not critical for tool selection or initial invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters with descriptions, including legacy/structured aliases and normalization behavior, so the schema carries the full semantic burden. The description does not need to add parameter-level detail; the only implicit addition, that the data is time-series with buckets, aligns with the required time_of_generation_min/max parameters. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Get'), a precise resource ('cache-hit-only latency time-series data'), and the exact fields returned (summary.total_latency, summary.avg_latency, per-bucket total/avg latency). It also explicitly differentiates itself from the sibling get_latency_analytics by the 'cache-hit-only' scope, so an agent can tell them apart without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit selection criteria: 'Use this to evaluate cached-response speed; use get_latency_analytics for all requests.' It also warns that the tool is Enterprise-gated and returns 403 on non-Enterprise plans, which tells the agent when it is likely to fail and why.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cache_hit_rateA
Read-onlyIdempotent

Get cache-effectiveness time-series data with summary.hit_rate, summary.total_hits, summary.total_misses, and per-bucket hits/misses/rate. Use this to measure cache effectiveness; use get_cache_hit_latency for speed rather than hit/miss ratio. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool read-only, idempotent, and non-destructive. The description adds valuable behavioral context beyond those annotations: this is enterprise-gated and returns 403 on non-Enterprise Portkey plans. It also clarifies the response shape in terms of time-series buckets and summary hit-rate metrics, which supplements the structured annotations meaningfully.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: the first states output and purpose, the second provides routing guidance to a sibling, and the third flags the enterprise restriction. It is front-loaded with the most decision-relevant information and contains no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input schema, existing annotations, and an output schema, the description adds exactly what an agent needs: what the metric means, how it differs from the latency sibling, and an important access restriction. Nothing critical to correct tool selection or invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 29 parameters thoroughly, including legacy vs structured aliases and normalization behavior. The description adds no parameter-specific semantics beyond naming the result fields, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Get cache-effectiveness time-series data') and identifies the concrete output fields, including summary.hit_rate, total_hits, total_misses, and per-bucket metrics. It also distinguishes itself from the sibling get_cache_hit_latency by contrasting hit/miss ratio with speed, making selection unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to measure cache effectiveness; use get_cache_hit_latency for speed rather than hit/miss ratio.' This directly tells the agent when to choose this tool and when to prefer the named alternative. No inference is required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cache_summaryA
Read-onlyIdempotent

Enterprise-gated. Get cache summary metrics for one workspace and time range: hits, average cache-hit latency, total requests, and percentage speedup. Use the graph cache tools when you need changes over time instead of one aggregate. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugYesWorkspace slug to analyze
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the description's extra value is the enterprise plan restriction and the 403 error on non-Enterprise plans, plus the aggregate-not-time-series behavior. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the enterprise restriction, but 'Enterprise-gated.' is repeated verbatim, which is avoidable waste. Otherwise the sentences are short and information-dense.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 29-parameter tool, the description covers the essential selection facts: single-workspace aggregate, time range, returned metrics, the alternative tool category, and plan-based auth failure. With a full output schema and 100% schema parameter coverage, the remaining optional filters do not need to be restated in prose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter descriptions, so the baseline is 3. The description only names the high-level workspace/time-range scope and does not add meaning beyond what the schema already states for the 29 parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get cache summary metrics') and names the scope ('one workspace and time range') and the concrete metrics returned (hits, average cache-hit latency, total requests, percentage speedup). It also distinguishes itself from 'graph cache tools' by offering a single aggregate instead of changes over time.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides an explicit routing rule: use the graph cache tools when changes over time are needed instead of a single aggregate. The alternative is somewhat generic ('graph cache tools') rather than naming exact sibling tools, and it does not mention other analytic siblings like get_cache_hit_rate, so it is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_collectionA
Read-onlyIdempotent

Fetch one collection by id or slug and return its name, slug, workspace, and timestamps. Use list_collections when browsing and get_collection when you already know the target.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idYesCollection ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, covering the safety profile. The description adds value by specifying the retrieval method (by id or slug) and the returned fields, which are not in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loads the verb and resource, and every sentence serves a clear purpose. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, safe read operation, existing output schema), the description provides all necessary context: what it does, how to use it, and when to prefer it over siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for 'collection_id' that already mentions 'Collection ID or slug'. The description does not add new semantic meaning beyond the schema, so baseline is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Fetch', the resource 'one collection by id or slug', and specifies the return fields. It distinguishes from the sibling tool 'list_collections' by mentioning the browsing vs. targeted retrieval use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use this tool ('when you already know the target') versus the alternative 'list_collections' ('when browsing'), providing clear guidance on selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_configA
Read-onlyIdempotent

Get one config by slug and return its routing, cache, retry, and target settings. Requires a known slug; use list_configs to discover one before editing.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique identifier (slug) of the configuration to retrieve. This can be found in the configuration's URL or from the list_configs tool response

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, idempotent, and open-world. Description adds value by specifying what settings are returned, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Essential information is front-loaded: action, resource, returned fields, and prerequisite.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, annotations, and output schema presence, the description covers all necessary aspects. It explains what the tool returns and how to obtain the required parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter (slug) with 100% schema description coverage. Description adds context about where to find the slug (URL or list_configs response), enhancing the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('config by slug'), and specifies the returned data: 'routing, cache, retry, and target settings'. It differentiates from sibling tools like list_configs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states prerequisite: 'Requires a known slug' and directs to list_configs for discovery. Provides clear when-to-use and alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cost_analyticsA
Read-onlyIdempotent

Get cost time-series data with summary.total_cost, summary.average_cost_per_request, and per-bucket total/avg cost. Use this for spend analysis and spike detection; use get_token_analytics when you need token volume instead of monetary cost. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the bar is lower. The description adds genuinely useful behavior beyond annotations: the tool is Enterprise-gated and 'Returns 403 on non-Enterprise Portkey plans,' which lets an agent predict and handle a common failure mode. It also discloses the output shape. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: purpose and return fields, usage routing to the sibling, and entitlement/error behavior. The core purpose is front-loaded in the first phrase, and there is zero filler or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a high-complexity tool (29 params), the description is well-supported: the output schema covers return values, the 100% schema coverage handles parameters, and annotations cover safety. The description adds the one contextual element not in structured fields — Enterprise gating and 403 behavior — plus the primary sibling router. A minor gap is that bucket granularity/aggregation behavior of the time-series isn't hinted at, but that is likely covered by the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and every one of the 29 parameters has a meaningful schema description (e.g., ISO8601 format for time bounds, legacy vs. structured alias guidance, cents units for cost filters). Per the baseline rule, the description need not add parameter details, and it doesn't attempt to — it adds nothing that would improve on the schema here, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource pair, 'Get cost time-series data,' and enumerates the concrete return fields (summary.total_cost, summary.average_cost_per_request, per-bucket total/avg cost). It also names the closest sibling (get_token_analytics) to differentiate monetary cost from token volume, so an agent can distinguish this tool from the 10+ analytics siblings without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit positive use cases ('spend analysis and spike detection') and an explicit routing condition with a named alternative ('use get_token_analytics when you need token volume instead of monetary cost'). It also forewarns about the Enterprise plan requirement, which is a hard precondition for invocation. This leaves little to inference, though it only names one of many analytics siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_deploymentA
Read-onlyIdempotent

Enterprise-gated. Get one registered Gateway deployment by UUID, or use self when authenticating as that deployment. Use list_deployments to resolve an ID; read responses contain only masked authentication and registry credential values. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesDeployment UUID or self
organisation_idNoOrganisation UUID scope

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds valuable behavioral context beyond annotations: enterprise plan gating with a 403 response, masked authentication and registry credentials in responses, and the specific meaning of the 'self' identifier.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with key information, but 'Enterprise-gated' appears twice, creating minor redundancy. Structure is otherwise logical: gating, main function, ID resolution, response masking, and error behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema and annotations, the description covers the essential access modes, credential masking, enterprise restriction, and ID resolution path. Nothing critical is missing for an agent to call this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by clarifying that 'self' is only valid when authenticating as that deployment and by emphasizing the 'registered' status requirement. It does not elaborate on organisation_id, but the schema already describes it clearly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches one registered Gateway deployment by UUID, with an explicit alternative of using 'self' when authenticating as that deployment. It also names list_deployments as the way to resolve an ID, effectively differentiating itself from the sibling listing tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance: use list_deployments to resolve an ID before calling this tool, and use 'self' only when authenticating as the deployment. It also provides a clear exclusion by stating this is Enterprise-gated and returns 403 on non-Enterprise Portkey plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_analyticsA
Read-onlyIdempotent

Get error-count time-series data with summary.total_errors and per-bucket counts. Use this for high-level error trends; use get_error_rate_analytics for percentages, or get_error_status_codes_analytics and get_error_stacks_analytics for breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile with readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds a meaningful behavioral caveat: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans,' which is not captured in annotations. It also discloses that results include summary.total_errors and per-bucket counts, contributing useful behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no filler. It front-loads the core function, immediately provides sibling differentiation, and ends with the enterprise gate caveat. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analytics tool with an output schema and a fully documented 29-parameter input schema, the description covers purpose, usage, alternatives, and access constraints. The enterprise-gating and 403 behavior are included, and the output shape is partly described and partly covered by the output schema. Nothing an agent needs to correctly invoke this tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 29 parameters with detailed descriptions. The description itself does not add parameter syntax, constraints, or examples beyond what the schema provides. Baseline 3 is appropriate because the schema carries the full parameter burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Get error-count time-series data with summary.total_errors and per-bucket counts.' It clearly distinguishes itself from sibling tools by naming get_error_rate_analytics, get_error_status_codes_analytics, and get_error_stacks_analytics as alternatives for different analytical needs. An agent can immediately understand what this tool does and how it differs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool ('Use this for high-level error trends') and when to use alternatives ('get_error_rate_analytics for percentages, or get_error_status_codes_analytics and get_error_stacks_analytics for breakdowns'). It also adds the enterprise-gating restriction with the 403 error, giving clear operational guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_rate_analyticsA
Read-onlyIdempotent

Get error-rate time-series data with summary.error_rate_percent and per-bucket percentages of total requests. Use this for reliability and SLA trends; use get_error_analytics for absolute error counts instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only/idempotent/non-destructive traits, so the description's added value is the Enterprise entitlement behavior (403 on non-Enterprise plans) and the exact response semantics (time-series with summary.error_rate_percent and per-bucket shares). This is meaningful behavioral context beyond the annotations, with no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: one defines the deliverable, one gives usage direction vs an alternative, and one flags the enterprise restriction. The most decision-relevant information is front-loaded with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a rich 29-parameter schema that is 100% documented and an output schema present, the description still covers the core differentiator, the sibling routing, and the critical error condition. Nothing necessary for selecting or invoking the tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and every one of the 29 parameters already carries a meaningful description. The tool description adds global context about the output being error-rate percentages but no parameter-specific detail, so the baseline 3 for full schema coverage is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Get error-rate time-series data') and goes further by naming the return shape (summary.error_rate_percent, per-bucket percentages). It also names the sibling tool get_error_analytics and contrasts against it, so an agent can disambiguate without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says when to use this tool ('reliability and SLA trends') and when not to ('use get_error_analytics for absolute error counts instead'). It also declares the Enterprise-gating precondition and the 403 failure on non-Enterprise plans, leaving no ambiguity about eligibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_stacks_analyticsA
Read-onlyIdempotent

Get stacked error-series data grouped by HTTP status code over time, with summary and per-code series. Use this to see which error classes dominate; use get_error_status_codes_analytics for distinct-code distribution instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry the safety profile (readOnlyHint, idempotentHint, destructiveHint=false, openWorldHint), so the bar is lower. The description adds genuinely new operational context: the response composition (summary plus per-code series over time) and the entitlement behavior ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'), which neither annotations nor schema convey. It omits pagination or limit details, but that is minor for a read-only analytics tool with a rich output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with zero waste: the operation and output shape, the usage guidance with sibling routing, and the gating caveat. The core action is front-loaded before any caveats.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 29-parameter analytics tool, the schema documents every parameter and an output schema exists, so the description need not explain return values. It covers what the tool computes, when to prefer it over the nearest sibling, and its critical failure mode (403 on non-Enterprise plans). Slightly more operational depth, such as data retention or date-range limits, would push it to a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and every parameter — including the two required time bounds with explicit ISO8601 format examples — is already documented in the schema. The description adds no parameter-level meaning beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause 'Get stacked error-series data grouped by HTTP status code over time, with summary and per-code series' names a specific verb, resource, and output shape. It explicitly distinguishes itself from get_error_status_codes_analytics, so an agent can tell the two analytics tools apart without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this to see which error classes dominate' states the intended use case, and 'use get_error_status_codes_analytics for distinct-code distribution instead' gives an explicit exclusion with a named alternative. The Enterprise-gated and 403 note also tells agents when the call will fail, which is directly relevant to choosing and invoking the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_status_codes_analyticsA
Read-onlyIdempotent

Get HTTP error-code distribution time-series data with summary and per-code series. Use this to see which codes occur most often; use get_error_stacks_analytics for stacked or cumulative breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds meaningful behavior beyond annotations: enterprise-gated access and a concrete 403 failure mode on non-Enterprise plans, plus the summary/per-code series output shape. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: first states the output, second gives the selection rule versus a sibling, third states access restriction and failure behavior. The key purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (29 parameters, output schema, annotations), the description covers the essential selection context: what the data contains, the sibling alternative, the enterprise requirement, and the 403 error. Remaining details like filter parameters are fully documented in the schema, so nothing critical is missing for selecting and invoking the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and each parameter already has a clear description including ISO8601 formats, legacy vs. structured aliases, and filter semantics. The tool description adds no parameter-level detail, but with full schema coverage the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource ('HTTP error-code distribution time-series data') and the output composition ('summary and per-code series'). It also names the sibling it is not, get_error_stacks_analytics, so an agent can distinguish them without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use this tool: to see which HTTP error codes occur most often. It also directs users to get_error_stacks_analytics for stacked or cumulative breakdowns, and it adds the Enterprise-gating restriction as a usage limitation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_analyticsA
Read-onlyIdempotent

Get feedback-submission time-series data with summary totals and per-bucket counts. Use this as the top-level feedback trend view; use get_feedback_models_analytics, get_feedback_scores_analytics, or get_feedback_weighted_analytics for breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety. The description adds meaningful context beyond annotations by disclosing Enterprise-gated access and the specific 403 error on non-Enterprise plans. This is exactly the kind of auth/error behavior that helps an agent handle failures. Slightly short of a 5 because it doesn't mention pagination or rate limits, but those are well covered by the output schema and readOnly annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: purpose first, then usage routing, then access constraint. No tangents, no repetition of schema or annotation content, and the most decision-relevant information is front-loaded. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analytics tool with rich schema (100% param descriptions) and an output schema, the description covers what the tool returns, when to use it, and the enterprise restriction. An agent has everything needed to invoke and interpret results. The only potential omission is pagination, but the output schema likely handles that, and the tool is a read-only aggregate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with all 29 parameters documented (e.g., time_of_generation_min, metadata_filter, provider_models). The tool description adds nothing about parameters, but the schema already carries that burden, so the baseline 3 applies. No deficiency in the description, but no value added either.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get'), names the exact resource ('feedback-submission time-series data'), and specifies the output shape ('summary totals and per-bucket counts'). It clearly distinguishes itself from sibling feedback analytics tools by positioning itself as the top-level trend view. An agent can immediately understand what this tool does and how it differs from related tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'Use this as the top-level feedback trend view' and names three sibling tools (get_feedback_models_analytics, get_feedback_scores_analytics, get_feedback_weighted_analytics) for breakdowns. It also warns about the Enterprise gating. This gives an agent clear direction on choosing between alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_models_analyticsA
Read-onlyIdempotent

Get feedback time-series data grouped by model, with per-model counts over time. Use this to compare feedback volume and satisfaction across models; use get_feedback_analytics for the overall total instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, openWorld, idempotent, and non-destructive behavior. The description adds meaningful context beyond those structured hints: the Enterprise gating and the concrete 403 behavior on non-Enterprise plans. It also clarifies the grouping and time-series output shape, which helps set expectations for what the tool actually returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no fluff: core function, usage guidance with alternative, and an important error/access caveat. Every sentence earns its place, and critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and annotations cover the safety profile, the description provides the remaining essential context: what the data is grouped by, when to use this instead of the sibling, and the Enterprise restriction. Nothing needed for correct selection or invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 29 parameters in detail. The description does not add parameter-level meaning beyond stating the model grouping and time-series nature, which is exactly the baseline situation where a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get feedback time-series data grouped by model, with per-model counts over time.' It also states the analytical use case ('compare feedback volume and satisfaction across models') and explicitly differentiates this tool from get_feedback_analytics, so an agent can identify the right sibling without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage context: 'Use this to compare feedback volume and satisfaction across models; use get_feedback_analytics for the overall total instead.' This tells the agent exactly when to choose this tool and when to pick the named alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_scores_analyticsA
Read-onlyIdempotent

Get raw feedback-score distribution time-series data with per-score buckets. Use this to understand sentiment mix; use get_feedback_weighted_analytics for calibrated scores with weighting. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly/idempotent/non-destructive, and the description adds the enterprise entitlement gate and its concrete failure mode ('Returns 403 on non-Enterprise Portkey plans'). This is useful behavioral context beyond the structured annotations, though it doesn't describe output shape or rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three short sentences with no filler: the capability, the usage recommendation, and the enterprise gate are each stated once. Key differentiators are front-loaded before the alternative and error condition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 29-parameter analytics tool, the description supplies the essential selection context (raw vs weighted, sentiment mix, enterprise gate) while the schema fully documents every parameter and an output schema exists. Nothing needed for correct selection or calling is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline applies and the description does not need to restate parameters. It also adds no parameter-level detail beyond the schema; the only extra cue, 'per-score buckets,' relates to the output rather than the inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get raw feedback-score distribution time-series data with per-score buckets.' This clearly identifies what the tool returns and the raw/per-score qualifiers distinguish it from the weighted feedback sibling without needing to inspect schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit usage context: 'Use this to understand sentiment mix; use get_feedback_weighted_analytics for calibrated scores with weighting.' This tells the agent when to call this tool and names the exact alternative, satisfying the when/when-not requirement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_weighted_analyticsA
Read-onlyIdempotent

Get weighted feedback-score time-series data using the weight recorded at feedback creation. Use this for calibrated quality metrics; use get_feedback_scores_analytics for the raw unweighted distribution. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the read-only, idempotent, non-destructive profile, so the description's job is to add context beyond that. It discloses the Enterprise plan requirement and the concrete failure mode ('Returns 403 on non-Enterprise Portkey plans'), which is valuable auth/entitlement context the annotations cannot convey. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each with a distinct job: core function, sibling routing, and access-gate warning. The primary purpose is front-loaded, and every sentence earns its place with zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 29-parameter analytics tool, the full package is complete: schema covers parameters and aliases, output schema covers return values, annotations cover safety/idempotency, and the description covers purpose, sibling differentiation, and the critical enterprise-gating failure mode. Nothing needed to invoke this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 29 parameters, including the legacy-vs-structured alias relationships. The description adds no parameter-level detail beyond what the schema provides, which lands at the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get weighted feedback-score time-series data') and clarifies the distinguishing mechanism ('weight recorded at feedback creation'). The explicit contrast with get_feedback_scores_analytics ('raw unweighted distribution') makes sibling differentiation immediate and precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use guidance ('Use this for calibrated quality metrics') and names the exact alternative with its own use case ('use get_feedback_scores_analytics for the raw unweighted distribution'). An agent can route between these tools without opening either schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrailA
Read-onlyIdempotent

Fetch one guardrail by id or slug with its full checks and actions; use list_guardrails to discover ids first. Use before update_guardrail or delete_guardrail when you need the exact enforcement policy, and returns the full check and action configuration alongside status and ownership.

ParametersJSON Schema
NameRequiredDescriptionDefault
guardrail_idYesThe guardrail UUID or slug (with guard_ prefix) to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds that it returns full check and action configuration, status, and ownership, which is valuable beyond annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states the purpose, second provides usage guidance. No unnecessary words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and an output schema (present but not shown), the description covers the purpose, usage context, and return content well. Slight deduction for not mentioning pagination or filtering, but not needed given the fetch-by-ID nature.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a clear description of guardrail_id. The description mentions 'by id or slug', which matches but does not add significant new semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies 'Fetch one guardrail' by id or slug, clearly defining the verb and resource. It distinguishes from sibling tool list_guardrails by mentioning to use that for discovery first.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states to use this tool before update_guardrail or delete_guardrail when the exact enforcement policy is needed, providing clear context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_integrationA
Read-onlyIdempotent

Fetch one integration by slug, including masked key, workspace access, allowed models, and configuration metadata. Use this before editing provider-specific settings or auditing access. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the integration to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds useful context about returned fields (masked key, workspace access, etc.) and enterprise restriction. No contradiction with annotations; additional transparency provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: purpose+details, usage context, enterprise restriction, error note. Front-loaded and concise with zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, usage, enterprise gating, and error codes. Has output schema, so return values not needed. Complete for a simple retrieval tool; could mention authentication but annotations cover readOnly. Sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage for the single parameter 'slug' with a complete description. The tool description only repeats 'by slug' without adding format or source details. Baseline 3 appropriate as description adds no extra value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Fetch' and specific resource 'one integration by slug', listing included data (masked key, workspace access, allowed models, configuration metadata). Differentiates from siblings like list_integrations, create_integration, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises use 'before editing provider-specific settings or auditing access', providing clear context. Also warns about Enterprise gating and 403 error, implying when not to use. No alternative tool named, but guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_latency_analyticsA
Read-onlyIdempotent

Get latency time-series data with summary.avg_latency_ms, summary.p50_latency_ms, summary.p90_latency_ms, summary.p99_latency_ms, and per-bucket latency percentiles in ms. Use this to spot slowdowns and regressions; use get_cache_hit_latency when you only want cache-hit latency. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, so the read-only/idempotent behavior is covered. The description adds genuinely useful behavioral context beyond annotations by disclosing Enterprise gating and the 403 failure mode on non-Enterprise plans. This is a meaningful behavioral disclosure not present in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact, with three sentences each serving a distinct purpose: what it returns, when to use it versus an alternative, and an important access constraint. The core purpose is front-loaded before the sibling distinction, and no sentences are wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analytics tool with a rich output schema, annotations covering safety, and a fully-documented input schema, the description covers the key contextual gaps: the specific latency metrics, the sibling distinction, and the Enterprise restriction. It could be more complete by noting pagination or summary statistics availability, but the output schema likely covers return structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline for this dimension is 3. The description names key metrics returned but does not add substantial meaning to the 29 parameters beyond what the schema already documents. The required time_of_generation_min/max parameters are not elaborated in the description, though the schema covers their format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as retrieving latency time-series data and enumerates the specific metrics included (avg, p50, p90, p99, per-bucket). It also names the sibling tool get_cache_hit_latency and distinguishes it by scope, which prevents confusion with the many other analytics tools in the sibling list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool to 'spot slowdowns and regressions' and directs users to use get_cache_hit_latency 'when you only want cache-hit latency'. This provides both a positive use case and an exclusion criterion naming the alternative, which is especially valuable given the large cluster of latency-related siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_logA
Read-onlyIdempotent

Get one gateway request log by ID, including its request, response, usage, cost, and metadata payload when available. For path_format v2, also provide the log's created_at timestamp.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoSet to hooks when retrieving a hook execution log
log_idYesGateway log ID to retrieve
created_atNoLog creation timestamp in ISO 8601 format; required for v2
path_formatNoLog storage path format; defaults to v1

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds meaningful context: the included fields are conditional ('when available'), and for path_format v2 the caller must provide created_at. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, and every clause contributes relevant information. It is appropriately concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-log retrieval with an output schema and strong annotations, the description covers essential behavior and the key path_format caveat. The output schema handles return format, so no further detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing a baseline of 3. The description adds the critical relationship that path_format v2 requires created_at, which is not marked required in the schema. It does not cover the 'type' parameter, but the schema fully documents it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get one gateway request log by ID' with a specific verb, resource, and scope, and lists the included payload elements (request, response, usage, cost, metadata). It distinguishes from sibling log-export tools by focusing on single-log retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when retrieving a specific gateway log by ID with full details. However, it does not explicitly name alternatives or provide exclusions, so it falls short of a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_log_exportA
Read-onlyIdempotent

Fetch one log export job by export_id and return its status, filters, requested fields, and file metadata. Use this when you already know the target; use list_log_exports for a workspace-wide overview. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful context beyond annotations: it discloses the return payload fields and the Enterprise plan requirement that may cause a 403. This gives the agent important behavioral and error-handling context without contradicting the structured hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each earning its place: function and return value, usage distinction, and access restriction. It is front-loaded with the primary action and avoids any filler. This is a model of concise, high-signal writing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one required parameter, a well-populated schema, an output schema, and rich annotations, this description provides all necessary context: what it retrieves, when to use it, and what plan is required. There are no significant gaps given the tool's low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully covers the single parameter export_id with a description ('The unique ID of the log export'). The description adds no additional parameter-level syntax or format detail, but because schema coverage is 100%, the baseline of 3 is appropriate; the tool's simple structure does not require further elaboration.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Fetch one log export job by export_id') and the resource (a single log export job), and lists what it returns (status, filters, requested fields, file metadata). This distinguishes it from the sibling tool list_log_exports, which is explicitly named as an alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance: 'Use this when you already know the target; use list_log_exports for a workspace-wide overview.' This clearly separates it from the alternative and also notes the Enterprise-gated access and 403 behavior, which helps set expectations for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_log_export_field_restrictionsA
Read-onlyIdempotent

Get the organisation-managed fields that a workspace is restricted from including in log exports. Use before create_log_export or update_log_export to avoid requesting disallowed fields. Requires an API key with logs.export scope; completion-log fields may additionally require completion scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesWorkspace ID whose export field restrictions should be read

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, but the description adds significant behavioral context beyond those: required API key scopes (logs.export, completion scope for some fields), enterprise gating, and the 403 error on non-Enterprise plans. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: primary purpose, usage directive, and necessary caveats. Front-loaded with the core function, no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema is present and annotations cover operational hints, the description fully rounds out context: it explains when to call (before exports), what scopes are needed, and enterprise gating. This is complete for a read-only, single-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter workspace_id is fully described in the schema ('Workspace ID whose export field restrictions should be read'), with 100% schema description coverage. The description does not add syntax or additional meaning to the parameter beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Get') and a precise resource ('organisation-managed fields that a workspace is restricted from including in log exports'). It clearly distinguishes the tool from sibling tools by referencing create_log_export and update_log_export, making its unique function obvious.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to use: 'Use before create_log_export or update_log_export to avoid requesting disallowed fields.' It also provides contextual caveats (API key scope, enterprise gating, 403 on non-Enterprise), giving clear guidance on prerequisites and expected failures.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_integrationA
Read-onlyIdempotent

Retrieve one MCP integration by id or slug. Returns the full Portkey-side config, including auth type, transport, and masked configuration keys; use get_mcp_integration_metadata for the server's self-reported metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context by specifying what is returned (full Portkey-side config including auth type, transport, masked keys) without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, and every word contributes. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one required parameter, output schema present), the description fully covers what the tool does, what it returns, and how it relates to a sibling tool. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already has 100% coverage with a description including 'ID or slug'. The description reiterates the same information without adding new parameter semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'one MCP integration' with identifier id or slug. It distinguishes from the sibling tool 'get_mcp_integration_metadata' by specifically mentioning that alternative for server-reported metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is provided: 'use get_mcp_integration_metadata for the server's self-reported metadata'. This clearly tells when to use this tool versus an alternative, making the usage context unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_integration_metadataA
Read-onlyIdempotent

Retrieve the external MCP server's self-reported metadata for an integration. Returns name, version, protocol, capability flags, and sync status; use get_mcp_integration for the Portkey-side connection config.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive. Description adds that it returns self-reported metadata with specific fields, enhancing agent understanding beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff, efficient communication of purpose and distinction.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with output schema and clear annotations, the description covers purpose, return values, and sibling differentiation. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and description does not add any additional meaning to the single 'id' parameter beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it retrieves external MCP server metadata, lists returned fields, and explicitly distinguishes from sibling tool get_mcp_integration for connection config.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Directly tells when to use this tool vs. get_mcp_integration, providing a clear alternative for a different purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_serverA
Read-onlyIdempotent

Retrieve one MCP server by id or slug. Returns server details including the parent integration, status, and created time; use get_mcp_server when you need the server record rather than the integration config.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description does not need to repeat these. It adds value by specifying the return fields (parent integration, status, created time), which gives behavioral insight beyond the schema. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two sentences, each earning its place. The first sentence states purpose and return info, the second provides usage guidance. No extraneous words, front-loaded effectively.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (single parameter, output schema exists), the description sufficiently covers what the tool does, what it returns, and when to use it. Annotations and output schema cover the rest, so no gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (the parameter 'id' has a clear description). The description adds only marginal value by mentioning 'by id or slug', which is already in the parameter description. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verb 'Retrieve' and resource 'one MCP server by id or slug', clearly stating the action and input. It distinguishes itself from get_mcp_integration by specifying when to use this tool for server records rather than integration config.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool vs. the alternative get_mcp_integration: 'use get_mcp_server when you need the server record rather than the integration config.' This provides clear context and an alternative, meeting the when/alternatives criterion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_model_pricingA
Read-onlyIdempotent

Get Portkey's current public pricing configuration for one exact provider/model pair. Prices are returned in USD cents per token or provider-specific unit and may include cache, audio, image, fine-tuning, and calculation metadata. Use this read-only catalog lookup before setting integration pricing_adjustments or custom-model pricing; it does not return your negotiated integration multiplier or require Portkey authentication.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesExact provider model identifier, such as gpt-4o
providerYesLowercase Portkey provider identifier, such as openai, anthropic, bedrock, or x-ai

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, open-world, and idempotent. The description adds valuable context on return semantics (USD cents per token or provider-specific unit), the types of metadata included (cache, audio, image, fine-tuning, calculation), and the exclusion of negotiated pricing. This goes beyond the structured annotations and helps the agent set expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, front-loaded with the core purpose, and every sentence adds critical information. There is no redundancy or filler, making it easy for an agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 params, read-only, output schema present), the description provides complete context: what it returns, its units, content categories, what it excludes, and when to use it. The output schema covers return structure, so no further detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond the schema, but it reinforces the 'exact pair' requirement. Since the schema already handles parameter semantics fully, this is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Get Portkey's current public pricing configuration') and clearly scopes it to one exact provider/model pair. This distinguishes it from any analytics tools or other pricing-related operations. The purpose is unambiguous and immediately understandably.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use the tool ('before setting integration pricing_adjustments or custom-model pricing') and what it does not do ('does not return your negotiated integration multiplier or require Portkey authentication'). This gives clear usage boundaries and prevents misuse, even without naming alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_organisation_defaultsA
Read-onlyIdempotent

Get the organisation-wide input and output guardrails that workspaces inherit by default. Use this before update_organisation_defaults or when auditing baseline enforcement; it does not include per-workspace exclusions, which are available from the directional exclusion list tools. Requires an organisation service API key with organisation_settings.read scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds valuable behavioral context by specifying auth scope, enterprise-gating, and the 403 response on non-Enterprise plans, plus clarifying that per-workspace exclusions are not included. It doesn't contradict annotations and adds useful information beyond them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the core purpose, and each sentence earns its place: purpose, usage, exclusions, auth, and error behavior. No redundant or extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters) and presence of an output schema, the description is complete: it specifies purpose, usage context, exclusions, access prerequisites, and error behavior. It fully equips an agent to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so per rubric the baseline is 4. The description doesn't need to add parameter details since there are none, and the schema coverage is effectively complete for an empty parameter set.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get the organisation-wide input and output guardrails that workspaces inherit by default' with a specific verb and resource, and explicitly distinguishes from related tools like update_organisation_defaults and exclusion list tools. It fully clarifies scope, making it unmistakable what this tool does relative to siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this before update_organisation_defaults or when auditing baseline enforcement' and states what is excluded and where to find it ('directional exclusion list tools'). Also includes required scope and enterprise-gating requirements, giving clear when-to-use and when-not-to-use context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_promptA
Read-onlyIdempotent

Fetch a prompt's full definition, active version, and version history. Use this before updating, publishing, rendering, or copying a prompt when you need the stored template and metadata. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, non-destructive, idempotent. The description adds that it returns active version and history, and gives specific advice for multi-message chat prompts, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and output, second provides usage guidance and a specific edge case. No redundant information, perfectly front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter tool with output schema and annotations, the description fully covers what the tool returns and when to use it, including a notable edge case for multi-message chats.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the baseline is 3. The description does not add additional semantic meaning to the prompt_id parameter beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it fetches a prompt's full definition, active version, and version history. This distinguishes it from sibling tools like get_prompt_version (specific version) and get_prompt_label (label metadata).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends using this tool before updating, publishing, rendering, or copying a prompt. It provides clear when-to-use context and an alternative approach for multi-message chats, though it does not explicitly state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_labelA
Read-onlyIdempotent

Fetch one label's full definition, including scope, color, and status. Use this when you already know the label_id; list_prompt_labels is better for browsing candidates.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to retrieve
workspace_idNoWorkspace ID for filtering
organisation_idNoOrganisation ID for filtering

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint, so safety is clear. Description adds no new behavioral traits beyond return content, which is expected for a fetch tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, then usage guidance. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given output schema exists and annotations are rich, the description is sufficient for a simple fetch tool. Minor omission: optional filter parameters not elaborated, but schema covers them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so description adds no meaning beyond what schema already provides for parameters. It mentions needing label_id but that is already required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'Fetch' and the resource 'one label's full definition', and lists included fields (scope, color, status). It explicitly distinguishes from sibling list_prompt_labels.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use this tool (when you already know the label_id) and when to use the alternative (list_prompt_labels for browsing).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_partialA
Read-onlyIdempotent

Fetch a partial's content and current version details. Use this before embedding, updating, or checking what {{> partial_name}} resolves to; returns the stored string plus version metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds value beyond the annotations by specifying what the tool returns ('the stored string plus version metadata') and its purpose (pre-embedding/update check). Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, so safety is clear. The description does not contradict any annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two sentences, both front-loaded with key information. The first sentence states the core action and objective. The second sentence provides usage guidance and return details. Every phrase earns its place; there is no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has one required parameter and an output schema. The description covers the main use case (fetching content and version details) and advises when to use it. Given the low complexity and the presence of an output schema, the description is sufficient. It could briefly mention that the partial is from a prompt_template, but that is not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has one parameter (prompt_partial_id) with a description already provided ('Prompt partial ID or slug to retrieve'). Schema description coverage is 100%. The tool description does not add additional meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Fetch a partial's content and current version details') and specifies the resource (a partial). It distinguishes itself from siblings like list_prompt_partials (which lists all partials) and get_prompt (which retrieves a prompt). The verb 'Fetch' is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use this tool: 'Use this before embedding, updating, or checking what {{> partial_name}} resolves to'. This provides clear context for invocation. It does not explicitly mention alternatives, but the context implies that for listing all partials one would use list_prompt_partials, and for mutations one would use create/update_prompt_partial.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_versionA
Read-onlyIdempotent

Retrieve a specific prompt version by its version UUID. Use list_prompt_versions to find the id first; returns the template, parameters, and model config for that version.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug
version_idYesVersion UUID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so description needs little behavioral context. It adds value by stating what fields are returned (template, parameters, model config), which is helpful but not critical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff. Every word earns its place, front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that output schema exists, the description appropriately highlights key return fields. It covers purpose, prerequisite steps, and output summary, making it complete for a retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters are fully described in the schema (100% coverage). The description adds no additional meaning to the parameters beyond usage guidance, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it retrieves a specific prompt version by UUID, distinguishing it from list_prompt_versions and get_prompt. The verb 'Retrieve' and resource 'prompt version' are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to use list_prompt_versions first to find the ID, providing clear when-to-use guidance and a direct reference to a sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_providerA
Read-onlyIdempotent

Fetch one provider by slug, including limits, rate settings, expiration, and reset status. Use this to check consumption or audit configuration before updating.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the provider to retrieve
workspace_idNoWorkspace ID - required when using organization admin keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable context beyond annotations by specifying that the response includes 'limits, rate settings, expiration, and reset status,' which informs the agent about the richness of the data. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences: the first states the action and included data, the second provides usage guidance. No extraneous words; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (not shown but indicated), the description is complete: it explains the tool's function, included data, and use case. No gaps given the low complexity of a single-resource fetch operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with descriptions for 'slug' and 'workspace_id'. The description adds the phrase 'by slug', but does not provide additional semantic detail beyond what the schema already states. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Fetch one provider by slug' – a specific verb+resource. It details the data returned (limits, rate settings, expiration, reset status) and distinguishes from sibling tools like list_providers and update_provider by focusing on a single entity and use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage context: 'Use this to check consumption or audit configuration before updating.' This guides the agent to use it for auditing or pre-update checks. While it doesn't explicitly name alternative tools, the context is sufficient to differentiate from list_providers or update_provider.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_rate_limitA
Read-onlyIdempotent

Get one rate-limit policy by id, including current conditions, grouping, unit, value, target, scope, and status. Pass archived status when retrieving a soft-deleted policy.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesRate-limit policy UUID
statusNoInclude an archived policy by status

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, and non-destructive behavior. The description adds value by revealing the response contents (current conditions, grouping, unit, value, target, scope, status) and by clarifying the archived-status behavior for soft-deleted policies, without contradicting any annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences deliver the core operation, the fields returned, and the important archived-status usage note. Every sentence earns its place and key information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only retrieval with two parameters, full schema coverage, rich annotations, and an output schema, the description is complete. It adds the only missing operational nuance (archived policy retrieval), and nothing else is needed for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines both parameters. The description enhances the 'status' parameter by explaining that 'archived' is needed to retrieve soft-deleted policies, providing context beyond the enum definition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and resource ('one rate-limit policy by id'), and lists the fields returned, making the tool's purpose precise. It distinguishes itself from list_rate_limits by emphasizing retrieval of a single item by id, and from get_usage_limit by clearly naming 'rate-limit policy'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear operational guidance by noting to pass 'archived' status when retrieving a soft-deleted policy. It does not explicitly name alternatives or exclusions, but the context makes it evident this is for fetching one specific policy rather than listing all.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_request_analyticsA
Read-onlyIdempotent

Get request-volume time-series data with summary.total_requests, summary.successful_requests, summary.failed_requests, and per-bucket total/success/failed counts. Use this for traffic and reliability trends; use get_error_analytics when you only need error counts. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the description does not need to restate those. It adds meaningful context by disclosing that the endpoint is Enterprise-gated and returns 403 on non-Enterprise plans, which is important operational behavior not present in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no filler. It front-loads the core purpose and output summary, then provides usage differentiation and gating constraints, all in a compact and scannable format.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 29-parameter schema and existing output schema, the description provides enough context for correct invocation: purpose, sibling distinction, and enterprise access requirement. It could be slightly more complete by mentioning how this relates to other analytics siblings or what time-bucket granularity is expected, but it is largely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all parameters with detailed descriptions (100% coverage), so the baseline is 3. The description itself does not add parameter-level guidance, but the schema already does the heavy lifting, including required time range fields and structured alias relationships.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool fetches request-volume time-series data and lists the key fields returned, making the verb, resource, and output scope clear. It also distinguishes itself from get_error_analytics by naming the sibling and the condition that selects it, which is strong differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives direct guidance on when to use this tool: for traffic and reliability trends. It also explicitly points to get_error_analytics as the alternative when only error counts are needed, giving the agent a clear routing decision.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_rescued_requests_analyticsA
Read-onlyIdempotent

Get rescued-request time-series data showing requests recovered by retry or fallback handling. Use this only when your configs include resilience features, and use it to measure how often recovery logic saved requests. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile needs no repetition. The description adds genuinely new behavioral context beyond annotations: the Enterprise plan gating and the 403 error on non-Enterprise Portkey plans, which informs the agent about entitlement-related failure modes before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, zero filler: what it does, when to use it, and the enterprise gating caveat. The core purpose is front-loaded in the first sentence, and each subsequent sentence adds a distinct piece of decision-relevant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists (return format need not be described), annotations cover the safety profile, and the 29-parameter schema is fully self-documenting, the description covers the remaining essentials: purpose, usage condition, and entitlement caveat. The only minor gap is not naming which sibling analytics tool to use when configs lack resilience features.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline of 3 applies. The description itself adds no parameter-level guidance, but it doesn't need to — the schema exhaustively documents legacy vs structured aliases, normalization behavior, formats (ISO8601), and range constraints for all 29 parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ('Get rescued-request time-series data') and defines the niche concept inline ('requests recovered by retry or fallback handling'). This clearly separates it from the ~20 analytics siblings (get_request_analytics, get_error_analytics, get_cache_hit_rate) by scoping to recovery logic specifically.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this only when your configs include resilience features' is an explicit conditional that tells the agent when the tool applies, and 'measure how often recovery logic saved requests' states the intended evaluation purpose. It stops short of naming a specific alternative for the non-resilience case, but the condition is clear enough to route selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_secret_referenceA
Read-onlyIdempotent

Retrieve one Secret Reference by UUID or slug. Portkey masks sensitive authentication fields for non-system users; the tool returns metadata and the masked auth configuration, not the resolved external secret value.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds important context beyond annotations: masking behavior for non-system users, returns metadata and masked auth, not the actual secret value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two precise sentences: first states action and identifier, second adds critical behavioral nuance. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers retrieval purpose, identification method, return content (metadata + masked auth), and what is NOT returned (secret value). Output schema handles return structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only parameter 'id' is fully documented in schema (100% coverage). Description reiterates 'UUID or slug' but adds no new semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Retrieve one Secret Reference' with identifier type (UUID or slug), distinguishing it from sibling list/create/update/delete tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies singular retrieval vs list by using 'one', but lacks explicit when-to-use vs alternatives. Context signals and siblings help differentiate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_token_analyticsA
Read-onlyIdempotent

Get token-usage time-series data with summary.total_tokens, summary.prompt_tokens, summary.completion_tokens, and per-bucket total/prompt/completion counts. Use this for consumption trends; use get_cost_analytics when you need spend instead of token volume. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds genuinely useful behavioral context beyond that: the enterprise plan restriction and the explicit 403 failure mode. This is meaningful operational transparency that annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with zero waste: output shape, usage routing, and failure condition. The core purpose is front-loaded, and each subsequent sentence adds distinct value rather than repeating the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 29-parameter schema with 100% coverage, an existing output schema, and safety annotations, the description covers the essential agent-facing concerns: what it returns, when to pick it over the sibling, and when it will fail. Minor gaps like time-bucket granularity or pagination behavior exist, but the output schema and rich parameter docs relieve the description of carrying those details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema carries the full parameter documentation burden, including legacy-vs-structured aliases, formats, examples, and ranges. The description adds only the time-series framing that loosely connects to the required time_of_generation_min/max parameters, but no parameter-level detail beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource — 'Get token-usage time-series data' — and goes further to enumerate the exact output shape (summary.total_tokens, summary.prompt_tokens, summary.completion_tokens, per-bucket counts). It explicitly differentiates from the closest sibling by stating this tool is for token volume while get_cost_analytics is for spend.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use guidance: 'Use this for consumption trends; use get_cost_analytics when you need spend instead of token volume.' The enterprise-gating note ('Returns 403 on non-Enterprise Portkey plans') also tells the agent the conditions under which the call will fail, so it can route to alternatives or set expectations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_usage_limitA
Read-onlyIdempotent

Get one cumulative usage-limit policy by id. Use list_usage_limits first when the id is unknown. Optionally include per-value usage counters and retrieve archived policies; scheduled reset timestamps are returned when present.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesUsage-limit policy UUID
statusNoInclude an archived policy by status
include_usageNoInclude per-value usage counters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish read-only, idempotent, non-destructive behavior, so the description is not solely responsible for safety disclosure. The description adds useful behavioral context beyond the schema by noting that 'scheduled reset timestamps are returned when present' and that archived policies can be retrieved via status.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences carry all essential information with no filler. The core purpose is front-loaded, and the optional parameter behaviors are summarized concisely without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, has an output schema, and is fully covered by rich annotations describing it as safe and idempotent. The description covers the primary lookup action, the fallback list call, and the optional behaviors, so nothing an agent needs to correctly invoke it is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds extra meaning by explaining 'include per-value usage counters' for include_usage and 'retrieve archived policies' for status, which clarifies the purpose of those parameters beyond the schema field names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get one cumulative usage-limit policy by id.' It clearly distinguishes this from sibling tools like 'list_usage_limits' and 'get_rate_limit' by emphasizing 'cumulative usage-limit policy' and a single-id lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage guidance is explicit: 'Use list_usage_limits first when the id is unknown' tells the agent exactly when to choose an alternative. It also clarifies when to use the optional parameters, such as including usage counters or retrieving archived policies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userA
Read-onlyIdempotent

Get one accepted user by id and return their profile, role, and timestamps. Use list_all_users to find the id if you only have a name or email, and get_user_invite for pending invitations. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint false, etc. The description adds context that the tool only returns 'accepted' users and is Enterprise-gated, providing useful behavioral detail beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose, no redundant information. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-id tool with one parameter and an output schema, the description covers the return fields, how to find the id, alternatives for different states, and access restrictions. Complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The sole parameter user_id is described in schema as 'The user ID to retrieve'. The description adds no new semantic meaning to the parameter itself, but does provide context on obtaining the id from list_all_users, which is more about usage than parameter semantics. Schema coverage is 100%, so baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'one accepted user', and distinguishes from siblings like list_all_users and get_user_invite by specifying the scope and alternative uses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly guides when to use list_all_users (to find id from name/email) and get_user_invite (for pending invitations), and warns about Enterprise gating and 403 response.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_inviteA
Read-onlyIdempotent

Get one invitation by invite id and return its email, role, status, and expiry. Use this for pending invites only; use get_user for accepted users.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that it returns specific fields (email, role, status, expiry) and that it's for pending invites. There is no contradiction, and the behavior is well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, no unnecessary words. It front-loads the purpose and includes usage guidance in a compact form.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single parameter, strong annotations, and the presence of an output schema (context shows has output schema: true), the description is sufficient. It covers what the tool does, what it returns, and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter is invite_id, which has a clear schema description ('The invite ID to retrieve'). The tool description does not add further semantic detail beyond what the schema already provides. With 100% schema coverage, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a single invitation by ID and lists the returned fields (email, role, status, expiry). It also distinguishes from get_user by explicitly stating 'Use this for pending invites only; use get_user for accepted users.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this for pending invites only; use get_user for accepted users.' This tells the agent when to use this tool vs. an alternative sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_requests_analyticsA
Read-onlyIdempotent

Get per-user request-count time-series data with counts grouped by user. Use this to find heavy users and traffic concentration; use get_users_analytics for aggregate active and new user trends instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, idempotent, and non-destructive behavior, so the bar is lower. The description adds a genuinely non-obvious behavioral fact beyond annotations: the tool is Enterprise-gated and returns 403 on non-Enterprise Portkey plans — an entitlement/error-trajectory disclosure that matters for invocation. No contradiction with annotations; it just doesn't describe pagination or time-bucket granularity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: core function, usage guidance with a named alternative, and the entitlement caveat. The most decision-relevant scoping ('grouped by user') is front-loaded in the first sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 29-parameter tool, the description is lean but the schema fully documents every parameter and the output schema covers return shape, so nothing critical is missing. The only gap is an explicit boundary against get_request_analytics and get_analytics_group_users, which also touch user/request data — minor given that get_users_analytics is already named as the alternative.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline of 3 applies; all 29 parameters are already documented, including legacy vs structured alias pairs like configs/config_slugs. The description adds only that the result is request-count data grouped by user, which frames the required time range rather than documenting parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with 'Get per-user request-count time-series data with counts grouped by user' — a specific verb, resource, and grouping semantics. This cleanly separates it from sibling analytics tools for cost, token, latency, and errors, and directly contrasts it with get_users_analytics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use it ('find heavy users and traffic concentration') and names the alternative with its condition ('use get_users_analytics for aggregate active and new user trends instead'). It also flags the Enterprise entitlement and the 403 failure mode, so an agent can anticipate the error.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_users_analyticsA
Read-onlyIdempotent

Get user-growth time-series data with summary.total_active_users, summary.total_new_users, and per-bucket active/new user counts. Use this for growth and adoption trends; use get_user_requests_analytics for per-user traffic or get_analytics_group_users for per-user cost and token detail. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the description adds value by disclosing the Enterprise gating requirement and the 403 error on non-Enterprise plans. It also clarifies the time-series output shape, which is useful beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly packed sentences: the first states what is returned, the second routes to alternatives, and the third notes the Enterprise gating and error behavior. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the high parameter count and complexity, the description covers purpose, usage boundaries, output nature, and an important error condition. The schema covers parameter details and the output schema covers return values, so nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all 29 parameters are already fully documented in the input schema. The description adds no parameter-specific semantics, which is acceptable because the schema carries that burden completely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Get user-growth time-series data') and names the exact resource and output fields (summary.total_active_users, summary.total_new_users, per-bucket counts). It also distinguishes itself from nearby siblings by name, so an agent can select it correctly without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use it for growth and adoption trends and directs agents to get_user_requests_analytics for per-user traffic and get_analytics_group_users for per-user cost/token detail. It also discloses Enterprise-gating, eliminating ambiguity about when it can be used.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_statsA
Read-onlyIdempotent

Return per-user request and cost analytics for a required time range. This is usage-by-user, not population metrics; use get_users_analytics for active-user or cohort trends. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
page_sizeNoNumber of results per page (max 100)
status_codeNoFilter by specific HTTP status codes (comma-separated)
virtual_keysNoFilter by specific virtual key slugs (comma-separated)
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context not covered by annotations, such as the Enterprise-gated access (returns 403 on non-Enterprise plans) and the distinction from population metrics, which clarifies the tool's scope and limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise with three sentences, each serving a distinct purpose: stating the function, differentiating from a sibling, and noting access restrictions. The key information is front-loaded, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of a rich input schema, annotations, and an output schema, the description provides sufficient context. It covers purpose, usage differentiation, access constraints, and required parameters. However, it could briefly mention that the output is per-user, but the high schema coverage compensates, so it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter description coverage, so the baseline is 3. The tool description mentions the required time range but does not add new semantic information beyond what is already in the schema. Hence, it provides no additional parameter insights.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns per-user request and cost analytics for a required time range, using specific verbs ('Return') and resource ('per-user request and cost analytics'). It also distinguishes from the sibling tool get_users_analytics by specifying 'This is usage-by-user, not population metrics'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool (for per-user analytics) and when not to (for population metrics, use get_users_analytics). It also provides access constraint information ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'), guiding the agent on proper usage and error handling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_virtual_keyA
Read-onlyIdempotent

Fetch one virtual key by slug, including metadata, a masked secret, limits, status, and model config. Use this before updating or to inspect the current configuration.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the virtual key to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only and idempotent. The description adds value by specifying exactly what data is returned (e.g., masked secret, limits), which is not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no redundancy. First sentence states purpose, second gives usage context. Efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, full schema coverage, and output schema present, the description is complete. It includes usage guidance and return field highlights.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single parameter 'slug' with a description. The description's mention of fetching by slug adds little beyond the schema, achieving the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches a virtual key by slug, listing the returned fields (metadata, masked secret, limits, status, model config). It distinguishes itself from sibling getters like get_api_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this before updating or to inspect the current configuration.' It does not mention alternatives but gives clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspaceA
Read-onlyIdempotent

Get one workspace by id and return its full details, including defaults and the complete member list. Use this when you need membership detail; use list_workspaces for an overview.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesThe unique identifier of the workspace to retrieve. This can be found in the workspace's URL or from the list_workspaces tool response

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint. Description adds that it returns 'full details, including defaults and the complete member list', providing useful context beyond the annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and what it returns, second gives usage guidance. Front-loaded and efficient with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, and description adequately states what is returned (full details including defaults and member list). Sufficient for a simple retrieval tool with one parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 1 parameter with 100% description coverage, so baseline is 3. Description does not add additional parameter semantics beyond what schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get one workspace by id' with specific verb and resource, and distinguishes from sibling tool list_workspaces by mentioning 'return its full details, including defaults and the complete member list'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Contains explicit guidance: 'Use this when you need membership detail; use list_workspaces for an overview.' Clearly tells when to use this tool vs alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspace_memberA
Read-onlyIdempotent

Get one workspace member by workspace_id and user_id. Use this when you already know both IDs; use list_workspace_members to browse the full roster.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to retrieve (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds a minimal behavioral fact: it returns a single member and requires both IDs. However, it does not disclose additional behavioral traits such as error handling, rate limits, or authentication requirements. With annotations carrying most of the burden, this description adds only marginal transparency beyond them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first states the purpose and the second gives usage guidance. It is front-loaded with the action and resource, contains zero filler, and every word earns its place. This is a model of conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple lookup tool, the description is complete. The input schema fully documents the two required parameters, an output schema exists to define the return value, and annotations cover safety. The description adds the critical context about when to use it versus list_workspace_members. There is no missing information that would prevent correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% since both parameters have descriptions. The user_id description is detailed (valid UUID, not email, from list_all_users), while workspace_id's is generic. The tool description restates the parameters but does not add new semantic meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Get one workspace member by workspace_id and user_id.' This specific verb + resource combination distinguishes it from sibling tools like list_workspace_members, add_workspace_member, and remove_workspace_member. It also explicitly names list_workspace_members as the alternative for browsing, reinforcing differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this when you already know both IDs; use list_workspace_members to browse the full roster.' This tells the agent exactly when to use this tool and points to the appropriate alternative, making the choice unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

insert_logA

Insert log records for requests that bypassed the gateway. This writes request, response, and trace metadata into Portkey immediately, and the call will fail if request_provider does not match a configured integration. Use the span fields to stitch trace hierarchies together.

ParametersJSON Schema
NameRequiredDescriptionDefault
request_urlNoThe endpoint URL being called
request_bodyNoRequest payload/body
metadata_userNoUser identifier for the log
response_bodyNoResponse payload/body
response_timeNoResponse latency in milliseconds
request_methodNoHTTP method used (defaults to 'post')post
streaming_modeNoWhether the response was streamed
metadata_customNoAdditional custom metadata key-value pairs
request_headersNoRequest headers as key-value pairs
response_statusNoHTTP response status code (defaults to 200)
metadata_span_idNoSpan ID for tracing
request_providerNoAI provider name (e.g., 'openai', 'anthropic')
response_headersNoResponse headers as key-value pairs
metadata_trace_idNoTrace ID for distributed tracing
metadata_span_nameNoSpan name for tracing
metadata_organizationNoOrganization identifier for the log
metadata_parent_span_idNoParent span ID for tracing

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, destructiveHint=false) indicate a write operation. The description adds that writes are immediate, fail on unmatching provider, and span fields enable trace hierarchy. All behaviors are disclosed without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences front-loaded with purpose and context. Every sentence provides meaningful information without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool complexity (17 parameters) and existing schema/annotations/output schema, the description covers purpose, usage condition, failure case, and tracing guidance. It could optionally mention what the function returns, but output schema likely handles that, so it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 17 parameters have descriptions in the schema (100% coverage), so baseline is 3. The description adds value by explaining span fields 'stitch trace hierarchies together' and the failure condition for request_provider, which goes beyond the schema's generic descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Insert log records for requests that bypassed the gateway' – a specific verb and resource with context. It distinguishes from sibling tools by focusing on bypassed requests, which is unique among the many create and log tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies when to use (bypassed gateway requests), includes a failure condition ('call will fail if request_provider does not match a configured integration'), and provides guidance on trace stitching. However, it does not explicitly state when not to use or compare to alternative logging methods.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

invite_userA

Invite a new org user and optionally provision workspace access and an API key in one call. Workspace assignments apply only after acceptance; use add_workspace_member or update_workspace_member later for follow-up changes.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesOrganization-level role: 'admin' for full access, 'member' for limited access
emailYesEmail address of the user to invite
last_nameNoUser's last name
first_nameNoUser's first name
workspacesYesList of workspaces and corresponding roles to grant to the user
workspace_api_key_detailsNoOptional API key to be created for the user

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate non-read-only, non-destructive, non-idempotent, and open-world. The description adds behavioral context by noting that workspace assignments are effective only after acceptance, which is not captured in annotations. This is valuable transparency beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that cover purpose, optional features, and important behavioral nuance. Every word adds value, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 parameters, nested objects, output schema exists), the description covers the main purpose, optional capabilities, and key behavioral point about delayed workspace assignment. It is sufficiently complete for an AI agent to understand the tool's role, though it could briefly mention that an invitation is sent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already describes all parameters. The description mentions optional provisioning of workspace access and API key, which aligns with the 'workspaces' and 'workspace_api_key_details' parameters but adds little extra meaning. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool invites a new org user and optionally provisions workspace access and an API key. It distinguishes from sibling tools like add_workspace_member by noting that workspace assignments apply only after acceptance, prompting use of separate tools for follow-ups.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the tool's use case (inviting a new org user) and clarifies when to use alternative tools for workspace changes after acceptance. This provides clear guidance on when and when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_all_usersA
Read-onlyIdempotent

List accepted org users with id, name, email, role, and timestamps. Use this to find a user_id before get_user, update_user, delete_user, or add_workspace_member; use list_user_invites for pending invitations. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoFilter by organisation role
emailNoFilter by exact email
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly, idempotent, non-destructive), the description discloses that this returns only accepted users, that it is Enterprise-gated, and that it returns a 403 on non-Enterprise plans. This adds meaningful operational context that the agent cannot infer from the annotations or schema alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: the first defines the tool's output, the second provides routing guidance, and the third warns about the enterprise restriction. Every sentence earns its place, and the critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, has full schema coverage, an output schema, and annotations that clarify safety and idempotency. The description supplies the missing context around accepted status, expected usage, and enterprise gating, making it complete for an agent to select and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters fully. The description does not add extra nuance about how role, email, page_size, or current_page behave beyond what the schema states, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List'), a clear resource ('accepted org users'), and the exact fields returned (id, name, email, role, timestamps). It also distinguishes itself from list_user_invites by explicitly noting that pending invitations are not included, so an agent can confidently tell this tool apart from similar list operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: use this tool to find a user_id before calling get_user, update_user, delete_user, or add_workspace_member. It also names the alternative for pending invites (list_user_invites), which directly addresses a common point of confusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_api_keysA
Read-onlyIdempotent

List Portkey API keys for auditing access, scopes, defaults, limits, and expiration. Use this for API keys only; use list_virtual_keys for provider keys. Returns total plus id, type, status, workspace/user scope, limits, defaults, alert emails, and creation mode.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true) already fully disclose the safety profile, and the description is consistent with them. The description adds value beyond annotations by enumerating the return payload ('total plus id, type, status, workspace/user scope, limits, defaults, alert emails, and creation mode'), which is useful behavioral context. No annotation contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero waste. The purpose is front-loaded, followed by a one-sentence sibling disambiguation and a compact return summary. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with zero required parameters, an output schema, and full annotation coverage, the definition is complete: annotations carry the safety profile, the schema documents pagination and filtering, and the description names the distinguishing sibling, summarizes the return shape, and states the use case. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: page_size, current_page, and workspace_id each carry their own descriptions, so the baseline is 3. The description does not add parameter-level detail beyond what the schema already provides, though it implicitly hints at what workspace/user scope filters relate to via the return-field list.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'List Portkey API keys'. The intended purpose ('for auditing access, scopes, defaults, limits, and expiration') is concrete, and the description distinguishes itself from the sibling tool list_virtual_keys by scope ('Use this for API keys only'). An agent can tell exactly what this tool does without inspecting the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: 'Use this for API keys only; use list_virtual_keys for provider keys.' This names the alternative and the condition that selects it, leaving nothing to inference. The audit framing also communicates the appropriate use context for this tool versus the many other key-related siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_audit_logsA
Read-onlyIdempotent

List audit log events for a Portkey workspace or organization. Returns paginated action-level records with actor, resource, metadata, and timestamps for compliance or incident review; use this instead of analytics when you need individual events, not aggregates. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionNoFilter by action type (e.g., 'create', 'update', 'delete', 'login')
actor_idNoFilter by the user ID who performed the action
end_timeNoEnd of time range filter (ISO 8601 format, e.g., '2024-01-31T23:59:59Z')
page_sizeNoNumber of results per page (max 100)
start_timeNoStart of time range filter (ISO 8601 format, e.g., '2024-01-01T00:00:00Z')
resource_idNoFilter by specific resource ID
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter audit logs by workspace ID
resource_typeNoFilter by resource type (e.g., 'user', 'workspace', 'config', 'virtual_key')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive behavior, so the bar is lower. The description adds useful behavioral context: results are paginated, records include actor/resource/metadata/timestamps, and the Enterprise plan requirement is disclosed. This goes beyond the schema and annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: the first states the core purpose and output shape, the second differentiates from analytics, and the third flags the enterprise restriction. No filler or redundant restating of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and the input schema is fully self-documenting, the description covers the key non-obvious aspects: individual vs aggregate data, pagination, and enterprise entitlement. Nothing essential for correct selection or invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and all parameters have meaningful descriptions. The tool description does not need to add parameter-level detail, and it does not. It does imply pagination and time-range use, but the schema already handles each parameter clearly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: 'List audit log events for a Portkey workspace or organization.' It also distinguishes the tool from analytics by emphasizing it returns individual action-level events rather than aggregates, which separates it from many similar list/analytics siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: 'use this instead of analytics when you need individual events, not aggregates.' It also provides an important exclusion by noting the tool is Enterprise-gated and returns 403 on non-Enterprise plans, helping the agent decide whether to attempt the call.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_collectionsA
Read-onlyIdempotent

List prompt collections in the workspace, optionally filtering by name or workspace. Returns ids, names, slugs, and timestamps so you can choose a collection_id before create_prompt, get_collection, or list_prompts.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch collections by name
page_sizeNoResults per page (max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the bar is lower. The description adds value beyond those hints by disclosing the output payload (ids, names, slugs, and timestamps) and the purpose of that output (selecting a collection_id for downstream calls). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero filler. The first sentence front-loads the action and optional filters; the second states the return fields and downstream use case. Every clause earns its place, and there's no repetition of schema or annotation content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only listing tool with an output schema and safety annotations, the description covers the essentials: what it lists, the filter options, what it returns, and how to use the result. It could go one step further by clarifying how workspace scoping works (e.g., what happens when workspace_id is omitted), but pagination and field details are already handled by the schema and output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 — the schema already documents search, page_size, current_page, and workspace_id with meaningful descriptions. The description adds marginal value by mapping 'filtering by name or workspace' to the intent of the search and workspace_id parameters, but it adds no new format, syntax, or behavioral detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource pair — 'List prompt collections in the workspace' — and further narrows scope with 'optionally filtering by name or workspace.' Against siblings like create_collection, get_collection, and list_workspaces, the distinction is unambiguous, and the stated return fields (ids, names, slugs, timestamps) leave no doubt about what this tool produces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit downstream context: 'so you can choose a collection_id before create_prompt, get_collection, or list_prompts.' This tells the agent the typical sequencing for using the tool. It stops short of stating exclusions or naming an alternative for the case where a collection_id is already known (e.g., get_collection), so it's clear context without explicit when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_configsA
Read-onlyIdempotent

List configs in the org with id, slug, name, status, workspace, and timestamps. Use this summary view to find a slug; use get_config for the full routing, cache, retry, and target settings before updating or deleting.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds useful behavioral context beyond annotations: this is a summary/listing view rather than a full-config view, and it scopes results to the org.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and resource, followed by precise routing guidance. Every sentence earns its place with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple paginated list with no required parameters, and an output schema exists. The description sufficiently covers scope, returned fields, and when to use the sibling get_config tool. Nothing critical is missing for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so page_size and current_page are already fully documented in the input schema. The description adds no additional parameter meaning, which is acceptable given complete schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List configs in the org' with concrete returned fields (id, slug, name, status, workspace, timestamps). It clearly distinguishes itself from get_config as a 'summary view' rather than a full-detail endpoint.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to use this tool to find a slug and to use get_config for full routing, cache, retry, and target settings before updating or deleting. This gives clear when-to-use guidance and names the relevant alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_config_versionsA
Read-onlyIdempotent

List every version of a config with version_id, structured config payload, updater, and timestamp. Use this to audit history or compare revisions before update_config or delete_config.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesConfiguration slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false. The description adds useful behavioral context by specifying the return fields and the exhaustive nature ('every version'), which goes beyond the annotations. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tightly worded sentences. The core behavior is front-loaded, and the usage guidance is provided efficiently without redundant detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single required parameter, a fully described schema, an output schema, and comprehensive annotations, the description is complete. It states what is listed, why to use it, and how it relates to update_config and delete_config. Nothing essential is missing for correct tool invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the single parameter 'slug' has a clear schema description: 'Configuration slug to list versions for.' The description does not add beyond the schema, but with full coverage the schema carries the burden effectively.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action: 'List every version of a config' with explicit returned fields (version_id, structured config payload, updater, timestamp). It differentiates from sibling tools like list_configs by focusing on versions of a single config rather than a list of configs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool: 'Use this to audit history or compare revisions before update_config or delete_config.' This gives the agent a concrete decision rule and connects it to the related mutation tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_deploymentsA
Read-onlyIdempotent

Enterprise-gated. List registered self-hosted Gateway deployments with status, type, default state, and connection health. Use this before get_deployment, update_deployment, or archive_deployment to resolve a deployment ID. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by deployment type
searchNoSearch deployment names
statusNoFilter by deployment status
workspace_slugNoFilter by one or more workspace slugs
organisation_idNoFilter by organisation UUID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a safe, idempotent read operation, and the description adds meaningful behavioral context: it is Enterprise-gated and returns 403 on non-Enterprise plans, plus it enumerates the returned fields. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is useful and front-loaded, but 'Enterprise-gated.' is repeated twice, which is redundant. Tightening the description by removing the duplication would make it more concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, usage context, access restrictions, error behavior, and key output fields. With an output schema present and rich annotations, nothing critical is missing for an agent to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well documented. The description does not add additional meaning about parameter usage beyond what the schema provides, which is expected but keeps this dimension at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation: listing registered self-hosted Gateway deployments and the specific fields returned. It also distinguishes itself from get_deployment, update_deployment, and archive_deployment by positioning this as the ID-resolution step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to use this tool before get_deployment, update_deployment, or archive_deployment to resolve a deployment ID. It also discloses the Enterprise plan requirement and the 403 failure mode, giving clear conditions for appropriate use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_guardrailsA
Read-onlyIdempotent

List guardrails in the org with id, slug, status, ownership, and optional workspace/org filters. Use this to find IDs and slugs before get_guardrail, update_guardrail, or delete_guardrail.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of items per page (1-100, default: 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter guardrails by workspace ID
organisation_idNoFilter guardrails by organization ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context by stating this lists guardrails at the org level and supports optional workspace/org filtering, without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler. The first sentence states the core purpose and returned fields; the second sentence gives actionable usage guidance. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool, the description is complete: it states the resource, return fields, applicable filters, and the downstream tools that depend on it. The pagination parameters are fully documented in the schema, and an output schema exists, so no return-format details are missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters. The description adds only minor semantic value by mentioning 'optional workspace/org filters', but it does not add meaningful detail beyond what the parameter descriptions already provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and identifies the resource ('guardrails') plus the fields returned (id, slug, status, ownership). It clearly distinguishes this list operation from singleton operations like get_guardrail, update_guardrail, and delete_guardrail, which are named as siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: before get_guardrail, update_guardrail, or delete_guardrail, to discover IDs and slugs. This gives clear routing guidance and names the downstream alternatives that require the output of this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_input_guardrail_workspace_exclusionsA
Read-onlyIdempotent

List workspaces excluded from organisation-wide input guardrails for one organisation. Use this to audit exceptions or establish the current state before the matching update tool; it reads input exclusions only and does not return the organisation's default guardrail list. Requires an organisation service API key with organisation_exclusions.list scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
organisation_idYesOrganisation ID whose workspace exclusions should be listed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only/idempotent/destructive hints. The description adds valuable context about required API key scope, Enterprise-gated availability, and the 403 behavior on non-Enterprise plans, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, all directly informative, with front-loaded primary action. No redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only list tool with an output schema, the description covers purpose, usage, auth, enterprise gating, and returns meaningful constraints. It is fully sufficient for an agent to select and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter, so the schema fully documents 'organisation_id'. The description adds no additional parameter-level detail, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists workspaces excluded from organisation-wide input guardrails for one organisation, with a specific verb and resource. It also distinguishes itself by noting it only reads input exclusions, not the default guardrail list, which differentiates it from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: to audit exceptions or establish current state before the matching update tool. It also clarifies what it does not do (does not return default guardrail list), providing clear context versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integration_modelsA
Read-onlyIdempotent

List models enabled on an integration. Use this to verify model availability before creating prompts or configs. Returns total plus model ids, display names, enabled state, and custom-model markers. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
page_sizeNoNumber of results per page
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly, idempotent, and non-destructive annotations, the description discloses meaningful behavioral traits: it is 'Enterprise-gated' and 'Returns 403 on non-Enterprise Portkey plans.' It also summarizes the return payload, providing useful operational context even though an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of three compact, non-redundant sentences: purpose, recommended use case, return fields, and access restriction. The core action is front-loaded, and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations, full schema parameter documentation, and presence of an output schema, the description covers the essential invocation context: what the tool does, why an agent would call it, what it returns, and an important auth/plan prerequisite. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: slug, page_size, and current_page are all documented in the input schema. The tool description adds no additional parameter-level detail beyond what the schema already provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List models enabled on an integration.' It clearly distinguishes this from sibling tools like list_integrations and get_integration by focusing on models belonging to an integration, so an agent can identify the correct tool without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear use case: 'Use this to verify model availability before creating prompts or configs.' However, it does not explicitly mention alternatives or when not to use this tool, so it stops short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integrationsA
Read-onlyIdempotent

List org-level AI provider connections with optional workspace or type filters. Use this to find integration slugs before model or workspace updates. Returns total plus id, name, slug, provider, status, description, workspace counts, and config summary.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by integration type: 'workspace', 'organisation', or 'all' (default)
page_sizeNoNumber of results per page (default 100, max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter integrations accessible by a specific workspace

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint, and non-destructive behavior, so the safety profile is well covered. The description adds context about the response containing totals and configuration summaries, but does not disclose deeper behavior like pagination limits or authorization requirements beyond what the schema provides.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences: what it lists, when to use it, and what it returns. Every sentence earns its place, and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with a full input schema and an output schema, the description covers purpose, filters, usage context, and key return fields. Minor ambiguity remains around 'org-level' versus the type enum's 'workspace' and 'all' values, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are already fully documented. The description reinforces the workspace/type filters but does not add meaning beyond what the schema already states; this is a standard baseline-3 case.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('List org-level AI provider connections'), clarifies it supports workspace/type filters, and states the return payload. This clearly differentiates it from sibling integration tools like list_integration_models and get_integration by emphasizing the collection-level slug lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit use case: 'Use this to find integration slugs before model or workspace updates.' This provides clear when-to-use guidance, though it does not explicitly name alternatives or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integration_workspacesA
Read-onlyIdempotent

List workspaces that can use an integration, with their limits. Use this to audit access or confirm per-workspace cost and rate settings. Returns total plus workspace ids, names, enabled state, usage limits, and rate limits. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
page_sizeNoNumber of results per page
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a safe, read-only, idempotent operation. The description adds valuable behavioral context beyond that: the read is enterprise-gated and returns 403 on non-Enterprise plans, and it discloses the exact response fields. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four concise sentences, each earning its place: what it does, when to use it, what it returns, and a critical access constraint. Information is front-loaded and free of fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list operation with a full output schema and well-documented parameters, the description covers purpose, usage, return contents, and an important entitlement error. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters. The description adds no parameter-specific detail beyond what the schema provides, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('List'), a specific resource ('workspaces that can use an integration'), and the accompanying scope ('with their limits'). This clearly differentiates it from sibling tools like list_integrations or list_workspaces, even without naming them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit use cases: 'audit access or confirm per-workspace cost and rate settings.' It provides clear context for when to invoke the tool, though it does not explicitly state when not to use it or name alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_log_exportsA
Read-onlyIdempotent

List log export jobs in a workspace with status, filters, and timestamps. Use this to find an export_id before calling get_log_export, start_log_export, cancel_log_export, or download_log_export. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesWorkspace ID to list exports for (required)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is read-only and idempotent, so the description's additional disclosure of the Enterprise-gating and 403 error behavior adds value. However, it doesn't mention pagination or other behavioral nuances that might matter for listing jobs, though with output schema present this is less critical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: the first sentence defines the action, the second provides usage context, and the third states a critical limitation. No wasted words; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with a simple single-parameter schema and an output schema, the description covers the core purpose, usage, and error condition. The only minor gap is not explicitly mentioning pagination or ordering, but the output schema likely covers return structure. Overall complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter, workspace_id, with a clear description. The tool description merely reiterates 'in a workspace' without adding further meaning beyond the schema, so it meets the baseline but doesn't exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists log export jobs in a workspace with status, filters, and timestamps. It distinguishes itself from related tools by explicitly positioning it as the way to find an export_id before using other log export operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: 'Use this to find an export_id before calling get_log_export, start_log_export, cancel_log_export, or download_log_export.' It also discloses a restriction (Enterprise-gated, returns 403 on non-Enterprise plans), which helps the agent decide if this tool is appropriate in context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integration_capabilitiesA
Read-onlyIdempotent

List capabilities exposed by the external MCP server for an integration. Returns total plus enabled-state entries so you can decide what to toggle; use before update_mcp_integration_capabilities when you need to compare the current surface.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only (readOnlyHint: true), non-destructive, idempotent, and open-world. The description adds value by specifying that it returns 'total plus enabled-state entries' and positions it as a preparatory step for toggling. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the primary purpose, and no redundant information. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with one parameter and an assumed output schema, the description provides the purpose, return summary, and usage context (preparation for update). It is complete enough for an agent to understand when and how to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single parameter 'id', and its description is adequate. The tool description does not add extra parameter details, but since the schema already covers it, a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List', the resource 'capabilities exposed by the external MCP server for an integration', and distinguishes it from siblings like 'update_mcp_integration_capabilities' and 'list_mcp_integrations'. It also specifies the return content (total plus enabled-state).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to 'use before update_mcp_integration_capabilities when you need to compare the current surface', providing clear context and an alternative. This helps the agent decide when to invoke this tool versus the update sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integrationsA
Read-onlyIdempotent

List MCP integrations in the organization. Returns paginated integration records plus total and has_more for discovering integration IDs; use get_mcp_integration for one integration's full Portkey-side config and list_mcp_servers for the servers under an integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by workspace, organisation, or all integration ownership
searchNoSearch integrations by name
page_sizeNoNumber of results per page (max 1000)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID
organisation_idNoFilter by organisation UUID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds behavioral value by stating the pagination output (total, has_more) and the use case of discovering integration IDs. This enriches the agent's understanding beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with three tightly packed clauses: core action, return/pagination details, and sibling routing. Every element earns its place and the most important information is front-loaded. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only listing tool with an output schema and full schema coverage for parameters, the description covers the key contextual needs: what it returns, pagination cues, and how to fetch more detail or related resources. The agent has enough information to select and call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 without any parameter info in the description. The description does not add meaning beyond what the schema already provides for the six parameters. No parameter detail is missing, but the description also does not enhance it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource: 'List MCP integrations in the organization.' It further clarifies the return shape (paginated records, total, has_more) and distinguishes itself from get_mcp_integration and list_mcp_servers. This makes the tool's purpose unambiguous and easily differentiated from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly names the alternatives and the conditions that select them: 'use get_mcp_integration for one integration's full Portkey-side config and list_mcp_servers for the servers under an integration.' This gives the agent clear routing guidance without needing to inspect schemas.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integration_workspacesA
Read-onlyIdempotent

List which workspaces can access an MCP integration. Returns the global access mode plus per-workspace enablement for audit or permission review; use before update_mcp_integration_workspaces.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only and idempotent behavior. The description adds valuable context by stating that it returns global access mode and per-workspace enablement, and that it should be used before updates. This goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that front-load the purpose and then add return details and usage guidance. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the presence of an output schema, the description explains the return contents (global access mode and per-workspace enablement), the use case (audit/permission review), and ties to the sibling update tool. For a simple list tool with one parameter, this is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully describes the single parameter 'id' with a clear description. The tool description does not add new meaning beyond referencing the integration context. With 100% schema coverage, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action ('List which workspaces can access an MCP integration') and specifies the return content (global access mode plus per-workspace enablement). It also distinguishes from the sibling update tool by explicitly recommending use before update_mcp_integration_workspaces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance: use for audit/permission review and before an update. However, it does not differentiate from the similar sibling tool 'list_integration_workspaces', which could cause confusion. The guidance is good but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_capabilitiesA
Read-onlyIdempotent

List capabilities exposed by an MCP server instance. Returns total plus the current tool, resource, and prompt surface; use this instead of the integration-level capability list when you need server-specific exposure.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only, idempotent, and non-destructive. The description adds useful behavioral context by stating that it returns a total count plus the current tool, resource, and prompt surface, going beyond what the annotations alone provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no filler. The core purpose and the differentiating usage guidance are front-loaded, making the description efficient and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations, schema coverage, and presence of an output schema, the description provides sufficient context for correct invocation. It clearly covers what the tool does, when to use it, and how it differs from alternatives.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (id, page_size, current_page) are already well documented in the schema. The description adds no additional parameter-specific meaning, which aligns with the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('List capabilities exposed by an MCP server instance') and identifies the resource. It also distinguishes itself from the integration-level capability list, making it easy for an agent to select the correct tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent when to use this tool: 'use this instead of the integration-level capability list when you need server-specific exposure.' This directly routes the agent away from the closely related sibling tool list_mcp_integration_capabilities.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_connectionsA
Read-onlyIdempotent

List active connection records for one Portkey-managed MCP server, including user, connected state, and connection timestamps. Use it to audit sessions or identify the user/workspace before disconnect_mcp_server_connection; service keys can list all users, while user keys default to their own user. Organisation admin keys must provide workspace_id. This reads live connection state and does not test upstream server reachability (use test_mcp_server for that).

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesMCP server ID or slug whose connections to list
user_idNoUser UUID to filter; user-scoped keys default to their own user when omitted
page_sizeNoConnections per page, from 1 through 500; defaults to 100
current_pageNoZero-based results page; defaults to 0
workspace_idNoWorkspace ID or slug; required with an organisation admin API key

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavioral context beyond these: it reads live connection state, does not test upstream reachability, and explains how authentication scopes affect results. It could go further by noting possible absence of fields or pagination behavior, but the output schema likely covers the return shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences deliver the essential purpose, primary use case, key-scoping behavior, and a critical exclusion, all without redundancy. The most important information is front-loaded, and every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, rich input schema, output schema, and annotations, the description covers everything an agent needs to invoke it correctly: what it returns, when to use it, how auth affects results, and how it differs from related tools. No significant gap remains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters. The description adds meaningful semantics around user_id and workspace_id by explaining how service keys, user keys, and organisation admin keys affect filtering and requirements. This goes beyond the base schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List active connection records for one Portkey-managed MCP server', and enumerates the returned fields (user, connected state, connection timestamps). It also distinguishes itself from siblings by explicitly naming test_mcp_server and disconnect_mcp_server_connection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use it ('audit sessions or identify the user/workspace before disconnect_mcp_server_connection') and when not to ('does not test upstream server reachability (use test_mcp_server for that)'). It also provides key-scoping rules: service keys list all users, user keys default to their own user, and organisation admin keys must provide workspace_id.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_serversA
Read-onlyIdempotent

List MCP servers in the organization. Returns paginated server records plus total for discovering server IDs; use get_mcp_server for one server's details and list_mcp_integrations for the parent integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnly/idempotent/non-destructive context, so the description adds value by disclosing that the call returns paginated server records plus a total, and that scoping is at the organization level. This goes beyond the structured hints without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler: the primary action and return shape are front-loaded, and sibling differentiators follow in the same sentence. Every clause contributes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a paginated list tool with full parameter documentation, an output schema, and safety annotations, the description covers purpose, scope, return shape, and sibling relationships. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all three parameters with descriptions and 100% coverage. The description's mention of pagination reinforces the page_size/current_page semantics but does not add detail beyond what the schema provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('List MCP servers in the organization') and immediately differentiates from siblings by naming get_mcp_server for details and list_mcp_integrations for the parent integration. An agent can tell exactly what this tool returns and how it differs from nearby read tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (discovering server IDs via paginated listing) and names the two alternatives with their intended purposes. This is direct routing guidance rather than leaving usage to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_user_accessA
Read-onlyIdempotent

List per-user access for an MCP server. Returns the default access mode, override flags, and connection status so you can audit who can use it; use before update_mcp_server_user_access.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the bar is lower. The description adds value beyond annotations by disclosing what the response reveals (default access mode, override flags, connection status) and that it serves an audit purpose. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero filler. The verb+resource is front-loaded, followed by the return-value context and the usage-order hint. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with full schema coverage, annotations, and an output schema, the description covers purpose, response contents, and usage ordering. Pagination and return structure are handled by the schema/output schema, so nothing critical is missing. A small gap: it doesn't hint at scale or how open-world results should be interpreted, but annotations cover that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — id, page_size, and current_page are all documented in the schema with type and semantics. The description does not add param-level detail, but it doesn't need to; the baseline of 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource: "List per-user access for an MCP server." This clearly distinguishes it from sibling tools like list_mcp_servers (listing servers themselves), get_mcp_server (fetching a single server), and list_mcp_server_capabilities/connections (other server dimensions). An agent can tell it apart without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context — "audit who can use it" — and explicitly names the follow-on tool update_mcp_server_user_access with the ordering "use before." It lacks explicit when-not guidance for the other list_mcp_server* siblings, but the main alternative is named and the audit scenario is concrete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_output_guardrail_workspace_exclusionsA
Read-onlyIdempotent

List workspaces excluded from organisation-wide output guardrails for one organisation. Use this to audit exceptions or establish the current state before the matching update tool; it reads output exclusions only and does not return the organisation's default guardrail list. Requires an organisation service API key with organisation_exclusions.list scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
organisation_idYesOrganisation ID whose workspace exclusions should be listed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds valuable behavioral context: it 'reads output exclusions only', states the required API key scope, emphasizes enterprise gating, and documents the 403 error on non-Enterprise plans. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each carrying substantive information (purpose, usage context, permissions/error behavior). No redundant phrasing. Well-structured and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, the description covers purpose, usage scenarios, exclusions vs. defaults, authentication scope, enterprise restriction, and error condition. An output schema exists, so return values need no further explanation. Description is complete for tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear description for organisation_id. The description reinforces the parameter by saying 'for one organisation', but does not add significant new semantic details beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List workspaces excluded'), the resource ('organisation-wide output guardrails'), and the scope ('for one organisation'). It distinguishes from the sibling tool list_input_guardrail_workspace_exclusions by explicitly specifying 'output guardrails' and from the matching update tool by using 'list' and 'reads'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this to audit exceptions or establish the current state before the matching update tool'. It also clarifies what the tool does not return ('does not return the organisation's default guardrail list'), differentiating it from get_organisation_defaults, and notes enterprise gating and 403 behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_partial_versionsA
Read-onlyIdempotent

List all versions for one partial, including version numbers, descriptions, status, and timestamps. Use this when you need history or want to choose a version_id before publish_partial.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, idempotent, and openWorld. Description adds that the tool returns specific fields (version numbers, descriptions, status, timestamps), giving extra clarity on output content beyond the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and output, second specifies usage scenario. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one simple parameter, full schema coverage, output schema present, annotations covering safety, and a clear link to a sibling tool (publish_partial), the description is complete and sufficient for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with clear parameter description. Description does not add significant extra meaning beyond the schema, but the mention of 'version_id' in context provides a slight connection.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it lists all versions for one partial, including specific fields (version numbers, descriptions, status, timestamps). It implicitly differentiates from sibling tools like list_prompt_versions by focusing on partials.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use this when you need history or want to choose a version_id before publish_partial,' providing clear when-to-use context and linking to a related action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_labelsA
Read-onlyIdempotent

List labels across the workspace or organisation, with optional search and scope filters. Returns ids, names, colors, status, and timestamps so you can choose a label_id before get_prompt_label or update_prompt_version.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch labels by name
page_sizeNoResults per page (max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID
organisation_idNoFilter by organisation ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds useful context about filtering by workspace/organisation and returning label metadata for downstream selection, but it does not disclose pagination behavior or other operational details beyond what annotations and schema provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The primary action and scope are front-loaded, followed by a concrete explanation of the returned fields and the practical downstream use, which is directly actionable for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich annotations, fully documented parameters, and an output schema, the description is complete enough. It explains what the tool lists, how it can be filtered, what data comes back, and why the agent would call it, leaving no critical operational gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters are already documented with their meanings and constraints. The description only loosely refers to 'optional search and scope filters', which mirrors search, workspace_id, and organisation_id without adding substantial semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'List labels across the workspace or organisation'. It also clarifies the tool's differentiating purpose by stating that it returns label metadata so the agent can choose a label_id before get_prompt_label or update_prompt_version.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear use context: use this tool when you need to discover or filter labels and select a label_id for subsequent operations. It names the downstream tools in the same domain, though it does not explicitly list exclusion cases such as when to use get_prompt_label directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_partialsA
Read-onlyIdempotent

List partials across collections, with optional collection filtering. Returns ids, slugs, names, collections, and status so you can choose a prompt_partial_id before get_prompt_partial, update_prompt_partial, delete_prompt_partial, or publish_partial.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idNoFilter by collection ID. Optional — omit to list all partials across collections

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by specifying returned fields and purpose, but does not disclose additional behavioral traits beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences) and front-loaded with the action. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 optional param, annotations present, output schema exists), the description covers the return fields and purpose adequately for agent invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter description is already clear. The description reiterates the optional parameter usage but does not add new semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it lists partials across collections with optional filtering, and mentions return fields. It distinguishes from sibling tools like get_prompt_partial and update_prompt_partial by indicating its role as a list operation for selection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explains when to use (to list partials and choose an ID for further actions) and mentions optional collection filtering. It doesn't explicitly state when not to use, but provides clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_promptsA
Read-onlyIdempotent

List prompts across the workspace, with optional collection, workspace, or search filters. Returns a paginated summary with id, name, slug, model, and status so you can choose a prompt_id before get_prompt, update_prompt, or render_prompt.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch prompts by name
page_sizeNoResults per page (max 100)
current_pageNoZero-based page number; the first page is 0
workspace_idNoFilter by workspace ID
collection_idNoFilter by collection ID (recommended for app-specific prompts)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds value beyond those by disclosing pagination behavior and the exact summary fields returned (id, name, slug, model, status), which is actionable context for downstream calls. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero waste: the first is front-loaded with the verb, resource, and filters; the second explains the return shape and its downstream purpose. Every clause earns its place, and the most decision-relevant information (what this is for) comes first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only listing tool with 5 optional parameters (all schema-documented), an output schema present, and safety annotations provided, the description is complete. It covers the action, the filter space, the paginated return format, and the workflow role (selecting prompt_id for subsequent tools). Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — every parameter (search, page_size, current_page, workspace_id, collection_id) already has a meaningful description, so the baseline of 3 applies. The description's mention of 'optional collection, workspace, or search filters' restates what the schema already documents without adding syntax, defaults, or format details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List'), a precise resource ('prompts'), and scope ('across the workspace'), then names the filters (collection, workspace, search). It differentiates itself from siblings by explicitly positioning it as the precursor to get_prompt, update_prompt, or render_prompt — the tool that supplies prompt_id for those operations. An agent can tell exactly what this does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear workflow context: use this to choose a prompt_id before get_prompt, update_prompt, or render_prompt. This tells the agent when in a multi-step flow this tool applies. It does not explicitly exclude close alternatives like list_prompt_versions or list_collections, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_versionsA
Read-onlyIdempotent

List all versions of one prompt, including version number, description, status, label, and a short template preview. Use this for history or to choose a version_id before publish_prompt or update_prompt_version.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint, destructiveHint, idempotentHint). Description adds that it returns specific fields but does not mention ordering or limits. Adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and output, second gives usage guidance. No unnecessary words, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present, description appropriately lists return fields. Usage guidance is complete for a simple list tool. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage with description for prompt_id. Description does not add extra semantics beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists all versions of one prompt, specifying the fields returned (version number, description, status, label, template preview). It distinguishes from siblings like list_prompts and get_prompt_version.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly suggests use for history or choosing a version_id before publish_prompt or update_prompt_version. Provides clear context but does not explicitly exclude alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_providersA
Read-onlyIdempotent

List workspace-scoped provider instances and their limits or status. Use this to find provider slugs for workspace-level updates; use list_integrations for the org-level source connection. Returns total plus provider name, slug, integration, status, limits, expiration, and reset flags.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100, default 50)
current_pageNoZero-based page number; the first page is 0
workspace_idNoWorkspace ID - required when using organization admin keys, optional with workspace API keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the safety profile is covered. The description adds useful behavioral context about workspace scoping and the output fields returned (total, name, slug, integration, status, limits, expiration, reset flags), going beyond what annotations alone provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, with the core purpose in the first sentence and usage guidance plus return fields in the second. Every sentence adds relevant information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations, complete parameter schema, and presence of an output schema, the description covers what the tool does, when to use it versus list_integrations, and what results to expect. Nothing critical is missing for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters are already documented with their meanings, defaults, and constraints. The description only lightly reinforces the workspace-scoped nature of the data but does not add substantial meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists workspace-scoped provider instances with limits and status, using a specific verb and resource. It explicitly distinguishes this from list_integrations, so an agent can differentiate the two tools without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance: use this tool to find provider slugs for workspace-level updates, and use list_integrations for the org-level source connection. This directly tells the agent when to choose this tool over the relevant alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_rate_limitsA
Read-onlyIdempotent

List rate-limit policies with their current condition, grouping, rate unit, target, status, and scope. Filter before retrieving or changing one policy, especially when archived policies should be included.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by counter type
unitNoFilter by reset interval
statusNoFilter by active or archived status
targetNoFilter by LLM or MCP-tool target
page_sizeNoNumber of policies to return per page
current_pageNoZero-based page number
workspace_idNoFilter by workspace UUID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false. The description adds useful context about the fields returned and the archived-policy nuance, but it does not disclose deeper behavior such as pagination defaults or whether archived policies are excluded by default. This is acceptable given strong annotations, but not a standout contribution.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no filler. The action and result fields are front-loaded, and the usage guidance follows naturally without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Seven optional parameters are fully described in the schema, and an output schema exists, so the description does not need to detail return values. It provides enough selection context for an agent to know when to call it. The archived-policy hint is slightly ambiguous about default behavior, but overall it is complete enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and all parameters have their own descriptions. The tool description reinforces the filtering idea and references status/type concepts, but it does not add meaning beyond the schema. A baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') and resource ('rate-limit policies') and enumerates the fields returned (condition, grouping, rate unit, target, status, scope). It also differentiates from the sibling get_rate_limit by positioning itself as the list/filter operation used before retrieving or changing a single policy.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Filter before retrieving or changing one policy' clearly indicates this tool is the discovery/filtering step before get/update/delete operations. It also calls out a concrete condition ('especially when archived policies should be included'), giving the agent actionable context, though it does not explicitly name the alternative sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scim_groupsA
Read-onlyIdempotent

Search and page through identity-provider groups synchronized to Portkey over SCIM. Use this to resolve a group ID or exact display name before create_scim_workspace_mapping; it reads directory groups only and does not show their workspace mappings or individual members. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoZero-based results page to retrieve; the first page is 0. These SCIM tools name the parameter page, not the current_page used by other list tools.
searchNoCase-insensitive text to match against SCIM group names
page_sizeNoSCIM groups per page, from 1 through 100

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnlyHint/idempotentHint/destructiveHint, so the bar is lower. The description adds enterprise-gating and the concrete 403 on non-Enterprise plans, plus the 'reads only, does not show mappings/members' limitation. The page-size cap of 100 already exists in the schema, so the description doesn't waste space repeating it. It lacks pagination/cursor details beyond the schema, but annotations cover the safety profile and the added constraints are genuinely useful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with no wasted words: purpose, usage context, scope limitation, and error/enterprise-gating. The most decision-relevant facts (what it does, what it lacks, when to use it) are front-loaded, and the 403 detail is a single clause at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with a rich output schema, zero required params, and full schema coverage, the description covers purpose, scope, exclusions, enterprise gating, and error behavior. There is no missing information an agent needs to select and call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds functional context by tying search to SCIM group names and by linking the tool's purpose to the parameters (resolving group ID/display name). It also clarifies the page parameter is zero-based through the schema itself, which is already encoded there. The description itself adds modest param semantics, so 4 is fair.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (search and page), resource (identity-provider groups synchronized to Portkey over SCIM), and explicit scope (directory groups only, not workspace mappings or members). Distinguishes itself from sibling list_scim_workspace_mappings even before the alternative is named. Nothing tautological or vague.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use this to resolve a group ID or exact display name before create_scim_workspace_mapping, and tells readers what it does not show (workspace mappings or individual members), so an agent knows when to pick it vs. list_scim_workspace_mappings. This is clear when-to-use guidance with an exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scim_workspace_mappingsA
Read-onlyIdempotent

List identity-provider SCIM group mappings that automatically grant Portkey workspace roles. Use it to audit provisioned access or obtain mapping_id before delete_scim_workspace_mapping; filter by workspace, group, or role and page through large directories. This reads mappings only and does not query individual workspace members. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoZero-based results page to retrieve; the first page is 0. These SCIM tools name the parameter page, not the current_page used by other list tools.
roleNoReturn only mappings that grant this workspace role
page_sizeNoMappings per page, from 1 through 100
workspace_idNoReturn only mappings for this Portkey workspace ID
scim_group_idNoReturn only mappings for this identity-provider SCIM group ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly/idempotent/non-destructive, and the description adds meaningful context beyond them: the operation is scoped to mappings only ('does not query individual workspace members'), it supports paging through large directories, and crucially it discloses an entitlement gate ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'). Disclosing the auth requirement and the specific failure mode is exactly the behavioral information an agent needs before invoking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: purpose, use cases plus related-tool reference, behavioral scope, and error/entitlement behavior. The core purpose is front-loaded in the first sentence, and there is zero redundant restating of the title or the schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a filtered list tool with 5 optional params, full schema coverage, an output schema, and safety annotations covering read-only/idempotency, the description covers every decision-relevant gap: purpose, use cases, workflow linkage to delete_scim_workspace_mapping, pagination, entitlement gating, and the 403 failure mode. Nothing an agent needs to decide whether to call this tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter (page, role, page_size, workspace_id, scim_group_id) already has a thorough schema description, including a disambiguation note about the page parameter naming. The description adds only a light grouping ('filter by workspace, group, or role and page through large directories') that maps to the schema. With the schema doing the heavy lifting, the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List identity-provider SCIM group mappings that automatically grant Portkey workspace roles.' The qualifier 'automatically grant... roles' distinguishes this from the sibling list_scim_groups (which lists groups, not mappings) and from list_workspace_members, reinforced later by 'does not query individual workspace members.' An agent can identify exactly what this tool does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit use cases are given: 'Use it to audit provisioned access or obtain mapping_id before delete_scim_workspace_mapping,' which names a concrete workflow and a sibling tool. It also states an explicit exclusion ('does not query individual workspace members') so an agent knows not to use it for member lookups, and it prescribes filter/pagination parameters. This is the strongest form of usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_secret_referencesA
Read-onlyIdempotent

List Secret References without returning authentication configuration or resolved secret values. Filter by manager type, tags, or name and use a returned UUID or slug with get_secret_reference.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoReturn references matching these tags
searchNoSearch references by name
page_sizeNoReferences per page, from 1 to 100; defaults to 20
current_pageNoZero-based page number; defaults to 0
manager_typeNoFilter by external secret manager: aws_sm, azure_kv, or hashicorp_vault

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, so the description's added value is the explicit guarantee that authentication configuration and resolved secret values are not returned. This is useful behavioral context beyond the annotations and helps set expectations for callers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff, and the most important behavioral constraint ('without returning authentication configuration or resolved secret values') is front-loaded. The filter options and follow-up usage are stated efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich annotations, fully described schema, and presence of an output schema, the description is complete. It adds the necessary context about what the list excludes and how to proceed to a detailed reference, leaving no critical gap for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all five parameters. The description adds a compact mapping of the main filter dimensions (manager type, tags, name) and notes the output contains UUIDs/slugs, providing extra semantic glue beyond raw parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'List Secret References' while explicitly excluding authentication configuration and resolved secret values. It also distinguishes itself from the sibling get_secret_reference by naming that tool as the follow-up for retrieving a specific reference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete filtering guidance (manager type, tags, name) and a clear next-step instruction to use a returned UUID or slug with get_secret_reference. It does not explicitly state when not to use this tool, but the context strongly implies it is for discovery/listing rather than fetching full secret details.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_usage_limit_entitiesA
Read-onlyIdempotent

List the values currently tracked by one usage-limit policy with each value key and current usage. Filter by active or exhausted state, search text, and pagination before resetting one counter.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch tracked values
statusNoFilter by enforcement state
limit_idYesUsage-limit policy UUID
page_sizeNoNumber of entities per page
current_pageNoZero-based page number

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, fully covering the safety profile, so the bar is lower. The description adds modest context: per-policy scoping, the return contents (value keys + current usage), and a workflow cue, but reveals no deeper behavior such as pagination defaults or open-world state effects beyond what annotations already signal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences with zero filler: the first states the action, resource, and return content; the second enumerates filters and the workflow position. Every clause earns its place, and the most important disambiguator ('one usage-limit policy') appears immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given rich annotations, 100% schema coverage, and a present output schema, the description supplies the essential selection and invocation context: required limit_id is implied by 'one usage-limit policy', and filters are enumerated. Explicit alternative routing (e.g., naming list_usage_limits) and pagination defaults are minor gaps partially covered by sibling names and the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — all five parameters (search, status, limit_id, page_size, current_page) carry meaningful descriptions such as 'Filter by enforcement state' and 'Zero-based page number'. The description merely echoes the filter set (status, search, pagination) without adding syntax or format details, so the high-coverage baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List the values currently tracked by one usage-limit policy', and states what is returned ('each value key and current usage'). Scoping to a single policy distinguishes it from the sibling `list_usage_limits` (which lists policies) and positions it relative to `reset_usage_limit_entity`. The purpose is unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear operational context: it is for inspecting the tracked values of one policy, with filters for active/exhausted state, search text, and pagination, and it is positioned before a reset action ('before resetting one counter'). It stops short of explicitly naming alternatives or stating when-not-to-use, relying on inference to separate it from list_usage_limits/get_usage_limit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_usage_limitsA
Read-onlyIdempotent

List cumulative cost or token usage-limit policies with current conditions, grouping, reset schedule, status, and scope. Filter by workspace, policy type, status, and pagination.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by budget type
statusNoFilter by active or archived status
page_sizeNoNumber of policies to return per page
current_pageNoZero-based page number
workspace_idNoFilter by workspace UUID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context by naming what the result contains: current conditions, grouping, reset schedule, status, and scope. It does not mention pagination defaults or auth requirements, but those are less critical given the existing annotations and output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler: the first front-loads the resource and expected result content, and the second compresses the filter dimensions. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema and full parameter documentation, the description covers the tool's core behavior and filter set sufficiently for an agent to call it correctly. It lacks explicit sibling disambiguation and pagination default behavior, but these are not essential for a read-only listing tool with well-specified schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters are already documented with meaningful descriptions in the schema. The description merely summarizes filters ('workspace, policy type, status, and pagination') without adding new semantic detail, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') with a clear resource ('cumulative cost or token usage-limit policies') and enumerates the key returned aspects: current conditions, grouping, reset schedule, status, and scope. It is semantically distinct from sibling tools like list_rate_limits and list_usage_limit_entities, even without naming them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies a list/filter use case via the available filters, but it does not explicitly state when to prefer this over get_usage_limit or list_usage_limit_entities, nor does it describe when not to use it. A cautious agent must infer the selection from the tool name and sibling context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_user_invitesA
Read-onlyIdempotent

List pending and sent invitations with id, email, role, status, and expiry. Use this to check invite state; use list_all_users for users who already accepted. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoFilter invitations by organisation role
emailNoFilter by invited email
statusNoFilter invitations by lifecycle status
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context beyond annotations: the tool is enterprise-gated and returns 403 on non-Enterprise plans, and it surfaces the returned fields. Minor omissions like pagination behavior are already covered by the schema, so this is strong but not maximal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: purpose and returned fields, usage/alternative, and gating/error behavior. Each sentence earns its place and the most decision-relevant information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with 100% schema coverage, an output schema, and rich annotations, the description is complete. It covers what is returned, when to use it, which sibling to use instead, and the enterprise restriction. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with all five parameters documented, so the schema carries the parameter-semantics burden. The description adds no parameter-specific meaning beyond the schema; it only mentions the resource and returned fields. Baseline 3 is appropriate because the description neither compensates nor detracts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource: 'List pending and sent invitations', and enumerates the returned fields (id, email, role, status, expiry). It also distinguishes itself from list_all_users, so an agent can immediately tell what this tool does versus its sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use this tool ('Use this to check invite state') and directs to the alternative for already-accepted users ('use list_all_users for users who already accepted'). It also warns about the Enterprise-gated restriction and the 403 result on non-Enterprise plans, leaving no ambiguity about eligibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_virtual_keysA
Read-onlyIdempotent

List provider API keys stored as virtual keys in your Portkey org. Use this to find slugs before wiring prompts/configs or auditing limits. Returns total plus name, slug, status, usage limits, rate limits, reset state, and model config.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context by disclosing the response contents: total plus name, slug, status, usage limits, rate limits, reset state, and model config. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, tightly written, with the core action and use case front-loaded. The return-value enumeration is useful and not redundant with the output schema because it names the key fields an agent would care about.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, read-only, paginated list tool with a full output schema and annotations already covering safety and idempotency, the description is complete. It explains the purpose, the practical use case, and the returned data shape. Nothing crucial for invoking it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters (page_size and current_page) with descriptions covering 100% of the schema. The description adds no additional parameter-level semantics, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'List provider API keys stored as virtual keys in your Portkey org.' It clearly identifies the object being listed and distinguishes this from sibling tools like get_virtual_key, create_virtual_key, and list_api_keys by emphasizing virtual keys and their role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete guidance on when to use the tool: 'Use this to find slugs before wiring prompts/configs or auditing limits.' It stops short of explicitly naming alternatives or saying when not to use it, but the context is clear enough for an agent to select it appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workspace_membersA
Read-onlyIdempotent

List every member in a workspace with organization role, workspace role, status, and timestamps. Use this to find a user_id before get_workspace_member, update_workspace_member, or remove_workspace_member.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoFilter by workspace role
emailNoFilter by member email
page_sizeNoNumber of members per page
current_pageNoZero-based page number
workspace_idYesThe workspace ID to list members for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds useful context about returned fields and the user_id lookup use case, but it does not disclose behavioral details like pagination defaults or response limits beyond what the schema already provides.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler. The first sentence states the action and result fields; the second sentence gives a concrete workflow purpose. It is front-loaded and every clause contributes value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description plus a 100%-covered schema and strong annotations is complete for a read-only list tool. It identifies the required workspace_id, optional filters, pagination parameters, output fields, and the follow-up operations it supports. No critical information needed to select or invoke the tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with workspace_id, role, email, page_size, and current_page all described in the input schema. The description does not add semantic meaning to the parameters themselves, so it earns the baseline score rather than extra credit.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List'), a clear resource ('every member in a workspace'), and names the returned fields ('organization role, workspace role, status, and timestamps'). It also distinguishes itself from singular member operations like get_workspace_member by framing itself as the enumeration step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool to find a user_id before get_workspace_member, update_workspace_member, or remove_workspace_member, giving concrete invocation context. It does not, however, contrast itself with other list tools such as list_all_users or state when not to use it, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workspacesA
Read-onlyIdempotent

List workspaces with id, name, slug, default settings, and timestamps. Use this to find a workspace_id before get_workspace, update_workspace, add_workspace_member, or remove_workspace_member.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoCase-sensitive name filter
statusNoFilter by workspace lifecycle status
page_sizeNoNumber of workspaces to return per page (max 100)
exact_nameNoExact workspace name filter
current_pageNoZero-based page number; the first page is 0

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds the returned-field context and positions the tool as a lookup step, but does not disclose additional behavior such as default page size or ordering. With annotations covering the risk profile, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler. The first sentence states the core behavior and output fields; the second provides routing value by naming the dependent operations. Everything earns its place and the key information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, rich annotations covering safety, and full parameter documentation, the description only needed to establish purpose and usage context — both are present. Nothing an agent needs to select and invoke this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters (name, status, page_size, exact_name, current_page) are already fully documented with types, bounds, enum values, and semantics. The description adds no parameter-level detail, but the schema carries the burden effectively, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') plus resource ('workspaces') and enumerates the returned fields (id, name, slug, default settings, timestamps). It also names the exact downstream operations it feeds (get_workspace, update_workspace, add_workspace_member, remove_workspace_member), making it easy to distinguish from siblings like list_workspace_members or list_scim_workspace_mappings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit, actionable context: 'Use this to find a workspace_id before' four named operations. This tells the agent exactly when to reach for this tool. It stops short of a full 5 because it doesn't state when not to use it or name alternative lookup tools, but the provided workflow guidance is concrete and sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

migrate_promptA
Idempotent

Create or update a prompt in one idempotent step for CI/CD and prompt-as-code flows, unlike create_prompt which always makes a new prompt. Looks up the prompt by name within collection_id: if missing it creates the prompt, if found with different content it adds a new version (nothing is overwritten), and if the content already matches it no-ops; dry_run reports what would happen without changing anything. Stores app/env in template_metadata; get collection_id from list_collections. Returns the action taken (created, updated, or unchanged), dry_run flag, message, prompt id, slug, and version id.

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp identifier (REQUIRED). Use your deployed app name, for example 'hourlink' or 'support-console'.
envYesEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
nameYesPrompt name to create or find for update
modelNoModel identifier
toolsNoTool definitions
stringNoLegacy prompt template string with {{variable}} mustache syntax.
dry_runNoWhen true, only check what action would be taken without making changes
messagesNoStructured chat template alias for migrations. Serialized to the legacy string format before the prompt is created or updated.
functionsNoFunction definitions
parametersYesDefault values for template variables
tool_choiceNoTool choice strategy
virtual_keyYesVirtual key slug for model access
collection_idYesCollection ID to search in and create under
is_raw_templateNoPreserve raw structural-template rendering semantics
template_metadataNoAdditional custom metadata
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false, and the description enriches those with exact semantics: creates if missing, adds a new version without overwriting if content differs, no-ops if content already matches, and dry_run changes nothing. It also discloses side-effect context ('Stores app/env in template_metadata') and the return payload. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but economical, packing purpose, sibling distinction, the idempotent state machine, dry-run behavior, metadata storage, collection_id sourcing, and return fields into four sentences. It is front-loaded with the core purpose and avoids redundant schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 16-parameter tool with an output schema, the description covers the non-obvious selection and invocation details: idempotency semantics, no-overwrite safety, where collection_id comes from, and what action/identifiers are returned. Nothing an agent needs to safely migrate a prompt is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds value by clarifying dry_run behavior, explaining that app/env are persisted into template_metadata, and providing a source for collection_id via list_collections. The remaining parameter meanings are adequately handled by the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Create or update a prompt') and immediately scopes it as an idempotent step for CI/CD and prompt-as-code flows. It explicitly differentiates itself from create_prompt, so an agent can distinguish it from the closest sibling without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use context ('for CI/CD and prompt-as-code flows') and names the key alternative with its distinguishing behavior ('unlike create_prompt which always makes a new prompt'). It also tells the caller where to obtain collection_id ('get collection_id from list_collections'), covering the main prerequisite for invoking the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

promote_promptA

Copy a prompt from one environment to another and create or update the target automatically. Use this for staged releases when you want the target prompt synchronized without manual edits, and it returns both source and target version ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
target_envYesEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
target_nameNoTarget prompt name (defaults to source name with env suffix replaced)
virtual_keyNoVirtual key ID to use (defaults to source prompt's virtual_key)
source_prompt_idYesSource prompt ID or slug (e.g., staging prompt)
target_collection_idYesTarget collection ID for the promoted prompt

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate mutation (readOnlyHint false) and non-destructiveness, and the description adds that it creates or updates the target and returns version IDs. However, given openWorldHint true, the description does not disclose potential side effects or required permissions, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise—just two sentences—with no extraneous information. It front-loads the primary action and then provides usage guidance, making it efficient and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's 5 parameters and mutation nature, the description covers core purpose, usage context, and return values. It does not mention error conditions or prerequisites, but the presence of a (presumed) output schema reduces the need for full detail. Overall, it is fairly complete for the use case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 5 parameters have descriptions in the schema (100% coverage), so the description adds minimal extra parameter-level detail. It mentions return values (version IDs) but not parameter specifics, warranting the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: copy a prompt between environments and auto-create/update the target. It explicitly distinguishes itself by mentioning staged releases and synchronization without manual edits, which sets it apart from siblings like create_prompt, update_prompt, and migrate_prompt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear when-to-use context: for staged releases when automated synchronization is desired. However, it does not explicitly state when not to use this tool or compare it to alternatives like migrate_prompt, which is present among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publish_partialA

Publish a specific partial version as the default, unlike update_prompt_partial which creates a new draft without activating it. Use after list_partial_versions to pick a version_id; this immediately changes what {{> partial_name}} resolves to for all prompts and replaces the previously active version without a rollback path.

ParametersJSON Schema
NameRequiredDescriptionDefault
versionYesVersion number to publish as default
prompt_partial_idYesPrompt partial ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that it immediately changes the default for all prompts, replaces the active version, and has no rollback path. This adds value beyond annotations (which indicate non-read-only behavior).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and differentiation, second gives usage guidance and consequences. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given simple two-param schema, annotations, and presence of an output schema, the description covers purpose, usage, effect, and caution. Sufficient for correct tool invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions. The description provides usage context (e.g., 'use after list_partial_versions') but adds no new meaning to the parameters themselves.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool publishes a specific partial version as default and distinguishes it from update_prompt_partial, which creates a draft. It specifies the effect on partial resolution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use after list_partial_versions to pick a version_id, describes immediate changes and no rollback path. Lacks explicit 'when not to use', but context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publish_promptA

Publish a specific version of a prompt as the active default, unlike promote_prompt which copies across environments or update_prompt which creates a new draft. This immediately routes all callers using the slug to that version and there is no rollback, so use list_prompt_versions to pick the version and update_prompt first if you need to create new content before promoting it.

ParametersJSON Schema
NameRequiredDescriptionDefault
versionYesVersion number to publish as the default
prompt_idYesPrompt ID or slug to publish

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only indicate non-read-only and non-destructive. The description adds crucial behavioral context: immediate routing of all callers and no rollback possibility. This is essential for understanding the irreversible impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise: three sentences that front-load the core action, contrast with siblings, explain impact, and give a recommended workflow. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains the effect, irreversibility, and workflow. It complements the existing schema and annotations well. Could optionally mention error conditions or prerequisites, but given the output schema exists and annotations are adequate, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameter descriptions. The description does not add further detail about the parameters themselves but provides contextual usage advice ('use list_prompt_versions to pick the version'). Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Publish a specific version of a prompt as the active default'. It also distinguishes itself from sibling tools 'promote_prompt' and 'update_prompt', making the action unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance by contrasting with 'promote_prompt' and 'update_prompt'. It also advises a workflow: use 'list_prompt_versions' to pick the version and 'update_prompt' if new content is needed before publishing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

register_deploymentA

Enterprise-gated. Register a self-hosted Gateway deployment when onboarding a new control-plane target; use list_deployments for existing registrations. The response can contain authentication and registry credentials exactly once, exposed to this MCP transcript, so store them securely immediately and never log them. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDeployment display name
slugNoOptional deployment slug
typeNoProduction or non-production deployment type
is_defaultNoMake this the default deployment
organisation_idNoOwning organisation UUID
gateway_base_urlNoSelf-hosted Gateway base URL
jwt_subs_allowedNoJWT subject values allowed to use the deployment
deployment_configNoGateway deployment configuration
workspaces_allowedNoWorkspace slugs this deployment may serve; empty allows all
mcp_gateway_base_urlNoMCP Gateway base URL
is_dataservice_hostedNoWhether the deployment hosts its own data service
jwt_sub_workspace_mappingNoJWT subject to workspace-slug mapping
is_playground_proxy_allowedNoWhether Playground proxy traffic is allowed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Delivers high-value behavior that annotations cannot express: the response contains authentication and registry credentials exactly once, they are exposed in the MCP transcript, and the agent must store them securely and never log them. Also discloses enterprise gating and the 403 error on ineligible plans. This goes well beyond the flag-only annotations (readOnlyHint=false, idempotentHint=false) and meaningfully shapes how the agent should handle the return value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and the structure flows well (purpose → context → security → error). However, 'Enterprise-gated' appears verbatim twice in a roughly 50-word description, which is redundant and sloppy. The credential warning is also somewhat wordy but earns its place given its criticality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the essential non-schema context: when to use it, the sibling alternative, the one-time credential exposure and storage imperative, the logging prohibition, the enterprise-plan restriction, and the 403 failure mode. With a 100%-covered 13-parameter schema and an output schema present, nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every one of the 13 parameters is already documented; baseline 3 applies. The description adds no parameter-level meaning beyond what the schema provides, though the credential warning indirectly implies the response varies with parameters. This is acceptable given the schema carries the full load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Register a self-hosted Gateway deployment') plus the precise triggering context ('when onboarding a new control-plane target'). It also names the sibling it is not ('use list_deployments for existing registrations'), so an agent can distinguish it from the deployment family without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly ties usage to a condition ('when onboarding a new control-plane target') and names the alternative for existing registrations (list_deployments). The 403-on-non-Enterprise caveat also tells an agent when the operation will fail. It does not explicitly route modifications of existing registrations to update_deployment, which is a minor gap given that sibling exists.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove_workspace_memberA
Destructive

Remove a user from a workspace and revoke workspace access. This does not delete the user from the organization; use delete_user for full removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to remove
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and readOnlyHint=false. The description adds valuable context by clarifying that workspace access is revoked but the user remains in the organization, which helps the agent understand the exact scope of destruction. It does not cover reversibility or permissions, but the essential behavioral traits are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary action, and contains no fluff. The second sentence adds important clarification about what the tool does not do, which is essential for disambiguation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for the tool's complexity: there is an output schema, the parameters are fully described in the schema, annotations cover safety traits, and the description provides the needed scope and alternative guidance. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both required parameters (user_id and workspace_id) already documented in the schema. The description does not add further parameter-level detail, but the schema carries the full semantic burden, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: 'Remove a user from a workspace and revoke workspace access.' It clearly distinguishes from the sibling delete_user by noting this action does not delete the user from the organization, making the tool's purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides usage guidance: use this tool to remove a user from a workspace, and use delete_user for full removal from the organization. This directly addresses when to use this tool versus an alternative, which is strong guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

render_promptA
Read-onlyIdempotent

Render a prompt by substituting variables and returning the final messages without calling the model. Use this to verify template output before a completion; run_prompt_completion is the tool that actually invokes the model.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to render
variablesYesVariable values to substitute into the template
hyperparametersNoOverride default hyperparameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive. Description adds important behavioral context: it substitutes variables, returns final messages, and does not call the model. This clarifies the exact operation beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first explains what the tool does, second provides usage guidance and distinguishes from a sibling. No unnecessary words, front-loaded with essential info.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with nested parameters and output schema, the description covers the core purpose, usage context, and comparison to sibling. It notes return of final messages, which suffices given output schema exists. Complete for informed selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description does not repeat parameter schemas but implicitly connects the 'variables' parameter to substitution. No extra detail beyond schema, but given full coverage, this is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it renders a prompt by substituting variables and returning final messages without calling the model. It explicitly distinguishes from the sibling tool run_prompt_completion, which invokes the model. The verb and resource are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool: 'Use this to verify template output before a completion'. Names the alternative tool run_prompt_completion for when model invocation is needed. Provides clear context that this is for verification only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resend_user_inviteA

Resend the email for a pending invite that has not been accepted, unlike invite_user which creates a new invite. This sends a fresh email without modifying the invite record, expiry, or role; use get_user_invite first if you are unsure whether the invite still exists and list_user_invites to discover invite_ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to resend

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description clarifies that the tool 'sends a fresh email without modifying the invite record, expiry, or role', which complements the annotations (readOnlyHint=false, destructiveHint=false) by specifying the non-destructive nature despite being a mutation. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences that efficiently deliver purpose, behavior, usage differentiation, and prerequisite tool recommendations without any redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple parameter set and comprehensive annotations, the description provides complete context including behavior, side effects (none), and usage hints. No gaps remain for an agent to safely invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (single parameter invite_id with clear description). The tool description does not add further semantic meaning beyond what the schema already provides for the parameter, thus baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Resend the email for a pending invite that has not been accepted') and contrasts with sibling tool 'invite_user' which creates a new invite, making the purpose and distinction unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (for resending pending invites) and when not (use invite_user to create new invites). Additionally, recommends calling get_user_invite first to verify invite existence and list_user_invites to discover IDs, providing excellent usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reset_usage_limit_entityA
Destructive

Reset the current usage counter for one tracked usage-limit entity. This changes enforcement immediately for that exact policy and entity, so use the entity id returned by list_usage_limit_entities.

ParametersJSON Schema
NameRequiredDescriptionDefault
limit_idYesUsage-limit policy UUID
entity_idYesTracked entity UUID to reset

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and readOnlyHint=false. The description adds meaningful behavioral context by warning that enforcement changes immediately and that the reset applies to exactly one policy-entity pairing, which goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The main action is stated first, followed by a critical scoping and sourcing caveat that earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation tool with a full input schema, output schema, and clear annotations, the description sufficiently explains the operation's effect, scope, and how to get the required entity id. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters with clear descriptions and 100% coverage. The description adds extra value by specifying that entity_id should come from list_usage_limit_entities, which helps the agent retrieve the correct value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Reset the current usage counter for one tracked usage-limit entity.' It also clarifies the scope ('that exact policy and entity'), distinguishing it from operations like deleting or updating the usage limit itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context about the immediate enforcement change and instructs the agent to use the entity id returned by list_usage_limit_entities. However, it does not explicitly state when to use this tool instead of related alternatives such as delete_usage_limit or update_usage_limit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rotate_api_keyA
Destructive

Rotate an API key without changing its identity or scopes. The new secret is returned once and exposed to this MCP transcript, while the previous secret remains valid until key_transition_expires_at. Store the new key securely, update callers during the transition window, and never log either secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesAPI key UUID obtained from list_api_keys
key_transition_period_msNoOverlap in milliseconds while the previous key remains valid; minimum 1,800,000 (30 minutes), for example 3,600,000 for 1 hour

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description reveals that the new secret is exposed in the transcript, the previous secret remains valid during a transition period, and both secrets must never be logged. This adds critical behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences: purpose, behavioral detail, and actionable guidance. No fluff; front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers essential aspects (purpose, transition window, security). Lacks explicit error conditions or prerequisites, but given the output schema and full parameter descriptions, it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds context about the transition period's effect (key_transition_expires_at) but does not significantly enhance parameter meaning beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Rotate an API key without changing its identity or scopes,' specifying the action and resource. It implicitly distinguishes from sibling tools like update_api_key by emphasizing identity and scope preservation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides post-rotation guidance (store securely, update callers, never log) and hints at the transition window, but does not explicitly state when to use rotate versus alternatives like create or update.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_prompt_completionA

Execute a prompt against the configured model and return the completion. This makes a billable model call, so use render_prompt first when you want to check the template and validate_completion_metadata when billing fields are uncertain.

ParametersJSON Schema
NameRequiredDescriptionDefault
metadataYesBilling metadata - client_id, app, env are REQUIRED for cost attribution
prompt_idYesPrompt ID or slug to execute
variablesYesVariable values to substitute into the template
hyperparametersNoOverride default hyperparameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that the tool makes a billable model call, adding context beyond annotations. Annotations already indicate readOnlyHint=false and destructiveHint=false. The description adds value but could mention other side effects like logging.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first defines purpose, second provides usage guidelines. No wasted words; front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present and extensive parameter descriptions in schema, the description is largely complete. It covers purpose and billing context but could mention error handling or rate limits for a more complete picture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds minimal parameter info beyond the schema, mostly restating that billing metadata is required. No significant enhancement.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states 'Execute a prompt against the configured model and return the completion.' This provides a specific verb and resource, and distinguishes from siblings by mentioning render_prompt and validate_completion_metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use render_prompt for template checking and validate_completion_metadata when billing fields are uncertain, given the billable nature of the call. Provides clear usage context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_log_exportA

Start processing a previously created log export job. This is asynchronous, only queues the export, and does not return rows or a download file; use get_log_export to poll progress and download_log_export after the job completes. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to start

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behavioral traits beyond annotations: asynchronous queueing, no immediate rows/download return, dependency on prior job creation, enterprise-only gating with 403 on non-Enterprise plans. This adds significant context not present in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences, each adding essential information: action, async behavior, alternative tools, and enterprise restriction. No redundant phrases or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter schema, existing output schema, and rich annotations, the description covers all critical aspects: what it does, how it behaves asynchronously, what it does not return, next steps, and error condition. Fully sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter export_id is already well-described in the schema. The description reinforces that it must be a previously created job, but this adds minimal new semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action: 'Start processing a previously created log export job.' It distinguishes itself from siblings like create_log_export and get_log_export by emphasizing that this is the start step, not creation or polling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides usage context: it operates on a previously created job, is asynchronous, and directs the agent to use get_log_export for polling and download_log_export after completion. Enterprise gating is also noted, covering when it is and isn't applicable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_mcp_serverA

Test connectivity to an MCP server. Sends a live check and returns success, response time, HTTP status, and any error; use this before changing configuration or when diagnosing reachability.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to test

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that it sends a live check and returns success, response time, HTTP status, and any error. Annotations indicate idempotentHint false and openWorldHint true, which the description supports by mentioning external interaction. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. The description is front-loaded with the purpose, followed by usage guidance. Highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple diagnostic tool with one parameter and an output schema, the description covers essential aspects: purpose, when to use, and what to expect in return. Complete given the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description does not add extra meaning beyond what the schema provides for the 'id' parameter. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Test connectivity to an MCP server' with a specific verb and resource, and the context of diagnosing reachability distinguishes it from sibling tools like get_mcp_server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises use before configuration changes or when diagnosing reachability, providing clear context. However, it does not explicitly name alternative tools for other use cases, though these can be inferred from sibling names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_api_keyA

Update an API key's name, description, scopes, defaults, or limits, unlike delete_api_key which revokes it or create_api_key which issues a new one. Changes take effect immediately for downstream callers, type and sub-type stay fixed after creation, and the call returns success without rotating the secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to update
nameNoNew display name for the key
scopesNoNew permission scopes for the key
expires_atNoNew expiration date in ISO 8601 format, or null to remove expiration
descriptionNoNew description for the key
rate_limitsNoReplacement request or token rate limits, or null to clear
reset_usageNoSet to a nonnegative acknowledgement value to reset usage
alert_emailsNoNew email addresses for alerts
credit_limitNoNew credit limit for usage
rate_limit_rpmNoNew rate limit in requests per minute
alert_thresholdNoNew alert threshold percentage (0-100)
rotation_policyNoReplacement rotation policy, or null to disable it
default_metadataNoNew default metadata key-value pairs
default_config_idNoNew default configuration ID
default_allow_config_overrideNoWhether callers may override the key's default config

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint=false, destructiveHint=false, and idempotentHint=false, and the description adds meaningful extra context: changes take effect immediately for downstream callers, type and sub-type are immutable after creation, and the call does not rotate the secret. These are non-obvious behaviors an agent needs to know, and nothing contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler. The primary action is front-loaded, sibling differentiation comes second, and the behavioral caveats are packed into the final clause. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 15-parameter tool with nested objects, the description covers the core behavioral essentials (immediate effect, fixed type/sub-type, no rotation) and the output schema relieves it of explaining return values. A minor gap: it does not explicitly state whether omitted fields are left unchanged, which is material for an update operation with so many optional fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description's field list ('name, description, scopes, defaults, or limits') provides a compressed high-level map and groups related parameters, but it adds little meaning beyond what the schema's per-parameter descriptions already state. It does not, for example, explain update semantics like partial vs. full replacement.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Update') plus resource ('API key') and enumerates the mutable attributes (name, description, scopes, defaults, limits). It explicitly names sibling tools it is not — delete_api_key (revokes) and create_api_key (issues new) — so an agent can disambiguate without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly contrasts update_api_key with delete_api_key and create_api_key, telling the agent when NOT to use it. The phrase 'returns success without rotating the secret' also implicitly routes secret-rotation intent to the rotate_api_key sibling. No inference is required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_collectionA

Update a collection's name or description only. This does not move prompts or change membership, so use it for metadata changes rather than reorganization.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the collection
descriptionNoNew description for the collection
collection_idYesCollection ID to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond annotations, specifying that only name/description can be updated and that it does not affect membership or prompts. Annotations already indicate non-destructive and non-read-only, so the description complements without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. The first sentence states the action and scope; the second provides exclusions and usage guidance. Front-loaded and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple update tool with 3 parameters and an output schema, the description is complete. It covers what is updated, what is not, and when to use the tool. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds meaning by stating 'name or description only', confirming the intended parameters. While the schema lists them, the description reinforces the limitation, adding some value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a collection's name or description only', specifying the verb (update), resource (collection), and scope (name or description only). It effectively distinguishes from sibling tools like update_prompt or update_workspace by highlighting its limited scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'use it for metadata changes rather than reorganization.' It also clarifies what the tool does not do ('does not move prompts or change membership'), helping the agent avoid misuse.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_configA

Update a config by slug; every call creates a new config version rather than overwriting, so earlier versions stay recoverable via list_config_versions. Only provided fields change: name, status, and all routing/cache/retry settings (cache_mode, cache_max_age, retry_attempts, retry_on_status_codes, strategy_mode, targets) are editable, while the slug stays fixed. Changes apply immediately to every API key and prompt referencing the config. Returns the update acknowledgement and new version_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the configuration
slugYesConfiguration slug to update
statusNoConfiguration status
targetsNoArray of target providers
cache_modeNoCache mode: 'simple' or 'semantic'
cache_max_ageNoCache max age in seconds
strategy_modeNoRouting strategy
retry_attemptsNoNumber of retry attempts (1-5)
retry_on_status_codesNoHTTP status codes to retry on

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only mark readiness and idempotency, so the description carries the burden and does it well. It discloses versioned writes instead of overwrites, recoverability via list_config_versions, immediate propagation to all API keys/prompts, and the returned version_id, all of which materially affect how an agent should reason about repeated calls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: purpose and key caveat first, then editable vs fixed fields, then impact and return value. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutation tool, the description covers the important context the schema and annotations cannot: versioning behavior, immediate global effect, slug immutability, and the version_id return. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3; the description adds value beyond it by giving patch semantics ('Only provided fields change'), marking the slug as fixed, and grouping the editable routing/cache/retry parameters. It does not need to repeat per-parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description names a specific verb and resource: update a config by slug, and immediately differentiates the operation from overwrite-style updates by stating every call creates a new config version. This distinguishes it from sibling tools like create_config, delete_config, and list_config_versions without needing the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clear context is provided: update by slug, only provided fields change, and the slug stays fixed, implying use a different creation path if a new slug is needed. It does not explicitly name create_config or state when not to call it, but the partial-update and versioning semantics effectively route the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_deploymentA

Enterprise-gated. Update a registered Gateway deployment, its workspace or JWT-sub access, or rotate its authentication secret. Use get_deployment first to inspect current settings. Rotation returns the new secret once in this MCP transcript, so store it securely immediately. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesDeployment UUID
nameNoReplacement display name
typeNoReplacement deployment type
statusNoReplacement deployment status
is_defaultNoWhether this is the default deployment
rotate_authNoRotate the one-time deployment authentication secret
gateway_base_urlNoSelf-hosted Gateway base URL
jwt_subs_allowedNoJWT subject values allowed to use the deployment
deployment_configNoReplacement Gateway configuration, or null to clear it
override_existingNoAllow replacement of existing deployment settings
workspaces_allowedNoWorkspace slugs this deployment may serve; empty allows all
remove_subs_allowedNoJWT subject values to remove from the allowlist
allow_all_workspacesNoClear workspace restrictions and allow every workspace
mcp_gateway_base_urlNoMCP Gateway base URL
is_dataservice_hostedNoWhether the deployment hosts its own data service
jwt_sub_workspace_mappingNoJWT subject to workspace-slug mapping
remove_workspaces_allowedNoWorkspace slugs to remove from the allowlist
is_playground_proxy_allowedNoWhether Playground proxy traffic is allowed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as a mutating, non-idempotent call, and the description adds two material behavioral facts beyond them: the operation is Enterprise-gated (403 on non-Enterprise plans) and secret rotation returns the new secret only once in this transcript, requiring immediate secure storage. That one-time disclosure is a high-stakes trait an agent must not miss. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose, sequencing, the one-time secret warning, and the error condition each occupy one purposeful sentence, properly front-loaded. The only flaw is that 'Enterprise-gated' appears twice, a redundant token cost in an otherwise tight description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 18-parameter nested-object tool, the description covers the essential operational surface: what types of updates are possible, the prerequisite inspection call, the critical one-time secret behavior, and the plan-gating error. The output schema covers return values, and annotations cover the safety profile, so nothing operationally critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the baseline is 3 and the schema already documents each of the 18 parameters. The description adds an organizing frame — workspace updates, JWT-sub allowlist updates, and auth-secret rotation — that groups the parameter space into update modalities, and it flags rotate_auth's special one-time-return behavior. This is meaningful value above the schema baseline, though it adds no per-parameter syntax detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb + resource ('Update a registered Gateway deployment') and enumerates three distinct update modalities: workspace access, JWT-sub access, and auth-secret rotation. It clearly differentiates from its sibling deployment tools (register_deployment, archive_deployment, get_deployment) by scope. Plan-gating and the 403 outcome further pin down what this call is for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit sequencing guidance — 'Use get_deployment first to inspect current settings' — which tells the agent when in a workflow this tool belongs. It does not, however, state explicit exclusions such as 'use register_deployment to create a new deployment' or 'use archive_deployment to remove one,' so alternatives are not fully spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_feedbackA

Update an existing feedback record by ID. Returns the updated status and feedback IDs, changes only value, weight, and metadata, and leaves the trace linkage immutable; use create_feedback only for a new record.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the feedback to update
valueNoNew feedback value/rating. Common patterns: 1 for positive, 0 for negative.
weightNoNew weighting factor for the feedback
metadataNoNew or updated custom metadata for the feedback

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate non-readonly, non-destructive, non-idempotent, open-world. The description adds that only value, weight, and metadata are changed, trace linkage is immutable, and it returns updated status and IDs. This provides useful context beyond annotations, though it could mention potential side effects (e.g., impact on analytics).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action and return, then constraints and alternative. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 parameters, output schema exists) and the presence of sibling alternatives, the description covers the essential aspects: purpose, modifiable fields, immutability of trace linkage, and usage guidance. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all four parameters. The description summarizes which parameters are modifiable but does not add new semantic details beyond the schema. The mention of return values is about output, not input parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update an existing feedback record by ID' with a specific verb and resource. It details what fields are changed (value, weight, metadata) and distinguishes from the sibling tool create_feedback, which is for new records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool (to update existing feedback) and when not (use create_feedback for a new record). It also clarifies that trace linkage is immutable, guiding parameter selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_guardrailA

Update a guardrail's name, checks, or actions, unlike create_guardrail which registers a new one or delete_guardrail which removes it. This creates a new version that takes effect immediately for dependent configs, so review list_guardrails first; returns the updated id, slug, and version_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the guardrail
checksNoUpdated array of checks to apply
actionsNoUpdated actions configuration
guardrail_idYesThe guardrail UUID or slug to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false and idempotentHint=false, but the description adds genuinely non-obvious behavior beyond them: the update creates a new version, takes effect immediately for dependent configs, and returns id/slug/version_id. This versioning side-effect is exactly the kind of context structured fields cannot convey. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero filler: purpose and sibling contrast first, then the versioning consequence and recommended pre-step, then the return contract. Every clause earns its place and the most decision-relevant information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex mutation tool with nested objects and an output schema, the description covers purpose, sibling routing, a behavioral warning (immediate effect on dependent configs), a recommended pre-step, and the return payload. The output schema covers return structure, so nothing an agent needs to invoke this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters richly, including the security caution on forwarded headers. The description only echoes the parameter categories (name, checks, actions) without adding syntax or format detail. This meets the baseline for high schema coverage but adds no parameter-level value beyond it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource+scope: 'Update a guardrail's name, checks, or actions', and immediately differentiates itself from siblings ('unlike create_guardrail which registers a new one or delete_guardrail which removes it'). An agent can distinguish this from create/delete/get/list variants without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the alternatives and the condition that selects them: create_guardrail for registration, delete_guardrail for removal. It goes further by prescribing a workflow step ('review list_guardrails first') because updates create a new version that affects dependent configs. This is direct when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_input_guardrail_workspace_exclusionsA
DestructiveIdempotent

Set workspace exclusions from organisation-wide input guardrails. Each entry excludes or restores one workspace; override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces. Review the matching list tool first because enforcement changes immediately. Repeating the same states is safe. Requires an organisation service API key with organisation_exclusions.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspacesYesWorkspace exclusion states to apply
organisation_idYesOrganisation ID whose workspace exclusions should be updated
override_existingNoReplace existing exclusion states instead of merging changes

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses merge vs. override behavior, immediate enforcement changes, idempotency of repeating states, authentication requirements, and 403 errors on non-Enterprise plans. This significantly enriches the annotations (readOnlyHint=false, destructiveHint=true, idempotentHint=true), which only hint at these behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and information-dense. Six short sentences each contribute unique value: purpose, entry semantics, merge behavior, guidance, auth, and error condition. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex mutation tool, the description covers all essential context: auth scope, Enterprise gating, immediate enforcement impact, merge vs. override semantics, and idempotency. With annotations and an output schema present, this description is fully sufficient for correct tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining that each entry excludes or restores one workspace and that default merge preserves unmentioned workspaces, which clarifies the workspaces array and override_existing parameter beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool sets workspace exclusions for organisation-wide input guardrails, with a specific verb and resource. It distinguishes itself from the sibling update_output_guardrail_workspace_exclusions by specifying 'input guardrails' and explains the core entry semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises reviewing the matching list tool first because enforcement changes immediately, and specifies required API key scope and Enterprise plan gating. While it doesn't name alternatives, the context strongly implies the input/output distinction and provides clear preconditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integrationA
DestructiveIdempotent

Update an integration's name, description, API key, provider config, Secret Reference mappings, or pricing adjustments by slug. Only provided fields change; key, secret mapping, and config changes take effect immediately and can disrupt dependent providers or live requests, while pricing multipliers change cost analytics and budget accounting. Review get_integration first. Model availability and workspace access remain separate in update_integration_models and update_integration_workspaces.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoNew API key for the provider
nameNoNew human-readable name for the integration
slugYesThe slug of the integration to update
aws_regionNoNew AWS region (for AWS Bedrock)
api_versionNoNew API version (for Azure OpenAI)
custom_hostNoNew custom base URL for the provider
descriptionNoNew description for the integration
resource_nameNoNew resource name (for Azure OpenAI)
vertex_regionNoNew GCP region (for Vertex AI)
deployment_nameNoNew deployment name (for Azure OpenAI)
secret_mappingsNoReplacement runtime Secret Reference mappings; each target_field must be unique
aws_access_key_idNoNew AWS access key ID (for AWS Bedrock)
vertex_project_idNoNew GCP project ID (for Vertex AI)
pricing_adjustmentsNoReplacement cost multiplier configuration, or null to clear adjustments
aws_secret_access_keyNoNew AWS secret access key (for AWS Bedrock)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

While annotations already flag destructiveHint and readOnlyHint, the description adds vital behavioral context: partial updates are honored, changes to key/secret/config are immediately effective and can disrupt live traffic, pricing adjustments modify cost analytics, and related concerns are deliberately isolated. This materially enriches what the bare annotations convey and aligns with them (no contradiction).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences deliver purpose, side effects, usage safety, and sibling differentiation without any filler. The most important behavioral caveat (disruption risk) is front-loaded, and every clause contributes information an agent needs before invoking the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's 15 parameters, rich schema, and existing output schema, the description covers purpose, mutation semantics, immediate effects, risk profile, and separation from closely related tools. It leaves no major gap: an agent knows what to call, what to expect, and how to avoid unintended consequences. The explicit pointer to get_integration for pre-review further rounds out operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already covers all 15 parameters with detailed per-field descriptions, so the baseline is a 3 per the rubric. The description adds value by grouping fields into behavioral categories (e.g., 'key, secret mapping, and config' vs. 'pricing adjustments') and clarifying partial update semantics ('Only provided fields change'), which helps the agent understand how parameter combinations behave even though individual properties are schema-documented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Update') and an explicit resource ('an integration'), then enumerates the exact updatable fields: name, description, API key, provider config, Secret Reference mappings, and pricing adjustments. It further distinguishes this tool from siblings by naming update_integration_models and update_integration_workspaces as separate concerns, making its scope unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete guidance: only provided fields change, key/secret/config changes take effect immediately and can disrupt providers or live requests, and pricing multiplier changes affect cost analytics. It also instructs the agent to review get_integration first and explicitly redirects model/workspace access changes to separate tools, providing both usage context and exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integration_modelsA
DestructiveIdempotent

Bulk enable or disable integration models, register custom or fine-tuned models, set per-model hosts and headers, and attach static token pricing. allow_all_models controls whether future provider models start enabled. These changes affect every workspace using the integration; inspect list_integration_models first and use get_model_pricing when deriving custom rates. Returns success and the number of models updated.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
modelsYesArray of model configurations to update
allow_all_modelsNoWhether newly available provider models are enabled by default

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true. The description adds valuable context beyond annotations: 'These changes affect every workspace using the integration' and 'Returns success and the number of models updated.' This improves transparency, though it doesn't cover all nuances like atomicity or rollback behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four sentences, each earning its place: first lists actions, second clarifies a parameter, third warns about scope and gives guidance, fourth notes the return. It is front-loaded and concise without waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 3 parameters and a nested models array, the description covers purpose, safety, scope, sequencing, alternatives, and return value. An output schema exists, so detailed return fields do not need to be explained. The description is complete and well-rounded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description summarizes parameter effects (e.g., 'allow_all_models controls whether future provider models start enabled') but does not add significant new meaning beyond the schema definitions. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb and resource: 'Bulk enable or disable integration models, register custom or fine-tuned models, set per-model hosts and headers, and attach static token pricing.' It enumerates distinct actions, which distinguishes it from sibling tools like list_integration_models or delete_integration_model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'inspect list_integration_models first' and 'use get_model_pricing when deriving custom rates.' It names alternative tools and gives sequencing context, fulfilling the when-to-use vs alternatives criterion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integration_workspacesA
DestructiveIdempotent

Control per-workspace and global access to an integration, including cost/rate limits, usage resets, and automatic default-provider creation. global_workspace_access_enabled affects current and future workspaces; override_existing_workspace_access determines whether it replaces explicit workspace settings. Per-workspace default-provider fields override top-level values. Review list_integration_workspaces first because access and limits change downstream usage immediately.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
workspacesYesArray of workspace configurations to update
global_credit_limitNoGlobal cost credit limit applied with global workspace access
default_provider_slugNoDefault slug for providers auto-created in workspaces
global_rate_limit_rpmNoGlobal requests-per-minute limit for workspace access
global_alert_thresholdNoGlobal cost alert threshold applied to workspace access
create_default_providerNoAuto-create providers when granting workspace access
global_workspace_access_enabledNoEnable or disable access for all current and future workspaces
override_existing_workspace_accessNoApply global settings over existing per-workspace access

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context beyond these: global_workspace_access_enabled affects current and future workspaces, override_existing_workspace_access determines replacement behavior, and per-workspace fields override top-level values. It also notes immediate downstream impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: first states purpose and scope, second explains key toggles, third clarifies precedence, fourth gives operational warning. Slightly dense but well structured and front-loaded with the main purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, global vs per-workspace settings), the description covers the critical nuances and includes a caution to review first. An output schema exists, so return values need not be explained. It is complete enough for a knowledgeable agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaningful interactions between parameters (e.g., precedence rules, override semantics) that are not apparent from individual property descriptions. This elevates the score to 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool controls per-workspace and global access to an integration, enumerating specific features (cost/rate limits, usage resets, provider creation). This distinguishes it from read-only siblings like list_integration_workspaces and the broader update_integration tool, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance to 'Review list_integration_workspaces first' and warns that access and limits change immediately, giving clear context on when and how to use the tool. It stops short of naming alternative tools for exclusions, but the prerequisite sequence is helpful.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_log_exportA

Update an existing log export configuration before or between export runs. Only workspace_id, time_of_generation_max, and requested_fields can change after creation, so use get_log_export to review the current job and start_log_export after the definition is ready. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to update
workspace_idNoWorkspace ID for the export
requested_fieldsNoFields to include in export: id, trace_id, created_at, request, response, is_success, ai_org, ai_model, req_units, res_units, total_units, request_url, cost, cost_currency, response_time, response_status_code, mode, config, prompt_slug, metadata
time_of_generation_maxNoMaximum time filter in date format (e.g., '2024-07-25' or ISO 8601)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral details beyond annotations: only specific fields can be changed after creation, the operation is suitable only before or between runs, and it returns 403 on non-Enterprise plans. This goes beyond the basic readOnly/destructive flags and helps the agent anticipate side effects and limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences pack all essential information: purpose, timing, mutable fields, alternative tools, and a critical error condition. No filler or redundant repetition of schema details. Front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity, complete parameter schema, and presence of an output schema, the description covers all necessary context: what can be changed, when to call it, what to do before/after, and a failure mode. An agent has enough to decide when and how to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explicitly identifying which parameters are mutable (workspace_id, time_of_generation_max, requested_fields) and implying export_id is the required target. This contextualizes parameter usage beyond the raw schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates an existing log export configuration, with a specific verb ('Update') and resource ('log export configuration'). It also distinguishes itself from sibling tools by specifying timing ('before or between export runs') and pointing to alternatives like get_log_export and start_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit usage guidance: which fields are mutable, when to use it (before or between export runs), and exactly when to use other tools ('use get_log_export to review the current job and start_log_export after the definition is ready'). It also notes Enterprise-gating and 403 behavior, giving clear context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integrationA
DestructiveIdempotent

Update an MCP integration's name, description, URL, auth, transport, headers, or runtime Secret Reference mappings. Only supplied fields change; URL, auth, header, and secret changes apply immediately and can break active clients, so inspect get_mcp_integration first. Use update_mcp_server when changing only a Portkey server instance's display metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to update
urlNoNew URL endpoint
nameNoNew display name
auth_typeNoNew authentication type. Current examples are 'none', 'headers' (custom headers), and 'oauth_auto' (OAuth); Portkey may accept additional values.
transportNoNew transport protocol
descriptionNoNew description
configurationsNoReplacement documented or forward-compatible configuration fields. configurations.custom_headers must be a string-to-string header map when present.
custom_headersNoNew custom headers for authentication. Sent via configurations.custom_headers
secret_mappingsNoReplacement runtime Secret Reference mappings; each target_field must be unique

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description meaningfully extends the annotations. destructiveHint=true is just a flag, but the description explains the concrete danger: 'URL, auth, header, and secret changes apply immediately and can break active clients.' It also discloses the partial-update semantics ('Only supplied fields change') and the recommended pre-flight inspection, none of which are in the annotations or schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: sentence one states purpose and scope, sentence two states the risk and pre-requisite, sentence three routes to the sibling alternative. The most important safety information (immediate breakage) is front-loaded ahead of the sibling routing. No filler or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutation tool with nested objects, the description covers the decision-relevant ground: what fields are affected, patch behavior, immediate-apply risk, a pre-inspection step, and sibling differentiation. An output schema exists so return values need not be described, and annotations already carry idempotence/open-world traits. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents each of the 9 parameters, setting a baseline of 3. The description adds one genuinely useful cross-cutting semantic: 'Only supplied fields change,' which clarifies that all parameters are optional patch operations, plus the risk profiling that groups URL/auth/header/secret changes as immediately effective. This exceeds the baseline without duplicating schema detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Update'), a specific resource ('an MCP integration'), and enumerates exactly which aspects change: name, description, URL, auth, transport, headers, or runtime Secret Reference mappings. It distinguishes itself from siblings like update_mcp_server, update_mcp_integration_capabilities, and update_mcp_integration_workspaces by delimiting its scope through the field list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use context (changing any of the listed core configuration fields) and explicitly names the alternative: 'Use update_mcp_server when changing only a Portkey server instance's display metadata.' It also supplies a prerequisite ('inspect get_mcp_integration first'), which tells the agent what to do before invoking this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integration_capabilitiesA
Idempotent

Bulk enable or disable capabilities (tools, prompts, resources) on an MCP integration. A reversible toggle, not a deletion: only the capabilities named in the array change state, the change hides or exposes them immediately for connected users, and re-running with enabled flipped restores them. Source the integration id from list_mcp_integrations and current capability names, types, and states from list_mcp_integration_capabilities. Returns a success confirmation message.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug
capabilitiesYesArray of capability updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations show readOnlyHint=false, destructiveHint=false, idempotentHint=true. The description adds meaningful context: reversible toggle, immediate exposure, and that only named capabilities are affected. No contradictions. It does not cover rate limits or auth, but annotations already indicate non-destructive and idempotent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, then key behavioral points. Every sentence is purposeful and concise. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (bulk update with multiple capability types), the description is self-sufficient. It explains purpose, usage, side effects, and prerequisite data sources. The output schema existence is not critical as it mentions a confirmation message.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear descriptions. The description adds value by explaining the effect of the array (state toggle), reversibility, and the immediate visibility. This enriches the agent's understanding beyond raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with 'Bulk enable or disable capabilities (tools, prompts, resources) on an MCP integration,' clearly specifying the verb (enable/disable), resource (capabilities on an MCP integration), and distinguishing it from sibling tools like update_mcp_integration and list_mcp_integration_capabilities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit context: reversible toggle, not deletion; only named capabilities change state; immediate for connected users; re-running with enabled flipped restores them. It also instructs to source the integration id from list_mcp_integrations and current states from list_mcp_integration_capabilities. No direct when-not-to-use, but implied through reversibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integration_workspacesA
Idempotent

Grant or revoke workspace access to an MCP integration in bulk. Reversible: only the workspaces listed change, access applies or is removed immediately for all users in those workspaces, and re-running with enabled flipped undoes a change. Source the integration id from list_mcp_integrations, workspace ids from list_workspaces, and the current access state from list_mcp_integration_workspaces. Returns a success confirmation message.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug
workspacesYesArray of workspace access updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explains idempotency ('re-running with enabled flipped undoes a change') and immediate application for all users, adding value beyond annotations (idempotentHint: true). It does not mention auth or rate limits but is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the purpose, and includes all essential information without filler. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema (though not shown), so return values are covered. The description covers purpose, parameters, behavior, and prerequisites. It could mention error handling but is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds context on sourcing IDs but does not significantly expand on the schema's descriptions of 'id' and 'workspaces'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Grant or revoke workspace access to an MCP integration in bulk,' specifying the verb and resource. It distinguishes from sibling tools like list_mcp_integration_workspaces by detailing how to source IDs and the bulk nature.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit context for when to use this tool (bulk granting/revoking) and gives sourcing guidance from related tools (list_mcp_integrations, list_workspaces, list_mcp_integration_workspaces). It lacks explicit when-not-to-use or alternatives but is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_serverA

Update an MCP server's name or description. Changes apply immediately, but URL and auth live on the parent integration, so use update_mcp_integration for those fields.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to update
nameNoNew display name
descriptionNoNew description

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description notes that changes apply immediately, adding behavioral context beyond annotations. No contradictions with annotations (readOnlyHint=false, destructiveHint=false). Adequate for a simple update tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second provides usage guidance. No redundant information, front-loaded with key action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, an output schema exists, and the description covers what fields are updatable, immediate effect, and sibling tool delegation. Complete for this context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for each parameter. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Update') and the resource ('MCP server') and specifies the fields affected (name or description). It distinguishes from the sibling tool 'update_mcp_integration' by noting that URL and auth are handled separately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent when to use this tool versus the sibling ('use update_mcp_integration for those fields'), providing clear guidance on tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_server_capabilitiesA

Enable or disable capabilities on an MCP server. Changes take effect immediately and override the integration-level settings for this server; use list_mcp_server_capabilities first to inspect the current surface.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
capabilitiesYesArray of capability updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=false and destructiveHint=false, so the description adds context about immediacy and overriding integration settings. This provides behavioral insight beyond annotations, but it doesn't detail potential side effects, authorization needs, or reversibility. Given annotations cover some aspects, a score of 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the action, and every sentence provides essential information. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with a complex schema and good annotations, the description covers key context: action, effect (immediate override), and prerequisite inspection step. It doesn't document output, but an output schema exists. It might be slightly lacking on preconditions (e.g., server existence) but overall is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the description doesn't need to elaborate much on parameters. It mentions 'enable or disable capabilities' which aligns with the 'enabled' field, but adds no new meaning beyond the schema. The description does not explain the structure of the capabilities array or enum values, but the schema already documents these.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool enables or disables capabilities on an MCP server, uses a specific verb ('update_mcp_server_capabilities' inherently suggests modification), and distinguishes from siblings like 'update_mcp_integration_capabilities' by noting it overrides integration-level settings. It also advises to use 'list_mcp_server_capabilities' first, further clarifying scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends using 'list_mcp_server_capabilities' first to inspect current capabilities, providing clear when-to-use guidance. It states changes take effect immediately and override integration settings, helping differentiate from other update tools. However, it does not explicitly mention when not to use this tool or list all alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_server_user_accessA

Grant or revoke individual user access to an MCP server. Changes take effect immediately and override the default access setting for the selected users; use list_mcp_server_user_access first if you need the current state.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
usersYesArray of user access updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a write operation with no destructive or idempotent guarantees. The description adds that changes 'take effect immediately' and 'override the default access setting', providing useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the core action, the second adds a crucial usage hint. No wasted words, front-loaded with purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the action, immediate effect, override behavior, and a recommended precursor call. With an output schema available (as noted in context signals), the description is sufficient for a simple mutation tool, though edge cases like non-existent users are not addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear parameter descriptions. The description does not add significant new meaning to the parameters beyond reiterating their purpose, so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('grant or revoke individual user access') and the resource ('MCP server'). It distinguishes itself from sibling tools like list_mcp_server_user_access and update_mcp_server by focusing on per-user access overrides.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance to use list_mcp_server_user_access first for current state, and explains that changes override default access. While it doesn't cover when not to use it, the alternative recommendation is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_organisation_defaultsA
Idempotent

Replace the organisation-wide default input and/or output guardrail lists inherited by workspaces. Only supplied directions change, but enforcement updates immediately across non-excluded workspaces; inspect get_organisation_defaults and the directional workspace exclusions first. Repeating the same lists is safe. Requires an organisation service API key with organisation_settings.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
input_guardrailsNoOrdered guardrail IDs or slugs to enforce on model input
output_guardrailsNoOrdered guardrail IDs or slugs to enforce on model output

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (idempotentHint, readOnlyHint=false), the description discloses immediate enforcement propagation, partial-update behavior, required API key scope, Enterprise gating, and the 403 error on non-Enterprise plans. This adds substantial operational context that annotations alone do not convey. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact—four sentences—with each sentence earning its place: function, behavior, safety, permissions, and error handling. It is front-loaded with the core purpose and avoids redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex mutation tool, the description covers all essential aspects: what it does, how it behaves (immediate propagation), prerequisites (inspect read tools), idempotency, authentication, and error condition. The existence of an output schema also covers return values. No significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics by stating "Only supplied directions change," clarifying that parameters are optional and act as partial updates. This is extra value beyond the schema descriptions, warranting a slightly higher score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb "Replace" and clearly identifies the resource: "organisation-wide default input and/or output guardrail lists inherited by workspaces." It distinguishes itself from sibling tools like update_input_guardrail_workspace_exclusions and update_output_guardrail_workspace_exclusions by focusing on the defaults rather than exclusions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description instructs the agent to "inspect get_organisation_defaults and the directional workspace exclusions first," providing clear prerequisite steps. It also notes that only supplied directions change and that repetition is safe, giving practical usage context. It stops short of explicitly naming alternatives or when-not-to-use scenarios, but the guidance is solid.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_output_guardrail_workspace_exclusionsA
DestructiveIdempotent

Set workspace exclusions from organisation-wide output guardrails. Each entry excludes or restores one workspace; override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces. Review the matching list tool first because enforcement changes immediately. Repeating the same states is safe. Requires an organisation service API key with organisation_exclusions.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspacesYesWorkspace exclusion states to apply
organisation_idYesOrganisation ID whose workspace exclusions should be updated
override_existingNoReplace existing exclusion states instead of merging changes

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations (which already mark destructiveHint=true and idempotentHint=true) by disclosing concrete behavioral traits: enforcement changes immediately, override_existing replaces prior states while default merge preserves unmentioned workspaces, requires an organisation service API key with organisation_exclusions.update scope, is Enterprise-gated, and returns 403 on non-Enterprise plans. It also clarifies that repeating the same states is safe. No contradiction with annotations is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place. It covers purpose, usage prerequisite, behavioral side effects, idempotency, authentication scope, enterprise gating, and expected error response—all in five concise sentences. The structure is front-loaded with the core action, followed by key behavioral nuance, then supporting operational details. No redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a potentially destructive mutation tool with an output schema present, the description is comprehensive. It covers the tool's side effects (immediate enforcement), merge/override semantics, idempotency, required auth scope, enterprise gating, and a specific error condition (403). The presence of an output schema means return-value details need not be in the description. The combination of schema coverage, annotations, and description provides a complete picture for an agent to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all parameters, which would normally set a baseline of 3. The description adds meaningful semantic detail beyond the schema: it explains that 'override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces', which clarifies the interaction between the override_existing boolean and the workspaces array. This goes beyond the schema's per-parameter descriptions, so a 4 is warranted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Set workspace exclusions from organisation-wide output guardrails', which is a specific verb+resource statement. The tool name includes 'output_guardrail_workspace_exclusions' and there is a sibling 'update_input_guardrail_workspace_exclusions', so the explicit 'output guardrails' wording clearly distinguishes it from the input-guardrail alternative. The purpose is unambiguous and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to 'Review the matching list tool first because enforcement changes immediately', providing a clear prerequisite and warning about side effects. It also notes that 'Repeating the same states is safe', which guides usage around idempotency. It does not explicitly state 'use this instead of the input guardrail tool', but the purpose clarity and context make the usage context well-defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_promptA

Update an existing prompt and create a new archived version. Only provided fields change, and publish_prompt is what makes the new version active. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the prompt
modelNoNew model identifier
toolsNoNew tool definitions
stringNoLegacy prompt template string. Use plain text for single-message prompts, or a JSON-encoded messages array string for multi-message chat prompts.
dry_runNoWhen true, validate without updating
messagesNoStructured chat template alias for updates. Serialized to the legacy string format before the prompt is updated.
functionsNoNew function definitions
prompt_idYesPrompt ID or slug to update
parametersNoNew default values for template variables
tool_choiceNoNew tool choice strategy
virtual_keyNoNew virtual key slug
collection_idNoMove to a different collection
is_raw_templateNoEnable or disable raw structural-template rendering
template_metadataNoNew metadata
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description reveals important behavior: updates create a new archived version rather than mutating in place, only provided fields change, and the new version is not active until published. This is exactly the kind of behavioral context agents need.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The most important behaviors—archiving, partial updates, and activation—are front-loaded, and the parameter guidance is concise and directly actionable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input and output schemas, the description covers the critical operational details: versioning, activation, partial updates, and the messages-versus-string distinction. Nothing essential for selecting or invoking the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying the preferred parameter for multi-message prompts and explaining the partial-update semantics, which supplements the schema's field-level documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation: updating an existing prompt and creating a new archived version. It also distinguishes behavior from the sibling publish_prompt by noting that publishing is what activates the version, making the tool's role unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the key workflow: update_prompt creates an archived version, but publish_prompt is required to make it active. It also gives concrete guidance for multi-message prompts, telling callers to prefer messages over a JSON-encoded string.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_labelA

Update a prompt label's name, description, or color only, unlike update_prompt_version which changes which label a version carries. This takes effect immediately for all versions already tagged with the label, but does not reassign labels or touch history; use list_prompt_labels to find the label_id first.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the label
label_idYesLabel ID to update
color_codeNoNew hex color code (e.g., '#FF5733')
descriptionNoNew description

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are basic (non-readonly, non-destructive). The description adds that changes take effect immediately for all tagged versions and do not reassign labels or touch history, offering valuable behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, followed by behavioral notes and prerequisite. No redundant information; every phrase adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 4 parameters (1 required) and an output schema, the description covers immediate effects, side effects, and prerequisite steps. No gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions. The description reinforces that only name, description, or color can be updated, adding operational context. While not adding new parameter details, it clarifies mutation scope.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a prompt label's name, description, or color only', specifying the verb and resource. It distinguishes from sibling update_prompt_version, which changes label assignments, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly contrasts with update_prompt_version and advises using list_prompt_labels to find the label_id first, providing clear when-to-use and prerequisite instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_partialA

Create a new version of a partial by updating its content or metadata. Only provided fields change, and the new version stays inactive until publish_partial makes it current.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the partial
statusNoNew status for the partial
stringNoNew content for the partial
descriptionNoDescription for this version
prompt_partial_idYesPrompt partial ID or slug to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are present and consistent (readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true). The description adds that only provided fields change and that the new version is inactive until published, which are important behaviors not covered by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, each adding value. The purpose is front-loaded, and there is no redundant or unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description does not need to explain return values. It covers the key behavioral context (partial update, new inactive version) and is complete for this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all parameters. The description adds value by stating 'Only provided fields change', clarifying partial update semantics, which goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (create) and resource (a new version of a partial), and implies the resource is a partial, distinguishing it from sibling tools like publish_partial that make a version current.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that only provided fields change and the new version stays inactive until publish_partial is used, providing clear context for when to use this tool and hinting at the alternative publish tool. However, it does not explicitly say 'use publish_partial to make it current'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_versionA

Update a specific prompt version's label assignment. This only assigns or removes a label, and null clears the label after you look up ids with list_prompt_labels.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to assign to this version, or null to remove the label
prompt_idYesPrompt ID or slug
version_idYesVersion UUID to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate mutation (readOnlyHint=false) but no destruction. The description adds that the tool only affects label assignment and clarifies null behavior ('null clears the label'). This adds meaningful context beyond the basic annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. The first states the core purpose, the second elaborates on null behavior and prerequisite workflow. No unnecessary words; all content earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with three required parameters and an output schema, the description covers the essential: what it does, how null works, and where to get label IDs. No gaps for this complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing baseline 3. The description adds value by explaining the label_id parameter's role in assigning/removing labels and guiding the agent to use 'list_prompt_labels' for ID lookup. This goes beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it updates a prompt version's label assignment, with specific verbs ('assigns', 'removes', 'clears'). It distinguishes itself from siblings like 'update_prompt' (which updates the prompt itself) and 'list_prompt_labels' (for looking up label IDs).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: it only assigns or removes a label, and recommends looking up IDs with 'list_prompt_labels'. It does not explicitly state when not to use it or list alternative tools, but the scope is well-defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_providerA

Update a provider's metadata, limits, or expiration. reset_usage clears accumulated usage counters immediately, so use it only when you intend to reset quota tracking. Returns the updated provider id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the provider
noteNoNew note or description for the provider
slugYesThe slug of the provider to update
expires_atNoNew expiration date in ISO 8601 format
reset_usageNoSet to true to reset accumulated usage metrics
credit_limitNoNew credit limit for usage
workspace_idNoWorkspace ID - required when using organization admin keys
periodic_resetNoPeriod for resetting usage limits: 'monthly' or 'weekly'. Defaults to 'monthly'.
alert_thresholdNoNew alert threshold percentage (0-100)
rate_limit_unitNoRate limit unit: 'rpm' (requests per minute), 'rph' (requests per hour), or 'rpd' (requests per day)
rate_limit_valueNoNew rate limit value
usage_limit_typeNoType of usage limit: 'cost' (monetary) or 'tokens' (token count). Defaults to 'cost'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that reset_usage clears counters immediately, adding behavioral context beyond the annotations (which only indicate non-destructive, non-idempotent). It also mentions the return value. This is good but could be more detailed about other side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, followed by a critical usage note and return value. Every word is necessary and no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 12 parameters and an existing output schema, the description covers the main categories and return value. It lacks details on authorization or behavior for omitted fields, but the output schema and schema descriptions compensate adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the baseline is 3. The description only generally mentions 'metadata, limits, or expiration' and specifically calls out reset_usage, but does not add significant meaning beyond what the schema's parameter descriptions already provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates a provider's metadata, limits, or expiration, with a specific verb and resource. While it differentiates from sibling update tools by targeting 'provider', it does not explicitly distinguish from other update tools, but the resource is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use the reset_usage parameter ('only when you intend to reset quota tracking'), but does not offer broader when-to-use or when-not-to-use guidance relative to other tools, though the context is sufficient given the tool's specificity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_rate_limitA

Update a rate-limit policy's name, unit, value, or non-empty conditions by id. The public contract does not allow changing its type, target, or grouping after creation.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesRate-limit policy UUID
nameNoReplacement display name
unitNoReplacement reset interval
valueNoReplacement maximum
conditionsNoReplacement conditions

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish that this is not read-only and not destructive. The description adds meaningful behavioral context beyond the annotations: the public contract forbids changing type, target, or grouping after creation. It does not describe partial-update semantics or response behavior, but the annotations and output schema lower the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences with no filler: the first states the action and allowed fields, the second states the key immutability constraint. The most important information is front-loaded and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations, full schema coverage, and presence of an output schema, the description covers the core updateable fields and the important contract restriction. A minor gap is that it does not explicitly state whether omitted fields are preserved, but the overall definition is sufficiently complete for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents id, name, unit, value, and conditions in detail. The description's mention of 'non-empty conditions' adds little beyond the schema's minItems: 1 constraint. No additional parameter-level meaning is provided that the schema does not already convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Update') and resource ('rate-limit policy'), and lists the exact mutable fields: name, unit, value, and non-empty conditions. It also distinguishes itself from creation by stating that type, target, and grouping cannot be changed after creation, which separates it clearly from sibling tools like create_rate_limit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes clear this tool updates an existing policy by id and explicitly warns that type, target, or grouping cannot be changed through this tool. However, it does not explicitly name an alternative like create_rate_limit for setting those immutable fields, so the routing guidance is strong but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_secret_referenceA
DestructiveIdempotent

Update at least one selected field on a Secret Reference. auth_config is merged into the existing manager-specific configuration. allowed_workspaces replaces all workspace mappings and sets allow_all_workspaces=false; allow_all_workspaces=true removes workspace-specific mappings. Credential values are sent to Portkey and exposed to the MCP transcript.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug
nameNoNew human-readable name
tagsNoReplacement string key-value tags, or null to clear them
secret_keyNoNew key within a structured secret, or null to use the whole value
auth_configNoProvide at least one changed authentication field to merge into the existing configuration; choose one AWS, Azure, or HashiCorp field family and do not mix families
descriptionNoNew description, or null to clear it
secret_pathNoNew path of the secret in the external secret manager
allowed_workspacesNoReplacement list of workspace UUIDs or slugs allowed to use this reference; supplying it sets allow_all_workspaces=false
allow_all_workspacesNoWhether every workspace may use this reference; cannot be true with allowed_workspaces

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds value beyond annotations by detailing that auth_config is merged, allowed_workspaces replaces mappings, and credential values are exposed to transcript. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three efficient sentences: purpose, key behavioral details, security note. Front-loaded with action, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers critical merge/replace behaviors and credential exposure. With output schema present, return value explanation is unnecessary. Minor gap: no prerequisites or error conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description adds meaning by specifying merge behavior for auth_config and replacement logic for allowed_workspaces, enhancing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Update at least one selected field on a Secret Reference', specifying verb and resource. Differentiates from sibling create/delete/get/list by emphasizing partial update.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explains merging vs replacing behavior for auth_config and allowed_workspaces, providing clear context for use. Lacks explicit when-not to use, but siblings are distinct.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_usage_limitA

Update a cumulative usage-limit policy's name, credit limit, alert threshold, reset schedule, or one grouped value's usage. Conditions and grouping aren't mutable in the public contract.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesUsage-limit policy UUID
nameNoReplacement display name
credit_limitNoReplacement cumulative maximum
periodic_resetNoReplacement reset schedule
alert_thresholdNoReplacement alert threshold
reset_usage_for_valueNoGrouped value whose usage should reset

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a non-read-only, non-idempotent operation, so the description's task is lighter. It adds useful behavioral context by clarifying which fields are updatable and explicitly stating that conditions/grouping cannot be modified, which is not evident from the schema alone. The description does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler. The primary action and resource are front-loaded, followed by the list of mutable fields, and then the key limitation. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, full parameter description coverage, and annotations covering the safety profile, the description provides sufficient context for a correct call. It clearly communicates the mutable surface and the one hard restriction. A minor gap is not distinguishing when to use update_usage_limit versus the closely related reset_usage_limit_entity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% coverage and each parameter already has a concise description. The tool description adds value by grouping the parameters semantically: name, credit limit, alert threshold, reset schedule are 'replacements,' while reset_usage_for_value is framed as resetting 'one grouped value's usage.' It also explains why there are no condition/grouping parameters: they are not mutable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Update') and identifies a clear resource ('cumulative usage-limit policy'), enumerating the exact mutable fields: name, credit limit, alert threshold, reset schedule, or one grouped value's usage. It further distinguishes itself by explicitly stating that conditions and grouping are not mutable, which prevents confusion with related usage-limit tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: this tool updates an existing usage-limit policy's allowed attributes. It also provides an explicit exclusion ('Conditions and grouping aren't mutable in the public contract'), signaling what cannot be changed. However, it does not name a specific alternative such as reset_usage_limit_entity for the grouped-value reset case, leaving some routing decision to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_userA

Update a user's first name, last name, or organization role by id. Email and workspace roles are not editable here; use update_workspace_member for workspace membership changes.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoNew organization-level role
user_idYesThe user ID to update
last_nameNoNew last name
first_nameNoNew first name

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate it's a mutation (readOnlyHint: false). The description confirms it modifies user fields and adds the important constraint that email and workspace roles are off-limits. It does not describe side effects, error conditions, or permissions, but given annotations already signal mutability, the description adds sufficient behavioral context for a simple update.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Delivers maximum information in two sentences: first sentence states the action and scope, second sentence specifies exclusions and suggests an alternative. No wasted words; front-loaded with essential details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with an output schema, the description covers what is updatable, what is not, and directs to an alternative. It lacks mention of prerequisites (e.g., user existence, authentication) or error behavior. However, given the tool's low complexity and the presence of openWorldHint, it is fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for each parameter. The overall description adds value by clarifying which parameters are and aren't editable, providing context beyond individual parameter descriptions. This compensates for the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Update', resource 'user', and lists specific fields (first name, last name, organization role). Explicitly distinguishes from update_workspace_member by stating what is not editable. This leaves no ambiguity about the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use and when-not-to-use guidance: 'Email and workspace roles are not editable here; use update_workspace_member for workspace membership changes.' Directly contrasts with a sibling tool, helping an agent select the correct tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_virtual_keyA

Update a virtual key's name, secret, note, or limits. Rotating the key takes effect immediately, and limit changes apply to downstream prompts and configs using this slug. Returns success when Portkey accepts the update.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoNew provider API key value
nameNoNew display name for the key
noteNoNew note or description
slugYesThe slug of the virtual key to update
credit_limitNoNew credit limit for usage
rate_limit_rpmNoNew rate limit in requests per minute
alert_thresholdNoNew alert threshold percentage (0-100)
secret_mappingsNoReplacement Secret Reference mappings
deployment_configurationsNoReplacement Azure deployment configurations

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark readOnlyHint=false, so the description does not need to state it is a write operation. The description adds valuable behavioral details: rotating the key takes effect immediately, limit changes apply to downstream prompts and configs, and success indicates Portkey accepted the update. It also implies that the operation is not idempotent by mentioning immediate effects. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, with the core action first, then immediate-effect behavior, then the success criterion. Every sentence contributes useful information: what can be updated, how changes propagate, and what the return indicates. No fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a rich input schema covering all 9 parameters and an output schema present, the description need not repeat parameter details. It provides the critical contextual facts about timing (immediate rotation) and propagation (downstream prompts/configs) that are not inferable from the schema. Slight gap: it does not mention idempotency or partial-update behavior, but the openWorldHint and schema's optional fields partially cover that. Overall complete enough for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema. The description adds a high-level summary of the key groups (name, secret, note, or limits) but does not add meaning beyond the schema. Baseline 3 is appropriate given the schema carries the detailed semantics, including nested objects like secret_mappings and deployment_configurations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Update a virtual key's name, secret, note, or limits.' It clearly indicates the tool modifies an existing virtual key. It does not explicitly differentiate itself from sibling tools like create_virtual_key or delete_virtual_key, but the update semantics and the mention of slug-based targeting provide adequate clarity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives useful context for when to use this tool: updating an existing virtual key identified by slug, with immediate effect for rotation and propagation of limit changes. It does not explicitly state 'use create_virtual_key for new keys' or 'use delete_virtual_key to remove,' but the update-focused language clearly implies the intended scenario. Slight gap: no explicit exclusion of alternatives, but context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_workspaceA

Update a workspace's name, slug, description, default flag, or metadata by id, unlike update_workspace_member which changes role assignments within a workspace. Only provided fields change and updates take effect immediately; changing the slug can break URLs, API key references, and other external links, so confirm no dependencies first.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the workspace
slugNoNew slug for the workspace
metadataNoNew metadata key-value pairs
is_defaultNoSet as default workspace (1 = yes, 0 = no)
descriptionNoNew description
rate_limitsNoReplacement workspace rate-limit settings
usage_limitsNoReplacement workspace usage-limit settings
workspace_idYesThe workspace ID to update
input_guardrailsNoGuardrail IDs applied to workspace model inputs
output_guardrailsNoGuardrail IDs applied to workspace model outputs
user_api_key_configNoDefault config slug for workspace-user API keys, or null to clear

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey readOnly=false, idempotent=false, and destructive=false. The description adds non-obvious behavior: partial-update semantics, immediate effect, and the side-effect risk of changing the slug. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with front-loaded purpose and immediately useful caveats. Every clause adds information; there is no filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter update with nested objects and an output schema, the description is compact but sufficient: it identifies the target, partial-update behavior, immediacy, and the key side effect. The fully covered schema handles the remaining parameter details, and the output schema makes return-value prose unnecessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all 11 parameters. The description reinforces that fields are optional and only provided ones are changed, but it does not add per-parameter meaning or format details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete operation ('Update a workspace's name, slug, description, default flag, or metadata by id') with a specific resource and target identifier. It also distinguishes itself from the closest sibling, update_workspace_member, preventing role-assignment confusion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names update_workspace_member as the alternative for role assignments, giving an explicit when-not condition. It also provides practical guardrails: only provided fields change, updates take effect immediately, and slug changes can break external references.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_workspace_memberA

Update a workspace member's role by workspace_id and user_id. Only the role changes here; use list_workspace_members or get_workspace_member to confirm the current assignment first.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesNew role in the workspace
user_idYesThe user ID to update (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=false, destructiveHint=false, and openWorldHint=true. The description adds the key behavioral constraint that only the role field changes and recommends a confirmation step before updating, which is useful context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two focused sentences: the first states the action and required keys, the second states the scope and a pre-check workflow. There is no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the schema covers all parameters, annotations cover the safety profile, and an output schema exists, the description is sufficient for selection and invocation. It includes the purpose, the specific keys, the role enum context, and a confirmation step, though it could briefly hint at alternative member-management tools for non-role updates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all three parameters thoroughly, including the role enum and user_id UUID constraints. The description reinforces that role is the only mutable field, adding semantic clarity beyond the schema fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Update') and names the exact resource ('workspace member's role') plus the identifying keys ('by workspace_id and user_id'). It distinguishes itself from sibling member tools by explicitly limiting the scope to role changes only.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states the tool is for role updates only and tells the agent to use list_workspace_members or get_workspace_member to confirm the current assignment first. It does not explicitly name alternative mutation tools (e.g., remove_workspace_member), but the scope exclusion ('Only the role changes here') provides clear when-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_completion_metadataA
Read-onlyIdempotent

Preflight billing metadata before run_prompt_completion. Validates required fields and values without making changes, so you can catch attribution errors before paying for the call.

ParametersJSON Schema
NameRequiredDescriptionDefault
appNoApp identifier (REQUIRED). Use your deployed app name, for example 'hourlink' or 'support-console'.
envNoEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
featureNoFeature name for tracking
client_idNoClient ID for billing attribution
project_idNoProject ID for granular billing

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true and idempotentHint=true, and the description reinforces 'without making changes'. It adds specific context about billing and attribution error checking, which goes beyond the annotations. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the key action, and every word adds value. No unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The explanation covers the purpose, usage context, and behavioral safety. Since the tool has an output schema, the description does not need to explain return values. It is sufficiently complete for a validation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter has a detailed description. The tool description does not add extra meaning beyond the schema; it only mentions 'validates required fields and values' without specifying which fields. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as a preflight validator for billing metadata before run_prompt_completion. It uses a specific verb ('validate') and resource ('billing metadata'), and distinguishes itself from the sibling 'run_prompt_completion' by being a read-only check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool ('before run_prompt_completion') and why ('catch attribution errors before paying for the call'). It implies not to use it when not about to call run_prompt_completion, but does not explicitly name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 76 tool updatesv0.11.5
    • Addedarchive_deployment
    • Changedcreate_api_key6 fields changed
      • addedInput schema / properties / default_allow_config_override
        Added value: +{
        +  "description": "Allow callers to override the default config",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / expires_at / format
        Added value: +"date-time"
      • addedInput schema / properties / expires_at / pattern
        Added value: +"^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$"
      • addedInput schema / properties / organisation_id
        Added value: +{
        +  "description": "Organisation UUID",
        +  "type": "string"
        +}
      • addedInput schema / properties / rate_limits
        Added value: +{
        +  "anyOf": [
        +    {
        +      "items": {
        +        "properties": {
        +          "type": {
        +            "description": "Whether the limit counts requests or tokens",
        +            "enum": [
        +              "requests",
        +              "tokens"
        +            ],
        +            "type": "string"
        +          },
        +          "unit": {
        +            "description": "Rate window unit for the request or token count",
        +            "enum": [
        +              "rpd",
        +              "rph",
        +              "rpm",
        +              "rps",
        +              "rpw"
        +            ],
        +            "type": "string"
        +          },
        +          "value": {
        +            "description": "Maximum count in the selected rate window",
        +            "maximum": 9007199254740991,
        +            "minimum": 0,
        +            "type": "integer"
        +          }
        +        },
        +        "required": [
        +          "type",
        +          "unit",
        +          "value"
        +        ],
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Request or token rate limits, or null to clear them"
        +}
      • addedInput schema / properties / rotation_policy
        Added value: +{
        +  "anyOf": [
        +    {
        +      "description": "Automatic API-key rotation policy, or null to remove it",
        +      "properties": {
        +        "key_transition_period_ms": {
        +          "description": "Overlap in milliseconds before the previous key expires",
        +          "maximum": 9007199254740991,
        +          "minimum": 1800000,
        +          "type": "integer"
        +        },
        +        "next_rotation_at": {
        +          "anyOf": [
        +            {
        +              "format": "date-time",
        +              "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +              "type": "string"
        +            },
        +            {
        +              "type": "null"
        +            }
        +          ],
        +          "description": "Explicit next rotation timestamp in ISO 8601 format"
        +        },
        +        "rotation_period": {
        +          "anyOf": [
        +            {
        +              "enum": [
        +                "weekly",
        +                "monthly"
        +              ],
        +              "type": "string"
        +            },
        +            {
        +              "type": "null"
        +            }
        +          ],
        +          "description": "Built-in weekly or monthly automatic rotation cadence"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Automatic API-key rotation policy, or null to disable it"
        +}
    • Changedcreate_guardrail1 field changed
      • changedInput schema / properties / checks / items / properties / parameters / description
        Previous value: -"Check-specific configuration parameters"New value: +"Check-specific configuration. Supported examples include requestParametersCheck and parameters.forwardHeaders; forwarded headers can expose sensitive values, so allow only the minimum required names."
    • Changedcreate_mcp_integration4 fields changed
      • changedInput schema / properties / auth_type / description
        Previous value: -"Authentication type: 'none', 'headers' (custom headers), or 'oauth_auto' (OAuth)"New value: +"Authentication type. Current examples are 'none', 'headers' (custom headers), and 'oauth_auto' (OAuth); Portkey may accept additional values."
      • removedInput schema / properties / auth_type / enum
        Removed value: -[
        -  "oauth_auto",
        -  "headers",
        -  "none"
        -]
      • addedInput schema / properties / auth_type / minLength
        Added value: +1
      • addedInput schema / properties / configurations
        Added value: +{
        +  "additionalProperties": {},
        +  "description": "Additional documented or forward-compatible configuration fields. For headers auth, configurations.custom_headers is a string-to-string header map.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedcreate_prompt1 field changed
      • addedInput schema / properties / is_raw_template
        Added value: +{
        +  "description": "Render the stored template as a raw structural template",
        +  "type": "boolean"
        +}
    • Changedcreate_rate_limit25 fields changed
      • changedInput schema / properties / conditions / description
        Previous value: -"Array of conditions that determine which requests this rate limit applies to"New value: +"Policy match conditions"
      • addedInput schema / properties / conditions / items / properties / excludes
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "items": {
        +        "type": "string"
        +      },
        +      "minItems": 1,
        +      "type": "array"
        +    }
        +  ],
        +  "description": "Values to exclude"
        +}
      • removedInput schema / properties / conditions / items / properties / field
        Removed value: -{
        -  "description": "The field to match on (e.g., 'virtual_key', 'api_key', 'user_id', 'metadata.key')",
        -  "type": "string"
        -}
      • addedInput schema / properties / conditions / items / properties / key
        Added value: +{
        +  "description": "Match key such as api_key, virtual_key, provider, config, prompt, model, endpoint_type, mcp_server, mcp_tool, workspace_id, or metadata.*",
        +  "minLength": 1,
        +  "type": "string"
        +}
      • removedInput schema / properties / conditions / items / properties / operator
        Removed value: -{
        -  "description": "The comparison operator (e.g., 'is', 'contains', 'is_not')",
        -  "type": "string"
        -}
      • addedInput schema / properties / conditions / items / properties / value / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "items": {
        +      "type": "string"
        +    },
        +    "minItems": 1,
        +    "type": "array"
        +  }
        +]
      • changedInput schema / properties / conditions / items / properties / value / description
        Previous value: -"The value to match against"New value: +"One value or several OR-matched values; use * for wildcard matching"
      • removedInput schema / properties / conditions / items / properties / value / type
        Removed value: -"string"
      • changedInput schema / properties / conditions / items / required
        Previous value: -[
        -  "field",
        -  "operator",
        -  "value"
        -]New value: +[
        +  "key",
        +  "value"
        +]
      • addedInput schema / properties / conditions / minItems
        Added value: +1
      • changedInput schema / properties / group_by / description
        Previous value: -"Array of fields to group the rate limit by (e.g., ['virtual_key'], ['api_key', 'user_id'])"New value: +"Counter grouping dimensions"
      • addedInput schema / properties / group_by / items / properties
        Added value: +{
        +  "key": {
        +    "description": "Counter grouping key such as api_key, workspace_id, virtual_key, provider, config, prompt, model, mcp_server, mcp_tool, or metadata.*",
        +    "minLength": 1,
        +    "type": "string"
        +  }
        +}
      • addedInput schema / properties / group_by / items / required
        Added value: +[
        +  "key"
        +]
      • changedInput schema / properties / group_by / items / type
        Previous value: -"string"New value: +"object"
      • addedInput schema / properties / group_by / minItems
        Added value: +1
      • changedInput schema / properties / name / description
        Previous value: -"Optional name for the rate limit"New value: +"Optional policy display name"
      • changedInput schema / properties / organisation_id / description
        Previous value: -"Organisation ID to scope the limit to"New value: +"Organisation UUID scope"
      • addedInput schema / properties / target
        Added value: +{
        +  "description": "LLM or MCP-tool enforcement target",
        +  "enum": [
        +    "llm",
        +    "mcp_tools"
        +  ],
        +  "type": "string"
        +}
      • changedInput schema / properties / type / description
        Previous value: -"What to rate limit: 'requests' or 'tokens'"New value: +"Counter type"
      • changedInput schema / properties / unit / description
        Previous value: -"Time unit: 'rpm' (per minute), 'rph' (per hour), or 'rpd' (per day)"New value: +"Counter reset interval"
      • changedInput schema / properties / unit / enum
        Previous value: -[
        -  "rpm",
        -  "rph",
        -  "rpd"
        -]New value: +[
        +  "rpm",
        +  "rph",
        +  "rpd",
        +  "rpw"
        +]
      • changedInput schema / properties / value / description
        Previous value: -"The maximum allowed value per unit (e.g., 100 rpm)"New value: +"Maximum requests or tokens"
      • removedInput schema / properties / value / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / value / minimum
        Added value: +0
      • changedInput schema / properties / workspace_id / description
        Previous value: -"Workspace ID to scope the limit to"New value: +"Workspace UUID scope"
    • Changedcreate_usage_limit29 fields changed
      • addedInput schema / properties / alert_threshold / anyOf
        Added value: +[
        +  {
        +    "minimum": 0,
        +    "type": "number"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • changedInput schema / properties / alert_threshold / description
        Previous value: -"Percentage threshold (0-100) at which to send an alert"New value: +"Optional alert threshold"
      • removedInput schema / properties / alert_threshold / type
        Removed value: -"number"
      • changedInput schema / properties / conditions / description
        Previous value: -"Array of conditions that determine which requests this usage limit applies to"New value: +"Policy match conditions"
      • addedInput schema / properties / conditions / items / properties / excludes
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "items": {
        +        "type": "string"
        +      },
        +      "minItems": 1,
        +      "type": "array"
        +    }
        +  ],
        +  "description": "Values to exclude"
        +}
      • removedInput schema / properties / conditions / items / properties / field
        Removed value: -{
        -  "description": "The field to match on (e.g., 'virtual_key', 'api_key', 'user_id', 'metadata.key')",
        -  "type": "string"
        -}
      • addedInput schema / properties / conditions / items / properties / key
        Added value: +{
        +  "description": "Match key such as api_key, virtual_key, provider, config, prompt, model, endpoint_type, mcp_server, mcp_tool, workspace_id, or metadata.*",
        +  "minLength": 1,
        +  "type": "string"
        +}
      • removedInput schema / properties / conditions / items / properties / operator
        Removed value: -{
        -  "description": "The comparison operator (e.g., 'is', 'contains', 'is_not')",
        -  "type": "string"
        -}
      • addedInput schema / properties / conditions / items / properties / value / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "items": {
        +      "type": "string"
        +    },
        +    "minItems": 1,
        +    "type": "array"
        +  }
        +]
      • changedInput schema / properties / conditions / items / properties / value / description
        Previous value: -"The value to match against"New value: +"One value or several OR-matched values; use * for wildcard matching"
      • removedInput schema / properties / conditions / items / properties / value / type
        Removed value: -"string"
      • changedInput schema / properties / conditions / items / required
        Previous value: -[
        -  "field",
        -  "operator",
        -  "value"
        -]New value: +[
        +  "key",
        +  "value"
        +]
      • addedInput schema / properties / conditions / minItems
        Added value: +1
      • changedInput schema / properties / credit_limit / description
        Previous value: -"The maximum allowed usage (cost in dollars or token count)"New value: +"Maximum cumulative usage"
      • removedInput schema / properties / credit_limit / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / credit_limit / minimum
        Added value: +0
      • changedInput schema / properties / group_by / description
        Previous value: -"Array of fields to group the usage limit by (e.g., ['virtual_key'], ['api_key', 'user_id'])"New value: +"Usage grouping dimensions"
      • addedInput schema / properties / group_by / items / properties
        Added value: +{
        +  "key": {
        +    "description": "Counter grouping key such as api_key, workspace_id, virtual_key, provider, config, prompt, model, mcp_server, mcp_tool, or metadata.*",
        +    "minLength": 1,
        +    "type": "string"
        +  }
        +}
      • addedInput schema / properties / group_by / items / required
        Added value: +[
        +  "key"
        +]
      • changedInput schema / properties / group_by / items / type
        Previous value: -"string"New value: +"object"
      • addedInput schema / properties / group_by / minItems
        Added value: +1
      • changedInput schema / properties / name / description
        Previous value: -"Optional name for the usage limit"New value: +"Optional policy display name"
      • changedInput schema / properties / organisation_id / description
        Previous value: -"Organisation ID to scope the limit to"New value: +"Organisation UUID scope"
      • addedInput schema / properties / periodic_reset / anyOf
        Added value: +[
        +  {
        +    "enum": [
        +      "monthly",
        +      "weekly"
        +    ],
        +    "type": "string"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • changedInput schema / properties / periodic_reset / description
        Previous value: -"Automatically reset usage counters on this schedule"New value: +"Optional reset schedule"
      • removedInput schema / properties / periodic_reset / enum
        Removed value: -[
        -  "monthly",
        -  "weekly"
        -]
      • removedInput schema / properties / periodic_reset / type
        Removed value: -"string"
      • changedInput schema / properties / type / description
        Previous value: -"What to limit: 'cost' (in dollars) or 'tokens'"New value: +"Budget type"
      • changedInput schema / properties / workspace_id / description
        Previous value: -"Workspace ID to scope the limit to"New value: +"Workspace UUID scope"
    • Changedcreate_virtual_key5 fields changed
      • addedInput schema / properties / deployment_configurations
        Added value: +{
        +  "description": "Azure deployment configurations with API versions and aliases",
        +  "items": {
        +    "properties": {
        +      "alias": {
        +        "description": "Optional model alias for the deployment",
        +        "type": "string"
        +      },
        +      "api_version": {
        +        "description": "Azure OpenAI API version",
        +        "minLength": 1,
        +        "type": "string"
        +      },
        +      "deployment_name": {
        +        "description": "Azure deployment name",
        +        "minLength": 1,
        +        "type": "string"
        +      },
        +      "is_default": {
        +        "description": "Whether this is the default Azure deployment",
        +        "type": "boolean"
        +      }
        +    },
        +    "required": [
        +      "api_version",
        +      "deployment_name"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / expires_at
        Added value: +{
        +  "description": "Expiration in ISO 8601 format",
        +  "format": "date-time",
        +  "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +  "type": "string"
        +}
      • changedInput schema / properties / key / description
        Previous value: -"The actual provider API key to store"New value: +"Provider API key; omit when secret_mappings supplies key"
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Secret Reference mappings for key or model_config fields",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key selected from a structured external secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference ID that owns the external secret",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Virtual Key field populated from the Secret Reference",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "json",
        +              "string"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Whether Portkey reads the mapped value as JSON or a string"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • changedInput schema / required
        Previous value: -[
        -  "name",
        -  "provider",
        -  "key"
        -]New value: +[
        +  "name",
        +  "provider"
        +]
    • Changedcreate_workspace3 fields changed
      • addedInput schema / properties / rate_limits
        Added value: +{
        +  "description": "Initial workspace rate-limit settings",
        +  "items": {
        +    "properties": {
        +      "type": {
        +        "description": "Whether the limit counts requests or tokens",
        +        "enum": [
        +          "requests",
        +          "tokens"
        +        ],
        +        "type": "string"
        +      },
        +      "unit": {
        +        "description": "Rate window: requests per day, hour, or minute",
        +        "enum": [
        +          "rpd",
        +          "rph",
        +          "rpm"
        +        ],
        +        "type": "string"
        +      },
        +      "value": {
        +        "description": "Maximum count in the selected rate window",
        +        "maximum": 9007199254740991,
        +        "minimum": 0,
        +        "type": "integer"
        +      }
        +    },
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / usage_limits
        Added value: +{
        +  "description": "Initial workspace usage-limit settings",
        +  "items": {
        +    "properties": {
        +      "alert_threshold": {
        +        "description": "Percentage of the limit that triggers an alert",
        +        "exclusiveMinimum": 0,
        +        "type": "number"
        +      },
        +      "credit_limit": {
        +        "description": "Maximum cost or token usage for the workspace",
        +        "exclusiveMinimum": 0,
        +        "type": "number"
        +      },
        +      "next_usage_reset_at": {
        +        "anyOf": [
        +          {
        +            "format": "date-time",
        +            "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Next reset timestamp in ISO 8601 format"
        +      },
        +      "periodic_reset": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "monthly",
        +              "weekly"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Built-in weekly or monthly reset cadence"
        +      },
        +      "periodic_reset_days": {
        +        "anyOf": [
        +          {
        +            "maximum": 365,
        +            "minimum": 1,
        +            "type": "integer"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Custom reset cadence in days, from 1 through 365"
        +      },
        +      "type": {
        +        "description": "Whether the limit measures cost or tokens",
        +        "enum": [
        +          "cost",
        +          "tokens"
        +        ],
        +        "type": "string"
        +      }
        +    },
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / users
        Added value: +{
        +  "description": "Existing user IDs to add",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
    • Changeddelete_rate_limit2 fields changed
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the rate limit"New value: +"Rate-limit policy UUID to delete"
      • addedInput schema / properties / id / minLength
        Added value: +1
    • Changeddelete_usage_limit2 fields changed
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the usage limit"New value: +"Usage-limit policy UUID to archive"
      • addedInput schema / properties / id / minLength
        Added value: +1
    • Changedget_analytics_group_metadata19 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_analytics_group_models19 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Addedget_analytics_group_providers
    • Changedget_analytics_group_users19 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_cache_hit_latency16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_cache_hit_rate16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Addedget_cache_summary
    • Changedget_cost_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Addedget_deployment
    • Changedget_error_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_error_rate_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_error_stacks_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_error_status_codes_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_feedback_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_feedback_models_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_feedback_scores_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_feedback_weighted_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_latency_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_rate_limit3 fields changed
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the rate limit"New value: +"Rate-limit policy UUID"
      • addedInput schema / properties / id / minLength
        Added value: +1
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Include an archived policy by status",
        +  "enum": [
        +    "active",
        +    "archived"
        +  ],
        +  "type": "string"
        +}
    • Changedget_request_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_rescued_requests_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_token_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_usage_limit4 fields changed
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the usage limit"New value: +"Usage-limit policy UUID"
      • addedInput schema / properties / id / minLength
        Added value: +1
      • addedInput schema / properties / include_usage
        Added value: +{
        +  "description": "Include per-value usage counters",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Include an archived policy by status",
        +  "enum": [
        +    "active",
        +    "archived"
        +  ],
        +  "type": "string"
        +}
    • Changedget_user_requests_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedget_users_analytics16 fields changed
      • removedInput schema / properties / completion_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_max / minimum
        Added value: +0
      • removedInput schema / properties / completion_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / completion_token_min / minimum
        Added value: +0
      • removedInput schema / properties / cost_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_max / minimum
        Added value: +0
      • removedInput schema / properties / cost_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / cost_min / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_max / minimum
        Added value: +0
      • removedInput schema / properties / prompt_token_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / prompt_token_min / minimum
        Added value: +0
      • removedInput schema / properties / total_units_max / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_max / minimum
        Added value: +0
      • removedInput schema / properties / total_units_min / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / total_units_min / minimum
        Added value: +0
    • Changedlist_all_users5 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • addedInput schema / properties / email
        Added value: +{
        +  "description": "Filter by exact email",
        +  "format": "email",
        +  "pattern": "^(?!\\.)(?!.*\\.\\.)([A-Za-z0-9_'+\\-\\.]*)[A-Za-z0-9_+-]@([A-Za-z0-9][A-Za-z0-9\\-]*\\.)+[A-Za-z]{2,}$",
        +  "type": "string"
        +}
      • addedInput schema / properties / role
        Added value: +{
        +  "description": "Filter by organisation role",
        +  "enum": [
        +    "admin",
        +    "member",
        +    "owner"
        +  ],
        +  "type": "string"
        +}
    • Changedlist_api_keys3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_audit_logs3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination (starts at 1)"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_collections3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_configs3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Addedlist_deployments
    • Changedlist_guardrails5 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • addedInput schema / properties / page_size / exclusiveMinimum
        Added value: +0
      • removedInput schema / properties / page_size / minimum
        Removed value: -1
    • Changedlist_integration_models3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_integration_workspaces3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_integrations3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_mcp_integrations8 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • addedInput schema / properties / organisation_id
        Added value: +{
        +  "description": "Filter by organisation UUID",
        +  "type": "string"
        +}
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of results per page (max 100)"New value: +"Number of results per page (max 1000)"
      • changedInput schema / properties / page_size / maximum
        Previous value: -100New value: +1000
      • addedInput schema / properties / search
        Added value: +{
        +  "description": "Search integrations by name",
        +  "type": "string"
        +}
      • addedInput schema / properties / type
        Added value: +{
        +  "description": "Filter by workspace, organisation, or all integration ownership",
        +  "enum": [
        +    "workspace",
        +    "organisation",
        +    "all"
        +  ],
        +  "type": "string"
        +}
    • Changedlist_mcp_server_capabilities3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_mcp_server_connections2 fields changed
      • addedInput schema / properties / page_size / exclusiveMinimum
        Added value: +0
      • removedInput schema / properties / page_size / minimum
        Removed value: -1
    • Changedlist_mcp_server_user_access3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_mcp_servers3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_prompt_labels3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_prompts3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_providers3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_rate_limits7 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Zero-based page number",
        +  "maximum": 9007199254740991,
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of policies to return per page",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "integer"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Filter by active or archived status",
        +  "enum": [
        +    "active",
        +    "archived"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / target
        Added value: +{
        +  "description": "Filter by LLM or MCP-tool target",
        +  "enum": [
        +    "llm",
        +    "mcp_tools"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / type
        Added value: +{
        +  "description": "Filter by counter type",
        +  "enum": [
        +    "requests",
        +    "tokens"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / unit
        Added value: +{
        +  "description": "Filter by reset interval",
        +  "enum": [
        +    "rpm",
        +    "rph",
        +    "rpd",
        +    "rpw"
        +  ],
        +  "type": "string"
        +}
      • changedInput schema / properties / workspace_id / description
        Previous value: -"Filter rate limits by workspace ID"New value: +"Filter by workspace UUID"
    • Changedlist_scim_groups1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."New value: +"Zero-based results page to retrieve; the first page is 0. These SCIM tools name the parameter page, not the current_page used by other list tools."
    • Changedlist_scim_workspace_mappings1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."New value: +"Zero-based results page to retrieve; the first page is 0. These SCIM tools name the parameter page, not the current_page used by other list tools."
    • Changedlist_secret_references2 fields changed
      • addedInput schema / properties / page_size / exclusiveMinimum
        Added value: +0
      • removedInput schema / properties / page_size / minimum
        Removed value: -1
    • Changedlist_usage_limit_entities6 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Zero-based page number",
        +  "maximum": 9007199254740991,
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • changedInput schema / properties / limit_id / description
        Previous value: -"Usage limit policy ID"New value: +"Usage-limit policy UUID"
      • addedInput schema / properties / limit_id / minLength
        Added value: +1
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of entities per page",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "integer"
        +}
      • addedInput schema / properties / search
        Added value: +{
        +  "description": "Search tracked values",
        +  "type": "string"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Filter by enforcement state",
        +  "enum": [
        +    "active",
        +    "exhausted"
        +  ],
        +  "type": "string"
        +}
    • Changedlist_usage_limits5 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Zero-based page number",
        +  "maximum": 9007199254740991,
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of policies to return per page",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "integer"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Filter by active or archived status",
        +  "enum": [
        +    "active",
        +    "archived"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / type
        Added value: +{
        +  "description": "Filter by budget type",
        +  "enum": [
        +    "cost",
        +    "tokens"
        +  ],
        +  "type": "string"
        +}
      • changedInput schema / properties / workspace_id / description
        Previous value: -"Filter usage limits by workspace ID"New value: +"Filter by workspace UUID"
    • Changedlist_user_invites6 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • addedInput schema / properties / email
        Added value: +{
        +  "description": "Filter by invited email",
        +  "format": "email",
        +  "pattern": "^(?!\\.)(?!.*\\.\\.)([A-Za-z0-9_'+\\-\\.]*)[A-Za-z0-9_+-]@([A-Za-z0-9][A-Za-z0-9\\-]*\\.)+[A-Za-z]{2,}$",
        +  "type": "string"
        +}
      • addedInput schema / properties / role
        Added value: +{
        +  "description": "Filter invitations by organisation role",
        +  "enum": [
        +    "admin",
        +    "member"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Filter invitations by lifecycle status",
        +  "enum": [
        +    "pending",
        +    "cancelled",
        +    "accepted",
        +    "expired"
        +  ],
        +  "type": "string"
        +}
    • Changedlist_virtual_keys3 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number for pagination"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
    • Changedlist_workspace_members4 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Zero-based page number",
        +  "maximum": 9007199254740991,
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / email
        Added value: +{
        +  "description": "Filter by member email",
        +  "format": "email",
        +  "pattern": "^(?!\\.)(?!.*\\.\\.)([A-Za-z0-9_'+\\-\\.]*)[A-Za-z0-9_+-]@([A-Za-z0-9][A-Za-z0-9\\-]*\\.)+[A-Za-z]{2,}$",
        +  "type": "string"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of members per page",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "integer"
        +}
      • addedInput schema / properties / role
        Added value: +{
        +  "description": "Filter by workspace role",
        +  "enum": [
        +    "admin",
        +    "manager",
        +    "member"
        +  ],
        +  "type": "string"
        +}
    • Changedlist_workspaces6 fields changed
      • changedInput schema / properties / current_page / description
        Previous value: -"Page number to retrieve when results are paginated"New value: +"Zero-based page number; the first page is 0"
      • removedInput schema / properties / current_page / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / current_page / minimum
        Added value: +0
      • addedInput schema / properties / exact_name
        Added value: +{
        +  "description": "Exact workspace name filter",
        +  "type": "string"
        +}
      • addedInput schema / properties / name
        Added value: +{
        +  "description": "Case-sensitive name filter",
        +  "type": "string"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Filter by workspace lifecycle status",
        +  "enum": [
        +    "active",
        +    "archived"
        +  ],
        +  "type": "string"
        +}
    • Changedmigrate_prompt1 field changed
      • addedInput schema / properties / is_raw_template
        Added value: +{
        +  "description": "Preserve raw structural-template rendering semantics",
        +  "type": "boolean"
        +}
    • Addedregister_deployment
    • Changedreset_usage_limit_entity4 fields changed
      • changedInput schema / properties / entity_id / description
        Previous value: -"Entity ID to reset usage for"New value: +"Tracked entity UUID to reset"
      • addedInput schema / properties / entity_id / minLength
        Added value: +1
      • changedInput schema / properties / limit_id / description
        Previous value: -"Usage limit policy ID"New value: +"Usage-limit policy UUID"
      • addedInput schema / properties / limit_id / minLength
        Added value: +1
    • Changedupdate_api_key5 fields changed
      • addedInput schema / properties / default_allow_config_override
        Added value: +{
        +  "description": "Whether callers may override the key's default config",
        +  "type": "boolean"
        +}
      • changedInput schema / properties / expires_at / anyOf
        Previous value: -[
        -  {
        -    "type": "string"
        -  },
        -  {
        -    "type": "null"
        -  }
        -]New value: +[
        +  {
        +    "format": "date-time",
        +    "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +    "type": "string"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • addedInput schema / properties / rate_limits
        Added value: +{
        +  "anyOf": [
        +    {
        +      "items": {
        +        "properties": {
        +          "type": {
        +            "description": "Whether the limit counts requests or tokens",
        +            "enum": [
        +              "requests",
        +              "tokens"
        +            ],
        +            "type": "string"
        +          },
        +          "unit": {
        +            "description": "Rate window unit for the request or token count",
        +            "enum": [
        +              "rpd",
        +              "rph",
        +              "rpm",
        +              "rps",
        +              "rpw"
        +            ],
        +            "type": "string"
        +          },
        +          "value": {
        +            "description": "Maximum count in the selected rate window",
        +            "maximum": 9007199254740991,
        +            "minimum": 0,
        +            "type": "integer"
        +          }
        +        },
        +        "required": [
        +          "type",
        +          "unit",
        +          "value"
        +        ],
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Replacement request or token rate limits, or null to clear"
        +}
      • addedInput schema / properties / reset_usage
        Added value: +{
        +  "description": "Set to a nonnegative acknowledgement value to reset usage",
        +  "minimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / rotation_policy
        Added value: +{
        +  "anyOf": [
        +    {
        +      "description": "Automatic API-key rotation policy, or null to remove it",
        +      "properties": {
        +        "key_transition_period_ms": {
        +          "description": "Overlap in milliseconds before the previous key expires",
        +          "maximum": 9007199254740991,
        +          "minimum": 1800000,
        +          "type": "integer"
        +        },
        +        "next_rotation_at": {
        +          "anyOf": [
        +            {
        +              "format": "date-time",
        +              "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +              "type": "string"
        +            },
        +            {
        +              "type": "null"
        +            }
        +          ],
        +          "description": "Explicit next rotation timestamp in ISO 8601 format"
        +        },
        +        "rotation_period": {
        +          "anyOf": [
        +            {
        +              "enum": [
        +                "weekly",
        +                "monthly"
        +              ],
        +              "type": "string"
        +            },
        +            {
        +              "type": "null"
        +            }
        +          ],
        +          "description": "Built-in weekly or monthly automatic rotation cadence"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Replacement rotation policy, or null to disable it"
        +}
    • Addedupdate_deployment
    • Changedupdate_guardrail1 field changed
      • changedInput schema / properties / checks / items / properties / parameters / description
        Previous value: -"Check-specific configuration parameters"New value: +"Check-specific configuration. Supported examples include requestParametersCheck and parameters.forwardHeaders; forwarded headers can expose sensitive values, so allow only the minimum required names."
    • Changedupdate_mcp_integration4 fields changed
      • changedInput schema / properties / auth_type / description
        Previous value: -"New authentication type"New value: +"New authentication type. Current examples are 'none', 'headers' (custom headers), and 'oauth_auto' (OAuth); Portkey may accept additional values."
      • removedInput schema / properties / auth_type / enum
        Removed value: -[
        -  "oauth_auto",
        -  "headers",
        -  "none"
        -]
      • addedInput schema / properties / auth_type / minLength
        Added value: +1
      • addedInput schema / properties / configurations
        Added value: +{
        +  "additionalProperties": {},
        +  "description": "Replacement documented or forward-compatible configuration fields. configurations.custom_headers must be a string-to-string header map when present.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedupdate_prompt1 field changed
      • addedInput schema / properties / is_raw_template
        Added value: +{
        +  "description": "Enable or disable raw structural-template rendering",
        +  "type": "boolean"
        +}
    • Changedupdate_rate_limit9 fields changed
      • addedInput schema / properties / conditions
        Added value: +{
        +  "description": "Replacement conditions",
        +  "items": {
        +    "properties": {
        +      "excludes": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "items": {
        +              "type": "string"
        +            },
        +            "minItems": 1,
        +            "type": "array"
        +          }
        +        ],
        +        "description": "Values to exclude"
        +      },
        +      "key": {
        +        "description": "Match key such as api_key, virtual_key, provider, config, prompt, model, endpoint_type, mcp_server, mcp_tool, workspace_id, or metadata.*",
        +        "minLength": 1,
        +        "type": "string"
        +      },
        +      "value": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "items": {
        +              "type": "string"
        +            },
        +            "minItems": 1,
        +            "type": "array"
        +          }
        +        ],
        +        "description": "One value or several OR-matched values; use * for wildcard matching"
        +      }
        +    },
        +    "required": [
        +      "key",
        +      "value"
        +    ],
        +    "type": "object"
        +  },
        +  "minItems": 1,
        +  "type": "array"
        +}
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the rate limit"New value: +"Rate-limit policy UUID"
      • addedInput schema / properties / id / minLength
        Added value: +1
      • changedInput schema / properties / name / description
        Previous value: -"New name for the rate limit"New value: +"Replacement display name"
      • changedInput schema / properties / unit / description
        Previous value: -"New time unit: 'rpm' (per minute), 'rph' (per hour), or 'rpd' (per day)"New value: +"Replacement reset interval"
      • changedInput schema / properties / unit / enum
        Previous value: -[
        -  "rpm",
        -  "rph",
        -  "rpd"
        -]New value: +[
        +  "rpm",
        +  "rph",
        +  "rpd",
        +  "rpw"
        +]
      • changedInput schema / properties / value / description
        Previous value: -"New maximum allowed value per unit"New value: +"Replacement maximum"
      • removedInput schema / properties / value / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / value / minimum
        Added value: +0
    • Changedupdate_usage_limit14 fields changed
      • addedInput schema / properties / alert_threshold / anyOf
        Added value: +[
        +  {
        +    "minimum": 0,
        +    "type": "number"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • changedInput schema / properties / alert_threshold / description
        Previous value: -"New alert threshold percentage (0-100)"New value: +"Replacement alert threshold"
      • removedInput schema / properties / alert_threshold / type
        Removed value: -"number"
      • changedInput schema / properties / credit_limit / description
        Previous value: -"New maximum allowed usage value"New value: +"Replacement cumulative maximum"
      • removedInput schema / properties / credit_limit / exclusiveMinimum
        Removed value: -0
      • addedInput schema / properties / credit_limit / minimum
        Added value: +0
      • changedInput schema / properties / id / description
        Previous value: -"The unique identifier of the usage limit"New value: +"Usage-limit policy UUID"
      • addedInput schema / properties / id / minLength
        Added value: +1
      • changedInput schema / properties / name / description
        Previous value: -"New name for the usage limit"New value: +"Replacement display name"
      • addedInput schema / properties / periodic_reset / anyOf
        Added value: +[
        +  {
        +    "enum": [
        +      "monthly",
        +      "weekly"
        +    ],
        +    "type": "string"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • changedInput schema / properties / periodic_reset / description
        Previous value: -"New periodic reset schedule"New value: +"Replacement reset schedule"
      • removedInput schema / properties / periodic_reset / enum
        Removed value: -[
        -  "monthly",
        -  "weekly"
        -]
      • removedInput schema / properties / periodic_reset / type
        Removed value: -"string"
      • changedInput schema / properties / reset_usage_for_value / description
        Previous value: -"Reset usage counters for a specific group_by value"New value: +"Grouped value whose usage should reset"
    • Changedupdate_virtual_key2 fields changed
      • addedInput schema / properties / deployment_configurations
        Added value: +{
        +  "description": "Replacement Azure deployment configurations",
        +  "items": {
        +    "properties": {
        +      "alias": {
        +        "description": "Optional model alias for the deployment",
        +        "type": "string"
        +      },
        +      "api_version": {
        +        "description": "Azure OpenAI API version",
        +        "minLength": 1,
        +        "type": "string"
        +      },
        +      "deployment_name": {
        +        "description": "Azure deployment name",
        +        "minLength": 1,
        +        "type": "string"
        +      },
        +      "is_default": {
        +        "description": "Whether this is the default Azure deployment",
        +        "type": "boolean"
        +      }
        +    },
        +    "required": [
        +      "api_version",
        +      "deployment_name"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Replacement Secret Reference mappings",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key selected from a structured external secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference ID that owns the external secret",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Virtual Key field populated from the Secret Reference",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "json",
        +              "string"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Whether Portkey reads the mapped value as JSON or a string"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedupdate_workspace5 fields changed
      • addedInput schema / properties / input_guardrails
        Added value: +{
        +  "description": "Guardrail IDs applied to workspace model inputs",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / output_guardrails
        Added value: +{
        +  "description": "Guardrail IDs applied to workspace model outputs",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / rate_limits
        Added value: +{
        +  "description": "Replacement workspace rate-limit settings",
        +  "items": {
        +    "properties": {
        +      "type": {
        +        "description": "Whether the limit counts requests or tokens",
        +        "enum": [
        +          "requests",
        +          "tokens"
        +        ],
        +        "type": "string"
        +      },
        +      "unit": {
        +        "description": "Rate window: requests per day, hour, or minute",
        +        "enum": [
        +          "rpd",
        +          "rph",
        +          "rpm"
        +        ],
        +        "type": "string"
        +      },
        +      "value": {
        +        "description": "Maximum count in the selected rate window",
        +        "maximum": 9007199254740991,
        +        "minimum": 0,
        +        "type": "integer"
        +      }
        +    },
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / usage_limits
        Added value: +{
        +  "description": "Replacement workspace usage-limit settings",
        +  "items": {
        +    "properties": {
        +      "alert_threshold": {
        +        "description": "Percentage of the limit that triggers an alert",
        +        "exclusiveMinimum": 0,
        +        "type": "number"
        +      },
        +      "credit_limit": {
        +        "description": "Maximum cost or token usage for the workspace",
        +        "exclusiveMinimum": 0,
        +        "type": "number"
        +      },
        +      "next_usage_reset_at": {
        +        "anyOf": [
        +          {
        +            "format": "date-time",
        +            "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$",
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Next reset timestamp in ISO 8601 format"
        +      },
        +      "periodic_reset": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "monthly",
        +              "weekly"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Built-in weekly or monthly reset cadence"
        +      },
        +      "periodic_reset_days": {
        +        "anyOf": [
        +          {
        +            "maximum": 365,
        +            "minimum": 1,
        +            "type": "integer"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Custom reset cadence in days, from 1 through 365"
        +      },
        +      "type": {
        +        "description": "Whether the limit measures cost or tokens",
        +        "enum": [
        +          "cost",
        +          "tokens"
        +        ],
        +        "type": "string"
        +      }
        +    },
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedInput schema / properties / user_api_key_config
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "string"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Default config slug for workspace-user API keys, or null to clear"
        +}
  2. 5 tool updatesv1.0.5
    • Changedget_workspace_member3 fields changed
      • changedInput schema / properties / user_id / description
        Previous value: -"The user ID to retrieve"New value: +"The user ID to retrieve (must be a valid UUID from list_all_users, not an email address)"
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
    • Changedlist_scim_groups1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0"New value: +"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."
    • Changedlist_scim_workspace_mappings1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0"New value: +"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."
    • Changedremove_workspace_member2 fields changed
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
    • Changedupdate_workspace_member3 fields changed
      • changedInput schema / properties / user_id / description
        Previous value: -"The user ID to update"New value: +"The user ID to update (must be a valid UUID from list_all_users, not an email address)"
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
  3. 23 tool updates
    • Changedcreate_config2 fields changed
      • addedInput schema / properties / targets / items / properties / provider / description
        Added value: +"Provider slug to route requests to"
      • addedInput schema / properties / targets / items / properties / virtual_key / description
        Added value: +"Virtual key slug to use for this routing target"
    • Changedcreate_integration4 fields changed
      • addedInput schema / properties / create_default_provider
        Added value: +{
        +  "description": "Create a provider automatically for a workspace-scoped integration; defaults to true",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / default_provider_slug
        Added value: +{
        +  "description": "Custom slug for the automatically created workspace provider",
        +  "maxLength": 255,
        +  "pattern": "^[a-zA-Z0-9_-]+$",
        +  "type": "string"
        +}
      • addedInput schema / properties / pricing_adjustments
        Added value: +{
        +  "anyOf": [
        +    {
        +      "properties": {
        +        "multiplier": {
        +          "description": "Per-unit multipliers applied over Portkey catalog pricing",
        +          "properties": {
        +            "additional_units": {
        +              "anyOf": [
        +                {
        +                  "additionalProperties": {
        +                    "anyOf": [
        +                      {
        +                        "minimum": 0,
        +                        "type": "number"
        +                      },
        +                      {
        +                        "type": "null"
        +                      }
        +                    ]
        +                  },
        +                  "propertyNames": {
        +                    "type": "string"
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Multipliers keyed by Portkey pricing unit, such as web_search or file_search"
        +            },
        +            "cache_read_audio_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read audio input token price multiplier"
        +            },
        +            "cache_read_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read image input token price multiplier"
        +            },
        +            "cache_read_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read input token price multiplier"
        +            },
        +            "cache_read_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read text input token price multiplier"
        +            },
        +            "cache_write_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write image input token price multiplier"
        +            },
        +            "cache_write_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write input token price multiplier"
        +            },
        +            "cache_write_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write text input token price multiplier"
        +            },
        +            "default": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Fallback multiplier for pricing units; 1 is unchanged, 0.8 is a 20% discount"
        +            },
        +            "image": {
        +              "anyOf": [
        +                {
        +                  "properties": {
        +                    "default": {
        +                      "anyOf": [
        +                        {
        +                          "minimum": 0,
        +                          "type": "number"
        +                        },
        +                        {
        +                          "type": "null"
        +                        }
        +                      ],
        +                      "description": "Default image-generation price multiplier"
        +                    }
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image-generation pricing multipliers"
        +            },
        +            "prediction_accepted_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Accepted predicted-output token price multiplier"
        +            },
        +            "prediction_rejected_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Rejected predicted-output token price multiplier"
        +            },
        +            "reasoning_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Reasoning token price multiplier"
        +            },
        +            "request_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio input token price multiplier"
        +            },
        +            "request_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image input token price multiplier"
        +            },
        +            "request_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text input token price multiplier"
        +            },
        +            "request_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Input token price multiplier"
        +            },
        +            "response_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio output token price multiplier"
        +            },
        +            "response_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image output token price multiplier"
        +            },
        +            "response_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text output token price multiplier"
        +            },
        +            "response_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Output token price multiplier"
        +            }
        +          },
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Negotiated discount or markup multipliers for cost accounting"
        +}
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Unique runtime Secret Reference mappings; mapping key allows the key field to be omitted",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to the workspace",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Integration field populated at runtime: key or configurations.<field>",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Treat the resolved value as a plain string or parsed JSON"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedcreate_mcp_integration1 field changed
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Runtime Secret Reference mappings; every configurations.<field> target must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to this integration",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Configuration field resolved at runtime, such as configurations.oauth_metadata",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Use json when the target configuration field expects a structured object"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Addedcreate_scim_workspace_mapping
    • Addeddelete_scim_workspace_mapping
    • Addeddisconnect_mcp_server_connection
    • Addedget_log
    • Addedget_log_export_field_restrictions
    • Addedget_model_pricing
    • Addedget_organisation_defaults
    • Addedlist_input_guardrail_workspace_exclusions
    • Addedlist_mcp_server_connections
    • Addedlist_output_guardrail_workspace_exclusions
    • Addedlist_scim_groups
    • Addedlist_scim_workspace_mappings
    • Changedupdate_config2 fields changed
      • addedInput schema / properties / targets / items / properties / provider / description
        Added value: +"Provider slug to route requests to"
      • addedInput schema / properties / targets / items / properties / virtual_key / description
        Added value: +"Virtual key slug to use for this routing target"
    • Addedupdate_input_guardrail_workspace_exclusions
    • Changedupdate_integration2 fields changed
      • addedInput schema / properties / pricing_adjustments
        Added value: +{
        +  "anyOf": [
        +    {
        +      "properties": {
        +        "multiplier": {
        +          "description": "Per-unit multipliers applied over Portkey catalog pricing",
        +          "properties": {
        +            "additional_units": {
        +              "anyOf": [
        +                {
        +                  "additionalProperties": {
        +                    "anyOf": [
        +                      {
        +                        "minimum": 0,
        +                        "type": "number"
        +                      },
        +                      {
        +                        "type": "null"
        +                      }
        +                    ]
        +                  },
        +                  "propertyNames": {
        +                    "type": "string"
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Multipliers keyed by Portkey pricing unit, such as web_search or file_search"
        +            },
        +            "cache_read_audio_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read audio input token price multiplier"
        +            },
        +            "cache_read_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read image input token price multiplier"
        +            },
        +            "cache_read_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read input token price multiplier"
        +            },
        +            "cache_read_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read text input token price multiplier"
        +            },
        +            "cache_write_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write image input token price multiplier"
        +            },
        +            "cache_write_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write input token price multiplier"
        +            },
        +            "cache_write_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write text input token price multiplier"
        +            },
        +            "default": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Fallback multiplier for pricing units; 1 is unchanged, 0.8 is a 20% discount"
        +            },
        +            "image": {
        +              "anyOf": [
        +                {
        +                  "properties": {
        +                    "default": {
        +                      "anyOf": [
        +                        {
        +                          "minimum": 0,
        +                          "type": "number"
        +                        },
        +                        {
        +                          "type": "null"
        +                        }
        +                      ],
        +                      "description": "Default image-generation price multiplier"
        +                    }
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image-generation pricing multipliers"
        +            },
        +            "prediction_accepted_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Accepted predicted-output token price multiplier"
        +            },
        +            "prediction_rejected_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Rejected predicted-output token price multiplier"
        +            },
        +            "reasoning_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Reasoning token price multiplier"
        +            },
        +            "request_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio input token price multiplier"
        +            },
        +            "request_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image input token price multiplier"
        +            },
        +            "request_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text input token price multiplier"
        +            },
        +            "request_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Input token price multiplier"
        +            },
        +            "response_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio output token price multiplier"
        +            },
        +            "response_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image output token price multiplier"
        +            },
        +            "response_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text output token price multiplier"
        +            },
        +            "response_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Output token price multiplier"
        +            }
        +          },
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Replacement cost multiplier configuration, or null to clear adjustments"
        +}
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Replacement runtime Secret Reference mappings; each target_field must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to the workspace",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Integration field populated at runtime: key or configurations.<field>",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Treat the resolved value as a plain string or parsed JSON"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedupdate_integration_models5 fields changed
      • addedInput schema / properties / allow_all_models
        Added value: +{
        +  "description": "Whether newly available provider models are enabled by default",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / models / items / properties / base_model_slug
        Added value: +{
        +  "description": "Base model slug for a fine-tuned model",
        +  "type": "string"
        +}
      • addedInput schema / properties / models / items / properties / configurations
        Added value: +{
        +  "description": "Per-model custom upstream host and headers",
        +  "properties": {
        +    "custom_headers": {
        +      "additionalProperties": {
        +        "type": "string"
        +      },
        +      "description": "Headers sent only for this model; values may contain secrets",
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "custom_host": {
        +      "description": "Custom upstream URL used only for this model",
        +      "format": "uri",
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
      • addedInput schema / properties / models / items / properties / is_finetune
        Added value: +{
        +  "description": "Whether this entry is a fine-tuned model",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / models / items / properties / pricing_config
        Added value: +{
        +  "description": "Static custom pricing for this integration model",
        +  "properties": {
        +    "pay_as_you_go": {
        +      "description": "Pay-as-you-go rates for this integration model",
        +      "properties": {
        +        "request_token": {
        +          "description": "Input token pricing",
        +          "properties": {
        +            "price": {
        +              "description": "Static price per token in the integration pricing unit",
        +              "minimum": 0,
        +              "type": "number"
        +            }
        +          },
        +          "required": [
        +            "price"
        +          ],
        +          "type": "object"
        +        },
        +        "response_token": {
        +          "description": "Output token pricing",
        +          "properties": {
        +            "price": {
        +              "description": "Static price per token in the integration pricing unit",
        +              "minimum": 0,
        +              "type": "number"
        +            }
        +          },
        +          "required": [
        +            "price"
        +          ],
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    "type": {
        +      "const": "static",
        +      "description": "Static per-token pricing configuration",
        +      "type": "string"
        +    }
        +  },
        +  "required": [
        +    "type"
        +  ],
        +  "type": "object"
        +}
    • Changedupdate_integration_workspaces10 fields changed
      • addedInput schema / properties / create_default_provider
        Added value: +{
        +  "description": "Auto-create providers when granting workspace access",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / default_provider_slug
        Added value: +{
        +  "description": "Default slug for providers auto-created in workspaces",
        +  "type": "string"
        +}
      • addedInput schema / properties / global_alert_threshold
        Added value: +{
        +  "description": "Global cost alert threshold applied to workspace access",
        +  "maximum": 100,
        +  "minimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_credit_limit
        Added value: +{
        +  "description": "Global cost credit limit applied with global workspace access",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_rate_limit_rpm
        Added value: +{
        +  "description": "Global requests-per-minute limit for workspace access",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_workspace_access_enabled
        Added value: +{
        +  "description": "Enable or disable access for all current and future workspaces",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / override_existing_workspace_access
        Added value: +{
        +  "description": "Apply global settings over existing per-workspace access",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / workspaces / items / properties / create_default_provider
        Added value: +{
        +  "description": "Override whether a provider is auto-created for this workspace",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / workspaces / items / properties / default_provider_slug
        Added value: +{
        +  "description": "Override the auto-created provider slug for this workspace",
        +  "type": "string"
        +}
      • addedInput schema / properties / workspaces / items / properties / reset_usage
        Added value: +{
        +  "description": "Reset exhausted usage status to active",
        +  "type": "boolean"
        +}
    • Changedupdate_mcp_integration1 field changed
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Replacement runtime Secret Reference mappings; each target_field must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to this integration",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Configuration field resolved at runtime, such as configurations.oauth_metadata",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Use json when the target configuration field expects a structured object"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Addedupdate_organisation_defaults
    • Addedupdate_output_guardrail_workspace_exclusions
  4. 29 tool updatesv0.5.0
    • Addedcreate_secret_reference
    • Addeddelete_secret_reference
    • Changedget_analytics_group_metadata2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedget_analytics_group_models2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedget_analytics_group_users2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Addedget_secret_reference
    • Changedget_user_stats1 field changed
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_all_users3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_api_keys3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_audit_logs3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_collections3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_configs3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_guardrails3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integration_models2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integration_workspaces2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integrations2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_integrations2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_server_capabilities3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_server_user_access3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_servers2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_prompt_labels3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_prompts3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_providers2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Addedlist_secret_references
    • Changedlist_user_invites3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_virtual_keys3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_workspaces3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Addedrotate_api_key
    • Addedupdate_secret_reference
  5. 23 tool updatesv1.0.2
    • Changedget_analytics_group_metadata1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids",
        -  "metadata_key"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max",
        +  "metadata_key"
        +]
    • Changedget_analytics_group_models1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_analytics_group_users1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cache_hit_latency1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cache_hit_rate1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cost_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_rate_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_stacks_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_status_codes_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_models_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_scores_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_weighted_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_latency_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_request_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_rescued_requests_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_token_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_user_requests_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_user_stats2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of results per page (for pagination)"New value: +"Number of results per page (max 100)"
      • addedInput schema / properties / page_size / maximum
        Added value: +100
    • Changedget_users_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedlist_guardrails2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of items per page (1-1000, default: 100)"New value: +"Number of items per page (1-100, default: 100)"
      • changedInput schema / properties / page_size / maximum
        Previous value: -1000New value: +100
    • Changedlist_workspaces2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of workspaces to return per page (default varies by endpoint)"New value: +"Number of workspaces to return per page (max 100)"
      • addedInput schema / properties / page_size / maximum
        Added value: +100
  6. 7 tool updatesv0.3.7
    • Changedget_request_analytics1 field changed
      • changedInput schema / properties / api_key_ids / description
        Previous value: -"API key UUIDs. Accepts the legacy comma-separated string or a structured array; normalized to the legacy Portkey query param before the request is sent."New value: +"Legacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form."
    • Changedlist_all_users2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_configs2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_mcp_server_capabilities2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_mcp_server_user_access2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_user_invites2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_virtual_keys2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
  7. 150 tool updatesv1.0.1
    • Addedadd_workspace_member
    • Addedcancel_log_export
    • Addedcreate_api_key
    • Addedcreate_collection
    • Addedcreate_config
    • Addedcreate_feedback
    • Addedcreate_guardrail
    • Addedcreate_integration
    • Addedcreate_log_export
    • Addedcreate_mcp_integration
    • Addedcreate_mcp_server
    • Addedcreate_prompt
    • Addedcreate_prompt_label
    • Addedcreate_prompt_partial
    • Addedcreate_provider
    • Addedcreate_rate_limit
    • Addedcreate_usage_limit
    • Addedcreate_virtual_key
    • Addedcreate_workspace
    • Addeddelete_api_key
    • Addeddelete_collection
    • Addeddelete_config
    • Addeddelete_guardrail
    • Addeddelete_integration
    • Addeddelete_integration_model
    • Addeddelete_mcp_integration
    • Addeddelete_mcp_server
    • Addeddelete_prompt
    • Addeddelete_prompt_label
    • Addeddelete_prompt_partial
    • Addeddelete_provider
    • Addeddelete_rate_limit
    • Addeddelete_usage_limit
    • Addeddelete_user
    • Addeddelete_user_invite
    • Addeddelete_virtual_key
    • Addeddelete_workspace
    • Addeddownload_log_export
    • Addedget_analytics_group_metadata
    • Addedget_analytics_group_models
    • Addedget_analytics_group_users
    • Addedget_api_key
    • Addedget_cache_hit_latency
    • Addedget_cache_hit_rate
    • Addedget_collection
    • Addedget_config
    • Addedget_cost_analytics
    • Addedget_error_analytics
    • Addedget_error_rate_analytics
    • Addedget_error_stacks_analytics
    • Addedget_error_status_codes_analytics
    • Addedget_feedback_analytics
    • Addedget_feedback_models_analytics
    • Addedget_feedback_scores_analytics
    • Addedget_feedback_weighted_analytics
    • Addedget_guardrail
    • Addedget_integration
    • Addedget_latency_analytics
    • Addedget_log_export
    • Addedget_mcp_integration
    • Addedget_mcp_integration_metadata
    • Addedget_mcp_server
    • Addedget_prompt
    • Addedget_prompt_label
    • Addedget_prompt_partial
    • Addedget_prompt_version
    • Addedget_provider
    • Addedget_rate_limit
    • Addedget_request_analytics
    • Addedget_rescued_requests_analytics
    • Addedget_token_analytics
    • Addedget_usage_limit
    • Addedget_user
    • Addedget_user_invite
    • Addedget_user_requests_analytics
    • Addedget_user_stats
    • Addedget_users_analytics
    • Addedget_virtual_key
    • Addedget_workspace
    • Addedget_workspace_member
    • Addedinsert_log
    • Addedinvite_user
    • Addedlist_all_users
    • Addedlist_api_keys
    • Addedlist_audit_logs
    • Addedlist_collections
    • Addedlist_config_versions
    • Addedlist_configs
    • Addedlist_guardrails
    • Addedlist_integration_models
    • Addedlist_integration_workspaces
    • Addedlist_integrations
    • Addedlist_log_exports
    • Addedlist_mcp_integration_capabilities
    • Addedlist_mcp_integration_workspaces
    • Addedlist_mcp_integrations
    • Addedlist_mcp_server_capabilities
    • Addedlist_mcp_server_user_access
    • Addedlist_mcp_servers
    • Addedlist_partial_versions
    • Addedlist_prompt_labels
    • Addedlist_prompt_partials
    • Addedlist_prompt_versions
    • Addedlist_prompts
    • Addedlist_providers
    • Addedlist_rate_limits
    • Addedlist_usage_limit_entities
    • Addedlist_usage_limits
    • Addedlist_user_invites
    • Addedlist_virtual_keys
    • Addedlist_workspace_members
    • Addedlist_workspaces
    • Addedmigrate_prompt
    • Addedpromote_prompt
    • Addedpublish_partial
    • Addedpublish_prompt
    • Addedremove_workspace_member
    • Addedrender_prompt
    • Addedresend_user_invite
    • Addedreset_usage_limit_entity
    • Addedrun_prompt_completion
    • Addedstart_log_export
    • Addedtest_mcp_server
    • Addedupdate_api_key
    • Addedupdate_collection
    • Addedupdate_config
    • Addedupdate_feedback
    • Addedupdate_guardrail
    • Addedupdate_integration
    • Addedupdate_integration_models
    • Addedupdate_integration_workspaces
    • Addedupdate_log_export
    • Addedupdate_mcp_integration
    • Addedupdate_mcp_integration_capabilities
    • Addedupdate_mcp_integration_workspaces
    • Addedupdate_mcp_server
    • Addedupdate_mcp_server_capabilities
    • Addedupdate_mcp_server_user_access
    • Addedupdate_prompt
    • Addedupdate_prompt_label
    • Addedupdate_prompt_partial
    • Addedupdate_prompt_version
    • Addedupdate_provider
    • Addedupdate_rate_limit
    • Addedupdate_usage_limit
    • Addedupdate_user
    • Addedupdate_virtual_key
    • Addedupdate_workspace
    • Addedupdate_workspace_member
    • Addedvalidate_completion_metadata
  8. 117 tool updates
    • Removedadd_workspace_member
    • Removedcancel_log_export
    • Removedcreate_api_key
    • Removedcreate_collection
    • Removedcreate_config
    • Removedcreate_feedback
    • Removedcreate_guardrail
    • Removedcreate_integration
    • Removedcreate_log_export
    • Removedcreate_prompt
    • Removedcreate_prompt_label
    • Removedcreate_prompt_partial
    • Removedcreate_provider
    • Removedcreate_rate_limit
    • Removedcreate_usage_limit
    • Removedcreate_virtual_key
    • Removedcreate_workspace
    • Removeddelete_api_key
    • Removeddelete_collection
    • Removeddelete_config
    • Removeddelete_guardrail
    • Removeddelete_integration
    • Removeddelete_integration_model
    • Removeddelete_prompt
    • Removeddelete_prompt_label
    • Removeddelete_prompt_partial
    • Removeddelete_provider
    • Removeddelete_rate_limit
    • Removeddelete_usage_limit
    • Removeddelete_user
    • Removeddelete_user_invite
    • Removeddelete_virtual_key
    • Removeddelete_workspace
    • Removeddownload_log_export
    • Removedget_api_key
    • Removedget_cache_hit_latency
    • Removedget_cache_hit_rate
    • Removedget_collection
    • Removedget_config
    • Removedget_cost_analytics
    • Removedget_error_analytics
    • Removedget_error_rate_analytics
    • Removedget_guardrail
    • Removedget_integration
    • Removedget_latency_analytics
    • Removedget_log_export
    • Removedget_prompt
    • Removedget_prompt_label
    • Removedget_prompt_partial
    • Removedget_provider
    • Removedget_rate_limit
    • Removedget_request_analytics
    • Removedget_token_analytics
    • Removedget_trace
    • Removedget_usage_limit
    • Removedget_user
    • Removedget_user_invite
    • Removedget_user_stats
    • Removedget_users_analytics
    • Removedget_virtual_key
    • Removedget_workspace
    • Removedget_workspace_member
    • Removedinsert_log
    • Removedinvite_user
    • Removedlist_all_users
    • Removedlist_api_keys
    • Removedlist_audit_logs
    • Removedlist_collections
    • Removedlist_config_versions
    • Removedlist_configs
    • Removedlist_guardrails
    • Removedlist_integration_models
    • Removedlist_integration_workspaces
    • Removedlist_integrations
    • Removedlist_log_exports
    • Removedlist_partial_versions
    • Removedlist_prompt_labels
    • Removedlist_prompt_partials
    • Removedlist_prompt_versions
    • Removedlist_prompts
    • Removedlist_providers
    • Removedlist_rate_limits
    • Removedlist_traces
    • Removedlist_usage_limits
    • Removedlist_user_invites
    • Removedlist_virtual_keys
    • Removedlist_workspace_members
    • Removedlist_workspaces
    • Removedmigrate_prompt
    • Removedpromote_prompt
    • Removedpublish_partial
    • Removedpublish_prompt
    • Removedremove_workspace_member
    • Removedrender_prompt
    • Removedresend_user_invite
    • Removedrun_prompt_completion
    • Removedstart_log_export
    • Removedupdate_api_key
    • Removedupdate_collection
    • Removedupdate_config
    • Removedupdate_feedback
    • Removedupdate_guardrail
    • Removedupdate_integration
    • Removedupdate_integration_models
    • Removedupdate_integration_workspaces
    • Removedupdate_log_export
    • Removedupdate_prompt
    • Removedupdate_prompt_label
    • Removedupdate_prompt_partial
    • Removedupdate_provider
    • Removedupdate_rate_limit
    • Removedupdate_usage_limit
    • Removedupdate_user
    • Removedupdate_virtual_key
    • Removedupdate_workspace
    • Removedupdate_workspace_member
    • Removedvalidate_completion_metadata
  9. 117 tool updates
    • First observedadd_workspace_member
    • First observedcancel_log_export
    • First observedcreate_api_key
    • First observedcreate_collection
    • First observedcreate_config
    • First observedcreate_feedback
    • First observedcreate_guardrail
    • First observedcreate_integration
    • First observedcreate_log_export
    • First observedcreate_prompt
    • First observedcreate_prompt_label
    • First observedcreate_prompt_partial
    • First observedcreate_provider
    • First observedcreate_rate_limit
    • First observedcreate_usage_limit
    • First observedcreate_virtual_key
    • First observedcreate_workspace
    • First observeddelete_api_key
    • First observeddelete_collection
    • First observeddelete_config
    • First observeddelete_guardrail
    • First observeddelete_integration
    • First observeddelete_integration_model
    • First observeddelete_prompt
    • First observeddelete_prompt_label
    • First observeddelete_prompt_partial
    • First observeddelete_provider
    • First observeddelete_rate_limit
    • First observeddelete_usage_limit
    • First observeddelete_user
    • First observeddelete_user_invite
    • First observeddelete_virtual_key
    • First observeddelete_workspace
    • First observeddownload_log_export
    • First observedget_api_key
    • First observedget_cache_hit_latency
    • First observedget_cache_hit_rate
    • First observedget_collection
    • First observedget_config
    • First observedget_cost_analytics
    • First observedget_error_analytics
    • First observedget_error_rate_analytics
    • First observedget_guardrail
    • First observedget_integration
    • First observedget_latency_analytics
    • First observedget_log_export
    • First observedget_prompt
    • First observedget_prompt_label
    • First observedget_prompt_partial
    • First observedget_provider
    • First observedget_rate_limit
    • First observedget_request_analytics
    • First observedget_token_analytics
    • First observedget_trace
    • First observedget_usage_limit
    • First observedget_user
    • First observedget_user_invite
    • First observedget_user_stats
    • First observedget_users_analytics
    • First observedget_virtual_key
    • First observedget_workspace
    • First observedget_workspace_member
    • First observedinsert_log
    • First observedinvite_user
    • First observedlist_all_users
    • First observedlist_api_keys
    • First observedlist_audit_logs
    • First observedlist_collections
    • First observedlist_config_versions
    • First observedlist_configs
    • First observedlist_guardrails
    • First observedlist_integration_models
    • First observedlist_integration_workspaces
    • First observedlist_integrations
    • First observedlist_log_exports
    • First observedlist_partial_versions
    • First observedlist_prompt_labels
    • First observedlist_prompt_partials
    • First observedlist_prompt_versions
    • First observedlist_prompts
    • First observedlist_providers
    • First observedlist_rate_limits
    • First observedlist_traces
    • First observedlist_usage_limits
    • First observedlist_user_invites
    • First observedlist_virtual_keys
    • First observedlist_workspace_members
    • First observedlist_workspaces
    • First observedmigrate_prompt
    • First observedpromote_prompt
    • First observedpublish_partial
    • First observedpublish_prompt
    • First observedremove_workspace_member
    • First observedrender_prompt
    • First observedresend_user_invite
    • First observedrun_prompt_completion
    • First observedstart_log_export
    • First observedupdate_api_key
    • First observedupdate_collection
    • First observedupdate_config
    • First observedupdate_feedback
    • First observedupdate_guardrail
    • First observedupdate_integration
    • First observedupdate_integration_models
    • First observedupdate_integration_workspaces
    • First observedupdate_log_export
    • First observedupdate_prompt
    • First observedupdate_prompt_label
    • First observedupdate_prompt_partial
    • First observedupdate_provider
    • First observedupdate_rate_limit
    • First observedupdate_usage_limit
    • First observedupdate_user
    • First observedupdate_virtual_key
    • First observedupdate_workspace
    • First observedupdate_workspace_member
    • First observedvalidate_completion_metadata

TDQS

A3.9/5.0

Scored across 178 tools

Disambiguation4/5

Most tools follow a clear resource+action pattern, and descriptions explicitly cross-reference related tools, so typical CRUD operations are easy to distinguish. However, the sheer number of analytics tools (cost/token/latency/error/cache/user/feedback) and the MCP integration/server/capability families create some boundary ambiguity that could lead to misselection.

Naming Consistency4/5

The overwhelming majority use consistent verb_noun (get_, list_, create_, update_, delete_) with resource nouns, and longer names like update_input_guardrail_workspace_exclusions are still readable. Occasional action verbs (publish, promote, migrate, rotate, test) and a few oddities like get_analytics_group_users vs get_users_analytics break perfect uniformity but don't undermine predictability.

Tool Count1/5

178 tools is far beyond typical MCP server scope and creates a massive surface area for an agent to navigate. Even with coherent organization, the count alone is an extreme mismatch for an admin toolset and will degrade selection performance.

Completeness4/5

The surface covers essentially the full lifecycle for users, workspaces, keys, configs, prompts, integrations, guardrails, limits, MCP servers, and logs, with list/get/create/update/delete plus specialized operations. Minor gaps like no dedicated get_feedback or delete_log_export are workarounds, so the domain is thoroughly covered.

Maintenance

ActivityActive
ResponsivenessWithin a week

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    F
    maintenance
    MCP server that exposes 300+ AI agents as tools via a single API key. Supports listing agents, invoking any agent with chat-completion style messages, checking agent health, and retrieving platform statistics.
    5
    3
    -
  • F
    license
    Not graded
    quality
    D
    maintenance
    Exposes LiteLLM Proxy admin APIs as MCP tools for managing internal users, virtual keys, and spend logs via streamable-http, enabling agents to administer LiteLLM without custom HTTP glue.
    1
    -
  • A
    license
    Not graded
    quality
    D
    maintenance
    MCP server that provides tools to interact with the LiteLLM proxy API, enabling LLM completions, embeddings, image generation, and admin operations.
    7 npm
    MIT