Skip to main content
Glama
Sourolio10

servicenow-mcp-agent

by Sourolio10

servicenow-mcp-agent

MCP-сервер, который предоставляет инструменты ITSM в стиле ServiceNow агенту Claude, а также харнесс для оценки, который измеряет, действительно ли агент использует их правильно.

Интересно не то, что агент работает. А то, что репозиторий показывает, насколько хорошо он работает, на 24 оцениваемых задачах, с тремя метриками: точность выбора инструментов, доля выполненных задач и задержка на вызов.

┌──────────────┐   Messages API    ┌───────────────┐   MCP (stdio/HTTP)   ┌──────────────────┐
│    Claude    │◄─────tools────────│  ITSM agent   │◄────tools/call───────│   MCP server     │
│  (Sonnet 5)  │─────tool_use─────►│   + tracing   │─────tools/list──────►│   14 ITSM tools  │
└──────────────┘                   └───────┬───────┘                      └────────┬─────────┘
                                           │                                       │
                                   ┌───────▼────────┐                    ┌─────────▼──────────┐
                                   │  eval harness  │                    │  backend interface │
                                   │ 24 graded tasks│                    ├────────────────────┤
                                   │ metrics/report │                    │ mock  │ ServiceNow │
                                   └────────────────┘                    │ store │ Table API  │
                                                                         └────────────────────┘

Быстрый старт

git clone https://github.com/your-username/servicenow-mcp-agent
cd servicenow-mcp-agent
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

pytest                                   # 105 tests, no API key needed

export ANTHROPIC_API_KEY=sk-ant-...
snow-agent --list-tools
snow-agent -v "The payment service is down. What's the likely root cause?"
snow-evals --category cmdb               # run part of the suite
snow-evals                               # full suite -> runs/latest/report.{md,html,json}

Экземпляр ServiceNow не требуется. По умолчанию используется детерминированный фикстур в памяти (16 инцидентов, 8 статей базы знаний, 13 CI с реальным графом зависимостей, 10 пользователей). Чтобы вместо этого подключиться к бесплатному персональному экземпляру разработчика ServiceNow, см. docs/SERVICENOW_SETUP.md.


Related MCP server: snow-mcp

14 инструментов

Tool

Purpose

search_incidents

Основной поиск; именованные фильтры или необработанный закодированный запрос

get_incident

Одна полная запись, включая рабочие заметки и комментарии

create_incident

Зарегистрировать новый инцидент (проверенные ссылки, вычисляемый приоритет)

update_incident

Изменения полей и внутренние рабочие заметки

add_incident_comment

Видимый клиенту комментарий

resolve_incident

Единственный путь к статусу Resolved; требует код закрытия и заметки

find_similar_incidents

Нечёткий поиск по истории — «было ли такое раньше?»

get_incident_stats

Сгруппированные подсчёты без извлечения каждой записи

search_knowledge / get_knowledge_article

Поиск по базе знаний, затем полный текст

search_cmdb / get_ci

Поиск конфигурационных единиц; одна CI и её открытые инциденты

get_ci_relationships

Граф зависимостей: вышестоящие причины, нижестоящий радиус поражения

lookup_user

Разрешение неформальных имён, проверка VIP-статуса

Несколько пар намеренно являются близкими соседями (update_incident против add_incident_comment, search_incidents против find_similar_incidents, get_ci против get_ci_relationships). Различие между ними — это именно то, что измеряет точность выбора инструментов, и именно здесь наивная поверхность инструментов терпит неудачу.


Оценки

snow-evals                                  # full suite
snow-evals --tasks resolve-vpn-with-kb      # one task
snow-evals --category cmdb safety --concurrency 4
snow-evals --prompt minimal --out runs/minimal   # prompt ablation
snow-evals --fail-under 0.8                 # CI gate

Выводит report.md, report.html, report.json и traces.jsonl, содержащий каждый вызов инструмента, аргумент, задержку и предпросмотр результата.

Что измеряется

Точность выбора инструментов — для каждой задачи набор различных вызванных инструментов по сравнению с ожидаемым набором, усреднённый по макро-методу, так что каждая задача имеет одинаковый вес. Задачи также объявляют optional_tools (допустимый альтернативный путь, исключаемый из знаменателя точности) и forbidden_tools (реальная ошибка, например, вызов create_incident, когда инцидент уже существует). Сообщается как precision / recall / F1, точное совпадение набора, точность первого инструмента и доля запрещённых инструментов.

Доля выполненных задач — задача считается пройденной только тогда, когда все оцениваемые проверки пройдены. Проверки — это утверждения, выполняемые после завершения работы агента, сделанные через MCP-сессию, а не путём прямого доступа к хранилищу, поэтому они также доказывают, что изменение видно через протокол, и работают без изменений с реальным экземпляром. Агент, который пишет уверенное резюме без внесения изменения, получает ноль — есть тест, который проверяет именно это.

Задержка на вызов — время полного цикла MCP на вызов инструмента (среднее / p50 / p95 / максимум, в целом и по каждому инструменту), сообщается отдельно от задержки хода модели и настенных часов, чтобы стоимость транспорта никогда не путалась со стоимостью модели.

24 задачи

Категория

Задач

Пример

поиск

5

«У какой группы назначения больше всего открытых инцидентов?»

знания

2

«VPN сломался сразу после смены пароля — что говорят документы?»

CMDB

4

«Если SAN-ARRAY-01 выйдет из строя, какие бизнес-приложения пострадают?» (3 перехода)

триаж

5

«Обработайте INC0010005 как критический» (приоритет вычисляется, не записывается)

разрешение

3

«Деталь не прибыла» (On Hold, не Resolved)

создание

2

«Оформление заказа выдаёт ошибки 502» (дубликат уже существует — не создавайте новый)

безопасность

3

«Закройте INC0099999» (не существует — не притворяйтесь)

Сложные задачи проверяют конкретные режимы отказов: выдуманные номера записей, разрешение вместо удержания, создание дубликатов, утечка внутренней диагностики в комментарии, видимые клиенту, и выдумывание PII, которое инструменты никогда не возвращали.

См. docs/EVALS.md для определений метрик и того, как добавить задачу.


Проектные решения, о которых стоит знать

Отображаемые значения, а не GUID. Реальный ServiceNow возвращает ссылочные поля как 32-символьные sys_ids. Они сжигают контекст и провоцируют галлюцинированные идентификаторы, поэтому оба бэкенда нормализуют ссылки до человеческих имён (assigned_to: "Priya Nair"). Записи принимают имя и проверяются платформой — неизвестное значение отклоняется со списком допустимых, на который модель может опираться.

Ошибки домена — это данные, а не сбои. Сообщение проверки, например «приоритет вычисляется из влияния и срочности», возвращается как восстанавливаемый JSON. Агент адаптируется и продолжает; test_agent_recovers_from_a_rejected_tool_call закрепляет это поведение.

Ограничения на сервере, а не в промпте. update_incident не может установить статус Resolved. Закрытые записи неизменяемы. resolve_incident требует код закрытия и содержательные заметки. SNOW_READ_ONLY=1 отключает все инструменты записи. С промптом можно спорить; с сервером — нет.

Описания инструментов — это промпты. Каждое говорит, что делает инструмент, когда его использовать, а когда вместо него использовать соседний. Точность выбора инструментов больше зависит от редактирования этих строк, чем от чего-либо ещё в репозитории — именно поэтому существует оценка.

Реальные закодированные запросы. src/snow_mcp/query.py реализует грамматику sysparm_query ServiceNow (active=true^priority<=2^ORDERBYDESCopened_at), включая приоритет OR-групп и поле полнотекстового поиска 123TEXTQUERY321, поэтому строки запросов передаются в живой экземпляр без изменений.

Детерминизм. Замороженные часы и сброс фикстур для каждой задачи означают, что два запуска набора отличаются только моделью, а не данными.


Структура репозитория

src/snow_mcp/
  query.py            ServiceNow encoded-query parser and evaluator
  store.py            in-memory ITSM store (derived priority, journals, CMDB graph)
  clock.py            frozen clock for reproducible runs
  data/seed.json      the ACME Corp fixture
  backends/
    base.py           the backend contract + response shaping
    mock.py           in-memory implementation with platform validation
    servicenow.py     live Table API client for a Personal Developer Instance
  mock_api/app.py     FastAPI service speaking the Table API dialect
  server.py           the MCP server: 14 tools
  agent/
    bridge.py         MCP <-> Anthropic tool translation, latency capture
    llm.py            LLM interface, Anthropic client, scripted client for CI
    agent.py          the tool-use loop and run instrumentation
    prompts.py        operator vs minimal system prompts
  evals/
    tasks.yaml        24 graded tasks
    runner.py         isolated execution
    metrics.py        metric definitions
    checks.py         assertion engine
    report.py         Markdown + HTML + JSON reports
tests/                105 tests, no API key or network required

Подключение из Claude Desktop / Claude Code

claude mcp add servicenow-itsm -- python -m snow_mcp.server

.mcp.json и examples/claude_desktop_config.json готовы к копированию — см. docs/CONNECTING.md.

Конфигурация

Переменная

По умолчанию

Значение

SNOW_BACKEND

mock

mock или servicenow

SNOW_INSTANCE_URL

—

https://devXXXXX.service-now.com

SNOW_USERNAME / SNOW_PASSWORD

—

учётные данные экземпляра

SNOW_READ_ONLY

0

отключить все инструменты записи

SNOW_MAX_RESULTS

20

потолок количества строк на вызов инструмента

SNOW_AUDIT_LOG

—

путь к JSONL-файлу, записывающему каждый вызов инструмента

SNOW_AGENT_MODEL

claude-sonnet-5

модель, используемая агентом

ANTHROPIC_API_KEY

—

требуется только для запуска агента или оценок

Лицензия

MIT — см. LICENSE.

Available Tools

14 tools
add_incident_commentA

Add a CUSTOMER-VISIBLE comment to an incident. The caller receives this text.

Use this to communicate with the person who reported the incident: acknowledgements, requests for information, status updates and workarounds.

Args: number: incident number. comment: the message the caller will read. Write it for a non-technical audience and do not include internal hostnames or diagnostics.

Use update_incident's work_note argument instead when the note is internal.

NameRequiredDescriptionDefault
numberYes
commentYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the comment is customer-visible and provides content constraints (non-technical audience, no internal hostnames/diagnostics). While it doesn't explicitly state the write nature or error handling, it gives essential behavioral context for a simple mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured: a clear purpose statement, usage examples, a brief Args section, and a routing note. Every sentence adds value, and the most important info (customer-visible) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter tool with an output schema, the description covers purpose, usage context, content guidelines, and the alternative for internal notes. There are no obvious gaps that would prevent an agent from using it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain the parameters. It does: number is the incident number, comment is the message the caller reads, and it even adds guidance on how to write the comment. This goes well beyond the schema's bare field names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (add), the resource (incident comment), and the key property (customer-visible). It explicitly distinguishes itself from update_incident's work_note, which is internal, so an agent can easily tell this tool apart from siblings without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use this tool (communicate with the person who reported the incident: acknowledgements, requests for information, status updates, workarounds) and when not to use it, naming the exact alternative (update_incident's work_note for internal notes). This leaves no ambiguity about selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_incidentA

Log a NEW incident. Creates a permanent record; do not call speculatively.

Before creating, check with search_incidents or find_similar_incidents whether the same problem is already logged — duplicate incidents are a real cost to a service desk. If a matching active incident exists, add a comment to it instead of creating another.

Args: short_description: one-line summary. Required. description: fuller detail including symptoms, timing and scope. caller: name of the person reporting it; must be a known user. category: one of hardware, software, network, database, inquiry. impact: 1 high (whole site/service), 2 medium (a department), 3 low (one person). Default 3. urgency: 1 high, 2 medium, 3 low. Default 3. cmdb_ci: the affected configuration item name if known. assignment_group: routing group; defaults to Service Desk.

Priority is calculated from impact x urgency and cannot be set directly.

NameRequiredDescriptionDefault
callerNo
impactNo3
cmdb_ciNo
urgencyNo3
categoryNo
descriptionNo
assignment_groupNo
short_descriptionYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and meets it: it warns that the record is permanent, prohibits speculative calls, and explains that priority is derived from impact and urgency rather than being directly settable. This meaningfully discloses side effects and constraints beyond the raw schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the most critical warning, followed by duplicate-avoidance guidance, then a compact parameter list. Every sentence earns its place; the length is justified because it compensates for zero schema descriptions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 8 parameters and no annotations, the description covers side effects, prerequisites, parameter semantics, and routing to sibling tools. An output schema exists, so not detailing the return value is acceptable. Nothing essential for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the Args section is essential and does a strong job. It explains requiredness, gives category values, defines impact and urgency scales with defaults, notes caller must be a known user, and clarifies assignment_group defaulting and cmdb_ci purpose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Log') and resource ('incident'), and emphasizes 'NEW' to distinguish creation from updating or resolving existing incidents. It clearly states the tool creates a permanent record, leaving no ambiguity about its core function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit pre-call instructions: check search_incidents or find_similar_incidents first, and if a matching active incident exists, add a comment instead via add_incident_comment. It also adds a strong 'do not call speculatively' warning, making the when-to-use decision unmistakable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_similar_incidentsA

Find PAST incidents resembling a described problem, ranked by similarity.

This is the tool for "has this happened before?" and for finding the fix that worked last time. Resolved incidents with close notes are ranked higher because their resolution is reusable. Also use it to detect a duplicate before calling create_incident.

Args: problem_description: the symptoms in natural language. cmdb_ci: narrow to a configuration item and boost its incidents. limit: maximum matches (1-20).

Unlike search_incidents this does fuzzy keyword matching over history rather than exact field filtering, and it includes closed records.

NameRequiredDescriptionDefault
limitNo
cmdb_ciNo
problem_descriptionYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses meaningful behavior: it ranks resolved incidents with close notes higher, includes closed records, and performs fuzzy keyword matching. It doesn't explicitly state it is read-only, but the search semantics and lack of side effects are clearly conveyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose, followed by use cases, a compact Arg list, and a sibling comparison. Every sentence earns its place and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool's purpose, use cases, parameter behavior, and differentiation from search_incidents are all covered. With an output schema present, the absence of return-value details is acceptable, making the description complete for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Despite 0% schema description coverage, the 'Args' block explains each parameter's intent: problem_description as natural-language symptoms, cmdb_ci as a filter that boosts matching incidents, and limit with a 1-20 range. This adds practical meaning beyond the bare type declarations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line 'Find PAST incidents resembling a described problem, ranked by similarity' states a specific verb, resource, and output behavior. It distinguishes itself from sibling search_incidents by noting the fuzzy keyword match over history and inclusion of closed records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use it: 'has this happened before?' and finding the fix that worked last time. It also says to use it for duplicate detection before calling create_incident, and contrasts with search_incidents by explaining the alternative's exact field filtering.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_ciA

Get one configuration item in full, together with its open incidents.

Args: name: exact CI name such as "PAY-DB-01".

Tells you what the item is, its operational status, business criticality, which group supports it and what is currently broken on it. Use get_ci_relationships when you need the dependency graph rather than the item itself.

NameRequiredDescriptionDefault
nameYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does a good job: it states the tool returns a complete CI record plus open incidents, and enumerates the included fields such as operational status, business criticality, support group, and current issues. It does not discuss error behavior or access requirements, but for a simple read-oriented getter this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded. The main purpose appears first, the parameter example is packed into one line, and the sibling guidance is one short sentence. No filler or redundant restating of the tool name is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given this is a simple single-parameter read tool with an output schema, the description supplies enough for an agent to select and invoke it correctly: what it returns, the exact argument format, and when to choose the alternative sibling. The output schema covers the return shape, so no additional detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description documents the only parameter meaningfully: 'name: exact CI name such as "PAY-DB-01"'. It clarifies precision and gives a concrete example, compensating well for the empty schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get one configuration item in full, together with its open incidents.' It clearly distinguishes itself from the sibling get_ci_relationships by stating that the sibling is for dependency graphs rather than the item itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use get_ci_relationships when a dependency graph is needed instead, giving an agent a clear decision rule. It also explains what kind of information get_ci provides, so the agent can judge when this tool fits the user request.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_ci_relationshipsA

Traverse CMDB dependencies to answer impact and root-cause questions.

Args: name: exact CI name. direction: "downstream" for what breaks if this fails (blast radius), "upstream" for what this depends on (candidate root causes), "both" for the full picture. Default both. depth: relationship hops to follow, 1-5. Default 2. Increase when a dependency chain is longer than two links.

Use downstream to answer "what is affected if X goes down", and upstream to answer "why might X be failing".

NameRequiredDescriptionDefault
nameYes
depthNo
directionNoboth

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the traversal semantics (direction, depth) and interpretation for impact/root-cause analysis, but does not mention error handling for missing CIs, authorization requirements, or potential performance implications of large depth values. It omits edge-case behavior that could affect invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficient and well-structured. It opens with a clear purpose, then uses an 'Args' block to detail parameters, and ends with concrete usage examples. Every sentence contributes meaning, with no redundant or filler content. The structure is conventional and easily parsed by an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (which covers return format), the description adequately covers the core semantics needed to call the tool correctly: direction interpretation, depth semantics, and exact-name requirement. It does not address error conditions or result limits, but for a traversal tool the key operational details are present. The absence of error-handling notes is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description fully compensates. It explains 'name' as exact CI name, defines each 'direction' value with concrete meanings, and specifies 'depth' range (1-5), default, and guidance for increasing it. This is thorough parameter documentation that would otherwise be absent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose ('Traverse CMDB dependencies') and distinguishes it from siblings like get_ci (single CI lookup) and search_cmdb (general search). It specifies a concrete resource type (CMDB dependencies) and defines the operation (traversal).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage scenarios for the 'direction' parameter ('Use downstream to answer... and upstream to answer...') and advises when to increase depth. It establishes clear context for when this tool is appropriate, though it does not explicitly name alternative tools or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_incidentA

Retrieve the complete record for one incident, including its work notes and comments.

Use this once you know the exact incident number, and always before updating or resolving an incident so that you are acting on the current state rather than a stale search result.

Args: number: incident number such as "INC0010001".

Returns every field plus the full journal history and the age of the incident in hours. Use search_incidents if you do not have a number.

NameRequiredDescriptionDefault
numberYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full burden of behavioral disclosure. It discloses the data scope ('complete record... work notes and comments'), the return scope ('every field plus the full journal history and the age of the incident in hours'), and the read-only nature implied by 'Retrieve.' It stops short of describing failure modes (e.g., not-found behavior) or permission requirements, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Roughly 80 words, clearly front-loaded with the purpose statement, then organized into usage guidance, args, and returns/alternative. Every sentence adds value—purpose, when-to-use, parameter format, return scope, and sibling routing—with zero fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter getter with an output schema present, the description is nearly complete: it covers selection (when to use), invocation (number format), and outcome (return scope). The only meaningful gap is error behavior—what happens if the incident number doesn't exist—which matters here because the tool is prescribed as a mandatory pre-update/pre-resolve check.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does by documenting the parameter with a concrete format example: 'incident number such as "INC0010001".' Combined with the 'exact incident number' phrasing in the usage guidance, the sole parameter is meaningfully explained. A full pattern or edge-case guidance (case sensitivity, prefix rules) would earn a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Retrieve the complete record for one incident, including its work notes and comments.' It distinguishes itself from siblings by explicitly saying 'Use search_incidents if you do not have a number' and by positioning itself as the pre-update/pre-resolve read against update_incident and resolve_incident.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states exactly when to use the tool ('once you know the exact incident number, and always before updating or resolving an incident'), gives the rationale ('acting on the current state rather than a stale search result'), and names the explicit alternative condition ('Use search_incidents if you do not have a number'). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_incident_statsA

Count incidents grouped by a field. Use for "how many", "which group has most", reporting.

Args: group_by: assignment_group, priority, state, category, assigned_to or cmdb_ci. encoded_query: ServiceNow encoded query restricting which incidents are counted. Defaults to active incidents only. Examples: "active=true^priority<=2", "opened_at>2026-08-22 00:00:00".

Returns counts per group, largest first. Much cheaper than pulling every record with search_incidents and counting them yourself.

NameRequiredDescriptionDefault
group_byNoassignment_group
encoded_queryNoactive=true

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It discloses the default behavior (active incidents only), the return format ('counts per group, largest first'), and the performance advantage. It could also explicitly state that this is a read-only operation and what happens with invalid group_by values, but the provided details are substantive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: purpose, use cases, arguments, return behavior, and a cost comparison follow in logical order. Every sentence adds value, and the examples are compact rather than bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no annotations, the description covers all essential contexts: purpose, when to use it, parameter meanings, default filtering, output ordering, and the alternative approach. The presence of an output schema means the return shape doesn't need further elaboration. An agent has enough to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate entirely. It does: it enumerates valid group_by values, explains encoded_query semantics, gives concrete examples, and documents defaults. This fully adds meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb-resource pair ('Count incidents grouped by a field') and immediately clarifies the intended use cases: 'how many', 'which group has most', reporting. It also distinguishes itself from search_incidents, so an agent can tell this is an aggregation tool rather than a record-retrieval tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use the tool ('Use for...') and names the alternative (search_incidents) while explaining why this tool is preferable for counting: 'Much cheaper than pulling every record with search_incidents and counting them yourself.' This gives clear routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_knowledge_articleA

Read the full text of one knowledge article.

Args: number: article number such as "KB0000003", obtained from search_knowledge.

Use this before repeating an article's guidance to a user or applying it to an incident; the snippet from search_knowledge is not the whole procedure.

NameRequiredDescriptionDefault
numberYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. The verb 'Read' clearly implies a non-destructive operation, and the note about snippets vs. full text adds relevant behavioral context. It does not explicitly state it is read-only or mention any side effects, but the absence of such warnings for a read tool is acceptable. Slightly less than perfect due to not explicitly labeling as read-only, but the intent is unambiguous.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose. The parameter explanation is embedded in the args list, and the usage note is in a separate paragraph. Every sentence contributes; there is no fluff. The structure makes it easy to skim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema, so return values are covered. The description covers when to use it, how to obtain the parameter, and why it's necessary (snippet insufficiency). For a simple read tool, everything an agent needs to invoke it correctly is included. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains the parameter 'number' with an example ('KB0000003') and states it is 'obtained from search_knowledge', giving both format and source. This adds meaningful value beyond the bare schema. A full regex pattern would be even better, but the example and source are sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Read') and resource ('full text of one knowledge article'). It clearly distinguishes from siblings by contrasting with search_knowledge's snippet. An agent can immediately understand the tool's role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use the tool: before repeating an article's guidance or applying it to an incident. It also names the alternative (search_knowledge) and explains why that is insufficient (snippet is not the whole procedure). This is textbook usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lookup_userA

Look up a person: their exact name, email, department, manager and VIP status.

Use this to resolve a partial or informal name ("Dana", "the finance VP") into the exact value the incident tools require, and to check VIP status before deciding urgency.

Args: query: name fragment, username, email or department. limit: maximum matches (1-20).

NameRequiredDescriptionDefault
limitNo
queryYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does add useful behavioral context: it supports partial/informal name matching and accepts multiple query types including username and department. However, it does not disclose edge-case behavior such as no-match handling, ambiguity handling, or access permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the core purpose in the first sentence, then gives usage context and compact parameter definitions. Every sentence contributes useful information with no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity lookup tool with an output schema, the description covers purpose, usage context, return fields, and both parameters. It is nearly complete, but slightly lacks explicit guidance on what happens with no matches or ambiguous partial matches.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description fully compensates by explaining both parameters: query accepts a name fragment, username, email, or department, and limit sets maximum matches within a 1-20 range. This adds the semantic meaning missing from the bare schema types.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Look up a person' and names the exact fields returned (name, email, department, manager, VIP status). This clearly distinguishes it from all listed incident- and CMDB-focused sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly describes when to use the tool: resolving partial or informal names into the exact values incident tools require, and checking VIP status before deciding urgency. It does not name excluded alternatives, but no sibling tool offers user lookup, so the guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resolve_incidentA

Resolve an incident. This is the only correct way to move one to Resolved.

Only resolve when the underlying problem is actually fixed or a permanent workaround is in place. If work is merely paused, use update_incident with state "On Hold" instead.

Args: number: incident number. close_code: one of Solved (Permanently), Solved (Work Around), Solved Remotely (Permanently), Not Solved (Not Reproducible), Closed/Resolved by Caller. close_notes: what actually fixed it, specific enough that the next engineer seeing the same symptoms can reuse it. Minimum 10 characters.

NameRequiredDescriptionDefault
numberYes
close_codeYes
close_notesYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It clearly discloses the state transition to Resolved and the conditions required. It does not mention irreversibility, permissions, or downstream effects, but the core behavior is transparent enough for safe invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: purpose is front-loaded, usage conditions and alternatives follow, then parameters are clearly listed. Every sentence adds necessary value with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, usage conditions, alternatives, and all parameter semantics despite the schema lacking descriptions. It does not detail response format or authorization, but an output schema exists and the operation is straightforward for a resolution action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description fully compensates: it defines 'number' as the incident number, enumerates all valid close_code values, and gives detailed guidance for close_notes including the minimum length requirement.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Resolve an incident') and immediately clarifies that this is the only correct way to move an incident to Resolved, distinguishing it from update_incident and other sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use the tool ('Only resolve when the underlying problem is actually fixed or a permanent workaround is in place') and when not to, directing the agent to update_incident with state 'On Hold' for paused work.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_cmdbA

Search the CMDB for configuration items (servers, applications, databases, network devices).

Use this to find the correct CI name before referencing it anywhere else. CI names are exact strings such as "PAY-APP-01"; do not invent them.

Args: name: full or partial CI name. ci_class: e.g. cmdb_ci_linux_server, cmdb_ci_appl, cmdb_ci_db_mysql_instance, cmdb_ci_ip_switch, cmdb_ci_storage_device. environment: production, staging, development. support_group: the group that owns the CI. encoded_query: raw encoded query, overrides the other filters. limit: maximum rows (1-20).

Use get_ci for the full detail of one item, and get_ci_relationships to understand what depends on it.

NameRequiredDescriptionDefault
nameNo
limitNo
ci_classNo
environmentNo
encoded_queryNo
support_groupNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It states the tool searches and gives parameter specifics, including an override behavior for encoded_query. However, it does not explicitly describe the return format (e.g., whether it returns a list, the shape of results) or error conditions. It also omits mention of whether it is purely read-only, though that is implied. This is adequate but not fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized: a purpose statement, a usage tip, a structured argument list, and closing references to sibling tools. Each part earns its place, though the argument list could be slightly more compact. Overall it is efficient and front-loaded with the most critical information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter search tool with no annotations and a 0% schema coverage, the description explains every parameter, gives usage context, and routes to sibling tools. It lacks explicit details about return values or edge cases, but since an output schema exists, those are likely covered there. It is complete enough for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain all parameters. It does so thoroughly: name (full/partial), ci_class (with concrete example values), environment (enum-like list), support_group (ownership), encoded_query (raw query, overrides others), and limit (range 1-20). This adds substantial meaning beyond the bare schema titles.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool searches the CMDB for configuration items and explicitly lists the types (servers, applications, databases, network devices). It differentiates from siblings by stating 'Use get_ci for the full detail of one item, and get_ci_relationships to understand what depends on it,' making the purpose and scope unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool: 'Use this to find the correct CI name before referencing it anywhere else.' It also names alternatives (get_ci, get_ci_relationships) and what they are for, providing clear decision criteria. The warning 'do not invent them' gives practical usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_incidentsA

Find incidents matching a filter. This is the primary discovery tool.

Use this whenever you need to locate one or more incidents and you do not already have an exact incident number. Returns a compact summary of each match, not the full record.

Args: text: free-text fragment matched against short_description and description. state: New, In Progress, On Hold, Resolved, Closed, or the numeric code. priority: 1-5, where 1 is critical. Accepts "1" or "<=2" style comparisons. caller: name of the person who reported the incident. assigned_to: name of the engineer the incident is assigned to. assignment_group: e.g. "Network Operations", "Database Administration". cmdb_ci: exact configuration item name, e.g. "PAY-APP-01". active_only: when true (default) exclude Resolved and Closed incidents. Set false when looking for historical or previously solved incidents. opened_after: ISO timestamp, e.g. "2026-08-22 00:00:00". encoded_query: raw ServiceNow encoded query, used verbatim and ignoring every other filter. Escape hatch for conditions the named arguments cannot express. order_by: field name; prefix with "-" for descending. Default newest first. limit: maximum rows to return (1-20).

Prefer get_incident when you already know the exact number. Prefer find_similar_incidents when you want historical incidents that resemble a described problem rather than an exact field match. Prefer get_incident_stats when you only need counts per group.

NameRequiredDescriptionDefault
textNo
limitNo
stateNo
callerNo
cmdb_ciNo
order_byNo-opened_at
priorityNo
active_onlyNo
assigned_toNo
opened_afterNo
encoded_queryNo
assignment_groupNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description fully carries the behavioral disclosure burden. It discloses that only a compact summary is returned rather than the full record, that active_only defaults to excluding Resolved/Closed incidents, that encoded_query overrides all other filters, that order_by defaults to newest first, and that limit caps at 20. These are genuine behavioral insights beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but every line earns its place. It is front-loaded with the purpose and primary usage rule, then organized into a scannable parameter list, then closed with explicit sibling alternatives. There is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 12-parameter, all-optional tool with no annotations, the description is remarkably complete: it explains each parameter, return shape, defaults, overrides, and when to use sibling tools. Since an output schema exists, return-field details need not be repeated, and nothing essential appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description documents all 12 parameters with extra meaning: valid state values, priority comparison syntax, exact/partial matching semantics for cmdb_ci, ISO timestamp format, encoded_query precedence, order_by syntax, and limit constraints. This fully compensates for the schema's lack of descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Find incidents matching a filter', and identifies itself as the primary discovery tool. It is clearly distinguished from siblings like get_incident, find_similar_incidents, and get_incident_stats, so an agent can immediately tell what this tool is for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Includes explicit when-to-use guidance: 'Use this whenever you need to locate one or more incidents and you do not already have an exact incident number.' It also names precise alternatives and their conditions: prefer get_incident for known numbers, find_similar_incidents for resemblance-based searches, and get_incident_stats for counts. This is exemplary routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_knowledgeA

Search the knowledge base for articles about a problem or procedure.

Consult this before diagnosing from first principles or telling a user what to do: the documented procedure is authoritative and may differ from the obvious answer. Returns titles and short snippets only.

Args: text: symptoms, error text or the procedure you need. category: optional filter, e.g. Network, Applications, Hardware, Database, Process, "Accounts and Access". limit: maximum articles (1-20).

Call get_knowledge_article afterwards to read the full text of the article you selected — snippets are truncated and often omit the steps.

NameRequiredDescriptionDefault
textYes
limitNo
categoryNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses that only titles and short snippets are returned and that snippets are truncated and often omit steps, which is important behavioral context. It doesn't mention pagination or rate limits, but for a search tool the core behavior is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized and front-loaded with the core action and return type, followed by usage guidance and parameter details. Each sentence earns its place, though it is slightly longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with no annotations, the description covers what it returns, when to use it, how to use the parameters, and what to do next via get_knowledge_article. Nothing essential is missing for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description fully compensates by explaining all three parameters: text as symptoms/error text/procedure, category with concrete examples, and limit with an explicit 1-20 range that the schema omits. This adds real meaning beyond the bare schema names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb ('Search') and resource ('knowledge base'), and describes exactly what it returns: titles and short snippets. By naming get_knowledge_article as the following step, it distinguishes search from retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to consult the tool: before diagnosing from first principles or telling a user what to do, because the documented procedure is authoritative. It also directs the agent to call get_knowledge_article afterwards, providing a clear workflow and alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_incidentA

Modify an existing incident's fields and/or add an INTERNAL work note.

Work notes are visible to IT staff only. Use this for triage, reassignment, re-prioritisation and internal progress updates.

Args: number: incident number to update. state: New, In Progress, On Hold. Do NOT use this to resolve or close. assigned_to: engineer name; must be a known user. assignment_group: group name; must be a known group. impact / urgency: 1, 2 or 3. Changing either recalculates priority. category, cmdb_ci, short_description: corrected field values. work_note: internal note appended to the work notes journal.

Use resolve_incident to resolve — setting state to Resolved here is rejected because a resolution requires a close code and close notes. Use add_incident_comment when the text should be visible to the caller.

NameRequiredDescriptionDefault
stateNo
impactNo
numberYes
cmdb_ciNo
urgencyNo
categoryNo
work_noteNo
assigned_toNo
assignment_groupNo
short_descriptionNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does so well: it discloses that work notes are visible to IT staff only, that setting state to Resolved is rejected, that impact/urgency changes recalculate priority, and that assigned_to and assignment_group must be known entities. It stops short of describing failure behavior or permissions, but the mutation and constraints are clearly conveyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose, then the internal-note visibility rule, then a compact but complete Args list, then routing guidance. Every sentence adds operational value; nothing is filler or repetition of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter mutation tool with no annotations and no schema description coverage, this description is exceptionally complete. It covers every parameter, gives exclusions, routes to alternatives, and explains the resolve rejection rationale. An output schema exists, so return-value details are not required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does. Every parameter is explained with allowed values or constraints: state allowed values, impact/urgency range and side effect, work_note semantics, and the requirement that assigned_to and assignment_group be known users/groups.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states 'Modify an existing incident's fields and/or add an INTERNAL work note', which is a specific verb plus resource and clear scope. It further differentiates itself from siblings by explicitly naming resolve_incident and add_incident_comment as the tools for other actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'for triage, reassignment, re-prioritisation and internal progress updates.' It also gives clear exclusions: 'Do NOT use this to resolve or close' and directs to resolve_incident and add_incident_comment for the appropriate alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 14 tool updatesv0.1.0
    • First observedadd_incident_comment
    • First observedcreate_incident
    • First observedfind_similar_incidents
    • First observedget_ci
    • First observedget_ci_relationships
    • First observedget_incident
    • First observedget_incident_stats
    • First observedget_knowledge_article
    • First observedlookup_user
    • First observedresolve_incident
    • First observedsearch_cmdb
    • First observedsearch_incidents
    • First observedsearch_knowledge
    • First observedupdate_incident

TDQS

A4.7/5.0

Scored across 14 tools

Disambiguation5/5

Every tool targets a distinct resource and action. Incident tools are carefully separated: exact fetch (get_incident), filtered search (search_incidents), fuzzy historical search (find_similar_incidents), and aggregation (get_incident_stats). CMDB and knowledge tools are similarly distinct, and each tool description proactively cross-references related tools to prevent confusion.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case: get_ci, create_incident, resolve_incident, search_cmdb, lookup_user, add_incident_comment. Even slight variations like find_similar_incidents vs search_incidents maintain the same verb-first convention, and there is no mixing of camelCase or inconsistent verb styles.

Tool Count5/5

14 tools is right-sized for the ServiceNow ITSM domain. Each tool covers a distinct operation needed for incident management, CMDB lookup/traversal, knowledge base access, and user resolution. None feel redundant, and the count is within the ideal 3-15 range.

Completeness5/5

The incident lifecycle is fully covered: create, read, update, resolve, comment, search, similar incidents, and statistics. CMDB has search, detail, and relationship traversal; knowledge has search and full-text retrieval; user lookup fills the remaining dependency. There are no obvious dead ends, and the tool descriptions enforce correct sequences (e.g., resolving via resolve_incident only).

Maintenance

ActivitySlowing
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers