Systems Intelligence Performative Commercial Benchmarking
MO§ES™ Enterprise — AI 操作者评估平台
在线访问:mos2es.org — 面向人机操作系统的系统智能。 AI 操作者在性能指标与基准测试领域的新标准。 为日常操作和 AI 工作流建立系统智能基线。 无内容的 token 遥测。绝不含你的提示词。
企业级 AI 操作者评估。衡量人们如何操作 AI——而非 AI 模型本身。
MO§ES™ 负责为企业构建系统智能——衡量人们在工具、任务、工作流和各类场景中驱动 AI 系统的效率。正如 BI 看见业务本身,MO§ES™ 看见业务如何操作 AI。
目录
Related MCP server: Signomy
什么是 MO§ES™?
MO§ES™ 是一个企业级 AI 操作者评估平台。它衡量人们如何操作 AI 系统——而非 AI 模型本身、使用量或自我报告的能力。借助无内容的 token 遥测技术(输入、输出、缓存读取、缓存写入——不包含提示词文本,也不包含响应文本),MO§ES™ 在以下层面建立系统智能基线:
操作者 — 个人驱动 AI 的效能
团队 — 队列分布与能力拓扑
工作流 — AI 在工作中的嵌入位置,而不仅仅是是否被使用
组织 — 跨团队基准测试与能力映射
该平台围绕你的工作流、角色和 AI 系统构建定制化评估,对照内部与外部参考人群进行性能基准测试,诊断能力差距与发散模式,测试有针对性的干预措施,并重新衡量变化。
定位
BI 看见业务本身。MO§ES™ 看见业务如何操作 AI。
LMSYS 对模型进行基准测试。Braintrust 评估产品输出。Langfuse 追踪 LLM 调用。WakaTime 记录时间。CostHawk 记录支出。它们都无法看见日常操作与 AI 工作流中的系统智能。MO§ES™ 正是缺失的那一层。
MO§ES™ 生态系统
仓库 / 站点 | 是什么 | URL |
MO§ES™ Enterprise(本仓库) | 该平台——Python 评估引擎、CLI、MCP 服务器、演示、宣传页、企业演示、Workers | |
SigRank SignalAF | 公开排行榜——按 token 级联效率对操作者进行排名 | |
SigRank MCP | 检测工具——提取 token 支柱、计算级联、提交到排行榜 |
|
SIGNOMY | 受治理的 AI 代理市场——排名代理组建团队、执行任务、赚取收入 | |
SigEconomy | 公开 LLM 操作者评估——只读排行榜、SEO/AEO 入口 |
线上入口
入口 | URL | 说明 |
宣传站点 | 营销、方法论、演示、对比、预约 | |
企业演示 | 交互式产品演示(评估 → 诊断 → 工作流 → 对比) | |
MCP 服务器 | 27 个工具的 MCP 服务器(22 个读取 + 5 个写入),支持流式 HTTP | |
MCP 服务器信息 | 服务器信息 JSON(版本、工具数量、传输方式) | |
MCP 服务器卡片 | 包含所有工具模式的完整服务器卡片 | |
OpenAPI 规范 | REST API 规范 | |
LLM 指引 | 供 AI 代理和爬虫使用的 llms.txt | |
站点地图 | XML 站点地图 | |
预约演示 | B2B 演示预约,含结构化信息收集表单 |
快速开始
运行演示(一条命令,无需克隆)
curl -sL https://mos2es.org/demo/run.py | python3 -这条命令将仓库克隆到临时目录,安装 rich,并运行完整的 11 步演示流程。需要 Python 3.10+ 和 git。
克隆并运行
git clone https://github.com/SunrisesIllneverSee/Moses_Enterprise_B2BPilot_.git
cd Moses_Enterprise_B2BPilot_/_01_platform
pip install rich
python3 -m src.cli.main demo full运行测试套件
cd _01_platform
python3 -m pytest tests/ -q
# 527 tests pass在本地运行 MCP 服务器
cd _01_platform
pip install mcp
python3 -m src.mcp_server.server使用远程 MCP 服务器
添加到 Claude Desktop 配置中(macOS 上为 ~/Library/Application Support/Claude/claude_desktop_config.json):
{
"mcpServers": {
"moses": {
"url": "https://mcp.mos2es.org/mcp",
"transport": "http"
}
}
}或直接调用工具:
curl -s -D /tmp/h -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' > /dev/null
SID=$(grep -i "^mcp-session-id:" /tmp/h | tr -d '\r\n' | sed 's/.*: //')
curl -s -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" -H "MCP-Session-Id: $SID" \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | python3 -m json.tool11 步演示流程
步骤 | 名称 | 说明 |
1 | LOAD | 加载 50 名操作者的合成队列 |
2 | EVALUATE | 计算每次观测的指标 |
3 | BENCHMARK | 计算百分位排名 |
4 | DIAGNOSE | 运行模式检测器 |
5 | OPERATOR×SYSTEM | 分解操作者与系统的影响 |
6 | INTERVENE | 分配有针对性的干预措施 |
7 | RE-EVALUATE | 干预后重新衡量 |
8 | OUTCOME LINEAGE | 追踪观测 → 转换 → 产物 → 结果 |
9 | REPORT | 生成 Markdown + PDF 试点报告 |
10 | VISUALIZE | 生成 9 张架构图 |
11 | DASHBOARD | 导出高管 HTML 仪表板 |
演示完全基于合成数据运行。没有真实操作者、真实提示词文本或真实 API 调用。结果具有确定性——运行两次会产生完全相同的结果。
5 项标准指标
指标 | 衡量内容 |
Yield | AI 输出中最终进入已提交状态的比例 |
Leverage | 每消耗一个 token 所产生的输出价值 |
Token SNR | token 使用中的信噪比 |
Construction | 通过增量构建而非替换的输出占比 |
Divergence | 使用模式与操作模式之间的差距 |
15 个评估家族。13 个基准类别。4 个测量层级(操作者、团队、工作流、组织)。
仓库结构
_01_platform/ Python platform (the product)
src/ Source code (domain, metrics, analysis, diagnostics, CLI, MCP server)
tests/ 527 tests
demo_data/ Synthetic demo cohort (50 operators, 1,668 observations)
scripts/ Utility scripts
schemas/ JSON schemas
_02_demo-website/ Enterprise demo site (enterprise.mos2es.org)
_03_promo-site/ Promo site (mos2es.org)
vs/ 16 competitor comparison pages
alternatives/ 8 alternatives listicle pages
concepts/ 10 concept explainer pages
guides/ 4 how-to guides
demo/run.py One-liner demo runner
.well-known/mcp.json MCP server discovery
_04_onepager/ One-pager site
_workers/ Cloudflare Workers
promo-worker/ mos2es.org worker (static assets + AEO/SEO/GEO headers)
moses-worker/ enterprise.mos2es.org worker (static assets)
onepager-worker/ One-pager worker (static assets)
mcp-worker/ MCP server worker (computes live from raw data, 27 tools)MCP 服务器
MCP 服务器通过流式 HTTP 在 https://mcp.mos2es.org/mcp 上提供 27 个工具(22 个读取 + 5 个写入)。公共服务器无需身份验证。
读取工具包括:队列统计、操作者画像、指标分布、基准排名、发散发现、干预结果、结果谱系、工作流适配度、组织拓扑等。
写入工具包括:创建干预、分配干预、记录结果、创建评估配置、创建试点配置。
服务器基于原始观测数据实时计算——没有预先计算的结果。每次调用都会运行实际的评分、基准测试和诊断引擎。
服务器卡片
curl -s https://mcp.mos2es.org/.well-known/mcp/server-card.json | python3 -m json.tool返回包含全部 27 个工具模式、传输信息和元数据的完整服务器卡片。
Cloudflare Workers
本仓库部署了四个 Workers:
Worker | 域名 | 说明 |
| mos2es.org | 宣传站点 + AEO/SEO/GEO 头 + 代理可发现性(llms.txt、sitemap、OpenAPI、MCP 链接) |
| enterprise.mos2es.org | 企业演示站点 |
| (workers.dev) | 单页宣传页 |
| mcp.mos2es.org | MCP 服务器(27 个工具,实时计算) |
部署
npm install -g wrangler
cd _workers/promo-worker && wrangler deploy
cd _workers/moses-worker && wrangler deploy
cd _workers/onepager-worker && wrangler deploy
cd _workers/mcp-worker && wrangler deployMCP worker 数据同步
MCP worker 从平台演示数据自动生成的原始数据文件(observations.js、lineages.js)进行实时计算。若要在更新演示数据后重新生成:
cd _workers/mcp-worker/src
python3 -c "
import json
with open('../../../_01_platform/demo_data/observations.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('observations.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
with open('../../../_01_platform/demo_data/lineages.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('lineages.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
"关键约定
**品牌:**MO§ES™(使用分节符 §)
所有结果声明均为 ASSOCIATION(关联性),绝非 CAUSATION(因果性),除非有受控实验支持
综合得分属于 DEVELOPMENTAL(发展性),绝非 PERSONNEL(人事性) —— 不得用于惩罚目的,不得用于员工排行榜
操作者相似度是指标相似度,而非性格匹配
无内容遥测 —— 观测中不包含提示文本、输出正文或代码内容
无提示内容监控 —— 仅凭 token 结构即可观察操作者表现
治理就绪 —— 每条输出均带有证据标签(DEVELOPMENTAL、HYPOTHESIS、ASSOCIATION)
互补,而非替代
MO§ES™ 与您现有的 BI、eval 套件、可观测性工具和生产力分析协同工作。它度量的是这些工具看不见的系统智能层。它不是替代品 —— 而是缺失的那一块。
五大支柱
# | 支柱 | 说明 |
1 | 系统智能 | MO§ES™ 以 BI 透视业务的方式,透视人-AI 操作系统。操作者表现、工作流契合度、工具选择、能力分布、干预结果 —— 全部从结构而非内容中度量。 |
2 | 标准操作绩效指标 | 5 项规范指标。15 个 eval 家族。13 个基准测试类别。4 个度量层级。一套开放、文档化的规范,用于度量人类如何操作 AI。参考实现为 MO§ES™。 |
3 | 定制企业级 Evals | 贵公司不应沿用他人对 AI 熟练度的定义。围绕您的工作流、角色、模型和绩效问题构建的 Evals。 |
4 | 互补,而非替代 | 与您现有的 BI、eval 套件、可观测性工具和生产力分析协同工作。是缺失的那一块,而非替代品。 |
5 | 隐私优先 / 治理 | 无内容 token 遥测。不含提示文本。无监控。每条输出均带有 DEVELOPMENTAL / HYPOTHESIS / ASSOCIATION 证据标签。开箱即用,治理就绪。 |
许可协议
PolyForm Noncommercial 1.0.0 —— 参见 LICENSE。允许个人使用、研究与非商业使用。商业使用须获得许可。专利申请中。
mos2es.org · 预约演示 · 运行演示 · MCP 服务器
由 Deric J. McHenry —— Ello Cello LLC 构建
Available Tools
27 toolsassign_interventionA
Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access.
| Name | Required | Description | Default |
|---|---|---|---|
| notes | No | Free-text notes about the intervention assignment | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| intervention_type | Yes | Intervention type from catalog (e.g., prompt_template, context_window_expansion, model_switch) |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | |
| error | Yes | |
| message | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey non-idempotent and non-destructive behavior, and the description adds a valuable behavioral constraint: authorization is required and pilot access must be arranged. It does not contradict the annotations, though it could say more about side effects of creating an assignment.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the first sentence states the action, and the second communicates the operational prerequisite and contact path. There is no redundant repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the full parameter schema, output schema, and annotations, the description covers the mechanics of invocation and the authorization barrier. However, it leaves selection context unaddressed—when to assign versus close an intervention—and offers limited detail on the assignment's effect, making it adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are fully documented in the input schema with descriptions and examples, including operator_id and intervention_type. The tool description itself adds no additional parameter semantics, so the baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the verb (assign), resource (targeted intervention), and recipient (operator), making the core operation clear. It is distinguishable from the sibling close_intervention by the opposing action, though it does not explicitly call out that contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no when-to-use guidance or alternative routing, and it does not mention relationships to siblings like close_intervention or create_experiment. The only usage-related information is the authorization prerequisite, which is not sufficient for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
attach_outcome_datasetA
Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION.
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | Data format (e.g., 'json', 'csv', 'jsonl') | |
| source | Yes | External outcome data source name (e.g., 'jira', 'github', 'linear') |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | |
| error | Yes | |
| message | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description adds meaningful behavioral context: it warns that outcome joins are association-only and explicitly flags that authorization is required. This is valuable for an agent deciding whether and how to invoke the tool. Annotations already cover idempotency and destructiveness, so the additional caveats push this above baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero filler. It front-loads the primary purpose and then packs the most important constraints—association-not-causation and authorization—into a compact, readable warning. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, full schema coverage, and the presence of an output schema, the description covers the key behavioral caveats an agent needs. It could theoretically mention the effect on downstream analysis or concrete when-not-to-use scenarios, but it is sufficiently complete for a tool that attaches a dataset.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters, including examples. The description adds no parameter-level detail, which is acceptable because the schema carries the burden. Per the baseline for high schema coverage, a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and resource: 'Attach external outcome dataset for join analysis.' It clearly conveys what the tool does and is distinguishable from the sibling tools, which are mostly read/get operations. It does not explicitly contrast itself with a specific sibling, but the verb+resource combination is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when an external outcome dataset needs to be attached for join analysis, but it does not explicitly state when to use this tool over alternatives or mention any exclusions. The causation warning and authorization requirement provide context, but no comparative guidance is given versus siblings like get_outcome_correlation or create_experiment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
close_interventionA
Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| outcome_notes | No | Free-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved | |
| intervention_id | Yes | Intervention ID to close (e.g., intv_001, intv_007). Must be an active intervention. |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| status | No | New status of the intervention (e.g., 'closed') |
| message | Yes | Human-readable status message |
| closed_at | No | ISO timestamp of closure (on success) |
| intervention_id | No | ID of the closed intervention (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent=false and destructive=false, and the description adds significant behavior: the intervention becomes ineligible for verify_change after closing, and in the synthetic demo the call returns an authorization notice rather than performing the close. This is valuable context beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, prerequisites, note content guidance, post-closing consequence, authorization requirement, and demo behavior. The structure is front-loaded with the core action and then expands with necessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers what the tool does, when it is valid, what happens after, who is allowed, and what to expect in a demo environment. With only two simple parameters, nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reinforces the meaning of outcome_notes by specifying what should be included, but it largely restates the schema rather than adding new detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('close'), explicit resource ('intervention'), and the outcome ('mark it complete'). The reference to verify_change comparisons distinguishes it from its most related sibling, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly specifies prerequisites (must exist and be active) and postconditions (no longer eligible for verify_change). The authorization requirement adds important gatekeeping context, though it does not explicitly name alternative tools for comparison.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_operator_to_referenceARead-only
Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field.
| Name | Required | Description | Default |
|---|---|---|---|
| reference | No | Reference population name | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| reference | Yes | |
| synthetic | Yes | |
| comparisons | Yes | |
| operator_id | Yes | |
| reference_version | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already signals a safe read operation, and the description adds useful behavioral context by stating it computes from raw metrics and reference field and returns specific comparison components. There is no contradiction with the annotation, and no destructive or side-effect behaviors need disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences, with the core purpose front-loaded in the first sentence. Every sentence adds distinct value (purpose, return contents, data source) without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a complete input schema and readOnlyHint annotation, the description sufficiently covers purpose and return categories; the presence of an output schema also relieves the need to detail return structures. Minor missing context includes how to select a reference population and potential relationships to sibling comparison tools, but these are not critical for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with operator_id and reference already documented. The description's phrase 'reference field' adds slight ambiguity but essentially restates the reference parameter; it does not provide meaningful additional semantic detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares an operator to a reference population and lists the return categories (benchmark selection, comparison group, metric comparison). However, it does not explicitly differentiate itself from sibling tools like get_operator_similarity or get_cohort_distribution, which may serve similar comparison purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an operator-to-reference comparison is needed, and the return types suggest analytical contexts. It offers no explicit when-to-use/when-not-to-use guidance or alternative tool references, leaving the selection inference to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_experimentA
Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha') | |
| configuration | No | Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration. |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| status | No | Initial status of the experiment (e.g., 'draft', 'configured') |
| message | Yes | Human-readable status message |
| created_at | No | ISO timestamp of creation (on success) |
| experiment_id | No | ID of the created experiment (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate openWorldHint, non-idempotent, non-destructive. The description adds valuable behavioral context: it enforces the ASSOCIATION-not-CAUSATION evidence standard, requires authorization, and discloses that in the synthetic demo it returns an authorization notice. This goes beyond the annotations and helps set expectations, though it does not detail side effects or persistence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise and well-structured. It opens with the core purpose, then explains the workflow and evidence standard, and ends with authorization and demo behavior. Every sentence adds new information, though the authorization sentence could arguably be more compact. Still, it is efficient and each sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema and well-documented parameters. The description covers key context: the prerequisite workflow, the evidence standard, authorization, and demo behavior. It does not explicitly mention return values, but the output schema exists to cover that. The only minor gap is the lack of details about what happens after creation (e.g., storage or side effects), but given the available structured schema and annotations, this is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% description coverage for both parameters. The description adds a small extra nuance by explaining that the 'configuration' parameter can be generated by create_pilot_configuration, and references list_pilot_options for available metrics. This is helpful, but the schema already carries the main semantic load, so a baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create an experiment configuration for controlled comparison studies.' It specifies the verb (create), the resource (experiment configuration), and the context (controlled comparison studies). It also distinguishes itself from the sibling create_pilot_configuration by explaining that experiments pair a pilot configuration with a hypothesis and measurement plan, and by explicitly instructing to use create_pilot_configuration first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: 'Use create_pilot_configuration first to build the config, then pass it here.' It also notes authorization requirements and the synthetic demo behavior. While it does not explicitly say 'when not to use,' it provides a clear workflow that guides an agent to the correct alternative when appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_pilot_configurationARead-only
Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard).
| Name | Required | Description | Default |
|---|---|---|---|
| metrics | No | Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog. | |
| cohort_size | No | Number of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size. | |
| duration_days | No | Pilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability. |
Output Schema
| Name | Required | Description |
|---|---|---|
| valid | Yes | Whether the configuration passed validation |
| warnings | No | Non-blocking validation warnings |
| synthetic | Yes | Whether data is synthetic |
| configuration | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation only declares readOnlyHint=true; the description adds that the result is a 'validated configuration object,' that it is 'saveable' rather than persisted, and that governance metadata is embedded (DEVELOPMENTAL label, no punitive use, association-not-causation). These are behavioral details beyond what annotations already provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences with the main action front-loaded, followed by workflow and governance context. There is no filler, though the governance metadata sentence is nonessential for invocation and makes it slightly longer than the leanest possible definition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a generation tool with complete schema documentation, an output schema, and read-only annotations, the description is self-sufficient: it covers prerequisites, downstream validation, return contents, and governance metadata. Nothing essential for selecting and calling the tool is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all three parameters with examples, giving 100% coverage, so the baseline is 3. The description merely restates 'cohort size, duration, selected metrics' at a high level and adds no parameter-specific semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Generate a saveable pilot configuration JSON from parameters.' It lists the returned object's components (cohort size, duration, selected metrics, eval families, benchmark classes) and distinguishes the tool from siblings by naming list_pilot_options and validate_pilot_configuration as adjacent workflow steps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit workflow context: 'Use list_pilot_options first to see available options' and 'can be passed to validate_pilot_configuration for pre-deployment checks.' It does not explicitly state when not to use this tool or name alternative tools for different scenarios, so it falls just short of full exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_usage_operation_divergenceARead-only
Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| all_operators | Yes | |
| divergent_operators | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description does not contradict this. It adds useful behavioral context by explaining that it computes usage percentiles from raw token totals, compares them to yield percentiles, and returns all 50 operators with divergence classes. This goes beyond the annotation without overpromising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences carry full value: the first states the core purpose, and the second explains the method and output. No filler or redundant restatement of the tool name, and key details like 'all 50 operators' and divergence class examples are included efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and an output schema present, the description covers what an agent needs to know: which operators are returned, how divergence is computed, and what categories are produced. There are no configuration choices, side effects, or ambiguous inputs that would require additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description cannot add parameter-level meaning. The schema already covers 100% of a nonexistent parameter set, making the baseline 4 appropriate. The description focuses on what the computation does, which is sufficient given there is nothing to configure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Find operators with usage-operation divergence.' It also explains the computation involved, distinguishing this from generic get_* siblings by focusing on divergence classification rather than simple retrieval. The mention of specific divergence classes further clarifies the intended output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The imperative 'Find operators with usage-operation divergence' clearly indicates when this tool is appropriate: when an agent needs to identify operators whose usage percentile diverges from yield percentile. It does not explicitly name alternative tools or exclusion conditions, but the context is unambiguous enough for an agent to select it correctly from the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_cohort_distributionARead-only
Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations.
| Name | Required | Description | Default |
|---|---|---|---|
| metric | No | Metric: leverage, yield, token_snr, log_leverage, construction | leverage |
Output Schema
| Name | Required | Description |
|---|---|---|
| max | No | |
| min | No | |
| p10 | No | |
| p25 | No | |
| p75 | No | |
| p90 | No | |
| std | No | |
| mean | No | |
| count | Yes | |
| median | Yes | |
| metric | Yes | |
| outliers | No | |
| synthetic | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers the safety profile, so the description does not need to restate that. 'Computed from raw observations' adds useful provenance context beyond the annotation, but the description does not clarify outlier definition, cohort fixedness, or behavior for empty/missing data. Acceptable but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no redundancy. The primary purpose and output statistics are front-loaded, and the second sentence adds a single high-value behavioral note about raw observations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional parameter, a read-only annotation, and an output schema, the description provides enough information to invoke it correctly. It could add context about the exact cohort definition or outlier computation, but those are not required for a successful call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already documents the metric parameter, its default ('leverage'), and allowed values. The description only refers to 'a given metric' and adds no parameter-level semantics beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States exactly what it computes ('Get cohort metric distribution') and lists the precise statistics returned (min, p10, p25, median, p75, p90, max, mean, std, outliers) plus the scope ('50-operator cohort'). This clearly distinguishes it from per-operator tools like get_operator_profile or aggregate summary tools, even though no sibling is named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'across the 50-operator cohort' phrasing gives clear context for when to use this tool: when a distribution-level view of a metric across the cohort is needed. It does not explicitly mention when not to use it or name alternative tools, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_composite_scoreARead-only
Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| name | No | |
| label | Yes | |
| score | Yes | |
| caveats | No | |
| score_id | No | |
| synthetic | Yes | |
| components | Yes | |
| operator_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already signals a safe read operation, and the description goes further by disclosing the computation method (raw metrics normalized via reference percentiles), the exact component weights, and the fact that data is synthetic. This is strong behavioral context beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the core purpose and scale come first, followed by computation details and caveats. Every sentence adds meaningful information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read-only tool with an output schema present, the description covers the scoring domain, input identification, weighting components, and data caveat. Nothing critical for correct invocation appears missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description and example for operator_id, so the baseline is 3. The description adds that the score is computed for an operator, but does not provide new parameter-level semantics beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Get'), resource ('developmental composite score'), scope ('for an operator'), and scale (0-100). It also distinguishes itself from personnel-related metrics and provides the component weights, making the tool's purpose unambiguous even without consulting siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like get_composite_score_summary or compare_operator_to_reference. The 'Labeled DEVELOPMENTAL, not PERSONNEL' line hints at appropriate use, but there is no explicit routing guidance or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_composite_score_summaryARead-only
Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| q1 | No | |
| q3 | No | |
| max | Yes | |
| min | Yes | |
| mean | Yes | |
| name | No | |
| count | Yes | |
| label | Yes | |
| median | Yes | |
| weights | No | |
| score_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already indicates a safe read operation, and the description adds meaningful context beyond that: it is computed from per-operator scores, it exposes no individual rankings, and it carries a developmental label. This gives the agent useful expectations about privacy and maturity without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tight and front-loaded: the first clause names the operation and resource, the middle lists the computed statistics, and the final clauses add caveats. Every sentence earns its place, with no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has no parameters and an output schema exists, the description is largely complete: it explains the aggregate nature, the source of computation, and the exposure boundary. The main omission is not routing the agent toward siblings or clarifying what the 'DEVELOPMENTAL' label means operationally.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema has 100% coverage by virtue of being empty. The description therefore does not need to explain parameter meanings. It still adds value by clarifying what the composite summary contains, which indirectly covers how the tool should be interpreted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Get cohort composite score summary' and enumerates the computed statistics (count, min, max, median, mean, Q1, Q3). It clearly indicates the tool returns an aggregate summary rather than raw scores. However, it does not explicitly differentiate itself from sibling tools like get_composite_score or get_cohort_distribution, so some ambiguity remains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_cohort_distribution. It does not state exclusions, prerequisites, or the intended cohort-analysis context. The 'DEVELOPMENTAL' label hints at caution but does not give usable selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_data_qualityARead-only
Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| issues | No | |
| coverage | Yes | |
| validity | Yes | |
| synthetic | Yes | |
| completeness | Yes | |
| operators_covered | Yes | |
| total_observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read operation. The description adds minimal behavioral context by stating the summary is 'computed from raw observations,' which is useful for understanding the data source but does not reveal much else about behavior such as aggregation scope or latency. With annotations covering safety, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the core purpose and then adds one relevant detail about the computation source. Every word earns its place, and there is no redundant or vague filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there are no parameters, the output schema is present, and the readOnlyHint annotation is provided, the description is complete enough for an agent to select and invoke this tool correctly. It specifies what kind of summary is returned and the key dimensions covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so there is no parameter documentation burden on the description. Baseline for 0 parameters is 4, and the description appropriately focuses on what the tool returns rather than parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'data quality summary,' and specifies the key dimensions: completeness, coverage, and validity across the cohort. It does not explicitly differentiate from sibling tools like get_diagnostics or get_composite_score, but 'data quality summary' is specific enough to identify the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used when a data quality summary over a cohort is needed, and the phrase 'across the cohort' gives some context. However, it provides no explicit guidance on when to use this tool versus alternatives such as get_diagnostics or get_cohort_distribution, and it lacks any exclusions or alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_diagnosticsARead-only
Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| status | Yes | |
| patterns | Yes | |
| diagnoses | Yes | |
| synthetic | Yes | |
| operator_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already communicates that this is a safe read operation. The description adds meaningful behavioral context by warning that all diagnoses are hypotheses, not facts, which is important for correct interpretation of results. There is no contradiction between the description and annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. The main action is front-loaded, the data source is provided, and the important caveat about hypothesis status is placed at the end without bloating the description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with a single well-documented parameter and an output schema, the description is complete. It conveys the purpose, data source, and a critical interpretive caveat, leaving no major gap that would prevent an agent from invoking the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with operator_id fully documented as a pseudonymous operator ID. The description itself does not add much parameter-level meaning, but the schema already carries the necessary semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: getting operator diagnostics, and it explains the source of those diagnostics (divergence analysis). It is clear what the tool does, though it does not explicitly distinguish it from related sibling tools like find_usage_operation_divergence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives such as find_usage_operation_divergence or get_data_quality. The context of "divergence analysis" provides only a weak implication of intended use, with no explicit exclusions or selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_executive_dashboardARead-only
Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| command | Yes | |
| message | Yes | |
| features | Yes | |
| governance | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already communicates that this is a safe read operation. The description adds useful context by explaining the dashboard is a self-contained HTML file produced by a CLI command, which suggests what the tool returns. It does not go deeper into output format or side effects, but with annotations and an output schema present, the additional behavior disclosure is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the action and resource, then adds one relevant detail about the dashboard being a generated HTML file. Every word earns its place, and there is no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless, read-only getter with an output schema available, the description is largely complete: it names the resource, explains its nature, and points to the CLI command that generates it. It could briefly mention what 'dashboard info' contains, but the output schema covers return structure, so the description is sufficient for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool accepts zero parameters, so there are no parameter semantics to clarify. Schema description coverage is trivially 100%, and the description does not need to explain any inputs. This is the appropriate baseline for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the verb ('Get') and resource ('executive dashboard info'), and adds a specific detail that the dashboard is a self-contained HTML file generated by the CLI. It is distinguishable from sibling tools like get_pilot_status or get_diagnostics because it names the executive dashboard specifically, though it does not explicitly contrast with any sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need the generated executive dashboard HTML file. Mentioning the CLI command 'enterprise export dashboard --output file.html' gives useful context about the artifact's origin. However, it does not provide explicit guidance on when to prefer this tool over alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_intervention_statusARead-only
Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| all | Yes | |
| count | Yes | |
| active | Yes | |
| closed | Yes | |
| synthetic | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already conveys that this is a safe read operation, and the description adds useful scope by specifying that all interventions and their fields are returned. Since there is no mutation or destructive behavior to disclose, this is adequate without being rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that front-loads the action and then enumerates the returned contents. Every part contributes useful information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only tool with an output schema, the description is nearly complete. It could be improved by naming a sibling alternative, but nothing an agent needs to invoke the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%, so there is no parameter meaning the description needs to add. A baseline of 4 is appropriate because there is nothing beyond the schema to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear read operation on the intervention resource and lists the returned fields, so an agent can understand it is a listing/status retrieval tool. It does not explicitly differentiate it from sibling get_pilot_status, but the verb and resource are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus siblings like get_pilot_status or close_intervention. It explains what is returned but not why an agent should select it over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_lineage_chainARead-only
Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_046) |
Output Schema
| Name | Required | Description |
|---|---|---|
| lineage | Yes | |
| synthetic | Yes | |
| data_window | Yes | |
| privacy_class | Yes | |
| reference_version | Yes | |
| validation_status | Yes | |
| metric_registry_version | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is consistent with the readOnlyHint annotation and adds one useful behavioral detail: the chain is 'built from raw lineage and outcome data.' It does not disclose error behavior, ordering guarantees, or data freshness, but the read-only annotation lowers the burden and the description adds some context beyond it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no filler. The main purpose and the data source are each stated once, and the chain definition is front-loaded. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read-only tool with an output schema, the description is nearly complete: it states what is returned, the chain shape, and the data source. The only notable gap is not explicitly framing the relationship to get_lineage_summary, but the output schema and annotations cover much of the remaining context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single parameter operator_id, including a type and an example. The description adds no parameter-specific detail, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Get') and resource ('full lineage chain for an operator'), and enumerates the exact chain steps STATE_A → BI_ACTION → ... → OUTCOME. The word 'full' and the explicit chain order clearly distinguish this from sibling get_lineage_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied: call this when you need the complete lineage chain for an operator. However, there is no explicit when-to-use or when-not-to-use guidance, and no mention of the closely related sibling get_lineage_summary, so an agent must infer the boundary between 'full' and 'summary'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_lineage_summaryARead-only
Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| data_window | Yes | |
| privacy_class | Yes | |
| lineage_summary | Yes | |
| reference_version | Yes | |
| validation_status | Yes | |
| metric_registry_version | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds the useful context that the summary is 'computed from raw lineage data' and exposes aggregate metrics, but it does not disclose other behavioral traits such as result shape, performance characteristics, or whether the summary is precomputed vs computed on demand. The description adds modest value beyond annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The primary action and scope are front-loaded, followed by a compact list of contained values, then a brief provenance note. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, a readOnly annotation, and an existing output schema, the description is complete for invoking the tool. An agent can determine what the tool returns (summary metrics, breakdown, linked outcomes) and that it is a safe read operation. No critical information is missing for a tool of this simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is nothing for the description to document. The input schema has 100% coverage by nature of being empty. The description's mention of cohort-wide aggregation is consistent with the stateless 'across the cohort' nature of the call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Get'), a resource ('lineage summary'), and a scope ('across the cohort'). It enumerates concrete contents (total lineages, workflow breakdown, average micro-eval metrics, outcomes linked), which distinguishes it from the sibling tool get_lineage_chain, which likely covers a single chain rather than an aggregate cohort summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'across the cohort' and nature of a summary imply aggregate use, and 'computed from raw lineage data' hints at a derived view. However, it does not explicitly state when to use this tool versus alternatives like get_lineage_chain, nor does it describe exclusions or cases where a different tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_profileARead-only
Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| team | No | |
| level | No | |
| pseudonym | No | |
| synthetic | Yes | |
| operator_id | Yes | |
| role_family | No | |
| measurements | Yes | |
| primary_platform | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark the tool as readOnlyHint=true, and the description adds useful behavioral context: metrics are 'computed from raw token observations,' operator IDs are pseudonymous, and the data is synthetic. This helps the agent set expectations about data provenance and privacy without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly structured sentence that front-loads the core purpose and then lists the key content areas. The second sentence adds relevant context about pseudonymity and synthetic data without any fluff. Every element earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter, read-only tool with an output schema present, the description covers the essential facts: what is returned, how metrics are derived, and important caveats about data provenance. The agent has enough information to decide whether to call it and to understand the response shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully documents the single parameter with description and examples, so the description does not need to add much. The description's mention of pseudonymous operator IDs reinforces the parameter format, but adds no fundamentally new semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Get operator profile') and enumerates the exact contents: operator details, 5 canonical metrics with values/percentiles/status, and benchmark availability. This clearly differentiates it from sibling tools like get_operator_similarity or get_composite_score, which target different aspects of operator data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this is the entry point for basic operator profile information, including metrics and benchmark availability. It does not explicitly state when to choose this tool over related siblings such as compare_operator_to_reference or get_operator_similarity, so the usage guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_similarityARead-only
Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching.
| Name | Required | Description | Default |
|---|---|---|---|
| n_neighbors | No | Number of nearest neighbors to return | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001) |
Output Schema
| Name | Required | Description |
|---|---|---|
| note | Yes | |
| normalization | Yes | |
| cluster_quality | Yes | |
| distance_metric | Yes | |
| nearest_neighbors | Yes | |
| query_operator_id | Yes | |
| cluster_description | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=true, so the description carries the burden of explaining behavior. It adds meaningful detail: percentile-rank normalization, Euclidean distance across 5 canonical metrics, computation from raw measurements, and the explicit warning that it does not perform personality matching. This goes well beyond the read-only annotation without contradicting it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences with no filler. The core purpose and method are front-loaded, the data source is clarified, and the crucial 'NOT personality matching' disambiguation is included without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return-value details are covered elsewhere. The description provides enough context for an agent to understand methodology, data source, and outcome scope. There is no explicit guidance on when to choose this over compare_operator_to_reference or get_cohort_distribution, but the core operational context is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description does not add meaningful parameter-level detail beyond what the schema provides, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific function: 'Nearest-neighbor operator search' with a defined methodology and output ('Returns comparable operators/cohorts'). It also distinguishes itself from what it is not ('NOT personality matching'), which differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when you need nearest-neighbor or similarity-based operator comparisons. It gives an explicit exclusion ('NOT personality matching') but does not name alternative tools or provide direct when-to-use versus when-not-to-use guidance beyond that single exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_system_decompositionARead-only
Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | No | Optional: filter to a single operator's decomposition |
Output Schema
| Name | Required | Description |
|---|---|---|
| metrics | Yes | |
| summary | Yes | |
| operator_id | No | |
| systems_compared | Yes | |
| operators_analyzed | Yes | |
| total_observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read operation. The description adds meaningful behavioral context by explaining the computation method, grouping by platform, and the interpretative output of the variance decomposition.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: it states the methodology first, then the data source, then the practical interpretation. Every sentence adds distinct value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only analytical tool with one optional, well-documented parameter and an output schema, the description provides sufficient context on purpose, computation, and interpretation. Nothing essential is missing for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single optional parameter, operator_id, with a clear filter description. The tool description adds no further parameter-level semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific analytical verb and resource: a two-way ANOVA-style decomposition partitioning variance into operator, system, and interaction effects. This clearly differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference, which address different questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: use this tool to determine whether operator capability or system choice drives performance. It does not explicitly mention exclusions or alternative tools, but the intended use case is well implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_org_topologyARead-only
Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| summary | Yes | |
| total_teams | Yes | |
| team_topologies | Yes | |
| total_operators | Yes | |
| platform_adoption | Yes | |
| capability_concentration | Yes | |
| single_points_of_failure | Yes | |
| cross_team_complementarity | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description notes the tool is 'Computed from raw measurements,' which indicates an aggregating, read-only computation consistent with the readOnlyHint annotation. It adds value beyond the annotation by describing provenance and the analytical nature of the result, though it does not address things like auth or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, information-dense sentence that front-loads the core concept and then lists specific value-adding capabilities. There is no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters, an output schema, and a readOnly annotation, the description is largely complete for selecting and invoking the tool correctly. It thoroughly describes the content of the topology map, though it could briefly mention what kinds of questions or scenarios it is best used for.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the baseline for such tools is 4. There is no parameter ambiguity to resolve, so the description does not need to compensate for missing parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific resource ('Organization-level AI topology map') and enumerates the exact analytical dimensions it provides, such as team-level metric distributions and Gini coefficient. This distinguishes it clearly from sibling tools like get_operator_profile and get_cohort_distribution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied through the organization-level scope and the list of team/capability metrics, but the description does not explicitly state when to prefer this tool over alternatives or mention any exclusions. It does not name sibling alternatives despite a large sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_outcome_correlationARead-only
Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| summary | Yes | |
| claim_status | Yes | |
| correlations | Yes | |
| evidence_grade | Yes | |
| operators_analyzed | Yes | |
| lineages_with_outcomes | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the readOnlyHint annotation by disclosing the statistical method (Pearson r), the data sources, and the critical interpretive boundary that results are ASSOCIATION/OBSERVATIONAL and never CAUSATION. This prevents misuse of the output and shows genuine behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences with no filler: the action, the computation method, and the interpretive caveat are all front-loaded and clearly organized. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, has an output schema, and carries a readOnly annotation, the description covers the essential behavioral and methodological context. It explains what is computed, from what data, and how results should be interpreted, leaving no critical gaps for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so there are no parameter semantics for the description to clarify. The baseline for zero-parameter tools is 4, and the description adds context about what data is used without needing to document arguments.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Correlate') and identifies the exact resources involved: micro-eval metrics, outcome quality scores, and cycle times via lineage. This clearly distinguishes it from sibling tools like get_lineage_summary or get_composite_score by stating the analytical method and result framing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use this tool: when the agent needs a Pearson correlation between micro-eval metrics and outcome/cycle-time data through lineage. It does not explicitly name alternatives or exclusions, but the context is strong enough that an agent would not confuse it with a general lineage or score retrieval tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_pilot_statusARead-only
Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| window | No | |
| cohort_id | Yes | |
| providers | No | |
| synthetic | Yes | |
| data_quality | No | |
| total_operators | Yes | |
| observation_count | Yes | |
| eligible_operators | No | |
| active_interventions | No | |
| metric_registry_version | No | |
| reference_field_version | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description states the status is computed from raw observations and that the data is from a labeled synthetic pilot. This tells the agent the output is an aggregate view rather than raw data, which adds meaningful behavioral context without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. The first sentence names the resource and its key contents, and the second adds data provenance and synthetic labeling. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters, a readOnlyHint annotation, and an output schema present, the description provides the necessary context: data source, synthetic nature, and aggregate computation. Nothing essential is missing for an agent to call this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters and the input schema has no properties, so there is no parameter burden for the description to carry. The baseline of 4 applies here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and names a concrete resource ('pilot status overview'), then enumerates the contents: cohort size, observation count, date range, data quality, and active interventions. It clearly conveys what the tool returns, though it does not explicitly differentiate it from sibling tools like get_data_quality or get_intervention_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives useful context—'Computed from raw observations' and '50-operator synthetic pilot'—which implies when this tool is appropriate. However, it does not explicitly state when to use it over alternatives or provide exclusions, so the guidance is implied rather than direct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_workflow_fitARead-only
Get workflow fit analysis — operator/workflow fit scores across workflow stages.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| note | No | |
| stages | Yes | |
| synthetic | Yes | |
| workflow_id | Yes | |
| observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already communicates that this is a safe read operation, and the description's 'Get' is consistent with that. The description adds modest context by stating the analysis spans workflow stages, but it does not describe any further behavioral traits such as aggregation level or data source.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence delivers the tool's purpose and scope with no filler. The main verb and resource are front-loaded, and the em-dash addition is the only elaboration, which is exactly the amount needed for a zero-parameter read tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read-only tool with an output schema, the description is sufficiently complete: it states what the tool returns and the domain (operator/workflow fit). It could add a short note on how to interpret the scores, but this is not necessary given the output schema and annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema description coverage is 100%, so there are no parameter semantics for the description to clarify. The 0-param baseline of 4 applies because the description need not compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb (Get) and a concrete resource (workflow fit analysis), and clarifies the content as operator/workflow fit scores across workflow stages. It does not explicitly contrast with any sibling tool, but 'workflow fit' is distinct enough among the listed siblings to identify the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_diagnostics. There are no exclusions, but also no explicit context or when-not-to-use conditions, so an agent must infer applicability from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_pilot_optionsARead-only
List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| eval_families | Yes | |
| benchmark_classes | Yes | |
| canonical_metrics | Yes | |
| intervention_types | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already establishes that this operation is safe and non-mutating, and the description's 'List' verb is consistent with that. The description adds some useful behavioral context by naming the categories and counts that will be returned, but it does not disclose details like response structure, ordering, or whether the counts are fixed or dynamic. This matches the baseline expectation for a read-only catalog tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the action ('List') and the object ('available pilot options'), then efficiently packs the key categorical details into an em-dash list. There is no filler, repetition, or unnecessary elaboration; every element earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless, read-only catalog tool with an output schema available, this description is complete enough for an agent to understand what it queries and what type of information it will receive. The enumeration of categories covers the tool's content, and the output schema handles return-value specifics. Nothing needed to invoke the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema has 100% coverage, so there is no parameter ambiguity to resolve. The 0-parameter baseline is 4, and the description appropriately does not need to explain parameter semantics that do not exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List') and resource ('available pilot options'), and further clarifies the scope by enumerating the four categories it covers: canonical metrics, eval families, benchmark classes, and intervention types. This clearly distinguishes it from the sibling tools, most of which are get/verify/create operations. However, it does not explicitly name a sibling or contrast itself, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention contexts like configuration setup, discovery workflows, or any exclusion cases. The intended use is only implied by the tool's name and simple read-only nature, which is not enough for strong usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
record_workflow_observationA
Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| notes | No | Free-text notes about the observation context — task type, AI system used, environmental factors | |
| fit_score | No | Workflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements. | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| workflow_id | Yes | Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing) |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| message | Yes | Human-readable status message |
| operator_id | No | Operator ID that was observed (on success) |
| recorded_at | No | ISO timestamp of recording (on success) |
| workflow_id | No | Workflow ID that was observed (on success) |
| observation_id | No | ID of the recorded observation (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide minimal safety hints (openWorld, non-idempotent, non-destructive). The description adds important behavioral context: REQUIRES AUTHORIZATION, contact for pilot access, and that the synthetic demo returns an authorization notice instead of performing the write. This goes beyond annotations and alerts the agent to the gated nature of the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, key concept, score range, read alternative, authorization, and demo behavior. The purpose is front-loaded, and the description is neither bloated nor under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, domain definition, score semantics, the appropriate read alternative, authorization requirements, and expected behavior in the demo environment. The output schema exists and likely covers return values; the description provides enough context for an agent to decide when and how to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The main description adds domain context (what workflow fit measures, score range) which is also present in the schema's property descriptions. No additional parameter-level detail is provided beyond what the schema contains.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb ('record') and clearly states the resource ('workflow fit observation linking an operator to a workflow stage with a fit score'). It explicitly names the read counterpart (get_workflow_fit), distinguishing this write operation from its sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the alternative for reading existing observations ('Use get_workflow_fit to read existing observations') and specifies authorization requirements, implying limited access. Does not provide an explicit 'when not to use' beyond the read case, but the write/read distinction is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_pilot_configurationARead-only
Validate a pilot configuration before deployment. Returns valid status with warnings and errors.
| Name | Required | Description | Default |
|---|---|---|---|
| configuration | No | Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes |
Output Schema
| Name | Required | Description |
|---|---|---|
| valid | Yes | |
| errors | Yes | |
| message | No | |
| warnings | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers safety; the description adds that the tool returns validation status with warnings and errors and is intended pre-deployment. It does not disclose details like required configuration properties or failure behavior, but with annotations and an output schema this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler; the key purpose is front-loaded and the return behavior is stated in one additional clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with one well-documented parameter, a readOnlyHint annotation, and an output schema, the description covers the essential decision: call it before deployment to get warnings/errors. It could mention prerequisites or valid value sources, but the schema already references list_pilot_options.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single 'configuration' parameter is fully described in the schema (JSON pilot configuration, link to list_pilot_options), and schema coverage is 100%. The description adds no parameter-level details, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action ('Validate'), resource ('pilot configuration'), and lifecycle context ('before deployment'), and describes the outcome ('Returns valid status with warnings and errors'). It is clearly a read-only validation step, but it does not explicitly distinguish itself from nearby siblings such as verify_change or create_pilot_configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'before deployment' provides a clear usage context, signaling that this tool is the pre-deployment check. It does not name alternatives or exclusion criteria, so an agent must infer when to use verify_change or list_pilot_options instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verify_changeARead-only
Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| intervention_id | No | Intervention ID (e.g., intv_001) |
Output Schema
| Name | Required | Description |
|---|---|---|
| label | Yes | |
| message | Yes | |
| synthetic | Yes | |
| operator_id | Yes | |
| composite_score | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers non-mutating behavior, and the description adds valuable interpretation guidance: results are associative, never causal. It also reveals the pre/post comparison methodology, which is beyond what the annotation or schema conveys.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence with a clear statistical caveat appended. Every part earns its place, and the core purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a full output schema and readOnlyHint annotation, the description covers the essential purpose and a key interpretive limitation. It does not explicitly discuss when to prefer this over correlated sibling tools, but the provided information is sufficient for most selection and invocation scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both operator_id and intervention_id. The description adds no additional parameter meaning, but the baseline of 3 applies because the schema carries the burden fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Verify') and resource ('a measured change after intervention') and clarifies the pre/post comparison scope. This clearly separates it from siblings like assign_intervention or get_intervention_status, which focus on assignment or status rather than change verification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it — after an intervention with pre/post measurements — but it does not explicitly state exclusions or name alternative tools. An agent can infer context, but there is no explicit routing guidance relative to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.4.3- Changed
close_intervention8 fields changed- changed
Input schema / properties / intervention_id / descriptionPrevious value: -"Intervention ID (e.g., intv_001)"New value: +"Intervention ID to close (e.g., intv_001, intv_007). Must be an active intervention." - changed
Input schema / properties / outcome_notes / descriptionPrevious value: -"Free-text notes about the intervention outcome"New value: +"Free-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved" - added
Output schema / properties / closed_atAdded value: +{ + "description": "ISO timestamp of closure (on success)", + "type": "string" +} - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / intervention_idAdded value: +{ + "description": "ID of the closed intervention (on success)", + "type": "string" +} - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / statusAdded value: +{ + "description": "New status of the intervention (e.g., 'closed')", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called"
- Changed
create_experiment8 fields changed- changed
Input schema / properties / configuration / descriptionPrevious value: -"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes"New value: +"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration." - changed
Input schema / properties / name / descriptionPrevious value: -"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison')"New value: +"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha')" - added
Output schema / properties / created_atAdded value: +{ + "description": "ISO timestamp of creation (on success)", + "type": "string" +} - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / experiment_idAdded value: +{ + "description": "ID of the created experiment (on success)", + "type": "string" +} - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / statusAdded value: +{ + "description": "Initial status of the experiment (e.g., 'draft', 'configured')", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called"
- Changed
create_pilot_configuration12 fields changed- changed
Input schema / properties / cohort_size / descriptionPrevious value: -"Number of operators in the pilot cohort (e.g., 25, 50, 100)"New value: +"Number of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size." - changed
Input schema / properties / duration_days / descriptionPrevious value: -"Pilot duration in days (e.g., 30, 60, 90)"New value: +"Pilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability." - changed
Input schema / properties / metrics / descriptionPrevious value: -"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction'])"New value: +"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog." - added
Output schema / properties / configuration / properties / benchmark_classesAdded value: +{ + "description": "Selected benchmark class IDs", + "items": { + "type": "string" + }, + "type": "array" +} - added
Output schema / properties / configuration / properties / cohort_size / descriptionAdded value: +"Number of operators in the cohort" - added
Output schema / properties / configuration / properties / duration_days / descriptionAdded value: +"Pilot duration in days" - added
Output schema / properties / configuration / properties / eval_familiesAdded value: +{ + "description": "Selected evaluation family IDs", + "items": { + "type": "string" + }, + "type": "array" +} - added
Output schema / properties / configuration / properties / governanceAdded value: +{ + "description": "Governance metadata including evidence labels and usage constraints", + "type": "object" +} - added
Output schema / properties / configuration / properties / metrics / descriptionAdded value: +"Selected metric IDs" - added
Output schema / properties / synthetic / descriptionAdded value: +"Whether data is synthetic" - added
Output schema / properties / valid / descriptionAdded value: +"Whether the configuration passed validation" - added
Output schema / properties / warningsAdded value: +{ + "description": "Non-blocking validation warnings", + "items": { + "type": "string" + }, + "type": "array" +}
- Changed
record_workflow_observation10 fields changed- changed
Input schema / properties / fit_score / descriptionPrevious value: -"Workflow fit score (0.0-1.0)"New value: +"Workflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements." - changed
Input schema / properties / notes / descriptionPrevious value: -"Free-text notes about the intervention assignment"New value: +"Free-text notes about the observation context — task type, AI system used, environmental factors" - changed
Input schema / properties / workflow_id / descriptionPrevious value: -"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture)"New value: +"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing)" - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / observation_idAdded value: +{ + "description": "ID of the recorded observation (on success)", + "type": "string" +} - added
Output schema / properties / operator_idAdded value: +{ + "description": "Operator ID that was observed (on success)", + "type": "string" +} - added
Output schema / properties / recorded_atAdded value: +{ + "description": "ISO timestamp of recording (on success)", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called" - added
Output schema / properties / workflow_idAdded value: +{ + "description": "Workflow ID that was observed (on success)", + "type": "string" +}
27 tool updates
v0.4.0- First observed
assign_intervention - First observed
attach_outcome_dataset - First observed
close_intervention - First observed
compare_operator_to_reference - First observed
create_experiment - First observed
create_pilot_configuration - First observed
find_usage_operation_divergence - First observed
get_cohort_distribution - First observed
get_composite_score - First observed
get_composite_score_summary - First observed
get_data_quality - First observed
get_diagnostics - First observed
get_executive_dashboard - First observed
get_intervention_status - First observed
get_lineage_chain - First observed
get_lineage_summary - First observed
get_operator_profile - First observed
get_operator_similarity - First observed
get_operator_system_decomposition - First observed
get_org_topology - First observed
get_outcome_correlation - First observed
get_pilot_status - First observed
get_workflow_fit - First observed
list_pilot_options - First observed
record_workflow_observation - First observed
validate_pilot_configuration - First observed
verify_change
TDQS
Scored across 27 tools
Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.
All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.
With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.
The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.
Maintenance
Related MCP Connectors
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for building and testing AI agents with multi-model experimentation and insights.
Related MCP Servers
- FlicenseNot gradedqualityNot gradedmaintenanceOpen-source AI governance toolkit. MCP servers & CLIs for scanning, auditing, and managing your AI environment-
- AlicenseAqualityAmaintenanceDescription Governed agent city-state. Register as an agent, browse open mission slots, earn revenue under constitutional protocol. Every action creates a SHA-256 provenance seed. Powered by MO§ES™.231MIT
- AlicenseNot gradedqualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.66MIT
- AlicenseAqualityAmaintenanceExposes the SigRank leaderboard as tools for agents, enabling token-only rank and submit operations with local privacy-first tokenpull reading.25527 npmMIT