Skip to main content
Glama
SunrisesIllNeverSee

Systems Intelligence Performative Commercial Benchmarking

MO§ES™ Enterprise: AI Operator Evaluation Platform

Live: mos2es.org: Systems intelligence for the human-AI operating system. The new standard in performative metrics and benchmarks for AI operators. Baselines system intelligence for everyday operations and AI workflows. Content-free token telemetry. Never your prompts.

Enterprise AI operator evaluation. Measure how people operate AI, not the AI model itself.

MO§ES™ curates a company's system intelligence: how effectively people drive AI systems across tools, tasks, workflows, and conditions. The same way BI sees the business, MO§ES™ sees how the business operates AI.

live enterprise MCP demo tests license deploy Python MCP Glama Smithery

Table of Contents


Related MCP server: Signomy

What is MO§ES™?

MO§ES™ is an enterprise AI operator evaluation platform. It measures how people operate AI systems, not the AI models themselves, not usage volume, not self-reported proficiency. Using content-free token telemetry (input, output, cache read, cache write, no prompt text, no response text), MO§ES™ baselines system intelligence across:

  • Operators: how effectively individuals drive AI

  • Teams: cohort distributions and capability topology

  • Workflows: where AI fits in the work, not just whether it's used

  • Organizations: cross-team benchmarking and capability mapping

The platform builds bespoke evals around your workflows, roles, and AI systems, benchmarks performance against internal and external reference populations, diagnoses capability gaps and divergence patterns, tests targeted interventions, and re-measures what changes.

The positioning

BI sees the business. MO§ES™ sees how the business operates AI.

LMSYS benchmarks models. Braintrust evaluates product outputs. Langfuse traces LLM calls. WakaTime tracks time. CostHawk tracks spend. None of them see the system intelligence of everyday operations and AI workflows. MO§ES™ is the missing layer.

The architecture

MO§ES™ → Upsilon → SigRank | SignalAF

  • MO§ES™ = governance framework / methodology (this repo)

  • Upsilon = measurement engine / enterprise product (the engine that measures)

  • SigRank = public leaderboard / benchmark / proof surface

  • SignalAF = public distribution / platform brand

  • Yield (Υ) = metric inside Upsilon: (cache_read × output) / input²

The MO§ES™ ecosystem

Repo / Site

What it is

URL

MO§ES™ Enterprise (this repo)

The platform: Python eval engine, CLI, MCP server, demo, promo, enterprise demo, workers. Powered by Upsilon.

mos2es.org

SigRank

Public leaderboard / benchmark / proof surface for AI operator evaluation

signalaf.com

SignalAF

Public brand and distribution surface

signalaf.com

SigRank MCP

The instrument: extracts token pillars, computes cascade, submits to leaderboard

npx sigrank

SIGNOMY

Governed AI agent marketplace: ranked agents form teams, run missions, earn revenue

signomy.xyz

SigEconomy

Public LLM operator evals: read-only leaderboard, SEO/AEO surface

sigeconomy.com

Live surfaces

Surface

URL

What

Promo site

https://mos2es.org

Marketing, methodology, demo, comparisons, booking

Enterprise demo

https://enterprise.mos2es.org

Interactive product walkthrough (evaluate → diagnose → workflow → compare)

MCP server

https://mcp.mos2es.org/mcp

27 tools (22 read + 5 write), 5 prompts, 6 resources, streamable HTTP

MCP server info

https://mcp.mos2es.org/

Server info JSON (version, tool count, transport)

MCP server card

https://mcp.mos2es.org/.well-known/mcp/server-card.json

Full server card with all tool schemas

OpenAPI spec

https://mos2es.org/openapi.json

REST API specification

LLM guidance

https://mos2es.org/llms.txt

llms.txt for AI agents and crawlers

Sitemap

https://mos2es.org/sitemap.xml

XML sitemap

Book a demo

https://mos2es.org/contact

B2B demo booking with structured intake form

Quick start

Run the demo (one-liner, no clone needed)

curl -sL https://mos2es.org/demo/run.py | python3 -

This clones the repo to a temp directory, installs rich, and runs the full 11-step demo pipeline. Requires Python 3.10+ and git.

Clone and run

git clone https://github.com/SunrisesIllneverSee/Moses_Enterprise_B2BPilot_.git
cd Moses_Enterprise_B2BPilot_/_01_platform
pip install rich
python3 -m src.cli.main demo full

Run the test suite

cd _01_platform
python3 -m pytest tests/ -q
# 676 tests pass

Run the MCP server locally

cd _01_platform
pip install mcp
python3 -m src.mcp_server.server

Use the remote MCP server

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "moses": {
      "url": "https://mcp.mos2es.org/mcp",
      "transport": "http"
    }
  }
}

Or call tools directly:

curl -s -D /tmp/h -X POST https://mcp.mos2es.org/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' > /dev/null
SID=$(grep -i "^mcp-session-id:" /tmp/h | tr -d '\r\n' | sed 's/.*: //')
curl -s -X POST https://mcp.mos2es.org/mcp \
  -H "Content-Type: application/json" -H "MCP-Session-Id: $SID" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | python3 -m json.tool

The 11-step demo pipeline

Step

Name

What it does

1

LOAD

Load 50-operator synthetic cohort

2

EVALUATE

Compute per-observation metrics

3

BENCHMARK

Compute percentile positions

4

DIAGNOSE

Run pattern detectors

5

OPERATOR×SYSTEM

Decompose operator vs system effects

6

INTERVENE

Assign targeted interventions

7

RE-EVALUATE

Re-measure post-intervention

8

OUTCOME LINEAGE

Trace observations → transformations → artifacts → outcomes

9

REPORT

Generate markdown + PDF pilot readout

10

VISUALIZE

Generate 9 architecture diagrams

11

DASHBOARD

Export executive HTML dashboard

The demo runs on fully synthetic data. No real operators, no real prompt text, no real API calls. Deterministic: running twice produces identical results.

The 8 canonical metrics

Metric

Formula

What it measures

Yield (Υ)

(R × O) / I²

Productive output relative to input squared

Leverage

R / I

Context reuse efficiency

Token SNR

O / (I + O)

Signal-to-noise ratio in token usage

Log Leverage (10xDEV)

log₁₀(R / I)

Leverage on a logarithmic scale

Construction

W / R

New context built vs reused

Velocity

O / I

Raw output efficiency

Scale V

log₁₀(I + O + W + R)

Total volume on a log scale

Efficiency

(R + W + O) / I / 4

Composite display metric

Where I = input, O = output, R = cache read, W = cache write.

15 evaluation families. 13 benchmark classes. 4 measurement levels (Operator, Team, Workflow, Organization).

Repository structure

_01_platform/          Python platform (the product)
  src/                 Source code (domain, metrics, analysis, diagnostics, CLI, MCP server)
  tests/               676 tests
  demo_data/           Synthetic demo cohort (50 operators, 1,668 observations)
  scripts/             Utility scripts
  schemas/             JSON schemas

_02_demo-website/      Enterprise demo site (enterprise.mos2es.org)
_03_promo-site/        Promo site (mos2es.org)
  vs/                  16 competitor comparison pages
  alternatives/        8 alternatives listicle pages
  concepts/            10 concept explainer pages
  guides/              4 how-to guides
  demo/run.py          One-liner demo runner
  .well-known/mcp.json MCP server discovery

_04_onepager/          One-pager site

_workers/              Cloudflare Workers
  promo-worker/        mos2es.org worker (static assets + AEO/SEO/GEO headers)
  moses-worker/        enterprise.mos2es.org worker (static assets)
  onepager-worker/     One-pager worker (static assets)
  mcp-worker/          MCP server worker (computes live from raw data, 27 tools)

MCP server

The MCP server exposes 27 tools (22 read + 5 write), 5 prompts, and 6 resources over streamable HTTP at https://mcp.mos2es.org/mcp. No authentication required for the public server.

Read tools include: cohort stats, operator profiles, metric distributions, benchmark positions, divergence findings, intervention outcomes, outcome lineage, workflow fit, org topology, and more.

Write tools include: create intervention, assign intervention, record outcome, create eval configuration, create pilot configuration.

Prompts include: operator evaluation summary, intervention recommendation, cohort health report, workflow fit analysis, pilot scoping guide.

Resources include: canonical metric definitions, metric registry, pilot status, pilot options, governance conventions, cohort operator list.

The server computes live from raw observation data, no pre-computed results. Every call runs the actual scoring, benchmarking, and diagnostic engines.

Server card

curl -s https://mcp.mos2es.org/.well-known/mcp/server-card.json | python3 -m json.tool

Returns the full server card with all 27 tool schemas, transport info, and metadata.

Cloudflare Workers

Four Workers deploy from this repo:

Worker

Domain

What

moses-promo

mos2es.org

Promo site + AEO/SEO/GEO headers + agent discoverability (llms.txt, sitemap, OpenAPI, MCP links)

moses

enterprise.mos2es.org

Enterprise demo site

moses-onepager

(workers.dev)

One-pager

moses-mcp

mcp.mos2es.org

MCP server (27 tools, live computation)

Deploy

npm install -g wrangler

cd _workers/promo-worker && wrangler deploy
cd _workers/moses-worker && wrangler deploy
cd _workers/onepager-worker && wrangler deploy
cd _workers/mcp-worker && wrangler deploy

MCP worker data sync

The MCP worker computes live from raw data files (observations.js, lineages.js) auto-generated from the platform's demo data. To regenerate after updating demo data:

cd _workers/mcp-worker/src
python3 -c "
import json
with open('../../../_01_platform/demo_data/observations.jsonl') as f:
    data = [json.loads(l) for l in f if l.strip()]
with open('observations.js', 'w') as f:
    f.write('export default '); json.dump(data, f); f.write(';\n')
with open('../../../_01_platform/demo_data/lineages.jsonl') as f:
    data = [json.loads(l) for l in f if l.strip()]
with open('lineages.js', 'w') as f:
    f.write('export default '); json.dump(data, f); f.write(';\n')
"

Key conventions

  • Brand: MO§ES™ (use the section sign §)

  • All outcome claims are ASSOCIATION, never CAUSATION unless backed by a controlled experiment

  • Composite score is DEVELOPMENTAL, never PERSONNEL: no punitive use, no employee leaderboard

  • Operator similarity is metric similarity, NOT personality matching

  • Content-free telemetry: no prompt text, no output bodies, no code content in observations

  • No prompt-content surveillance: operator performance is observable from token structure alone

  • Governance-ready: evidence labels (DEVELOPMENTAL, HYPOTHESIS, ASSOCIATION) on every output

Complement, don't replace

MO§ES™ works alongside your existing BI, eval suites, observability tools, and productivity analytics. It measures the systems intelligence layer they can't see. Not a replacement: the missing piece.

Five pillars

#

Pillar

What

1

Systems Intelligence

MO§ES™ sees the human-AI operating system the way BI sees the business. Operator performance, workflow fit, tool selection, capability distribution, intervention outcomes, all measured from structure, not content.

2

Standard Operational Performative Metrics

8 canonical metrics. 15 eval families. 13 benchmark classes. 4 measurement levels. An open, documented spec for measuring how humans operate AI. The reference implementation is MO§ES™.

3

Bespoke Enterprise Evals

Your company should not inherit someone else's definition of AI proficiency. Evals built around your workflows, roles, models, and performance questions.

4

Complement, Don't Replace

Works alongside your existing BI, eval suites, observability tools, and productivity analytics. The missing piece, not a replacement.

5

Privacy-First / Governance

Content-free token telemetry. No prompt text. No surveillance. DEVELOPMENTAL / HYPOTHESIS / ASSOCIATION evidence labels on every output. Governance-ready out of the box.

License

Apache License 2.0: see LICENSE. Patent pending: Serial No. 63/877,177. For enterprise pilots and commercial support: pilots@mos2es.org.


mos2es.org · Book a demo · Run the demo · MCP server

Built by Deric J. McHenry, Ello Cello LLC

Available Tools

27 tools
assign_interventionA

Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access.

ParametersJSON Schema
NameRequiredDescriptionDefault
notesNoFree-text notes about the intervention assignment
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)
intervention_typeYesIntervention type from catalog (e.g., prompt_template, context_window_expansion, model_switch)

Output Schema

ParametersJSON Schema
NameRequiredDescription
toolYes
errorYes
messageYes

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey non-idempotent and non-destructive behavior, and the description adds a valuable behavioral constraint: authorization is required and pilot access must be arranged. It does not contradict the annotations, though it could say more about side effects of creating an assignment.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the first sentence states the action, and the second communicates the operational prerequisite and contact path. There is no redundant repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the full parameter schema, output schema, and annotations, the description covers the mechanics of invocation and the authorization barrier. However, it leaves selection context unaddressed—when to assign versus close an intervention—and offers limited detail on the assignment's effect, making it adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All three parameters are fully documented in the input schema with descriptions and examples, including operator_id and intervention_type. The tool description itself adds no additional parameter semantics, so the baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the verb (assign), resource (targeted intervention), and recipient (operator), making the core operation clear. It is distinguishable from the sibling close_intervention by the opposing action, though it does not explicitly call out that contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no when-to-use guidance or alternative routing, and it does not mention relationships to siblings like close_intervention or create_experiment. The only usage-related information is the authorization prerequisite, which is not sufficient for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

attach_outcome_datasetA

Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION.

ParametersJSON Schema
NameRequiredDescriptionDefault
formatNoData format (e.g., 'json', 'csv', 'jsonl')
sourceYesExternal outcome data source name (e.g., 'jira', 'github', 'linear')

Output Schema

ParametersJSON Schema
NameRequiredDescription
toolYes
errorYes
messageYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description adds meaningful behavioral context: it warns that outcome joins are association-only and explicitly flags that authorization is required. This is valuable for an agent deciding whether and how to invoke the tool. Annotations already cover idempotency and destructiveness, so the additional caveats push this above baseline.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero filler. It front-loads the primary purpose and then packs the most important constraints—association-not-causation and authorization—into a compact, readable warning. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, full schema coverage, and the presence of an output schema, the description covers the key behavioral caveats an agent needs. It could theoretically mention the effect on downstream analysis or concrete when-not-to-use scenarios, but it is sufficiently complete for a tool that attaches a dataset.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters, including examples. The description adds no parameter-level detail, which is acceptable because the schema carries the burden. Per the baseline for high schema coverage, a 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'Attach external outcome dataset for join analysis.' It clearly conveys what the tool does and is distinguishable from the sibling tools, which are mostly read/get operations. It does not explicitly contrast itself with a specific sibling, but the verb+resource combination is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when an external outcome dataset needs to be attached for join analysis, but it does not explicitly state when to use this tool over alternatives or mention any exclusions. The causation warning and authorization requirement provide context, but no comparative guidance is given versus siblings like get_outcome_correlation or create_experiment.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_interventionA

Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

ParametersJSON Schema
NameRequiredDescriptionDefault
outcome_notesNoFree-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved
intervention_idYesIntervention ID to close (e.g., intv_001, intv_007). Must be an active intervention.

Output Schema

ParametersJSON Schema
NameRequiredDescription
toolYesTool name that was called
errorYesError type if authorization fails (e.g., 'authorization_required')
statusNoNew status of the intervention (e.g., 'closed')
messageYesHuman-readable status message
closed_atNoISO timestamp of closure (on success)
intervention_idNoID of the closed intervention (on success)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate idempotent=false and destructive=false, and the description adds significant behavior: the intervention becomes ineligible for verify_change after closing, and in the synthetic demo the call returns an authorization notice rather than performing the close. This is valuable context beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: purpose, prerequisites, note content guidance, post-closing consequence, authorization requirement, and demo behavior. The structure is front-loaded with the core action and then expands with necessary details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, when it is valid, what happens after, who is allowed, and what to expect in a demo environment. With only two simple parameters, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description reinforces the meaning of outcome_notes by specifying what should be included, but it largely restates the schema rather than adding new detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('close'), explicit resource ('intervention'), and the outcome ('mark it complete'). The reference to verify_change comparisons distinguishes it from its most related sibling, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly specifies prerequisites (must exist and be active) and postconditions (no longer eligible for verify_change). The authorization requirement adds important gatekeeping context, though it does not explicitly name alternative tools for comparison.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_operator_to_referenceA
Read-only

Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field.

ParametersJSON Schema
NameRequiredDescriptionDefault
referenceNoReference population name
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)

Output Schema

ParametersJSON Schema
NameRequiredDescription
referenceYes
syntheticYes
comparisonsYes
operator_idYes
reference_versionNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already signals a safe read operation, and the description adds useful behavioral context by stating it computes from raw metrics and reference field and returns specific comparison components. There is no contradiction with the annotation, and no destructive or side-effect behaviors need disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three concise sentences, with the core purpose front-loaded in the first sentence. Every sentence adds distinct value (purpose, return contents, data source) without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a complete input schema and readOnlyHint annotation, the description sufficiently covers purpose and return categories; the presence of an output schema also relieves the need to detail return structures. Minor missing context includes how to select a reference population and potential relationships to sibling comparison tools, but these are not critical for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with operator_id and reference already documented. The description's phrase 'reference field' adds slight ambiguity but essentially restates the reference parameter; it does not provide meaningful additional semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compares an operator to a reference population and lists the return categories (benchmark selection, comparison group, metric comparison). However, it does not explicitly differentiate itself from sibling tools like get_operator_similarity or get_cohort_distribution, which may serve similar comparison purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when an operator-to-reference comparison is needed, and the return types suggest analytical contexts. It offers no explicit when-to-use/when-not-to-use guidance or alternative tool references, leaving the selection inference to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_experimentA

Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesExperiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha')
configurationNoPilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration.

Output Schema

ParametersJSON Schema
NameRequiredDescription
toolYesTool name that was called
errorYesError type if authorization fails (e.g., 'authorization_required')
statusNoInitial status of the experiment (e.g., 'draft', 'configured')
messageYesHuman-readable status message
created_atNoISO timestamp of creation (on success)
experiment_idNoID of the created experiment (on success)

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate openWorldHint, non-idempotent, non-destructive. The description adds valuable behavioral context: it enforces the ASSOCIATION-not-CAUSATION evidence standard, requires authorization, and discloses that in the synthetic demo it returns an authorization notice. This goes beyond the annotations and helps set expectations, though it does not detail side effects or persistence.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is moderately concise and well-structured. It opens with the core purpose, then explains the workflow and evidence standard, and ends with authorization and demo behavior. Every sentence adds new information, though the authorization sentence could arguably be more compact. Still, it is efficient and each sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema and well-documented parameters. The description covers key context: the prerequisite workflow, the evidence standard, authorization, and demo behavior. It does not explicitly mention return values, but the output schema exists to cover that. The only minor gap is the lack of details about what happens after creation (e.g., storage or side effects), but given the available structured schema and annotations, this is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% description coverage for both parameters. The description adds a small extra nuance by explaining that the 'configuration' parameter can be generated by create_pilot_configuration, and references list_pilot_options for available metrics. This is helpful, but the schema already carries the main semantic load, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Create an experiment configuration for controlled comparison studies.' It specifies the verb (create), the resource (experiment configuration), and the context (controlled comparison studies). It also distinguishes itself from the sibling create_pilot_configuration by explaining that experiments pair a pilot configuration with a hypothesis and measurement plan, and by explicitly instructing to use create_pilot_configuration first.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage context: 'Use create_pilot_configuration first to build the config, then pass it here.' It also notes authorization requirements and the synthetic demo behavior. While it does not explicitly say 'when not to use,' it provides a clear workflow that guides an agent to the correct alternative when appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_pilot_configurationA
Read-only

Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard).

ParametersJSON Schema
NameRequiredDescriptionDefault
metricsNoArray of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog.
cohort_sizeNoNumber of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size.
duration_daysNoPilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability.

Output Schema

ParametersJSON Schema
NameRequiredDescription
validYesWhether the configuration passed validation
warningsNoNon-blocking validation warnings
syntheticYesWhether data is synthetic
configurationYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotation only declares readOnlyHint=true; the description adds that the result is a 'validated configuration object,' that it is 'saveable' rather than persisted, and that governance metadata is embedded (DEVELOPMENTAL label, no punitive use, association-not-causation). These are behavioral details beyond what annotations already provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with the main action front-loaded, followed by workflow and governance context. There is no filler, though the governance metadata sentence is nonessential for invocation and makes it slightly longer than the leanest possible definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a generation tool with complete schema documentation, an output schema, and read-only annotations, the description is self-sufficient: it covers prerequisites, downstream validation, return contents, and governance metadata. Nothing essential for selecting and calling the tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all three parameters with examples, giving 100% coverage, so the baseline is 3. The description merely restates 'cohort size, duration, selected metrics' at a high level and adds no parameter-specific semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Generate a saveable pilot configuration JSON from parameters.' It lists the returned object's components (cohort size, duration, selected metrics, eval families, benchmark classes) and distinguishes the tool from siblings by naming list_pilot_options and validate_pilot_configuration as adjacent workflow steps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit workflow context: 'Use list_pilot_options first to see available options' and 'can be passed to validate_pilot_configuration for pre-deployment checks.' It does not explicitly state when not to use this tool or name alternative tools for different scenarios, so it falls just short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_usage_operation_divergenceA
Read-only

Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
syntheticYes
all_operatorsYes
divergent_operatorsYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, and the description does not contradict this. It adds useful behavioral context by explaining that it computes usage percentiles from raw token totals, compares them to yield percentiles, and returns all 50 operators with divergence classes. This goes beyond the annotation without overpromising side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences carry full value: the first states the core purpose, and the second explains the method and output. No filler or redundant restatement of the tool name, and key details like 'all 50 operators' and divergence class examples are included efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters and an output schema present, the description covers what an agent needs to know: which operators are returned, how divergence is computed, and what categories are produced. There are no configuration choices, side effects, or ambiguous inputs that would require additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description cannot add parameter-level meaning. The schema already covers 100% of a nonexistent parameter set, making the baseline 4 appropriate. The description focuses on what the computation does, which is sufficient given there is nothing to configure.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Find operators with usage-operation divergence.' It also explains the computation involved, distinguishing this from generic get_* siblings by focusing on divergence classification rather than simple retrieval. The mention of specific divergence classes further clarifies the intended output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The imperative 'Find operators with usage-operation divergence' clearly indicates when this tool is appropriate: when an agent needs to identify operators whose usage percentile diverges from yield percentile. It does not explicitly name alternative tools or exclusion conditions, but the context is unambiguous enough for an agent to select it correctly from the sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cohort_distributionA
Read-only

Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations.

ParametersJSON Schema
NameRequiredDescriptionDefault
metricNoMetric: leverage, yield, token_snr, log_leverage, constructionleverage

Output Schema

ParametersJSON Schema
NameRequiredDescription
maxNo
minNo
p10No
p25No
p75No
p90No
stdNo
meanNo
countYes
medianYes
metricYes
outliersNo
syntheticYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already covers the safety profile, so the description does not need to restate that. 'Computed from raw observations' adds useful provenance context beyond the annotation, but the description does not clarify outlier definition, cohort fixedness, or behavior for empty/missing data. Acceptable but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no redundancy. The primary purpose and output statistics are front-loaded, and the second sentence adds a single high-value behavioral note about raw observations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one optional parameter, a read-only annotation, and an output schema, the description provides enough information to invoke it correctly. It could add context about the exact cohort definition or outlier computation, but those are not required for a successful call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already documents the metric parameter, its default ('leverage'), and allowed values. The description only refers to 'a given metric' and adds no parameter-level semantics beyond what the schema provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States exactly what it computes ('Get cohort metric distribution') and lists the precise statistics returned (min, p10, p25, median, p75, p90, max, mean, std, outliers) plus the scope ('50-operator cohort'). This clearly distinguishes it from per-operator tools like get_operator_profile or aggregate summary tools, even though no sibling is named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'across the 50-operator cohort' phrasing gives clear context for when to use this tool: when a distribution-level view of a metric across the cohort is needed. It does not explicitly mention when not to use it or name alternative tools, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_composite_scoreA
Read-only

Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)

Output Schema

ParametersJSON Schema
NameRequiredDescription
nameNo
labelYes
scoreYes
caveatsNo
score_idNo
syntheticYes
componentsYes
operator_idYes

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already signals a safe read operation, and the description goes further by disclosing the computation method (raw metrics normalized via reference percentiles), the exact component weights, and the fact that data is synthetic. This is strong behavioral context beyond the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the core purpose and scale come first, followed by computation details and caveats. Every sentence adds meaningful information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only tool with an output schema present, the description covers the scoring domain, input identification, weighting components, and data caveat. Nothing critical for correct invocation appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear description and example for operator_id, so the baseline is 3. The description adds that the score is computed for an operator, but does not provide new parameter-level semantics beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Get'), resource ('developmental composite score'), scope ('for an operator'), and scale (0-100). It also distinguishes itself from personnel-related metrics and provides the component weights, making the tool's purpose unambiguous even without consulting siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives like get_composite_score_summary or compare_operator_to_reference. The 'Labeled DEVELOPMENTAL, not PERSONNEL' line hints at appropriate use, but there is no explicit routing guidance or exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_composite_score_summaryA
Read-only

Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
q1No
q3No
maxYes
minYes
meanYes
nameNo
countYes
labelYes
medianYes
weightsNo
score_idYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already indicates a safe read operation, and the description adds meaningful context beyond that: it is computed from per-operator scores, it exposes no individual rankings, and it carries a developmental label. This gives the agent useful expectations about privacy and maturity without contradicting the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tight and front-loaded: the first clause names the operation and resource, the middle lists the computed statistics, and the final clauses add caveats. Every sentence earns its place, with no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the tool has no parameters and an output schema exists, the description is largely complete: it explains the aggregate nature, the source of computation, and the exposure boundary. The main omission is not routing the agent toward siblings or clarifying what the 'DEVELOPMENTAL' label means operationally.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema has 100% coverage by virtue of being empty. The description therefore does not need to explain parameter meanings. It still adds value by clarifying what the composite summary contains, which indirectly covers how the tool should be interpreted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get cohort composite score summary' and enumerates the computed statistics (count, min, max, median, mean, Q1, Q3). It clearly indicates the tool returns an aggregate summary rather than raw scores. However, it does not explicitly differentiate itself from sibling tools like get_composite_score or get_cohort_distribution, so some ambiguity remains.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_cohort_distribution. It does not state exclusions, prerequisites, or the intended cohort-analysis context. The 'DEVELOPMENTAL' label hints at caution but does not give usable selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_data_qualityA
Read-only

Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
issuesNo
coverageYes
validityYes
syntheticYes
completenessYes
operators_coveredYes
total_observationsYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already establishes that this is a safe read operation. The description adds minimal behavioral context by stating the summary is 'computed from raw observations,' which is useful for understanding the data source but does not reveal much else about behavior such as aggregation scope or latency. With annotations covering safety, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the core purpose and then adds one relevant detail about the computation source. Every word earns its place, and there is no redundant or vague filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there are no parameters, the output schema is present, and the readOnlyHint annotation is provided, the description is complete enough for an agent to select and invoke this tool correctly. It specifies what kind of summary is returned and the key dimensions covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter documentation burden on the description. Baseline for 0 parameters is 4, and the description appropriately focuses on what the tool returns rather than parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'data quality summary,' and specifies the key dimensions: completeness, coverage, and validity across the cohort. It does not explicitly differentiate from sibling tools like get_diagnostics or get_composite_score, but 'data quality summary' is specific enough to identify the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool should be used when a data quality summary over a cohort is needed, and the phrase 'across the cohort' gives some context. However, it provides no explicit guidance on when to use this tool versus alternatives such as get_diagnostics or get_cohort_distribution, and it lacks any exclusions or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_diagnosticsA
Read-only

Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)

Output Schema

ParametersJSON Schema
NameRequiredDescription
statusYes
patternsYes
diagnosesYes
syntheticYes
operator_idYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotation readOnlyHint=true already communicates that this is a safe read operation. The description adds meaningful behavioral context by warning that all diagnoses are hypotheses, not facts, which is important for correct interpretation of results. There is no contradiction between the description and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no filler. The main action is front-loaded, the data source is provided, and the important caveat about hypothesis status is placed at the end without bloating the description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only tool with a single well-documented parameter and an output schema, the description is complete. It conveys the purpose, data source, and a critical interpretive caveat, leaving no major gap that would prevent an agent from invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with operator_id fully documented as a pseudonymous operator ID. The description itself does not add much parameter-level meaning, but the schema already carries the necessary semantics, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: getting operator diagnostics, and it explains the source of those diagnostics (divergence analysis). It is clear what the tool does, though it does not explicitly distinguish it from related sibling tools like find_usage_operation_divergence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as find_usage_operation_divergence or get_data_quality. The context of "divergence analysis" provides only a weak implication of intended use, with no explicit exclusions or selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_executive_dashboardA
Read-only

Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
commandYes
messageYes
featuresYes
governanceYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already communicates that this is a safe read operation. The description adds useful context by explaining the dashboard is a self-contained HTML file produced by a CLI command, which suggests what the tool returns. It does not go deeper into output format or side effects, but with annotations and an output schema present, the additional behavior disclosure is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that immediately states the action and resource, then adds one relevant detail about the dashboard being a generated HTML file. Every word earns its place, and there is no redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless, read-only getter with an output schema available, the description is largely complete: it names the resource, explains its nature, and points to the CLI command that generates it. It could briefly mention what 'dashboard info' contains, but the output schema covers return structure, so the description is sufficient for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool accepts zero parameters, so there are no parameter semantics to clarify. Schema description coverage is trivially 100%, and the description does not need to explain any inputs. This is the appropriate baseline for a parameterless tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the verb ('Get') and resource ('executive dashboard info'), and adds a specific detail that the dashboard is a self-contained HTML file generated by the CLI. It is distinguishable from sibling tools like get_pilot_status or get_diagnostics because it names the executive dashboard specifically, though it does not explicitly contrast with any sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: when you need the generated executive dashboard HTML file. Mentioning the CLI command 'enterprise export dashboard --output file.html' gives useful context about the artifact's origin. However, it does not provide explicit guidance on when to prefer this tool over alternatives or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_intervention_statusA
Read-only

Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
allYes
countYes
activeYes
closedYes
syntheticYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotation readOnlyHint=true already conveys that this is a safe read operation, and the description adds useful scope by specifying that all interventions and their fields are returned. Since there is no mutation or destructive behavior to disclose, this is adequate without being rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single efficient sentence that front-loads the action and then enumerates the returned contents. Every part contributes useful information with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only tool with an output schema, the description is nearly complete. It could be improved by naming a sibling alternative, but nothing an agent needs to invoke the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema coverage is 100%, so there is no parameter meaning the description needs to add. A baseline of 4 is appropriate because there is nothing beyond the schema to explain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear read operation on the intervention resource and lists the returned fields, so an agent can understand it is a listing/status retrieval tool. It does not explicitly differentiate it from sibling get_pilot_status, but the verb and resource are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus siblings like get_pilot_status or close_intervention. It explains what is returned but not why an agent should select it over alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_lineage_chainA
Read-only

Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idYesPseudonymous operator ID (e.g., op_046)

Output Schema

ParametersJSON Schema
NameRequiredDescription
lineageYes
syntheticYes
data_windowYes
privacy_classYes
reference_versionYes
validation_statusYes
metric_registry_versionYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description is consistent with the readOnlyHint annotation and adds one useful behavioral detail: the chain is 'built from raw lineage and outcome data.' It does not disclose error behavior, ordering guarantees, or data freshness, but the read-only annotation lowers the burden and the description adds some context beyond it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, no filler. The main purpose and the data source are each stated once, and the chain definition is front-loaded. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only tool with an output schema, the description is nearly complete: it states what is returned, the chain shape, and the data source. The only notable gap is not explicitly framing the relationship to get_lineage_summary, but the output schema and annotations cover much of the remaining context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage for the single parameter operator_id, including a type and an example. The description adds no parameter-specific detail, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Get') and resource ('full lineage chain for an operator'), and enumerates the exact chain steps STATE_A → BI_ACTION → ... → OUTCOME. The word 'full' and the explicit chain order clearly distinguish this from sibling get_lineage_summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied: call this when you need the complete lineage chain for an operator. However, there is no explicit when-to-use or when-not-to-use guidance, and no mention of the closely related sibling get_lineage_summary, so an agent must infer the boundary between 'full' and 'summary'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_lineage_summaryA
Read-only

Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
syntheticYes
data_windowYes
privacy_classYes
lineage_summaryYes
reference_versionYes
validation_statusYes
metric_registry_versionYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds the useful context that the summary is 'computed from raw lineage data' and exposes aggregate metrics, but it does not disclose other behavioral traits such as result shape, performance characteristics, or whether the summary is precomputed vs computed on demand. The description adds modest value beyond annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. The primary action and scope are front-loaded, followed by a compact list of contained values, then a brief provenance note. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero parameters, a readOnly annotation, and an existing output schema, the description is complete for invoking the tool. An agent can determine what the tool returns (summary metrics, breakdown, linked outcomes) and that it is a safe read operation. No critical information is missing for a tool of this simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is nothing for the description to document. The input schema has 100% coverage by nature of being empty. The description's mention of cohort-wide aggregation is consistent with the stateless 'across the cohort' nature of the call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Get'), a resource ('lineage summary'), and a scope ('across the cohort'). It enumerates concrete contents (total lineages, workflow breakdown, average micro-eval metrics, outcomes linked), which distinguishes it from the sibling tool get_lineage_chain, which likely covers a single chain rather than an aggregate cohort summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'across the cohort' and nature of a summary imply aggregate use, and 'computed from raw lineage data' hints at a derived view. However, it does not explicitly state when to use this tool versus alternatives like get_lineage_chain, nor does it describe exclusions or cases where a different tool would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_operator_profileA
Read-only

Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)

Output Schema

ParametersJSON Schema
NameRequiredDescription
teamNo
levelNo
pseudonymNo
syntheticYes
operator_idYes
role_familyNo
measurementsYes
primary_platformNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already mark the tool as readOnlyHint=true, and the description adds useful behavioral context: metrics are 'computed from raw token observations,' operator IDs are pseudonymous, and the data is synthetic. This helps the agent set expectations about data provenance and privacy without contradicting the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tightly structured sentence that front-loads the core purpose and then lists the key content areas. The second sentence adds relevant context about pseudonymity and synthetic data without any fluff. Every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter, read-only tool with an output schema present, the description covers the essential facts: what is returned, how metrics are derived, and important caveats about data provenance. The agent has enough information to decide whether to call it and to understand the response shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully documents the single parameter with description and examples, so the description does not need to add much. The description's mention of pseudonymous operator IDs reinforces the parameter format, but adds no fundamentally new semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Get operator profile') and enumerates the exact contents: operator details, 5 canonical metrics with values/percentiles/status, and benchmark availability. This clearly differentiates it from sibling tools like get_operator_similarity or get_composite_score, which target different aspects of operator data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is the entry point for basic operator profile information, including metrics and benchmark availability. It does not explicitly state when to choose this tool over related siblings such as compare_operator_to_reference or get_operator_similarity, so the usage guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_operator_similarityA
Read-only

Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching.

ParametersJSON Schema
NameRequiredDescriptionDefault
n_neighborsNoNumber of nearest neighbors to return
operator_idYesPseudonymous operator ID (e.g., op_001)

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteYes
normalizationYes
cluster_qualityYes
distance_metricYes
nearest_neighborsYes
query_operator_idYes
cluster_descriptionYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true, so the description carries the burden of explaining behavior. It adds meaningful detail: percentile-rank normalization, Euclidean distance across 5 canonical metrics, computation from raw measurements, and the explicit warning that it does not perform personality matching. This goes well beyond the read-only annotation without contradicting it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three short sentences with no filler. The core purpose and method are front-loaded, the data source is clarified, and the crucial 'NOT personality matching' disambiguation is included without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return-value details are covered elsewhere. The description provides enough context for an agent to understand methodology, data source, and outcome scope. There is no explicit guidance on when to choose this over compare_operator_to_reference or get_cohort_distribution, but the core operational context is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description does not add meaningful parameter-level detail beyond what the schema provides, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific function: 'Nearest-neighbor operator search' with a defined methodology and output ('Returns comparable operators/cohorts'). It also distinguishes itself from what it is not ('NOT personality matching'), which differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when you need nearest-neighbor or similarity-based operator comparisons. It gives an explicit exclusion ('NOT personality matching') but does not name alternative tools or provide direct when-to-use versus when-not-to-use guidance beyond that single exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_operator_system_decompositionA
Read-only

Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idNoOptional: filter to a single operator's decomposition

Output Schema

ParametersJSON Schema
NameRequiredDescription
metricsYes
summaryYes
operator_idNo
systems_comparedYes
operators_analyzedYes
total_observationsYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already establishes that this is a safe read operation. The description adds meaningful behavioral context by explaining the computation method, grouping by platform, and the interpretative output of the variance decomposition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured: it states the methodology first, then the data source, then the practical interpretation. Every sentence adds distinct value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only analytical tool with one optional, well-documented parameter and an output schema, the description provides sufficient context on purpose, computation, and interpretation. Nothing essential is missing for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage for the single optional parameter, operator_id, with a clear filter description. The tool description adds no further parameter-level semantics, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific analytical verb and resource: a two-way ANOVA-style decomposition partitioning variance into operator, system, and interaction effects. This clearly differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference, which address different questions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: use this tool to determine whether operator capability or system choice drives performance. It does not explicitly mention exclusions or alternative tools, but the intended use case is well implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_org_topologyA
Read-only

Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryYes
total_teamsYes
team_topologiesYes
total_operatorsYes
platform_adoptionYes
capability_concentrationYes
single_points_of_failureYes
cross_team_complementarityYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description notes the tool is 'Computed from raw measurements,' which indicates an aggregating, read-only computation consistent with the readOnlyHint annotation. It adds value beyond the annotation by describing provenance and the analytical nature of the result, though it does not address things like auth or rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence that front-loads the core concept and then lists specific value-adding capabilities. There is no filler or redundant phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With zero parameters, an output schema, and a readOnly annotation, the description is largely complete for selecting and invoking the tool correctly. It thoroughly describes the content of the topology map, though it could briefly mention what kinds of questions or scenarios it is best used for.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the baseline for such tools is 4. There is no parameter ambiguity to resolve, so the description does not need to compensate for missing parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource ('Organization-level AI topology map') and enumerates the exact analytical dimensions it provides, such as team-level metric distributions and Gini coefficient. This distinguishes it clearly from sibling tools like get_operator_profile and get_cohort_distribution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied through the organization-level scope and the list of team/capability metrics, but the description does not explicitly state when to prefer this tool over alternatives or mention any exclusions. It does not name sibling alternatives despite a large sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_outcome_correlationA
Read-only

Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryYes
claim_statusYes
correlationsYes
evidence_gradeYes
operators_analyzedYes
lineages_with_outcomesYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the readOnlyHint annotation by disclosing the statistical method (Pearson r), the data sources, and the critical interpretive boundary that results are ASSOCIATION/OBSERVATIONAL and never CAUSATION. This prevents misuse of the output and shows genuine behavioral transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences with no filler: the action, the computation method, and the interpretive caveat are all front-loaded and clearly organized. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters, has an output schema, and carries a readOnly annotation, the description covers the essential behavioral and methodological context. It explains what is computed, from what data, and how results should be interpreted, leaving no critical gaps for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there are no parameter semantics for the description to clarify. The baseline for zero-parameter tools is 4, and the description adds context about what data is used without needing to document arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Correlate') and identifies the exact resources involved: micro-eval metrics, outcome quality scores, and cycle times via lineage. This clearly distinguishes it from sibling tools like get_lineage_summary or get_composite_score by stating the analytical method and result framing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use this tool: when the agent needs a Pearson correlation between micro-eval metrics and outcome/cycle-time data through lineage. It does not explicitly name alternatives or exclusions, but the context is strong enough that an agent would not confuse it with a general lineage or score retrieval tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_pilot_statusA
Read-only

Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
windowNo
cohort_idYes
providersNo
syntheticYes
data_qualityNo
total_operatorsYes
observation_countYes
eligible_operatorsNo
active_interventionsNo
metric_registry_versionNo
reference_field_versionNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description states the status is computed from raw observations and that the data is from a labeled synthetic pilot. This tells the agent the output is an aggregate view rather than raw data, which adds meaningful behavioral context without contradicting the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no wasted words. The first sentence names the resource and its key contents, and the second adds data provenance and synthetic labeling. It is well-structured and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters, a readOnlyHint annotation, and an output schema present, the description provides the necessary context: data source, synthetic nature, and aggregate computation. Nothing essential is missing for an agent to call this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and the input schema has no properties, so there is no parameter burden for the description to carry. The baseline of 4 applies here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and names a concrete resource ('pilot status overview'), then enumerates the contents: cohort size, observation count, date range, data quality, and active interventions. It clearly conveys what the tool returns, though it does not explicitly differentiate it from sibling tools like get_data_quality or get_intervention_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives useful context—'Computed from raw observations' and '50-operator synthetic pilot'—which implies when this tool is appropriate. However, it does not explicitly state when to use it over alternatives or provide exclusions, so the guidance is implied rather than direct.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workflow_fitA
Read-only

Get workflow fit analysis — operator/workflow fit scores across workflow stages.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteNo
stagesYes
syntheticYes
workflow_idYes
observationsYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotation readOnlyHint=true already communicates that this is a safe read operation, and the description's 'Get' is consistent with that. The description adds modest context by stating the analysis spans workflow stages, but it does not describe any further behavioral traits such as aggregation level or data source.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence delivers the tool's purpose and scope with no filler. The main verb and resource are front-loaded, and the em-dash addition is the only elaboration, which is exactly the amount needed for a zero-parameter read tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless read-only tool with an output schema, the description is sufficiently complete: it states what the tool returns and the domain (operator/workflow fit). It could add a short note on how to interpret the scores, but this is not necessary given the output schema and annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, and the schema description coverage is 100%, so there are no parameter semantics for the description to clarify. The 0-param baseline of 4 applies because the description need not compensate for schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb (Get) and a concrete resource (workflow fit analysis), and clarifies the content as operator/workflow fit scores across workflow stages. It does not explicitly contrast with any sibling tool, but 'workflow fit' is distinct enough among the listed siblings to identify the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_diagnostics. There are no exclusions, but also no explicit context or when-not-to-use conditions, so an agent must infer applicability from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_pilot_optionsA
Read-only

List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
syntheticYes
eval_familiesYes
benchmark_classesYes
canonical_metricsYes
intervention_typesYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotation readOnlyHint=true already establishes that this operation is safe and non-mutating, and the description's 'List' verb is consistent with that. The description adds some useful behavioral context by naming the categories and counts that will be returned, but it does not disclose details like response structure, ordering, or whether the counts are fixed or dynamic. This matches the baseline expectation for a read-only catalog tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the action ('List') and the object ('available pilot options'), then efficiently packs the key categorical details into an em-dash list. There is no filler, repetition, or unnecessary elaboration; every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless, read-only catalog tool with an output schema available, this description is complete enough for an agent to understand what it queries and what type of information it will receive. The enumeration of categories covers the tool's content, and the output schema handles return-value specifics. Nothing needed to invoke the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema has 100% coverage, so there is no parameter ambiguity to resolve. The 0-parameter baseline is 4, and the description appropriately does not need to explain parameter semantics that do not exist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') and resource ('available pilot options'), and further clarifies the scope by enumerating the four categories it covers: canonical metrics, eval families, benchmark classes, and intervention types. This clearly distinguishes it from the sibling tools, most of which are get/verify/create operations. However, it does not explicitly name a sibling or contrast itself, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention contexts like configuration setup, discovery workflows, or any exclusion cases. The intended use is only implied by the tool's name and simple read-only nature, which is not enough for strong usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

record_workflow_observationA

Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

ParametersJSON Schema
NameRequiredDescriptionDefault
notesNoFree-text notes about the observation context — task type, AI system used, environmental factors
fit_scoreNoWorkflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements.
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)
workflow_idYesWorkflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing)

Output Schema

ParametersJSON Schema
NameRequiredDescription
toolYesTool name that was called
errorYesError type if authorization fails (e.g., 'authorization_required')
messageYesHuman-readable status message
operator_idNoOperator ID that was observed (on success)
recorded_atNoISO timestamp of recording (on success)
workflow_idNoWorkflow ID that was observed (on success)
observation_idNoID of the recorded observation (on success)

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide minimal safety hints (openWorld, non-idempotent, non-destructive). The description adds important behavioral context: REQUIRES AUTHORIZATION, contact for pilot access, and that the synthetic demo returns an authorization notice instead of performing the write. This goes beyond annotations and alerts the agent to the gated nature of the operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: purpose, key concept, score range, read alternative, authorization, and demo behavior. The purpose is front-loaded, and the description is neither bloated nor under-specified.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, domain definition, score semantics, the appropriate read alternative, authorization requirements, and expected behavior in the demo environment. The output schema exists and likely covers return values; the description provides enough context for an agent to decide when and how to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so each parameter already has a description. The main description adds domain context (what workflow fit measures, score range) which is also present in the schema's property descriptions. No additional parameter-level detail is provided beyond what the schema contains.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses a specific verb ('record') and clearly states the resource ('workflow fit observation linking an operator to a workflow stage with a fit score'). It explicitly names the read counterpart (get_workflow_fit), distinguishing this write operation from its sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the alternative for reading existing observations ('Use get_workflow_fit to read existing observations') and specifies authorization requirements, implying limited access. Does not provide an explicit 'when not to use' beyond the read case, but the write/read distinction is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_pilot_configurationA
Read-only

Validate a pilot configuration before deployment. Returns valid status with warnings and errors.

ParametersJSON Schema
NameRequiredDescriptionDefault
configurationNoPilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes

Output Schema

ParametersJSON Schema
NameRequiredDescription
validYes
errorsYes
messageNo
warningsYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already covers safety; the description adds that the tool returns validation status with warnings and errors and is intended pre-deployment. It does not disclose details like required configuration properties or failure behavior, but with annotations and an output schema this is acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler; the key purpose is front-loaded and the return behavior is stated in one additional clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a validation tool with one well-documented parameter, a readOnlyHint annotation, and an output schema, the description covers the essential decision: call it before deployment to get warnings/errors. It could mention prerequisites or valid value sources, but the schema already references list_pilot_options.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single 'configuration' parameter is fully described in the schema (JSON pilot configuration, link to list_pilot_options), and schema coverage is 100%. The description adds no parameter-level details, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('Validate'), resource ('pilot configuration'), and lifecycle context ('before deployment'), and describes the outcome ('Returns valid status with warnings and errors'). It is clearly a read-only validation step, but it does not explicitly distinguish itself from nearby siblings such as verify_change or create_pilot_configuration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'before deployment' provides a clear usage context, signaling that this tool is the pre-deployment check. It does not name alternatives or exclusion criteria, so an agent must infer when to use verify_change or list_pilot_options instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_changeA
Read-only

Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION.

ParametersJSON Schema
NameRequiredDescriptionDefault
operator_idYesPseudonymous operator ID (e.g., op_001, op_003, op_034)
intervention_idNoIntervention ID (e.g., intv_001)

Output Schema

ParametersJSON Schema
NameRequiredDescription
labelYes
messageYes
syntheticYes
operator_idYes
composite_scoreYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already covers non-mutating behavior, and the description adds valuable interpretation guidance: results are associative, never causal. It also reveals the pre/post comparison methodology, which is beyond what the annotation or schema conveys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence with a clear statistical caveat appended. Every part earns its place, and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema and readOnlyHint annotation, the description covers the essential purpose and a key interpretive limitation. It does not explicitly discuss when to prefer this over correlated sibling tools, but the provided information is sufficient for most selection and invocation scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both operator_id and intervention_id. The description adds no additional parameter meaning, but the baseline of 3 applies because the schema carries the burden fully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Verify') and resource ('a measured change after intervention') and clarifies the pre/post comparison scope. This clearly separates it from siblings like assign_intervention or get_intervention_status, which focus on assignment or status rather than change verification.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it — after an intervention with pre/post measurements — but it does not explicitly state exclusions or name alternative tools. An agent can infer context, but there is no explicit routing guidance relative to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.4.3
    • Changedclose_intervention8 fields changed
      • changedInput schema / properties / intervention_id / description
        Previous value: -"Intervention ID (e.g., intv_001)"New value: +"Intervention ID to close (e.g., intv_001, intv_007). Must be an active intervention."
      • changedInput schema / properties / outcome_notes / description
        Previous value: -"Free-text notes about the intervention outcome"New value: +"Free-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved"
      • addedOutput schema / properties / closed_at
        Added value: +{
        +  "description": "ISO timestamp of closure (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / error / description
        Added value: +"Error type if authorization fails (e.g., 'authorization_required')"
      • addedOutput schema / properties / intervention_id
        Added value: +{
        +  "description": "ID of the closed intervention (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / message / description
        Added value: +"Human-readable status message"
      • addedOutput schema / properties / status
        Added value: +{
        +  "description": "New status of the intervention (e.g., 'closed')",
        +  "type": "string"
        +}
      • addedOutput schema / properties / tool / description
        Added value: +"Tool name that was called"
    • Changedcreate_experiment8 fields changed
      • changedInput schema / properties / configuration / description
        Previous value: -"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes"New value: +"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration."
      • changedInput schema / properties / name / description
        Previous value: -"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison')"New value: +"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha')"
      • addedOutput schema / properties / created_at
        Added value: +{
        +  "description": "ISO timestamp of creation (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / error / description
        Added value: +"Error type if authorization fails (e.g., 'authorization_required')"
      • addedOutput schema / properties / experiment_id
        Added value: +{
        +  "description": "ID of the created experiment (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / message / description
        Added value: +"Human-readable status message"
      • addedOutput schema / properties / status
        Added value: +{
        +  "description": "Initial status of the experiment (e.g., 'draft', 'configured')",
        +  "type": "string"
        +}
      • addedOutput schema / properties / tool / description
        Added value: +"Tool name that was called"
    • Changedcreate_pilot_configuration12 fields changed
      • changedInput schema / properties / cohort_size / description
        Previous value: -"Number of operators in the pilot cohort (e.g., 25, 50, 100)"New value: +"Number of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size."
      • changedInput schema / properties / duration_days / description
        Previous value: -"Pilot duration in days (e.g., 30, 60, 90)"New value: +"Pilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability."
      • changedInput schema / properties / metrics / description
        Previous value: -"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction'])"New value: +"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog."
      • addedOutput schema / properties / configuration / properties / benchmark_classes
        Added value: +{
        +  "description": "Selected benchmark class IDs",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • addedOutput schema / properties / configuration / properties / cohort_size / description
        Added value: +"Number of operators in the cohort"
      • addedOutput schema / properties / configuration / properties / duration_days / description
        Added value: +"Pilot duration in days"
      • addedOutput schema / properties / configuration / properties / eval_families
        Added value: +{
        +  "description": "Selected evaluation family IDs",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • addedOutput schema / properties / configuration / properties / governance
        Added value: +{
        +  "description": "Governance metadata including evidence labels and usage constraints",
        +  "type": "object"
        +}
      • addedOutput schema / properties / configuration / properties / metrics / description
        Added value: +"Selected metric IDs"
      • addedOutput schema / properties / synthetic / description
        Added value: +"Whether data is synthetic"
      • addedOutput schema / properties / valid / description
        Added value: +"Whether the configuration passed validation"
      • addedOutput schema / properties / warnings
        Added value: +{
        +  "description": "Non-blocking validation warnings",
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
    • Changedrecord_workflow_observation10 fields changed
      • changedInput schema / properties / fit_score / description
        Previous value: -"Workflow fit score (0.0-1.0)"New value: +"Workflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements."
      • changedInput schema / properties / notes / description
        Previous value: -"Free-text notes about the intervention assignment"New value: +"Free-text notes about the observation context — task type, AI system used, environmental factors"
      • changedInput schema / properties / workflow_id / description
        Previous value: -"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture)"New value: +"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing)"
      • addedOutput schema / properties / error / description
        Added value: +"Error type if authorization fails (e.g., 'authorization_required')"
      • addedOutput schema / properties / message / description
        Added value: +"Human-readable status message"
      • addedOutput schema / properties / observation_id
        Added value: +{
        +  "description": "ID of the recorded observation (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / operator_id
        Added value: +{
        +  "description": "Operator ID that was observed (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / recorded_at
        Added value: +{
        +  "description": "ISO timestamp of recording (on success)",
        +  "type": "string"
        +}
      • addedOutput schema / properties / tool / description
        Added value: +"Tool name that was called"
      • addedOutput schema / properties / workflow_id
        Added value: +{
        +  "description": "Workflow ID that was observed (on success)",
        +  "type": "string"
        +}
  2. 27 tool updatesv0.4.0
    • First observedassign_intervention
    • First observedattach_outcome_dataset
    • First observedclose_intervention
    • First observedcompare_operator_to_reference
    • First observedcreate_experiment
    • First observedcreate_pilot_configuration
    • First observedfind_usage_operation_divergence
    • First observedget_cohort_distribution
    • First observedget_composite_score
    • First observedget_composite_score_summary
    • First observedget_data_quality
    • First observedget_diagnostics
    • First observedget_executive_dashboard
    • First observedget_intervention_status
    • First observedget_lineage_chain
    • First observedget_lineage_summary
    • First observedget_operator_profile
    • First observedget_operator_similarity
    • First observedget_operator_system_decomposition
    • First observedget_org_topology
    • First observedget_outcome_correlation
    • First observedget_pilot_status
    • First observedget_workflow_fit
    • First observedlist_pilot_options
    • First observedrecord_workflow_observation
    • First observedvalidate_pilot_configuration
    • First observedverify_change

TDQS

A3.8/5.0

Scored across 27 tools

Disambiguation4/5

Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.

Tool Count2/5

With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.

Completeness4/5

The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers