Systems Intelligence Performative Commercial Benchmarking
This server is a read-and-write MCP endpoint for evaluating and piloting how people operate AI systems, using synthetic operator cohorts and content-free token telemetry to compute metrics, benchmarks, diagnostics, interventions, lineage, and organizational insights.
Inspect pilot health: get cohort size, observation counts, date ranges, data quality, and active interventions.
Review operator performance: pull pseudonymous operator profiles with the 5 canonical metrics (yield, leverage, token SNR, construction, divergence), percentiles, and benchmark positions.
Benchmark and compare: get cohort metric distributions, composite scores, reference-population comparisons, and nearest-neighbor operator similarity.
Diagnose and decompose: run pattern detectors, usage-operation divergence analysis, workflow fit analysis, and operator-vs-system effect decomposition.
Intervene and verify: assign/close interventions, record workflow observations, create experiments, and verify pre/post changes (labeled ASSOCIATION, never causation).
Trace outcomes and lineage: follow full lineage chains from state through transformations to committed outcomes, summarize lineage across the cohort, and correlate metrics with outcome quality.
Explore organizational topology: see team-level distributions, capability concentration, platform adoption, single points of failure, and cross-team complementarity.
Generate configurations: list pilot options, validate or create pilot configurations, and get executive dashboard generation commands.
Write with authorization: create experiments, assign interventions, record observations, and attach external outcome datasets — all requiring pilot authorization.
Allows attaching external outcome datasets from GitHub for join analysis, enabling correlation of GitHub activity with operator performance metrics.
MO§ES™ Enterprise: AI Operator Evaluation Platform
Live: mos2es.org: Systems intelligence for the human-AI operating system. The new standard in performative metrics and benchmarks for AI operators. Baselines system intelligence for everyday operations and AI workflows. Content-free token telemetry. Never your prompts.
Enterprise AI operator evaluation. Measure how people operate AI, not the AI model itself.
MO§ES™ curates a company's system intelligence: how effectively people drive AI systems across tools, tasks, workflows, and conditions. The same way BI sees the business, MO§ES™ sees how the business operates AI.
Table of Contents
Related MCP server: Signomy
What is MO§ES™?
MO§ES™ is an enterprise AI operator evaluation platform. It measures how people operate AI systems, not the AI models themselves, not usage volume, not self-reported proficiency. Using content-free token telemetry (input, output, cache read, cache write, no prompt text, no response text), MO§ES™ baselines system intelligence across:
Operators: how effectively individuals drive AI
Teams: cohort distributions and capability topology
Workflows: where AI fits in the work, not just whether it's used
Organizations: cross-team benchmarking and capability mapping
The platform builds bespoke evals around your workflows, roles, and AI systems, benchmarks performance against internal and external reference populations, diagnoses capability gaps and divergence patterns, tests targeted interventions, and re-measures what changes.
The positioning
BI sees the business. MO§ES™ sees how the business operates AI.
LMSYS benchmarks models. Braintrust evaluates product outputs. Langfuse traces LLM calls. WakaTime tracks time. CostHawk tracks spend. None of them see the system intelligence of everyday operations and AI workflows. MO§ES™ is the missing layer.
The architecture
MO§ES™ → Upsilon → SigRank | SignalAF
MO§ES™ = governance framework / methodology (this repo)
Upsilon = measurement engine / enterprise product (the engine that measures)
SigRank = public leaderboard / benchmark / proof surface
SignalAF = public distribution / platform brand
Yield (Υ) = metric inside Upsilon:
(cache_read × output) / input²
The MO§ES™ ecosystem
Repo / Site | What it is | URL |
MO§ES™ Enterprise (this repo) | The platform: Python eval engine, CLI, MCP server, demo, promo, enterprise demo, workers. Powered by Upsilon. | |
SigRank | Public leaderboard / benchmark / proof surface for AI operator evaluation | |
SignalAF | Public brand and distribution surface | |
SigRank MCP | The instrument: extracts token pillars, computes cascade, submits to leaderboard |
|
SIGNOMY | Governed AI agent marketplace: ranked agents form teams, run missions, earn revenue | |
SigEconomy | Public LLM operator evals: read-only leaderboard, SEO/AEO surface |
Live surfaces
Surface | URL | What |
Promo site | Marketing, methodology, demo, comparisons, booking | |
Enterprise demo | Interactive product walkthrough (evaluate → diagnose → workflow → compare) | |
MCP server | 27 tools (22 read + 5 write), 5 prompts, 6 resources, streamable HTTP | |
MCP server info | Server info JSON (version, tool count, transport) | |
MCP server card | Full server card with all tool schemas | |
OpenAPI spec | REST API specification | |
LLM guidance | llms.txt for AI agents and crawlers | |
Sitemap | XML sitemap | |
Book a demo | B2B demo booking with structured intake form |
Quick start
Run the demo (one-liner, no clone needed)
curl -sL https://mos2es.org/demo/run.py | python3 -This clones the repo to a temp directory, installs rich, and runs the full 11-step demo pipeline. Requires Python 3.10+ and git.
Clone and run
git clone https://github.com/SunrisesIllneverSee/Moses_Enterprise_B2BPilot_.git
cd Moses_Enterprise_B2BPilot_/_01_platform
pip install rich
python3 -m src.cli.main demo fullRun the test suite
cd _01_platform
python3 -m pytest tests/ -q
# 676 tests passRun the MCP server locally
cd _01_platform
pip install mcp
python3 -m src.mcp_server.serverUse the remote MCP server
Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
{
"mcpServers": {
"moses": {
"url": "https://mcp.mos2es.org/mcp",
"transport": "http"
}
}
}Or call tools directly:
curl -s -D /tmp/h -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' > /dev/null
SID=$(grep -i "^mcp-session-id:" /tmp/h | tr -d '\r\n' | sed 's/.*: //')
curl -s -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" -H "MCP-Session-Id: $SID" \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | python3 -m json.toolThe 11-step demo pipeline
Step | Name | What it does |
1 | LOAD | Load 50-operator synthetic cohort |
2 | EVALUATE | Compute per-observation metrics |
3 | BENCHMARK | Compute percentile positions |
4 | DIAGNOSE | Run pattern detectors |
5 | OPERATOR×SYSTEM | Decompose operator vs system effects |
6 | INTERVENE | Assign targeted interventions |
7 | RE-EVALUATE | Re-measure post-intervention |
8 | OUTCOME LINEAGE | Trace observations → transformations → artifacts → outcomes |
9 | REPORT | Generate markdown + PDF pilot readout |
10 | VISUALIZE | Generate 9 architecture diagrams |
11 | DASHBOARD | Export executive HTML dashboard |
The demo runs on fully synthetic data. No real operators, no real prompt text, no real API calls. Deterministic: running twice produces identical results.
The 8 canonical metrics
Metric | Formula | What it measures |
Yield (Υ) |
| Productive output relative to input squared |
Leverage |
| Context reuse efficiency |
Token SNR |
| Signal-to-noise ratio in token usage |
Log Leverage (10xDEV) |
| Leverage on a logarithmic scale |
Construction |
| New context built vs reused |
Velocity |
| Raw output efficiency |
Scale V |
| Total volume on a log scale |
Efficiency |
| Composite display metric |
Where I = input, O = output, R = cache read, W = cache write.
15 evaluation families. 13 benchmark classes. 4 measurement levels (Operator, Team, Workflow, Organization).
Repository structure
_01_platform/ Python platform (the product)
src/ Source code (domain, metrics, analysis, diagnostics, CLI, MCP server)
tests/ 676 tests
demo_data/ Synthetic demo cohort (50 operators, 1,668 observations)
scripts/ Utility scripts
schemas/ JSON schemas
_02_demo-website/ Enterprise demo site (enterprise.mos2es.org)
_03_promo-site/ Promo site (mos2es.org)
vs/ 16 competitor comparison pages
alternatives/ 8 alternatives listicle pages
concepts/ 10 concept explainer pages
guides/ 4 how-to guides
demo/run.py One-liner demo runner
.well-known/mcp.json MCP server discovery
_04_onepager/ One-pager site
_workers/ Cloudflare Workers
promo-worker/ mos2es.org worker (static assets + AEO/SEO/GEO headers)
moses-worker/ enterprise.mos2es.org worker (static assets)
onepager-worker/ One-pager worker (static assets)
mcp-worker/ MCP server worker (computes live from raw data, 27 tools)MCP server
The MCP server exposes 27 tools (22 read + 5 write), 5 prompts, and 6 resources over streamable HTTP at https://mcp.mos2es.org/mcp. No authentication required for the public server.
Read tools include: cohort stats, operator profiles, metric distributions, benchmark positions, divergence findings, intervention outcomes, outcome lineage, workflow fit, org topology, and more.
Write tools include: create intervention, assign intervention, record outcome, create eval configuration, create pilot configuration.
Prompts include: operator evaluation summary, intervention recommendation, cohort health report, workflow fit analysis, pilot scoping guide.
Resources include: canonical metric definitions, metric registry, pilot status, pilot options, governance conventions, cohort operator list.
The server computes live from raw observation data, no pre-computed results. Every call runs the actual scoring, benchmarking, and diagnostic engines.
Server card
curl -s https://mcp.mos2es.org/.well-known/mcp/server-card.json | python3 -m json.toolReturns the full server card with all 27 tool schemas, transport info, and metadata.
Cloudflare Workers
Four Workers deploy from this repo:
Worker | Domain | What |
| mos2es.org | Promo site + AEO/SEO/GEO headers + agent discoverability (llms.txt, sitemap, OpenAPI, MCP links) |
| enterprise.mos2es.org | Enterprise demo site |
| (workers.dev) | One-pager |
| mcp.mos2es.org | MCP server (27 tools, live computation) |
Deploy
npm install -g wrangler
cd _workers/promo-worker && wrangler deploy
cd _workers/moses-worker && wrangler deploy
cd _workers/onepager-worker && wrangler deploy
cd _workers/mcp-worker && wrangler deployMCP worker data sync
The MCP worker computes live from raw data files (observations.js, lineages.js) auto-generated from the platform's demo data. To regenerate after updating demo data:
cd _workers/mcp-worker/src
python3 -c "
import json
with open('../../../_01_platform/demo_data/observations.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('observations.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
with open('../../../_01_platform/demo_data/lineages.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('lineages.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
"Key conventions
Brand: MO§ES™ (use the section sign §)
All outcome claims are ASSOCIATION, never CAUSATION unless backed by a controlled experiment
Composite score is DEVELOPMENTAL, never PERSONNEL: no punitive use, no employee leaderboard
Operator similarity is metric similarity, NOT personality matching
Content-free telemetry: no prompt text, no output bodies, no code content in observations
No prompt-content surveillance: operator performance is observable from token structure alone
Governance-ready: evidence labels (DEVELOPMENTAL, HYPOTHESIS, ASSOCIATION) on every output
Complement, don't replace
MO§ES™ works alongside your existing BI, eval suites, observability tools, and productivity analytics. It measures the systems intelligence layer they can't see. Not a replacement: the missing piece.
Five pillars
# | Pillar | What |
1 | Systems Intelligence | MO§ES™ sees the human-AI operating system the way BI sees the business. Operator performance, workflow fit, tool selection, capability distribution, intervention outcomes, all measured from structure, not content. |
2 | Standard Operational Performative Metrics | 8 canonical metrics. 15 eval families. 13 benchmark classes. 4 measurement levels. An open, documented spec for measuring how humans operate AI. The reference implementation is MO§ES™. |
3 | Bespoke Enterprise Evals | Your company should not inherit someone else's definition of AI proficiency. Evals built around your workflows, roles, models, and performance questions. |
4 | Complement, Don't Replace | Works alongside your existing BI, eval suites, observability tools, and productivity analytics. The missing piece, not a replacement. |
5 | Privacy-First / Governance | Content-free token telemetry. No prompt text. No surveillance. DEVELOPMENTAL / HYPOTHESIS / ASSOCIATION evidence labels on every output. Governance-ready out of the box. |
License
Apache License 2.0: see LICENSE. Patent pending: Serial No. 63/877,177. For enterprise pilots and commercial support: pilots@mos2es.org.
mos2es.org · Book a demo · Run the demo · MCP server
Built by Deric J. McHenry, Ello Cello LLC
Available Tools
27 toolsassign_interventionA
Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access.
| Name | Required | Description | Default |
|---|---|---|---|
| notes | No | Free-text notes about the intervention assignment | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| intervention_type | Yes | Intervention type from catalog (e.g., prompt_template, context_window_expansion, model_switch) |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | |
| error | Yes | |
| message | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey non-idempotent and non-destructive behavior, and the description adds a valuable behavioral constraint: authorization is required and pilot access must be arranged. It does not contradict the annotations, though it could say more about side effects of creating an assignment.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the first sentence states the action, and the second communicates the operational prerequisite and contact path. There is no redundant repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the full parameter schema, output schema, and annotations, the description covers the mechanics of invocation and the authorization barrier. However, it leaves selection context unaddressed—when to assign versus close an intervention—and offers limited detail on the assignment's effect, making it adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are fully documented in the input schema with descriptions and examples, including operator_id and intervention_type. The tool description itself adds no additional parameter semantics, so the baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the verb (assign), resource (targeted intervention), and recipient (operator), making the core operation clear. It is distinguishable from the sibling close_intervention by the opposing action, though it does not explicitly call out that contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no when-to-use guidance or alternative routing, and it does not mention relationships to siblings like close_intervention or create_experiment. The only usage-related information is the authorization prerequisite, which is not sufficient for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
attach_outcome_datasetA
Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION.
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | Data format (e.g., 'json', 'csv', 'jsonl') | |
| source | Yes | External outcome data source name (e.g., 'jira', 'github', 'linear') |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | |
| error | Yes | |
| message | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description adds meaningful behavioral context: it warns that outcome joins are association-only and explicitly flags that authorization is required. This is valuable for an agent deciding whether and how to invoke the tool. Annotations already cover idempotency and destructiveness, so the additional caveats push this above baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero filler. It front-loads the primary purpose and then packs the most important constraints—association-not-causation and authorization—into a compact, readable warning. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, full schema coverage, and the presence of an output schema, the description covers the key behavioral caveats an agent needs. It could theoretically mention the effect on downstream analysis or concrete when-not-to-use scenarios, but it is sufficiently complete for a tool that attaches a dataset.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters, including examples. The description adds no parameter-level detail, which is acceptable because the schema carries the burden. Per the baseline for high schema coverage, a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and resource: 'Attach external outcome dataset for join analysis.' It clearly conveys what the tool does and is distinguishable from the sibling tools, which are mostly read/get operations. It does not explicitly contrast itself with a specific sibling, but the verb+resource combination is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when an external outcome dataset needs to be attached for join analysis, but it does not explicitly state when to use this tool over alternatives or mention any exclusions. The causation warning and authorization requirement provide context, but no comparative guidance is given versus siblings like get_outcome_correlation or create_experiment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
close_interventionA
Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| outcome_notes | No | Free-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved | |
| intervention_id | Yes | Intervention ID to close (e.g., intv_001, intv_007). Must be an active intervention. |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| status | No | New status of the intervention (e.g., 'closed') |
| message | Yes | Human-readable status message |
| closed_at | No | ISO timestamp of closure (on success) |
| intervention_id | No | ID of the closed intervention (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent=false and destructive=false, and the description adds significant behavior: the intervention becomes ineligible for verify_change after closing, and in the synthetic demo the call returns an authorization notice rather than performing the close. This is valuable context beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, prerequisites, note content guidance, post-closing consequence, authorization requirement, and demo behavior. The structure is front-loaded with the core action and then expands with necessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers what the tool does, when it is valid, what happens after, who is allowed, and what to expect in a demo environment. With only two simple parameters, nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reinforces the meaning of outcome_notes by specifying what should be included, but it largely restates the schema rather than adding new detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('close'), explicit resource ('intervention'), and the outcome ('mark it complete'). The reference to verify_change comparisons distinguishes it from its most related sibling, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly specifies prerequisites (must exist and be active) and postconditions (no longer eligible for verify_change). The authorization requirement adds important gatekeeping context, though it does not explicitly name alternative tools for comparison.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_operator_to_referenceARead-only
Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field.
| Name | Required | Description | Default |
|---|---|---|---|
| reference | No | Reference population name | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| reference | Yes | |
| synthetic | Yes | |
| comparisons | Yes | |
| operator_id | Yes | |
| reference_version | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already signals a safe read operation, and the description adds useful behavioral context by stating it computes from raw metrics and reference field and returns specific comparison components. There is no contradiction with the annotation, and no destructive or side-effect behaviors need disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences, with the core purpose front-loaded in the first sentence. Every sentence adds distinct value (purpose, return contents, data source) without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a complete input schema and readOnlyHint annotation, the description sufficiently covers purpose and return categories; the presence of an output schema also relieves the need to detail return structures. Minor missing context includes how to select a reference population and potential relationships to sibling comparison tools, but these are not critical for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with operator_id and reference already documented. The description's phrase 'reference field' adds slight ambiguity but essentially restates the reference parameter; it does not provide meaningful additional semantic detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares an operator to a reference population and lists the return categories (benchmark selection, comparison group, metric comparison). However, it does not explicitly differentiate itself from sibling tools like get_operator_similarity or get_cohort_distribution, which may serve similar comparison purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an operator-to-reference comparison is needed, and the return types suggest analytical contexts. It offers no explicit when-to-use/when-not-to-use guidance or alternative tool references, leaving the selection inference to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_experimentA
Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha') | |
| configuration | No | Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration. |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| status | No | Initial status of the experiment (e.g., 'draft', 'configured') |
| message | Yes | Human-readable status message |
| created_at | No | ISO timestamp of creation (on success) |
| experiment_id | No | ID of the created experiment (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate openWorldHint, non-idempotent, non-destructive. The description adds valuable behavioral context: it enforces the ASSOCIATION-not-CAUSATION evidence standard, requires authorization, and discloses that in the synthetic demo it returns an authorization notice. This goes beyond the annotations and helps set expectations, though it does not detail side effects or persistence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise and well-structured. It opens with the core purpose, then explains the workflow and evidence standard, and ends with authorization and demo behavior. Every sentence adds new information, though the authorization sentence could arguably be more compact. Still, it is efficient and each sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema and well-documented parameters. The description covers key context: the prerequisite workflow, the evidence standard, authorization, and demo behavior. It does not explicitly mention return values, but the output schema exists to cover that. The only minor gap is the lack of details about what happens after creation (e.g., storage or side effects), but given the available structured schema and annotations, this is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% description coverage for both parameters. The description adds a small extra nuance by explaining that the 'configuration' parameter can be generated by create_pilot_configuration, and references list_pilot_options for available metrics. This is helpful, but the schema already carries the main semantic load, so a baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create an experiment configuration for controlled comparison studies.' It specifies the verb (create), the resource (experiment configuration), and the context (controlled comparison studies). It also distinguishes itself from the sibling create_pilot_configuration by explaining that experiments pair a pilot configuration with a hypothesis and measurement plan, and by explicitly instructing to use create_pilot_configuration first.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: 'Use create_pilot_configuration first to build the config, then pass it here.' It also notes authorization requirements and the synthetic demo behavior. While it does not explicitly say 'when not to use,' it provides a clear workflow that guides an agent to the correct alternative when appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_pilot_configurationARead-only
Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard).
| Name | Required | Description | Default |
|---|---|---|---|
| metrics | No | Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog. | |
| cohort_size | No | Number of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size. | |
| duration_days | No | Pilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability. |
Output Schema
| Name | Required | Description |
|---|---|---|
| valid | Yes | Whether the configuration passed validation |
| warnings | No | Non-blocking validation warnings |
| synthetic | Yes | Whether data is synthetic |
| configuration | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation only declares readOnlyHint=true; the description adds that the result is a 'validated configuration object,' that it is 'saveable' rather than persisted, and that governance metadata is embedded (DEVELOPMENTAL label, no punitive use, association-not-causation). These are behavioral details beyond what annotations already provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences with the main action front-loaded, followed by workflow and governance context. There is no filler, though the governance metadata sentence is nonessential for invocation and makes it slightly longer than the leanest possible definition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a generation tool with complete schema documentation, an output schema, and read-only annotations, the description is self-sufficient: it covers prerequisites, downstream validation, return contents, and governance metadata. Nothing essential for selecting and calling the tool is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all three parameters with examples, giving 100% coverage, so the baseline is 3. The description merely restates 'cohort size, duration, selected metrics' at a high level and adds no parameter-specific semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Generate a saveable pilot configuration JSON from parameters.' It lists the returned object's components (cohort size, duration, selected metrics, eval families, benchmark classes) and distinguishes the tool from siblings by naming list_pilot_options and validate_pilot_configuration as adjacent workflow steps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit workflow context: 'Use list_pilot_options first to see available options' and 'can be passed to validate_pilot_configuration for pre-deployment checks.' It does not explicitly state when not to use this tool or name alternative tools for different scenarios, so it falls just short of full exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_usage_operation_divergenceARead-only
Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| all_operators | Yes | |
| divergent_operators | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description does not contradict this. It adds useful behavioral context by explaining that it computes usage percentiles from raw token totals, compares them to yield percentiles, and returns all 50 operators with divergence classes. This goes beyond the annotation without overpromising side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences carry full value: the first states the core purpose, and the second explains the method and output. No filler or redundant restatement of the tool name, and key details like 'all 50 operators' and divergence class examples are included efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and an output schema present, the description covers what an agent needs to know: which operators are returned, how divergence is computed, and what categories are produced. There are no configuration choices, side effects, or ambiguous inputs that would require additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description cannot add parameter-level meaning. The schema already covers 100% of a nonexistent parameter set, making the baseline 4 appropriate. The description focuses on what the computation does, which is sufficient given there is nothing to configure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Find operators with usage-operation divergence.' It also explains the computation involved, distinguishing this from generic get_* siblings by focusing on divergence classification rather than simple retrieval. The mention of specific divergence classes further clarifies the intended output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The imperative 'Find operators with usage-operation divergence' clearly indicates when this tool is appropriate: when an agent needs to identify operators whose usage percentile diverges from yield percentile. It does not explicitly name alternative tools or exclusion conditions, but the context is unambiguous enough for an agent to select it correctly from the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_cohort_distributionARead-only
Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations.
| Name | Required | Description | Default |
|---|---|---|---|
| metric | No | Metric: leverage, yield, token_snr, log_leverage, construction | leverage |
Output Schema
| Name | Required | Description |
|---|---|---|
| max | No | |
| min | No | |
| p10 | No | |
| p25 | No | |
| p75 | No | |
| p90 | No | |
| std | No | |
| mean | No | |
| count | Yes | |
| median | Yes | |
| metric | Yes | |
| outliers | No | |
| synthetic | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers the safety profile, so the description does not need to restate that. 'Computed from raw observations' adds useful provenance context beyond the annotation, but the description does not clarify outlier definition, cohort fixedness, or behavior for empty/missing data. Acceptable but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no redundancy. The primary purpose and output statistics are front-loaded, and the second sentence adds a single high-value behavioral note about raw observations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional parameter, a read-only annotation, and an output schema, the description provides enough information to invoke it correctly. It could add context about the exact cohort definition or outlier computation, but those are not required for a successful call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already documents the metric parameter, its default ('leverage'), and allowed values. The description only refers to 'a given metric' and adds no parameter-level semantics beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States exactly what it computes ('Get cohort metric distribution') and lists the precise statistics returned (min, p10, p25, median, p75, p90, max, mean, std, outliers) plus the scope ('50-operator cohort'). This clearly distinguishes it from per-operator tools like get_operator_profile or aggregate summary tools, even though no sibling is named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'across the 50-operator cohort' phrasing gives clear context for when to use this tool: when a distribution-level view of a metric across the cohort is needed. It does not explicitly mention when not to use it or name alternative tools, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_composite_scoreARead-only
Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| name | No | |
| label | Yes | |
| score | Yes | |
| caveats | No | |
| score_id | No | |
| synthetic | Yes | |
| components | Yes | |
| operator_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already signals a safe read operation, and the description goes further by disclosing the computation method (raw metrics normalized via reference percentiles), the exact component weights, and the fact that data is synthetic. This is strong behavioral context beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the core purpose and scale come first, followed by computation details and caveats. Every sentence adds meaningful information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read-only tool with an output schema present, the description covers the scoring domain, input identification, weighting components, and data caveat. Nothing critical for correct invocation appears missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description and example for operator_id, so the baseline is 3. The description adds that the score is computed for an operator, but does not provide new parameter-level semantics beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Get'), resource ('developmental composite score'), scope ('for an operator'), and scale (0-100). It also distinguishes itself from personnel-related metrics and provides the component weights, making the tool's purpose unambiguous even without consulting siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like get_composite_score_summary or compare_operator_to_reference. The 'Labeled DEVELOPMENTAL, not PERSONNEL' line hints at appropriate use, but there is no explicit routing guidance or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_composite_score_summaryARead-only
Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| q1 | No | |
| q3 | No | |
| max | Yes | |
| min | Yes | |
| mean | Yes | |
| name | No | |
| count | Yes | |
| label | Yes | |
| median | Yes | |
| weights | No | |
| score_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already indicates a safe read operation, and the description adds meaningful context beyond that: it is computed from per-operator scores, it exposes no individual rankings, and it carries a developmental label. This gives the agent useful expectations about privacy and maturity without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tight and front-loaded: the first clause names the operation and resource, the middle lists the computed statistics, and the final clauses add caveats. Every sentence earns its place, with no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has no parameters and an output schema exists, the description is largely complete: it explains the aggregate nature, the source of computation, and the exposure boundary. The main omission is not routing the agent toward siblings or clarifying what the 'DEVELOPMENTAL' label means operationally.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema has 100% coverage by virtue of being empty. The description therefore does not need to explain parameter meanings. It still adds value by clarifying what the composite summary contains, which indirectly covers how the tool should be interpreted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Get cohort composite score summary' and enumerates the computed statistics (count, min, max, median, mean, Q1, Q3). It clearly indicates the tool returns an aggregate summary rather than raw scores. However, it does not explicitly differentiate itself from sibling tools like get_composite_score or get_cohort_distribution, so some ambiguity remains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_cohort_distribution. It does not state exclusions, prerequisites, or the intended cohort-analysis context. The 'DEVELOPMENTAL' label hints at caution but does not give usable selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_data_qualityARead-only
Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| issues | No | |
| coverage | Yes | |
| validity | Yes | |
| synthetic | Yes | |
| completeness | Yes | |
| operators_covered | Yes | |
| total_observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read operation. The description adds minimal behavioral context by stating the summary is 'computed from raw observations,' which is useful for understanding the data source but does not reveal much else about behavior such as aggregation scope or latency. With annotations covering safety, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the core purpose and then adds one relevant detail about the computation source. Every word earns its place, and there is no redundant or vague filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there are no parameters, the output schema is present, and the readOnlyHint annotation is provided, the description is complete enough for an agent to select and invoke this tool correctly. It specifies what kind of summary is returned and the key dimensions covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so there is no parameter documentation burden on the description. Baseline for 0 parameters is 4, and the description appropriately focuses on what the tool returns rather than parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'data quality summary,' and specifies the key dimensions: completeness, coverage, and validity across the cohort. It does not explicitly differentiate from sibling tools like get_diagnostics or get_composite_score, but 'data quality summary' is specific enough to identify the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used when a data quality summary over a cohort is needed, and the phrase 'across the cohort' gives some context. However, it provides no explicit guidance on when to use this tool versus alternatives such as get_diagnostics or get_cohort_distribution, and it lacks any exclusions or alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_diagnosticsARead-only
Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| status | Yes | |
| patterns | Yes | |
| diagnoses | Yes | |
| synthetic | Yes | |
| operator_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already communicates that this is a safe read operation. The description adds meaningful behavioral context by warning that all diagnoses are hypotheses, not facts, which is important for correct interpretation of results. There is no contradiction between the description and annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. The main action is front-loaded, the data source is provided, and the important caveat about hypothesis status is placed at the end without bloating the description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with a single well-documented parameter and an output schema, the description is complete. It conveys the purpose, data source, and a critical interpretive caveat, leaving no major gap that would prevent an agent from invoking the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with operator_id fully documented as a pseudonymous operator ID. The description itself does not add much parameter-level meaning, but the schema already carries the necessary semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: getting operator diagnostics, and it explains the source of those diagnostics (divergence analysis). It is clear what the tool does, though it does not explicitly distinguish it from related sibling tools like find_usage_operation_divergence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives such as find_usage_operation_divergence or get_data_quality. The context of "divergence analysis" provides only a weak implication of intended use, with no explicit exclusions or selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_executive_dashboardARead-only
Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| command | Yes | |
| message | Yes | |
| features | Yes | |
| governance | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already communicates that this is a safe read operation. The description adds useful context by explaining the dashboard is a self-contained HTML file produced by a CLI command, which suggests what the tool returns. It does not go deeper into output format or side effects, but with annotations and an output schema present, the additional behavior disclosure is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately states the action and resource, then adds one relevant detail about the dashboard being a generated HTML file. Every word earns its place, and there is no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless, read-only getter with an output schema available, the description is largely complete: it names the resource, explains its nature, and points to the CLI command that generates it. It could briefly mention what 'dashboard info' contains, but the output schema covers return structure, so the description is sufficient for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool accepts zero parameters, so there are no parameter semantics to clarify. Schema description coverage is trivially 100%, and the description does not need to explain any inputs. This is the appropriate baseline for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the verb ('Get') and resource ('executive dashboard info'), and adds a specific detail that the dashboard is a self-contained HTML file generated by the CLI. It is distinguishable from sibling tools like get_pilot_status or get_diagnostics because it names the executive dashboard specifically, though it does not explicitly contrast with any sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need the generated executive dashboard HTML file. Mentioning the CLI command 'enterprise export dashboard --output file.html' gives useful context about the artifact's origin. However, it does not provide explicit guidance on when to prefer this tool over alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_intervention_statusARead-only
Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| all | Yes | |
| count | Yes | |
| active | Yes | |
| closed | Yes | |
| synthetic | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already conveys that this is a safe read operation, and the description adds useful scope by specifying that all interventions and their fields are returned. Since there is no mutation or destructive behavior to disclose, this is adequate without being rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that front-loads the action and then enumerates the returned contents. Every part contributes useful information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only tool with an output schema, the description is nearly complete. It could be improved by naming a sibling alternative, but nothing an agent needs to invoke the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%, so there is no parameter meaning the description needs to add. A baseline of 4 is appropriate because there is nothing beyond the schema to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear read operation on the intervention resource and lists the returned fields, so an agent can understand it is a listing/status retrieval tool. It does not explicitly differentiate it from sibling get_pilot_status, but the verb and resource are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus siblings like get_pilot_status or close_intervention. It explains what is returned but not why an agent should select it over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_lineage_chainARead-only
Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_046) |
Output Schema
| Name | Required | Description |
|---|---|---|
| lineage | Yes | |
| synthetic | Yes | |
| data_window | Yes | |
| privacy_class | Yes | |
| reference_version | Yes | |
| validation_status | Yes | |
| metric_registry_version | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is consistent with the readOnlyHint annotation and adds one useful behavioral detail: the chain is 'built from raw lineage and outcome data.' It does not disclose error behavior, ordering guarantees, or data freshness, but the read-only annotation lowers the burden and the description adds some context beyond it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no filler. The main purpose and the data source are each stated once, and the chain definition is front-loaded. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read-only tool with an output schema, the description is nearly complete: it states what is returned, the chain shape, and the data source. The only notable gap is not explicitly framing the relationship to get_lineage_summary, but the output schema and annotations cover much of the remaining context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single parameter operator_id, including a type and an example. The description adds no parameter-specific detail, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Get') and resource ('full lineage chain for an operator'), and enumerates the exact chain steps STATE_A → BI_ACTION → ... → OUTCOME. The word 'full' and the explicit chain order clearly distinguish this from sibling get_lineage_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied: call this when you need the complete lineage chain for an operator. However, there is no explicit when-to-use or when-not-to-use guidance, and no mention of the closely related sibling get_lineage_summary, so an agent must infer the boundary between 'full' and 'summary'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_lineage_summaryARead-only
Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| data_window | Yes | |
| privacy_class | Yes | |
| lineage_summary | Yes | |
| reference_version | Yes | |
| validation_status | Yes | |
| metric_registry_version | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds the useful context that the summary is 'computed from raw lineage data' and exposes aggregate metrics, but it does not disclose other behavioral traits such as result shape, performance characteristics, or whether the summary is precomputed vs computed on demand. The description adds modest value beyond annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The primary action and scope are front-loaded, followed by a compact list of contained values, then a brief provenance note. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, a readOnly annotation, and an existing output schema, the description is complete for invoking the tool. An agent can determine what the tool returns (summary metrics, breakdown, linked outcomes) and that it is a safe read operation. No critical information is missing for a tool of this simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is nothing for the description to document. The input schema has 100% coverage by nature of being empty. The description's mention of cohort-wide aggregation is consistent with the stateless 'across the cohort' nature of the call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Get'), a resource ('lineage summary'), and a scope ('across the cohort'). It enumerates concrete contents (total lineages, workflow breakdown, average micro-eval metrics, outcomes linked), which distinguishes it from the sibling tool get_lineage_chain, which likely covers a single chain rather than an aggregate cohort summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'across the cohort' and nature of a summary imply aggregate use, and 'computed from raw lineage data' hints at a derived view. However, it does not explicitly state when to use this tool versus alternatives like get_lineage_chain, nor does it describe exclusions or cases where a different tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_profileARead-only
Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) |
Output Schema
| Name | Required | Description |
|---|---|---|
| team | No | |
| level | No | |
| pseudonym | No | |
| synthetic | Yes | |
| operator_id | Yes | |
| role_family | No | |
| measurements | Yes | |
| primary_platform | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark the tool as readOnlyHint=true, and the description adds useful behavioral context: metrics are 'computed from raw token observations,' operator IDs are pseudonymous, and the data is synthetic. This helps the agent set expectations about data provenance and privacy without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly structured sentence that front-loads the core purpose and then lists the key content areas. The second sentence adds relevant context about pseudonymity and synthetic data without any fluff. Every element earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter, read-only tool with an output schema present, the description covers the essential facts: what is returned, how metrics are derived, and important caveats about data provenance. The agent has enough information to decide whether to call it and to understand the response shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully documents the single parameter with description and examples, so the description does not need to add much. The description's mention of pseudonymous operator IDs reinforces the parameter format, but adds no fundamentally new semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Get operator profile') and enumerates the exact contents: operator details, 5 canonical metrics with values/percentiles/status, and benchmark availability. This clearly differentiates it from sibling tools like get_operator_similarity or get_composite_score, which target different aspects of operator data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this is the entry point for basic operator profile information, including metrics and benchmark availability. It does not explicitly state when to choose this tool over related siblings such as compare_operator_to_reference or get_operator_similarity, so the usage guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_similarityARead-only
Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching.
| Name | Required | Description | Default |
|---|---|---|---|
| n_neighbors | No | Number of nearest neighbors to return | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001) |
Output Schema
| Name | Required | Description |
|---|---|---|
| note | Yes | |
| normalization | Yes | |
| cluster_quality | Yes | |
| distance_metric | Yes | |
| nearest_neighbors | Yes | |
| query_operator_id | Yes | |
| cluster_description | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=true, so the description carries the burden of explaining behavior. It adds meaningful detail: percentile-rank normalization, Euclidean distance across 5 canonical metrics, computation from raw measurements, and the explicit warning that it does not perform personality matching. This goes well beyond the read-only annotation without contradicting it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences with no filler. The core purpose and method are front-loaded, the data source is clarified, and the crucial 'NOT personality matching' disambiguation is included without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return-value details are covered elsewhere. The description provides enough context for an agent to understand methodology, data source, and outcome scope. There is no explicit guidance on when to choose this over compare_operator_to_reference or get_cohort_distribution, but the core operational context is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description does not add meaningful parameter-level detail beyond what the schema provides, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific function: 'Nearest-neighbor operator search' with a defined methodology and output ('Returns comparable operators/cohorts'). It also distinguishes itself from what it is not ('NOT personality matching'), which differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: when you need nearest-neighbor or similarity-based operator comparisons. It gives an explicit exclusion ('NOT personality matching') but does not name alternative tools or provide direct when-to-use versus when-not-to-use guidance beyond that single exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_operator_system_decompositionARead-only
Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | No | Optional: filter to a single operator's decomposition |
Output Schema
| Name | Required | Description |
|---|---|---|
| metrics | Yes | |
| summary | Yes | |
| operator_id | No | |
| systems_compared | Yes | |
| operators_analyzed | Yes | |
| total_observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already establishes that this is a safe read operation. The description adds meaningful behavioral context by explaining the computation method, grouping by platform, and the interpretative output of the variance decomposition.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: it states the methodology first, then the data source, then the practical interpretation. Every sentence adds distinct value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only analytical tool with one optional, well-documented parameter and an output schema, the description provides sufficient context on purpose, computation, and interpretation. Nothing essential is missing for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single optional parameter, operator_id, with a clear filter description. The tool description adds no further parameter-level semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific analytical verb and resource: a two-way ANOVA-style decomposition partitioning variance into operator, system, and interaction effects. This clearly differentiates it from sibling tools like get_operator_profile or compare_operator_to_reference, which address different questions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: use this tool to determine whether operator capability or system choice drives performance. It does not explicitly mention exclusions or alternative tools, but the intended use case is well implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_org_topologyARead-only
Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| summary | Yes | |
| total_teams | Yes | |
| team_topologies | Yes | |
| total_operators | Yes | |
| platform_adoption | Yes | |
| capability_concentration | Yes | |
| single_points_of_failure | Yes | |
| cross_team_complementarity | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description notes the tool is 'Computed from raw measurements,' which indicates an aggregating, read-only computation consistent with the readOnlyHint annotation. It adds value beyond the annotation by describing provenance and the analytical nature of the result, though it does not address things like auth or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, information-dense sentence that front-loads the core concept and then lists specific value-adding capabilities. There is no filler or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters, an output schema, and a readOnly annotation, the description is largely complete for selecting and invoking the tool correctly. It thoroughly describes the content of the topology map, though it could briefly mention what kinds of questions or scenarios it is best used for.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the baseline for such tools is 4. There is no parameter ambiguity to resolve, so the description does not need to compensate for missing parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific resource ('Organization-level AI topology map') and enumerates the exact analytical dimensions it provides, such as team-level metric distributions and Gini coefficient. This distinguishes it clearly from sibling tools like get_operator_profile and get_cohort_distribution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied through the organization-level scope and the list of team/capability metrics, but the description does not explicitly state when to prefer this tool over alternatives or mention any exclusions. It does not name sibling alternatives despite a large sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_outcome_correlationARead-only
Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| summary | Yes | |
| claim_status | Yes | |
| correlations | Yes | |
| evidence_grade | Yes | |
| operators_analyzed | Yes | |
| lineages_with_outcomes | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the readOnlyHint annotation by disclosing the statistical method (Pearson r), the data sources, and the critical interpretive boundary that results are ASSOCIATION/OBSERVATIONAL and never CAUSATION. This prevents misuse of the output and shows genuine behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences with no filler: the action, the computation method, and the interpretive caveat are all front-loaded and clearly organized. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, has an output schema, and carries a readOnly annotation, the description covers the essential behavioral and methodological context. It explains what is computed, from what data, and how results should be interpreted, leaving no critical gaps for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so there are no parameter semantics for the description to clarify. The baseline for zero-parameter tools is 4, and the description adds context about what data is used without needing to document arguments.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Correlate') and identifies the exact resources involved: micro-eval metrics, outcome quality scores, and cycle times via lineage. This clearly distinguishes it from sibling tools like get_lineage_summary or get_composite_score by stating the analytical method and result framing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use this tool: when the agent needs a Pearson correlation between micro-eval metrics and outcome/cycle-time data through lineage. It does not explicitly name alternatives or exclusions, but the context is strong enough that an agent would not confuse it with a general lineage or score retrieval tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_pilot_statusARead-only
Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| window | No | |
| cohort_id | Yes | |
| providers | No | |
| synthetic | Yes | |
| data_quality | No | |
| total_operators | Yes | |
| observation_count | Yes | |
| eligible_operators | No | |
| active_interventions | No | |
| metric_registry_version | No | |
| reference_field_version | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description states the status is computed from raw observations and that the data is from a labeled synthetic pilot. This tells the agent the output is an aggregate view rather than raw data, which adds meaningful behavioral context without contradicting the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. The first sentence names the resource and its key contents, and the second adds data provenance and synthetic labeling. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters, a readOnlyHint annotation, and an output schema present, the description provides the necessary context: data source, synthetic nature, and aggregate computation. Nothing essential is missing for an agent to call this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters and the input schema has no properties, so there is no parameter burden for the description to carry. The baseline of 4 applies here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and names a concrete resource ('pilot status overview'), then enumerates the contents: cohort size, observation count, date range, data quality, and active interventions. It clearly conveys what the tool returns, though it does not explicitly differentiate it from sibling tools like get_data_quality or get_intervention_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives useful context—'Computed from raw observations' and '50-operator synthetic pilot'—which implies when this tool is appropriate. However, it does not explicitly state when to use it over alternatives or provide exclusions, so the guidance is implied rather than direct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_workflow_fitARead-only
Get workflow fit analysis — operator/workflow fit scores across workflow stages.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| note | No | |
| stages | Yes | |
| synthetic | Yes | |
| workflow_id | Yes | |
| observations | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already communicates that this is a safe read operation, and the description's 'Get' is consistent with that. The description adds modest context by stating the analysis spans workflow stages, but it does not describe any further behavioral traits such as aggregation level or data source.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence delivers the tool's purpose and scope with no filler. The main verb and resource are front-loaded, and the em-dash addition is the only elaboration, which is exactly the amount needed for a zero-parameter read tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read-only tool with an output schema, the description is sufficiently complete: it states what the tool returns and the domain (operator/workflow fit). It could add a short note on how to interpret the scores, but this is not necessary given the output schema and annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema description coverage is 100%, so there are no parameter semantics for the description to clarify. The 0-param baseline of 4 applies because the description need not compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb (Get) and a concrete resource (workflow fit analysis), and clarifies the content as operator/workflow fit scores across workflow stages. It does not explicitly contrast with any sibling tool, but 'workflow fit' is distinct enough among the listed siblings to identify the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as get_composite_score or get_diagnostics. There are no exclusions, but also no explicit context or when-not-to-use conditions, so an agent must infer applicability from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_pilot_optionsARead-only
List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| synthetic | Yes | |
| eval_families | Yes | |
| benchmark_classes | Yes | |
| canonical_metrics | Yes | |
| intervention_types | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation readOnlyHint=true already establishes that this operation is safe and non-mutating, and the description's 'List' verb is consistent with that. The description adds some useful behavioral context by naming the categories and counts that will be returned, but it does not disclose details like response structure, ordering, or whether the counts are fixed or dynamic. This matches the baseline expectation for a read-only catalog tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the action ('List') and the object ('available pilot options'), then efficiently packs the key categorical details into an em-dash list. There is no filler, repetition, or unnecessary elaboration; every element earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless, read-only catalog tool with an output schema available, this description is complete enough for an agent to understand what it queries and what type of information it will receive. The enumeration of categories covers the tool's content, and the output schema handles return-value specifics. Nothing needed to invoke the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema has 100% coverage, so there is no parameter ambiguity to resolve. The 0-parameter baseline is 4, and the description appropriately does not need to explain parameter semantics that do not exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List') and resource ('available pilot options'), and further clarifies the scope by enumerating the four categories it covers: canonical metrics, eval families, benchmark classes, and intervention types. This clearly distinguishes it from the sibling tools, most of which are get/verify/create operations. However, it does not explicitly name a sibling or contrast itself, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention contexts like configuration setup, discovery workflows, or any exclusion cases. The intended use is only implied by the tool's name and simple read-only nature, which is not enough for strong usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
record_workflow_observationA
Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
| Name | Required | Description | Default |
|---|---|---|---|
| notes | No | Free-text notes about the observation context — task type, AI system used, environmental factors | |
| fit_score | No | Workflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements. | |
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| workflow_id | Yes | Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing) |
Output Schema
| Name | Required | Description |
|---|---|---|
| tool | Yes | Tool name that was called |
| error | Yes | Error type if authorization fails (e.g., 'authorization_required') |
| message | Yes | Human-readable status message |
| operator_id | No | Operator ID that was observed (on success) |
| recorded_at | No | ISO timestamp of recording (on success) |
| workflow_id | No | Workflow ID that was observed (on success) |
| observation_id | No | ID of the recorded observation (on success) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide minimal safety hints (openWorld, non-idempotent, non-destructive). The description adds important behavioral context: REQUIRES AUTHORIZATION, contact for pilot access, and that the synthetic demo returns an authorization notice instead of performing the write. This goes beyond annotations and alerts the agent to the gated nature of the operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, key concept, score range, read alternative, authorization, and demo behavior. The purpose is front-loaded, and the description is neither bloated nor under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, domain definition, score semantics, the appropriate read alternative, authorization requirements, and expected behavior in the demo environment. The output schema exists and likely covers return values; the description provides enough context for an agent to decide when and how to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The main description adds domain context (what workflow fit measures, score range) which is also present in the schema's property descriptions. No additional parameter-level detail is provided beyond what the schema contains.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb ('record') and clearly states the resource ('workflow fit observation linking an operator to a workflow stage with a fit score'). It explicitly names the read counterpart (get_workflow_fit), distinguishing this write operation from its sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names the alternative for reading existing observations ('Use get_workflow_fit to read existing observations') and specifies authorization requirements, implying limited access. Does not provide an explicit 'when not to use' beyond the read case, but the write/read distinction is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_pilot_configurationARead-only
Validate a pilot configuration before deployment. Returns valid status with warnings and errors.
| Name | Required | Description | Default |
|---|---|---|---|
| configuration | No | Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes |
Output Schema
| Name | Required | Description |
|---|---|---|
| valid | Yes | |
| errors | Yes | |
| message | No | |
| warnings | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers safety; the description adds that the tool returns validation status with warnings and errors and is intended pre-deployment. It does not disclose details like required configuration properties or failure behavior, but with annotations and an output schema this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler; the key purpose is front-loaded and the return behavior is stated in one additional clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with one well-documented parameter, a readOnlyHint annotation, and an output schema, the description covers the essential decision: call it before deployment to get warnings/errors. It could mention prerequisites or valid value sources, but the schema already references list_pilot_options.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single 'configuration' parameter is fully described in the schema (JSON pilot configuration, link to list_pilot_options), and schema coverage is 100%. The description adds no parameter-level details, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action ('Validate'), resource ('pilot configuration'), and lifecycle context ('before deployment'), and describes the outcome ('Returns valid status with warnings and errors'). It is clearly a read-only validation step, but it does not explicitly distinguish itself from nearby siblings such as verify_change or create_pilot_configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'before deployment' provides a clear usage context, signaling that this tool is the pre-deployment check. It does not name alternatives or exclusion criteria, so an agent must infer when to use verify_change or list_pilot_options instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verify_changeARead-only
Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION.
| Name | Required | Description | Default |
|---|---|---|---|
| operator_id | Yes | Pseudonymous operator ID (e.g., op_001, op_003, op_034) | |
| intervention_id | No | Intervention ID (e.g., intv_001) |
Output Schema
| Name | Required | Description |
|---|---|---|
| label | Yes | |
| message | Yes | |
| synthetic | Yes | |
| operator_id | Yes | |
| composite_score | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The readOnlyHint annotation already covers non-mutating behavior, and the description adds valuable interpretation guidance: results are associative, never causal. It also reveals the pre/post comparison methodology, which is beyond what the annotation or schema conveys.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence with a clear statistical caveat appended. Every part earns its place, and the core purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a full output schema and readOnlyHint annotation, the description covers the essential purpose and a key interpretive limitation. It does not explicitly discuss when to prefer this over correlated sibling tools, but the provided information is sufficient for most selection and invocation scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both operator_id and intervention_id. The description adds no additional parameter meaning, but the baseline of 3 applies because the schema carries the burden fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Verify') and resource ('a measured change after intervention') and clarifies the pre/post comparison scope. This clearly separates it from siblings like assign_intervention or get_intervention_status, which focus on assignment or status rather than change verification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it — after an intervention with pre/post measurements — but it does not explicitly state exclusions or name alternative tools. An agent can infer context, but there is no explicit routing guidance relative to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.4.3- Changed
close_intervention8 fields changed- changed
Input schema / properties / intervention_id / descriptionPrevious value: -"Intervention ID (e.g., intv_001)"New value: +"Intervention ID to close (e.g., intv_001, intv_007). Must be an active intervention." - changed
Input schema / properties / outcome_notes / descriptionPrevious value: -"Free-text notes about the intervention outcome"New value: +"Free-text notes about the intervention outcome — observed changes, unintended effects, whether the target metric moved" - added
Output schema / properties / closed_atAdded value: +{ + "description": "ISO timestamp of closure (on success)", + "type": "string" +} - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / intervention_idAdded value: +{ + "description": "ID of the closed intervention (on success)", + "type": "string" +} - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / statusAdded value: +{ + "description": "New status of the intervention (e.g., 'closed')", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called"
- Changed
create_experiment8 fields changed- changed
Input schema / properties / configuration / descriptionPrevious value: -"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes"New value: +"Pilot configuration object (JSON) — see list_pilot_options for available metrics, eval families, and benchmark classes. Can be generated by create_pilot_configuration." - changed
Input schema / properties / name / descriptionPrevious value: -"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison')"New value: +"Experiment name (e.g., 'Q3 Claude vs ChatGPT operator comparison', 'Context window expansion pilot — Team Alpha')" - added
Output schema / properties / created_atAdded value: +{ + "description": "ISO timestamp of creation (on success)", + "type": "string" +} - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / experiment_idAdded value: +{ + "description": "ID of the created experiment (on success)", + "type": "string" +} - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / statusAdded value: +{ + "description": "Initial status of the experiment (e.g., 'draft', 'configured')", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called"
- Changed
create_pilot_configuration12 fields changed- changed
Input schema / properties / cohort_size / descriptionPrevious value: -"Number of operators in the pilot cohort (e.g., 25, 50, 100)"New value: +"Number of operators in the pilot cohort (e.g., 25, 50, 100). Determines statistical power and minimum detectable effect size." - changed
Input schema / properties / duration_days / descriptionPrevious value: -"Pilot duration in days (e.g., 30, 60, 90)"New value: +"Pilot duration in days (e.g., 30, 60, 90). Longer windows improve intervention re-evaluation stability." - changed
Input schema / properties / metrics / descriptionPrevious value: -"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction'])"New value: +"Array of metric IDs to include (e.g., ['leverage', 'yield', 'token_snr', 'construction']). See list_pilot_options for the full catalog." - added
Output schema / properties / configuration / properties / benchmark_classesAdded value: +{ + "description": "Selected benchmark class IDs", + "items": { + "type": "string" + }, + "type": "array" +} - added
Output schema / properties / configuration / properties / cohort_size / descriptionAdded value: +"Number of operators in the cohort" - added
Output schema / properties / configuration / properties / duration_days / descriptionAdded value: +"Pilot duration in days" - added
Output schema / properties / configuration / properties / eval_familiesAdded value: +{ + "description": "Selected evaluation family IDs", + "items": { + "type": "string" + }, + "type": "array" +} - added
Output schema / properties / configuration / properties / governanceAdded value: +{ + "description": "Governance metadata including evidence labels and usage constraints", + "type": "object" +} - added
Output schema / properties / configuration / properties / metrics / descriptionAdded value: +"Selected metric IDs" - added
Output schema / properties / synthetic / descriptionAdded value: +"Whether data is synthetic" - added
Output schema / properties / valid / descriptionAdded value: +"Whether the configuration passed validation" - added
Output schema / properties / warningsAdded value: +{ + "description": "Non-blocking validation warnings", + "items": { + "type": "string" + }, + "type": "array" +}
- Changed
record_workflow_observation10 fields changed- changed
Input schema / properties / fit_score / descriptionPrevious value: -"Workflow fit score (0.0-1.0)"New value: +"Workflow fit score from 0.0 (no alignment) to 1.0 (perfect alignment). Computed from operator metric profile vs workflow requirements." - changed
Input schema / properties / notes / descriptionPrevious value: -"Free-text notes about the intervention assignment"New value: +"Free-text notes about the observation context — task type, AI system used, environmental factors" - changed
Input schema / properties / workflow_id / descriptionPrevious value: -"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture)"New value: +"Workflow ID (e.g., wf_debugging, wf_code_review, wf_architecture, wf_refactor, wf_testing)" - added
Output schema / properties / error / descriptionAdded value: +"Error type if authorization fails (e.g., 'authorization_required')" - added
Output schema / properties / message / descriptionAdded value: +"Human-readable status message" - added
Output schema / properties / observation_idAdded value: +{ + "description": "ID of the recorded observation (on success)", + "type": "string" +} - added
Output schema / properties / operator_idAdded value: +{ + "description": "Operator ID that was observed (on success)", + "type": "string" +} - added
Output schema / properties / recorded_atAdded value: +{ + "description": "ISO timestamp of recording (on success)", + "type": "string" +} - added
Output schema / properties / tool / descriptionAdded value: +"Tool name that was called" - added
Output schema / properties / workflow_idAdded value: +{ + "description": "Workflow ID that was observed (on success)", + "type": "string" +}
27 tool updates
v0.4.0- First observed
assign_intervention - First observed
attach_outcome_dataset - First observed
close_intervention - First observed
compare_operator_to_reference - First observed
create_experiment - First observed
create_pilot_configuration - First observed
find_usage_operation_divergence - First observed
get_cohort_distribution - First observed
get_composite_score - First observed
get_composite_score_summary - First observed
get_data_quality - First observed
get_diagnostics - First observed
get_executive_dashboard - First observed
get_intervention_status - First observed
get_lineage_chain - First observed
get_lineage_summary - First observed
get_operator_profile - First observed
get_operator_similarity - First observed
get_operator_system_decomposition - First observed
get_org_topology - First observed
get_outcome_correlation - First observed
get_pilot_status - First observed
get_workflow_fit - First observed
list_pilot_options - First observed
record_workflow_observation - First observed
validate_pilot_configuration - First observed
verify_change
TDQS
Scored across 27 tools
Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.
All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.
With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.
The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.
Maintenance
Related MCP Connectors
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server that lets AI assistants use all OneSchema features exposed via the public API.
Related MCP Servers
- FlicenseNot gradedqualityNot gradedmaintenanceOpen-source AI governance toolkit. MCP servers & CLIs for scanning, auditing, and managing your AI environment-
- AlicenseAqualityAmaintenanceDescription Governed agent city-state. Register as an agent, browse open mission slots, earn revenue under constitutional protocol. Every action creates a SHA-256 provenance seed. Powered by MO§ES™.6231MIT
- AlicenseNot gradedqualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.64MIT
- AlicenseAqualityAmaintenanceExposes the SigRank leaderboard as tools for agents, enabling token-only rank and submit operations with local privacy-first tokenpull reading.251,430 npmMIT