Systems Intelligence Performative Commercial Benchmarking
This server is a read-and-write MCP endpoint for evaluating and piloting how people operate AI systems, using synthetic operator cohorts and content-free token telemetry to compute metrics, benchmarks, diagnostics, interventions, lineage, and organizational insights.
Inspect pilot health: get cohort size, observation counts, date ranges, data quality, and active interventions.
Review operator performance: pull pseudonymous operator profiles with the 5 canonical metrics (yield, leverage, token SNR, construction, divergence), percentiles, and benchmark positions.
Benchmark and compare: get cohort metric distributions, composite scores, reference-population comparisons, and nearest-neighbor operator similarity.
Diagnose and decompose: run pattern detectors, usage-operation divergence analysis, workflow fit analysis, and operator-vs-system effect decomposition.
Intervene and verify: assign/close interventions, record workflow observations, create experiments, and verify pre/post changes (labeled ASSOCIATION, never causation).
Trace outcomes and lineage: follow full lineage chains from state through transformations to committed outcomes, summarize lineage across the cohort, and correlate metrics with outcome quality.
Explore organizational topology: see team-level distributions, capability concentration, platform adoption, single points of failure, and cross-team complementarity.
Generate configurations: list pilot options, validate or create pilot configurations, and get executive dashboard generation commands.
Write with authorization: create experiments, assign interventions, record observations, and attach external outcome datasets — all requiring pilot authorization.
Allows attaching external outcome datasets from GitHub for join analysis, enabling correlation of GitHub activity with operator performance metrics.
MO§ES™ Enterprise: AI Operator Evaluation Platform
Live: mos2es.org: Systems intelligence for the human-AI operating system. The new standard in performative metrics and benchmarks for AI operators. Baselines system intelligence for everyday operations and AI workflows. Content-free token telemetry. Never your prompts.
Enterprise AI operator evaluation. Measure how people operate AI, not the AI model itself.
MO§ES™ curates a company's system intelligence: how effectively people drive AI systems across tools, tasks, workflows, and conditions. The same way BI sees the business, MO§ES™ sees how the business operates AI.
Table of Contents
Related MCP server: Signomy
What is MO§ES™?
MO§ES™ is an enterprise AI operator evaluation platform. It measures how people operate AI systems, not the AI models themselves, not usage volume, not self-reported proficiency. Using content-free token telemetry (input, output, cache read, cache write, no prompt text, no response text), MO§ES™ baselines system intelligence across:
Operators: how effectively individuals drive AI
Teams: cohort distributions and capability topology
Workflows: where AI fits in the work, not just whether it's used
Organizations: cross-team benchmarking and capability mapping
The platform builds bespoke evals around your workflows, roles, and AI systems, benchmarks performance against internal and external reference populations, diagnoses capability gaps and divergence patterns, tests targeted interventions, and re-measures what changes.
The positioning
BI sees the business. MO§ES™ sees how the business operates AI.
LMSYS benchmarks models. Braintrust evaluates product outputs. Langfuse traces LLM calls. WakaTime tracks time. CostHawk tracks spend. None of them see the system intelligence of everyday operations and AI workflows. MO§ES™ is the missing layer.
The architecture
MO§ES™ → Upsilon → SigRank | SignalAF
MO§ES™ = governance framework / methodology (this repo)
Upsilon = measurement engine / enterprise product (the engine that measures)
SigRank = public leaderboard / benchmark / proof surface
SignalAF = public distribution / platform brand
Yield (Υ) = metric inside Upsilon:
(cache_read × output) / input²
The MO§ES™ ecosystem
Repo / Site | What it is | URL |
MO§ES™ Enterprise (this repo) | The platform: Python eval engine, CLI, MCP server, demo, promo, enterprise demo, workers. Powered by Upsilon. | |
SigRank | Public leaderboard / benchmark / proof surface for AI operator evaluation | |
SignalAF | Public brand and distribution surface | |
SigRank MCP | The instrument: extracts token pillars, computes cascade, submits to leaderboard |
|
SIGNOMY | Governed AI agent marketplace: ranked agents form teams, run missions, earn revenue | |
SigEconomy | Public LLM operator evals: read-only leaderboard, SEO/AEO surface |
Live surfaces
Surface | URL | What |
Promo site | Marketing, methodology, demo, comparisons, booking | |
Enterprise demo | Interactive product walkthrough (evaluate → diagnose → workflow → compare) | |
MCP server | 27 tools (22 read + 5 write), 5 prompts, 6 resources, streamable HTTP | |
MCP server info | Server info JSON (version, tool count, transport) | |
MCP server card | Full server card with all tool schemas | |
OpenAPI spec | REST API specification | |
LLM guidance | llms.txt for AI agents and crawlers | |
Sitemap | XML sitemap | |
Book a demo | B2B demo booking with structured intake form |
Quick start
Run the demo (one-liner, no clone needed)
curl -sL https://mos2es.org/demo/run.py | python3 -This clones the repo to a temp directory, installs rich, and runs the full 11-step demo pipeline. Requires Python 3.10+ and git.
Clone and run
git clone https://github.com/SunrisesIllneverSee/Moses_Enterprise_B2BPilot_.git
cd Moses_Enterprise_B2BPilot_/_01_platform
pip install rich
python3 -m src.cli.main demo fullRun the test suite
cd _01_platform
python3 -m pytest tests/ -q
# 676 tests passRun the MCP server locally
cd _01_platform
pip install mcp
python3 -m src.mcp_server.serverUse the remote MCP server
Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
{
"mcpServers": {
"moses": {
"url": "https://mcp.mos2es.org/mcp",
"transport": "http"
}
}
}Or call tools directly:
curl -s -D /tmp/h -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' > /dev/null
SID=$(grep -i "^mcp-session-id:" /tmp/h | tr -d '\r\n' | sed 's/.*: //')
curl -s -X POST https://mcp.mos2es.org/mcp \
-H "Content-Type: application/json" -H "MCP-Session-Id: $SID" \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | python3 -m json.toolThe 11-step demo pipeline
Step | Name | What it does |
1 | LOAD | Load 50-operator synthetic cohort |
2 | EVALUATE | Compute per-observation metrics |
3 | BENCHMARK | Compute percentile positions |
4 | DIAGNOSE | Run pattern detectors |
5 | OPERATOR×SYSTEM | Decompose operator vs system effects |
6 | INTERVENE | Assign targeted interventions |
7 | RE-EVALUATE | Re-measure post-intervention |
8 | OUTCOME LINEAGE | Trace observations → transformations → artifacts → outcomes |
9 | REPORT | Generate markdown + PDF pilot readout |
10 | VISUALIZE | Generate 9 architecture diagrams |
11 | DASHBOARD | Export executive HTML dashboard |
The demo runs on fully synthetic data. No real operators, no real prompt text, no real API calls. Deterministic: running twice produces identical results.
The 8 canonical metrics
Metric | Formula | What it measures |
Yield (Υ) |
| Productive output relative to input squared |
Leverage |
| Context reuse efficiency |
Token SNR |
| Signal-to-noise ratio in token usage |
Log Leverage (10xDEV) |
| Leverage on a logarithmic scale |
Construction |
| New context built vs reused |
Velocity |
| Raw output efficiency |
Scale V |
| Total volume on a log scale |
Efficiency |
| Composite display metric |
Where I = input, O = output, R = cache read, W = cache write.
15 evaluation families. 13 benchmark classes. 4 measurement levels (Operator, Team, Workflow, Organization).
Repository structure
_01_platform/ Python platform (the product)
src/ Source code (domain, metrics, analysis, diagnostics, CLI, MCP server)
tests/ 676 tests
demo_data/ Synthetic demo cohort (50 operators, 1,668 observations)
scripts/ Utility scripts
schemas/ JSON schemas
_02_demo-website/ Enterprise demo site (enterprise.mos2es.org)
_03_promo-site/ Promo site (mos2es.org)
vs/ 16 competitor comparison pages
alternatives/ 8 alternatives listicle pages
concepts/ 10 concept explainer pages
guides/ 4 how-to guides
demo/run.py One-liner demo runner
.well-known/mcp.json MCP server discovery
_04_onepager/ One-pager site
_workers/ Cloudflare Workers
promo-worker/ mos2es.org worker (static assets + AEO/SEO/GEO headers)
moses-worker/ enterprise.mos2es.org worker (static assets)
onepager-worker/ One-pager worker (static assets)
mcp-worker/ MCP server worker (computes live from raw data, 27 tools)MCP server
The MCP server exposes 27 tools (22 read + 5 write), 5 prompts, and 6 resources over streamable HTTP at https://mcp.mos2es.org/mcp. No authentication required for the public server.
Read tools include: cohort stats, operator profiles, metric distributions, benchmark positions, divergence findings, intervention outcomes, outcome lineage, workflow fit, org topology, and more.
Write tools include: create intervention, assign intervention, record outcome, create eval configuration, create pilot configuration.
Prompts include: operator evaluation summary, intervention recommendation, cohort health report, workflow fit analysis, pilot scoping guide.
Resources include: canonical metric definitions, metric registry, pilot status, pilot options, governance conventions, cohort operator list.
The server computes live from raw observation data, no pre-computed results. Every call runs the actual scoring, benchmarking, and diagnostic engines.
Server card
curl -s https://mcp.mos2es.org/.well-known/mcp/server-card.json | python3 -m json.toolReturns the full server card with all 27 tool schemas, transport info, and metadata.
Cloudflare Workers
Four Workers deploy from this repo:
Worker | Domain | What |
| mos2es.org | Promo site + AEO/SEO/GEO headers + agent discoverability (llms.txt, sitemap, OpenAPI, MCP links) |
| enterprise.mos2es.org | Enterprise demo site |
| (workers.dev) | One-pager |
| mcp.mos2es.org | MCP server (27 tools, live computation) |
Deploy
npm install -g wrangler
cd _workers/promo-worker && wrangler deploy
cd _workers/moses-worker && wrangler deploy
cd _workers/onepager-worker && wrangler deploy
cd _workers/mcp-worker && wrangler deployMCP worker data sync
The MCP worker computes live from raw data files (observations.js, lineages.js) auto-generated from the platform's demo data. To regenerate after updating demo data:
cd _workers/mcp-worker/src
python3 -c "
import json
with open('../../../_01_platform/demo_data/observations.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('observations.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
with open('../../../_01_platform/demo_data/lineages.jsonl') as f:
data = [json.loads(l) for l in f if l.strip()]
with open('lineages.js', 'w') as f:
f.write('export default '); json.dump(data, f); f.write(';\n')
"Key conventions
Brand: MO§ES™ (use the section sign §)
All outcome claims are ASSOCIATION, never CAUSATION unless backed by a controlled experiment
Composite score is DEVELOPMENTAL, never PERSONNEL: no punitive use, no employee leaderboard
Operator similarity is metric similarity, NOT personality matching
Content-free telemetry: no prompt text, no output bodies, no code content in observations
No prompt-content surveillance: operator performance is observable from token structure alone
Governance-ready: evidence labels (DEVELOPMENTAL, HYPOTHESIS, ASSOCIATION) on every output
Complement, don't replace
MO§ES™ works alongside your existing BI, eval suites, observability tools, and productivity analytics. It measures the systems intelligence layer they can't see. Not a replacement: the missing piece.
Five pillars
# | Pillar | What |
1 | Systems Intelligence | MO§ES™ sees the human-AI operating system the way BI sees the business. Operator performance, workflow fit, tool selection, capability distribution, intervention outcomes, all measured from structure, not content. |
2 | Standard Operational Performative Metrics | 8 canonical metrics. 15 eval families. 13 benchmark classes. 4 measurement levels. An open, documented spec for measuring how humans operate AI. The reference implementation is MO§ES™. |
3 | Bespoke Enterprise Evals | Your company should not inherit someone else's definition of AI proficiency. Evals built around your workflows, roles, models, and performance questions. |
4 | Complement, Don't Replace | Works alongside your existing BI, eval suites, observability tools, and productivity analytics. The missing piece, not a replacement. |
5 | Privacy-First / Governance | Content-free token telemetry. No prompt text. No surveillance. DEVELOPMENTAL / HYPOTHESIS / ASSOCIATION evidence labels on every output. Governance-ready out of the box. |
License
Apache License 2.0: see LICENSE. Patent pending: Serial No. 63/877,177. For enterprise pilots and commercial support: pilots@mos2es.org.
mos2es.org · Book a demo · Run the demo · MCP server
Built by Deric J. McHenry, Ello Cello LLC
Available Tools
27 toolsassign_interventionA
Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access.
Inputs: operator_id (required), intervention_type (required), notes. Returns: tool, error, message
attach_outcome_datasetA
Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION.
Inputs: source (required), format. Returns: tool, error, message
close_interventionA
Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
Inputs: intervention_id (required), outcome_notes. Returns: tool, error, status, message, +2 more
compare_operator_to_referenceARead-only
Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field.
Inputs: operator_id (required), reference. Returns: reference, synthetic, comparisons, operator_id, +1 more
create_experimentA
Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
Inputs: name (required), configuration. Returns: tool, error, status, message, +2 more
create_pilot_configurationARead-only
Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard).
Inputs: metrics, cohort_size, duration_days. Returns: valid, warnings, synthetic, configuration
find_usage_operation_divergenceARead-only
Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.).
Inputs: no named parameters. Returns: synthetic, all_operators, divergent_operators
get_cohort_distributionARead-only
Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations.
Inputs: metric. Returns: max, min, p10, p25, +9 more
get_composite_scoreARead-only
Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic.
Inputs: operator_id (required). Returns: name, label, score, caveats, +4 more
get_composite_score_summaryARead-only
Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL.
Inputs: no named parameters. Returns: q1, q3, max, min, +7 more
get_data_qualityARead-only
Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations.
Inputs: no named parameters. Returns: issues, coverage, validity, synthetic, +3 more
get_diagnosticsARead-only
Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact.
Inputs: operator_id (required). Returns: status, patterns, diagnoses, synthetic, +1 more
get_executive_dashboardARead-only
Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html).
Inputs: no named parameters. Returns: command, message, features, governance
get_intervention_statusARead-only
Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes.
Inputs: no named parameters. Returns: all, count, active, closed, +1 more
get_lineage_chainARead-only
Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data.
Inputs: operator_id (required). Returns: lineage, synthetic, data_window, privacy_class, +3 more
get_lineage_summaryARead-only
Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data.
Inputs: no named parameters. Returns: synthetic, data_window, privacy_class, lineage_summary, +3 more
get_operator_profileARead-only
Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic.
Inputs: operator_id (required). Returns: team, level, pseudonym, synthetic, +4 more
get_operator_similarityARead-only
Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching.
Inputs: operator_id (required), n_neighbors. Returns: note, normalization, cluster_quality, distance_metric, +3 more
get_operator_system_decompositionARead-only
Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance.
Inputs: operator_id. Returns: metrics, summary, operator_id, systems_compared, +2 more
get_org_topologyARead-only
Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements.
Inputs: no named parameters. Returns: summary, total_teams, team_topologies, total_operators, +4 more
get_outcome_correlationARead-only
Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION.
Inputs: no named parameters. Returns: summary, claim_status, correlations, evidence_grade, +2 more
get_pilot_statusARead-only
Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic).
Inputs: no named parameters. Returns: window, cohort_id, providers, synthetic, +7 more
get_workflow_fitARead-only
Get workflow fit analysis — operator/workflow fit scores across workflow stages.
Inputs: no named parameters. Returns: note, stages, synthetic, workflow_id, +1 more
list_pilot_optionsARead-only
List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types.
Inputs: no named parameters. Returns: synthetic, eval_families, benchmark_classes, canonical_metrics, +1 more
record_workflow_observationA
Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.
Inputs: operator_id (required), workflow_id (required), notes, fit_score. Returns: tool, error, message, operator_id, +3 more
validate_pilot_configurationARead-only
Validate a pilot configuration before deployment. Returns valid status with warnings and errors.
Inputs: configuration. Returns: valid, errors, message, warnings
verify_changeARead-only
Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION.
Inputs: operator_id (required), intervention_id. Returns: label, message, synthetic, operator_id, +1 more
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 27 tools
Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.
All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.
With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.
The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.
Maintenance
Related MCP Connectors
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for building and testing AI agents with multi-model experimentation and insights.
Related MCP Servers
- FlicenseNot gradedqualityNot gradedmaintenanceOpen-source AI governance toolkit. MCP servers & CLIs for scanning, auditing, and managing your AI environment-
- AlicenseAqualityBmaintenanceDescription Governed agent city-state. Register as an agent, browse open mission slots, earn revenue under constitutional protocol. Every action creates a SHA-256 provenance seed. Powered by MO§ES™.231MIT
- AlicenseNot gradedqualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.68MIT
- AlicenseAqualityAmaintenanceExposes the SigRank leaderboard as tools for agents, enabling token-only rank and submit operations with local privacy-first tokenpull reading.25412 npmMIT