Skip to main content
Glama
Systems Intelligence Performative Commercial Benchmarking

Systems Intelligence Performative Commercial Benchmarking

MO§ES™ Enterprise: AI Operator Evaluation Platform

Live: mos2es.org: Systems intelligence for the human-AI operating system. The new standard in performative metrics and benchmarks for AI operators. Baselines system intelligence for everyday operations and AI workflows. Content-free token telemetry. Never your prompts.

Enterprise AI operator evaluation. Measure how people operate AI, not the AI model itself.

MO§ES™ curates a company's system intelligence: how effectively people drive AI systems across tools, tasks, workflows, and conditions. The same way BI sees the business, MO§ES™ sees how the business operates AI.

live enterprise MCP demo tests license deploy Python MCP Glama Smithery

Table of Contents


Related MCP server: Signomy

What is MO§ES™?

MO§ES™ is an enterprise AI operator evaluation platform. It measures how people operate AI systems, not the AI models themselves, not usage volume, not self-reported proficiency. Using content-free token telemetry (input, output, cache read, cache write, no prompt text, no response text), MO§ES™ baselines system intelligence across:

  • Operators: how effectively individuals drive AI

  • Teams: cohort distributions and capability topology

  • Workflows: where AI fits in the work, not just whether it's used

  • Organizations: cross-team benchmarking and capability mapping

The platform builds bespoke evals around your workflows, roles, and AI systems, benchmarks performance against internal and external reference populations, diagnoses capability gaps and divergence patterns, tests targeted interventions, and re-measures what changes.

The positioning

BI sees the business. MO§ES™ sees how the business operates AI.

LMSYS benchmarks models. Braintrust evaluates product outputs. Langfuse traces LLM calls. WakaTime tracks time. CostHawk tracks spend. None of them see the system intelligence of everyday operations and AI workflows. MO§ES™ is the missing layer.

The architecture

MO§ES™ → Upsilon → SigRank | SignalAF

  • MO§ES™ = governance framework / methodology (this repo)

  • Upsilon = measurement engine / enterprise product (the engine that measures)

  • SigRank = public leaderboard / benchmark / proof surface

  • SignalAF = public distribution / platform brand

  • Yield (Υ) = metric inside Upsilon: (cache_read × output) / input²

The MO§ES™ ecosystem

Repo / Site

What it is

URL

MO§ES™ Enterprise (this repo)

The platform: Python eval engine, CLI, MCP server, demo, promo, enterprise demo, workers. Powered by Upsilon.

mos2es.org

SigRank

Public leaderboard / benchmark / proof surface for AI operator evaluation

signalaf.com

SignalAF

Public brand and distribution surface

signalaf.com

SigRank MCP

The instrument: extracts token pillars, computes cascade, submits to leaderboard

npx sigrank

SIGNOMY

Governed AI agent marketplace: ranked agents form teams, run missions, earn revenue

signomy.xyz

SigEconomy

Public LLM operator evals: read-only leaderboard, SEO/AEO surface

sigeconomy.com

Live surfaces

Surface

URL

What

Promo site

https://mos2es.org

Marketing, methodology, demo, comparisons, booking

Enterprise demo

https://enterprise.mos2es.org

Interactive product walkthrough (evaluate → diagnose → workflow → compare)

MCP server

https://mcp.mos2es.org/mcp

27 tools (22 read + 5 write), 5 prompts, 6 resources, streamable HTTP

MCP server info

https://mcp.mos2es.org/

Server info JSON (version, tool count, transport)

MCP server card

https://mcp.mos2es.org/.well-known/mcp/server-card.json

Full server card with all tool schemas

OpenAPI spec

https://mos2es.org/openapi.json

REST API specification

LLM guidance

https://mos2es.org/llms.txt

llms.txt for AI agents and crawlers

Sitemap

https://mos2es.org/sitemap.xml

XML sitemap

Book a demo

https://mos2es.org/contact

B2B demo booking with structured intake form

Quick start

Run the demo (one-liner, no clone needed)

curl -sL https://mos2es.org/demo/run.py | python3 -

This clones the repo to a temp directory, installs rich, and runs the full 11-step demo pipeline. Requires Python 3.10+ and git.

Clone and run

git clone https://github.com/SunrisesIllneverSee/Moses_Enterprise_B2BPilot_.git
cd Moses_Enterprise_B2BPilot_/_01_platform
pip install rich
python3 -m src.cli.main demo full

Run the test suite

cd _01_platform
python3 -m pytest tests/ -q
# 676 tests pass

Run the MCP server locally

cd _01_platform
pip install mcp
python3 -m src.mcp_server.server

Use the remote MCP server

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "moses": {
      "url": "https://mcp.mos2es.org/mcp",
      "transport": "http"
    }
  }
}

Or call tools directly:

curl -s -D /tmp/h -X POST https://mcp.mos2es.org/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' > /dev/null
SID=$(grep -i "^mcp-session-id:" /tmp/h | tr -d '\r\n' | sed 's/.*: //')
curl -s -X POST https://mcp.mos2es.org/mcp \
  -H "Content-Type: application/json" -H "MCP-Session-Id: $SID" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | python3 -m json.tool

The 11-step demo pipeline

Step

Name

What it does

1

LOAD

Load 50-operator synthetic cohort

2

EVALUATE

Compute per-observation metrics

3

BENCHMARK

Compute percentile positions

4

DIAGNOSE

Run pattern detectors

5

OPERATOR×SYSTEM

Decompose operator vs system effects

6

INTERVENE

Assign targeted interventions

7

RE-EVALUATE

Re-measure post-intervention

8

OUTCOME LINEAGE

Trace observations → transformations → artifacts → outcomes

9

REPORT

Generate markdown + PDF pilot readout

10

VISUALIZE

Generate 9 architecture diagrams

11

DASHBOARD

Export executive HTML dashboard

The demo runs on fully synthetic data. No real operators, no real prompt text, no real API calls. Deterministic: running twice produces identical results.

The 8 canonical metrics

Metric

Formula

What it measures

Yield (Υ)

(R × O) / I²

Productive output relative to input squared

Leverage

R / I

Context reuse efficiency

Token SNR

O / (I + O)

Signal-to-noise ratio in token usage

Log Leverage (10xDEV)

log₁₀(R / I)

Leverage on a logarithmic scale

Construction

W / R

New context built vs reused

Velocity

O / I

Raw output efficiency

Scale V

log₁₀(I + O + W + R)

Total volume on a log scale

Efficiency

(R + W + O) / I / 4

Composite display metric

Where I = input, O = output, R = cache read, W = cache write.

15 evaluation families. 13 benchmark classes. 4 measurement levels (Operator, Team, Workflow, Organization).

Repository structure

_01_platform/          Python platform (the product)
  src/                 Source code (domain, metrics, analysis, diagnostics, CLI, MCP server)
  tests/               676 tests
  demo_data/           Synthetic demo cohort (50 operators, 1,668 observations)
  scripts/             Utility scripts
  schemas/             JSON schemas

_02_demo-website/      Enterprise demo site (enterprise.mos2es.org)
_03_promo-site/        Promo site (mos2es.org)
  vs/                  16 competitor comparison pages
  alternatives/        8 alternatives listicle pages
  concepts/            10 concept explainer pages
  guides/              4 how-to guides
  demo/run.py          One-liner demo runner
  .well-known/mcp.json MCP server discovery

_04_onepager/          One-pager site

_workers/              Cloudflare Workers
  promo-worker/        mos2es.org worker (static assets + AEO/SEO/GEO headers)
  moses-worker/        enterprise.mos2es.org worker (static assets)
  onepager-worker/     One-pager worker (static assets)
  mcp-worker/          MCP server worker (computes live from raw data, 27 tools)

MCP server

The MCP server exposes 27 tools (22 read + 5 write), 5 prompts, and 6 resources over streamable HTTP at https://mcp.mos2es.org/mcp. No authentication required for the public server.

Read tools include: cohort stats, operator profiles, metric distributions, benchmark positions, divergence findings, intervention outcomes, outcome lineage, workflow fit, org topology, and more.

Write tools include: create intervention, assign intervention, record outcome, create eval configuration, create pilot configuration.

Prompts include: operator evaluation summary, intervention recommendation, cohort health report, workflow fit analysis, pilot scoping guide.

Resources include: canonical metric definitions, metric registry, pilot status, pilot options, governance conventions, cohort operator list.

The server computes live from raw observation data, no pre-computed results. Every call runs the actual scoring, benchmarking, and diagnostic engines.

Server card

curl -s https://mcp.mos2es.org/.well-known/mcp/server-card.json | python3 -m json.tool

Returns the full server card with all 27 tool schemas, transport info, and metadata.

Cloudflare Workers

Four Workers deploy from this repo:

Worker

Domain

What

moses-promo

mos2es.org

Promo site + AEO/SEO/GEO headers + agent discoverability (llms.txt, sitemap, OpenAPI, MCP links)

moses

enterprise.mos2es.org

Enterprise demo site

moses-onepager

(workers.dev)

One-pager

moses-mcp

mcp.mos2es.org

MCP server (27 tools, live computation)

Deploy

npm install -g wrangler

cd _workers/promo-worker && wrangler deploy
cd _workers/moses-worker && wrangler deploy
cd _workers/onepager-worker && wrangler deploy
cd _workers/mcp-worker && wrangler deploy

MCP worker data sync

The MCP worker computes live from raw data files (observations.js, lineages.js) auto-generated from the platform's demo data. To regenerate after updating demo data:

cd _workers/mcp-worker/src
python3 -c "
import json
with open('../../../_01_platform/demo_data/observations.jsonl') as f:
    data = [json.loads(l) for l in f if l.strip()]
with open('observations.js', 'w') as f:
    f.write('export default '); json.dump(data, f); f.write(';\n')
with open('../../../_01_platform/demo_data/lineages.jsonl') as f:
    data = [json.loads(l) for l in f if l.strip()]
with open('lineages.js', 'w') as f:
    f.write('export default '); json.dump(data, f); f.write(';\n')
"

Key conventions

  • Brand: MO§ES™ (use the section sign §)

  • All outcome claims are ASSOCIATION, never CAUSATION unless backed by a controlled experiment

  • Composite score is DEVELOPMENTAL, never PERSONNEL: no punitive use, no employee leaderboard

  • Operator similarity is metric similarity, NOT personality matching

  • Content-free telemetry: no prompt text, no output bodies, no code content in observations

  • No prompt-content surveillance: operator performance is observable from token structure alone

  • Governance-ready: evidence labels (DEVELOPMENTAL, HYPOTHESIS, ASSOCIATION) on every output

Complement, don't replace

MO§ES™ works alongside your existing BI, eval suites, observability tools, and productivity analytics. It measures the systems intelligence layer they can't see. Not a replacement: the missing piece.

Five pillars

#

Pillar

What

1

Systems Intelligence

MO§ES™ sees the human-AI operating system the way BI sees the business. Operator performance, workflow fit, tool selection, capability distribution, intervention outcomes, all measured from structure, not content.

2

Standard Operational Performative Metrics

8 canonical metrics. 15 eval families. 13 benchmark classes. 4 measurement levels. An open, documented spec for measuring how humans operate AI. The reference implementation is MO§ES™.

3

Bespoke Enterprise Evals

Your company should not inherit someone else's definition of AI proficiency. Evals built around your workflows, roles, models, and performance questions.

4

Complement, Don't Replace

Works alongside your existing BI, eval suites, observability tools, and productivity analytics. The missing piece, not a replacement.

5

Privacy-First / Governance

Content-free token telemetry. No prompt text. No surveillance. DEVELOPMENTAL / HYPOTHESIS / ASSOCIATION evidence labels on every output. Governance-ready out of the box.

License

Apache License 2.0: see LICENSE. Patent pending: Serial No. 63/877,177. For enterprise pilots and commercial support: pilots@mos2es.org.


mos2es.org · Book a demo · Run the demo · MCP server

Built by Deric J. McHenry, Ello Cello LLC

Available Tools

27 tools
assign_interventionA

Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access.

Inputs: operator_id (required), intervention_type (required), notes. Returns: tool, error, message

attach_outcome_datasetA

Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION.

Inputs: source (required), format. Returns: tool, error, message

close_interventionA

Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

Inputs: intervention_id (required), outcome_notes. Returns: tool, error, status, message, +2 more

compare_operator_to_referenceA
Read-only

Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field.

Inputs: operator_id (required), reference. Returns: reference, synthetic, comparisons, operator_id, +1 more

create_experimentA

Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

Inputs: name (required), configuration. Returns: tool, error, status, message, +2 more

create_pilot_configurationA
Read-only

Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard).

Inputs: metrics, cohort_size, duration_days. Returns: valid, warnings, synthetic, configuration

find_usage_operation_divergenceA
Read-only

Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.).

Inputs: no named parameters. Returns: synthetic, all_operators, divergent_operators

get_cohort_distributionA
Read-only

Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations.

Inputs: metric. Returns: max, min, p10, p25, +9 more

get_composite_scoreA
Read-only

Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic.

Inputs: operator_id (required). Returns: name, label, score, caveats, +4 more

get_composite_score_summaryA
Read-only

Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL.

Inputs: no named parameters. Returns: q1, q3, max, min, +7 more

get_data_qualityA
Read-only

Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations.

Inputs: no named parameters. Returns: issues, coverage, validity, synthetic, +3 more

get_diagnosticsA
Read-only

Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact.

Inputs: operator_id (required). Returns: status, patterns, diagnoses, synthetic, +1 more

get_executive_dashboardA
Read-only

Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html).

Inputs: no named parameters. Returns: command, message, features, governance

get_intervention_statusA
Read-only

Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes.

Inputs: no named parameters. Returns: all, count, active, closed, +1 more

get_lineage_chainA
Read-only

Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data.

Inputs: operator_id (required). Returns: lineage, synthetic, data_window, privacy_class, +3 more

get_lineage_summaryA
Read-only

Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data.

Inputs: no named parameters. Returns: synthetic, data_window, privacy_class, lineage_summary, +3 more

get_operator_profileA
Read-only

Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic.

Inputs: operator_id (required). Returns: team, level, pseudonym, synthetic, +4 more

get_operator_similarityA
Read-only

Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching.

Inputs: operator_id (required), n_neighbors. Returns: note, normalization, cluster_quality, distance_metric, +3 more

get_operator_system_decompositionA
Read-only

Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance.

Inputs: operator_id. Returns: metrics, summary, operator_id, systems_compared, +2 more

get_org_topologyA
Read-only

Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements.

Inputs: no named parameters. Returns: summary, total_teams, team_topologies, total_operators, +4 more

get_outcome_correlationA
Read-only

Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION.

Inputs: no named parameters. Returns: summary, claim_status, correlations, evidence_grade, +2 more

get_pilot_statusA
Read-only

Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic).

Inputs: no named parameters. Returns: window, cohort_id, providers, synthetic, +7 more

get_workflow_fitA
Read-only

Get workflow fit analysis — operator/workflow fit scores across workflow stages.

Inputs: no named parameters. Returns: note, stages, synthetic, workflow_id, +1 more

list_pilot_optionsA
Read-only

List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types.

Inputs: no named parameters. Returns: synthetic, eval_families, benchmark_classes, canonical_metrics, +1 more

record_workflow_observationA

Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice.

Inputs: operator_id (required), workflow_id (required), notes, fit_score. Returns: tool, error, message, operator_id, +3 more

validate_pilot_configurationA
Read-only

Validate a pilot configuration before deployment. Returns valid status with warnings and errors.

Inputs: configuration. Returns: valid, errors, message, warnings

verify_changeA
Read-only

Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION.

Inputs: operator_id (required), intervention_id. Returns: label, message, synthetic, operator_id, +1 more

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 27 tools

    Disambiguation4/5

    Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.

    Naming Consistency5/5

    All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.

    Tool Count2/5

    With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.

    Completeness4/5

    The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.

    Maintenance

    ActivityNo data
    ResponsivenessSyncing

    Related MCP Connectors

    Related MCP Servers