Systems Intelligence Performative Commercial Benchmarking
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {} |
| resources | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| get_pilot_statusA | Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic). |
| get_operator_profileA | Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic. |
| get_cohort_distributionA | Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations. |
| get_composite_scoreA | Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic. |
| get_composite_score_summaryA | Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL. |
| get_diagnosticsA | Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact. |
| get_data_qualityA | Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations. |
| find_usage_operation_divergenceA | Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.). |
| get_workflow_fitA | Get workflow fit analysis — operator/workflow fit scores across workflow stages. |
| get_intervention_statusA | Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes. |
| list_pilot_optionsA | List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types. |
| validate_pilot_configurationA | Validate a pilot configuration before deployment. Returns valid status with warnings and errors. |
| compare_operator_to_referenceA | Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field. |
| get_executive_dashboardA | Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html). |
| verify_changeA | Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION. |
| create_pilot_configurationA | Generate a saveable pilot configuration JSON from parameters. Returns a validated configuration object with cohort size, duration, selected metrics, eval families, and benchmark classes. Use list_pilot_options first to see available options. The returned configuration can be passed to validate_pilot_configuration for pre-deployment checks. Configuration includes governance metadata (DEVELOPMENTAL label, no punitive use, association-not-causation evidence standard). |
| assign_interventionA | Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access. |
| close_interventionA | Close an intervention with outcome notes and mark it complete. The intervention must exist and be active. Outcome notes should describe observed changes, unintended effects, and whether the target metric moved. After closing, the intervention is no longer eligible for verify_change comparisons. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice. |
| create_experimentA | Create an experiment configuration for controlled comparison studies. Experiments pair a pilot configuration with a hypothesis and measurement plan. Use create_pilot_configuration first to build the config, then pass it here. Experiments enforce the ASSOCIATION-not-CAUSATION evidence standard — controlled experiments may upgrade evidence to CAUSATION only with proper design. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice. |
| record_workflow_observationA | Record a workflow fit observation linking an operator to a workflow stage with a fit score. Workflow fit measures how well an operator's AI usage patterns align with a specific workflow stage (e.g., debugging, code review, architecture). Fit scores range 0.0-1.0 where 1.0 indicates perfect alignment. Use get_workflow_fit to read existing observations. REQUIRES AUTHORIZATION — contact pilots@mos2es.org for pilot access. In the synthetic demo, this returns an authorization notice. |
| attach_outcome_datasetA | Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION. |
| get_operator_system_decompositionA | Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance. |
| get_lineage_chainA | Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data. |
| get_lineage_summaryA | Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data. |
| get_outcome_correlationA | Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION. |
| get_org_topologyA | Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements. |
| get_operator_similarityA | Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 27 tools
Most tools are clearly distinct, focusing on specific entities like operator, cohort, lineage, intervention, or data quality. A few like get_composite_score and get_composite_score_summary are closely related but differ in scope (individual vs. cohort), and get_operator_profile overlaps slightly with other metric tools, but descriptions are sufficient to disambiguate.
All tool names follow a consistent verb_noun pattern in snake_case, using verbs like get, create, validate, list, assign, close, verify, find, compare, record, and attach. No mixed conventions or ambiguous verbs.
With 27 tools, the server exceeds the 25-tool threshold for 'too many.' While each tool has a defined purpose, the high count is driven by many read-only analytics endpoints that could likely be consolidated (e.g., multiple get_* summary variants), making the surface feel overloaded.
The core pilot lifecycle is well covered: configuration creation/validation, intervention assignment/closure, workflow observations, and post-intervention verification. Analytics are rich across lineage, outcomes, and org topology. Minor gaps include no way to retrieve or list experiments created via create_experiment, and no update operation for interventions or configurations.