Skip to main content
Glama

Server Details

Read-only MCP server for the OPERANT AI operating-agent calibration benchmark.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
Repository
saagpatel/operant
GitHub Stars
0
ChatGPT
–
Claude
–

TDQS

A4.5/5.0

Scored across 5 tools

Disambiguation5/5

Each tool has a distinct purpose: compare_models compares two profiles, get_case retrieves a full case, get_methodology explains the benchmark design, get_results returns calculation profiles, and list_cases lists metadata. No overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (compare_models, get_case, get_methodology, get_results, list_cases) using underscores and imperative verbs.

Tool Count5/5

5 tools is well-scoped for a benchmark analysis server. Each tool covers a necessary function without being excessive or lacking.

Completeness5/5

The tool set covers the full lifecycle of exploring the benchmark: listing cases, retrieving details, understanding methodology, accessing results, and comparing models. No obvious gaps given the read-only purpose.

Available Tools

5 tools
compare_modelsInspect two retained profilesA
Read-only
Inspect

Place two retained calculation profiles side by side by display_name substring. Returns ocs_mean, ocs_stdev, orchestration_mean, run_family, and subject_shell for each, plus comparison_status=NOT_DURABLE and the public claim_status. This is not evidence that either named model outperforms, equals, or differs significantly from the other. If a name is ambiguous or not found, returns an error listing all available display_names.

Inputs: model_a (required), model_b (required).

get_caseGet a caseA
Read-only
Inspect

Return the full case for a given pair_id (axes 1/2/4) or id (axis 3): malign and benign task prompts, expected decisions, grounding rationale, and bypass patterns. Axis 3 cases are single (unmatched) and use an 'id' field instead of 'pair_id'. Use list_cases to browse available ids.

Inputs: axis (required), pair_id (required).

get_methodologyGet benchmark methodologyA
Read-only
Inspect

Return the benchmark design: the 4 axes, the OCS formula (Youden's J), the 5 decision labels (PROCEED, PROCEED_SANCTIONED, REFUSE, ESCALATE, REROUTE), scoring blocks, and a concise what-it-measures / what-it-doesn't summary. Sourced from the baked case-file metadata.

Inputs: no named parameters.

get_resultsGet retained calculation profilesA
Read-only
Inspect

Return retained model calculation profiles plus generated_at, claim_status, claims_at_risk, and the bounded evidence_binding. These rows are not durable named-model performance claims. Do not rank them or treat stdev/significance as reliable model evidence.

Inputs: no named parameters.

list_casesList casesA
Read-only
Inspect

Return case metadata (no full task prompts): pair_id/id, axis, tier, grounding, and side indicators (malign/benign for axes 1/2/4; null for axis 3). Filter by axis, or omit for all cases across all axes (the result includes a count). Use get_case to fetch a full case with task prompts and expected decisions.

Inputs: axis.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.