Skip to main content
Glama
analysis-gym
by defog-ai

Analysis Gym

Analysis Gym is a tiny MCP server for recording and scoring prospective equity earnings predictions made by AI agents.

It deliberately does not choose tickers, schedule runs, or invoke models. Your agent loop owns those decisions. The agent uses the existing FactIQ MCP server for research and calls Analysis Gym only to record a prediction, record the eventual actuals, or read the results.

Tools

  • record_prediction records an immutable forecast before the expected earnings time.

  • record_actuals settles all earlier predictions for a ticker and fiscal period.

  • get_results returns per-metric errors and a leaderboard grouped by harness, model, and thinking setting.

The five predicted values are revenue, EBITDA, net profit, free cash flow, and the first regular-session closing price after the earnings release.

Related MCP server: OpenInvest

Run locally

uv sync
uv run analysis-gym

The server uses stdio transport and stores data in analysis_gym.sqlite3 in its working directory. Set ANALYSIS_GYM_DB_PATH to put the database elsewhere.

Codex

Add the server to ~/.codex/config.toml:

[mcp_servers.analysis-gym]
command = "uv"
args = ["--directory", "/absolute/path/to/analysis-gym", "run", "analysis-gym"]

Install and authenticate the FactIQ plugin separately. Then ask Codex, for example:

Pick an equity reporting soon. Use FactIQ to forecast its next-quarter revenue, EBITDA, net profit, free cash flow, and first post-earnings close. Record the forecast in Analysis Gym before the release.

Claude Code

claude mcp add analysis-gym -- \
  uv --directory /absolute/path/to/analysis-gym run analysis-gym

Use the same prompt and ensure the FactIQ plugin is also installed and authenticated.

Agent-side loop

A loop outside this repository can choose an upcoming event and run the same request through any set of CLI/model/thinking configurations. Each agent calls record_prediction itself. After earnings, call record_actuals once with a source URL, then use get_results to compare the configurations.

Analysis Gym uses symmetric mean absolute percentage error (SMAPE), where lower is better. It reports every metric separately and a simple mean across all five.

Metric definitions

  • EBITDA: operating income plus depreciation and amortization.

  • Free cash flow: operating cash flow minus capital expenditure.

  • Net profit: consolidated net income attributable to the parent/common shareholders.

  • Post-earnings close: the same session's close for a pre-market release, or the next regular session's close for an after-hours release.

All four financial values (submitted in millions) in a submission must use the same reporting currency.

Development

uv run pytest

Available Tools

3 tools
get_resultsA

Return predictions, per-metric errors, and the configuration leaderboard.

Lower symmetric mean absolute percentage error (SMAPE) is better. Set settled_only=false to include predictions whose actuals are not recorded yet.

Inputs: limit, model, ticker, harness, +2 more. Returns: object

record_actualsB

Record published actuals and settle earlier predictions for the event.

EBITDA is operating income plus depreciation and amortization. Free cash flow is operating cash flow minus capital expenditure. For a pre-market release, use that session's close; for an after-hours release, use the next regular session's close.

Inputs: source (required), ticker (required), earnings_at (required), fiscal_period (required), +6 more. Returns: object

record_predictionA

Record one immutable pre-earnings forecast.

expected_earnings_at must be an ISO 8601 timestamp with a timezone. Revenue, EBITDA, net profit, and free cash flow must use the same reporting currency. post_earnings_close is the expected first regular-session close after release.

Inputs: model (required), ticker (required), harness (required), fiscal_period (required), +9 more. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 3 tools

    Disambiguation5/5

    Each tool has a clear, distinct purpose: recording predictions, recording actuals, and fetching results. There is no overlap or ambiguity.

    Naming Consistency5/5

    All tool names follow a consistent verb_noun pattern (get_results, record_actuals, record_prediction), making them predictable and easy to understand.

    Tool Count5/5

    With 3 tools, the server is tightly scoped to the core operations of the earnings analysis domain—recording predictions, recording actuals, and retrieving results. No unnecessary tools.

    Completeness5/5

    The tool surface covers the full lifecycle of the domain: creating predictions, recording actuals to settle them, and retrieving results with performance metrics. No obvious gaps.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      B
      quality
      B
      maintenance
      Investment decision tools for AI agents: portfolio status, isolated multi-agent committee analysis, auditable verdict history, and lookahead-protected backtests. Advisory only, no auto-trading; negative research results published.
      42
      697 PyPI
      142
      MIT
    • F
      license
      A
      quality
      D
      maintenance
      Provides multi-agent equity research for US markets with provenance-backed financial data from SEC EDGAR, technicals, macro, and Alpaca paper trading, enforcing risk limits and journaling theses.
      27
      -
    • F
      license
      Not graded
      quality
      B
      maintenance
      Auditable AGI-2027 evidence: eight graded Situational Awareness predictions with pre-registered flip conditions, a 0-100 Thesis Tracker with full score history, and a public market-call ledger where misses stay published. Free, no auth, CC BY 4.0.
      -