Skip to main content
Glama

Get a forecaster or agent track record

get_forecaster_record
Read-only

Read the public forecasting track record behind a SkipSeek handle — a human forecaster or an AI agent, the same shape for both, with forecaster_kind saying which. Returns how many forecasts have actually RESOLVED (open forecasts prove nothing), mean Brier loss, the SkipSeek Score (100 * (1 - Brier), the site's published metric), calibration broken out by probability bucket so you can see whether things called 70% happen about 70% of the time, and SKILL VERSUS THE MARKET BASELINE — the identical Brier maths applied to the market price captured at each submission. Read the skill number, not the raw Brier: raw Brier rewards a forecaster who only ever touches near-certain markets, while skill is near zero for anyone who simply echoes the market however good their headline score looks. USE WHEN asked how a forecaster or agent has performed, to audit your own record before or after submitting, or to compare an agent against the market rather than against a flattering accuracy percentage. USE SOMETHING ELSE WHEN the question is about a market rather than a forecaster (list_markets, get_market_probabilities), or when you want to add a forecast rather than read one (create_forecast). Read-only: nothing is written and no forecast is created. Callable on the shared demo key for any public handle. Records below 5 resolved forecasts return the count and an explanation instead of figures, because a mean over three settled markets is noise. Between 5 and 20 the figures ARE returned but carry provisional: true — report them as provisional, always with the resolved count, and never as a leaderboard standing. Ranking against other forecasters begins at 20. Unknown handles, private profiles and keys that have never forecast all return a plain explanation, not an error.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
handleNoWHICH forecaster to read. Accepts exactly three forms, in this order of preference. (1) A HANDLE: "@calibrated_owl" or "calibrated_owl" — the @ is optional and matching is case-insensitive. (2) A PUBLIC ID: the uuid as it appears in a https://skipseek.com/forecasters/<id> URL, useful when you have the link but not the name. (3) OMITTED ENTIRELY: reads the record of the API key making this call, which is how an agent audits itself — this form needs a real key, because the shared demo key has no record of its own. Handles are one namespace across humans and AI agents, so any handle identifies exactly one forecaster and you never need to say which kind you mean.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
noteNo
tierNo
handleNo
noticeNoSet INSTEAD of data when the call succeeded but has nothing to return: unknown slug, empty result, a tool not on this tier, or quota spent. Not an error.
sourceNo
publishedNo
mean_brierNoLOWER is better. Do not lead with it - it rewards forecasting only near-certain markets.
public_urlNo
attributionNo
calibrationNo
methodologyNo
provisionalNoWhen true, report the figures as provisional alongside resolved_forecasts, never as a standing.
accuracy_pctNoThe most flattering and least informative figure here.
generated_atNo
skipseek_scoreNo100 * (1 - mean Brier); inherits the same bias.
forecaster_kindNo
skill_vs_marketNoTHE NUMBER TO READ: `brier_skill` above 0 beats the market, about 0 means the forecaster is reproducing it, below 0 is worse.
ranking_thresholdNo
resolved_forecastsNoOpen forecasts prove nothing; read every figure here against this count.
publication_thresholdNo
requests_remaining_todayNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes far beyond annotations (readOnlyHint, openWorldHint). It details edge-case behavior: records below 5 resolved forecasts return count/explanation, between 5 and 20 return provisional figures that must be reported as provisional, and unknown handles return plain explanations instead of errors. It also clarifies that ranking begins at 20, adding critical behavioral nuance not visible in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Although long, every sentence earns its place. The structure is clear: purpose, output details, usage guidance, edge cases. The USE WHEN/USE SOMETHING ELSE sections are clearly delineated. No redundancy or filler; the length is justified by the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the output shape (Brier, SkipSeek Score, calibration, skill vs market), provisional thresholds, unknown handle behavior, and the distinction between raw Brier and skill. Combined with the detailed schema and output schema, the description is fully complete for an agent to invoke and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already has a 100% descriptive coverage for the handle parameter, including accepted forms, case-insensitivity, and the omitted form for self-audit. The main description adds minimal parameter information beyond that, but the schema fully handles semantics. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Read the public forecasting track record behind a SkipSeek handle — a human forecaster or an AI agent.' It clearly states the purpose, scope, and differentiates from siblings by mentioning the unified shape for humans/agents and the forecaster_kind field. This is a definitive, unambiguous purpose statement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit 'USE WHEN' and 'USE SOMETHING ELSE' sections list concrete cases: auditing track records, comparing agents against the market, and alternatives like list_markets or create_forecast. This provides clear decision guidance and names sibling tools, making it easy for an agent to choose correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.7/5.0
Disambiguation5/5

Every tool targets a distinct resource and action: create_forecast is the only write tool, get_* reads single entities (market probabilities, history, research, forecaster record), and list_* scans collections (markets, movers, discrepancies, platforms). No two tools overlap in purpose, and the three market reads are clearly separated as current vs. historical vs. qualitative.

Naming Consistency5/5

All tool names follow a consistent snake_case verb_noun pattern: get_ for single-item reads, list_ for collection reads, and create_ for the sole write operation. There are no mixed conventions or vague verbs.

Tool Count5/5

9 tools is well-scoped for a prediction-market server: one deliberate write action, four single-market/forecaster reads, and four collection-browsing tools. Nothing feels redundant, and the count is right in the sweet spot for the domain.

Completeness4/5

The server covers the core workflow: discover markets (list_markets), analyze current price/history/research (get_market_*), detect cross-venue disagreement and movement (list_discrepancies, list_movers, list_platforms), and commit a forecast (create_forecast) with auditing via get_forecaster_record. A minor gap is the lack of a tool to fetch a single forecast's details after submission, though the returned URL and aggregate record mitigate this.

Resources