Skip to main content
Glama

Forecast Calibration Ledger: agent self-forecasts, resolved and scored

fcl_calibration

Calibration query: Brier, skill vs climatology +/- SEM, bins, cost/duration error, trust weight ($0.005/query; included with fleet plan). Requires api_token, forecaster, cohort; optional x_payment.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "api_token": "<api_token>",
      "forecaster": "<forecaster>",
      "cohort": "<cohort>"
    },
    "name": "fcl_calibration"
  }
}

Input Schema

NameRequiredDescriptionDefault
cohortYes
api_tokenYes
x_paymentNo
forecasterYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses the per-query cost, fleet-plan inclusion, authentication requirement, and returned metrics, but it does not state whether the operation is read-only, how errors or missing permissions are handled, or any rate-limit behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the core purpose before listing metrics, billing, and parameters. The semicolon-separated density is efficient, though slightly terse for an agent parsing multiple concepts.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description names the metrics returned, which is important because there is no output schema, and it discloses cost and required authentication. However, with no annotations and 0% schema coverage, it still leaves key behavioral and parameter details ambiguous.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain parameter meaning. It names api_token, forecaster, cohort, and x_payment and marks required versus optional, but it does not explain what a forecaster or cohort represents, expected formats, or what x_payment should contain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and operation: a calibration query returning Brier score, skill versus climatology, bins, cost/duration error, and trust weight. It is clear what the tool produces, but it does not explicitly distinguish itself from siblings such as fcl_forecast or fcl_weight.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists required and optional parameters but gives no guidance on when to use this tool versus the sibling tools. An agent must infer its usage from the phrase 'Calibration query' without explicit conditions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources