Skip to main content
Glama

forecast_uncertainty

Compares forecast uncertainty by lead time between a fitted multivariate VARMA model and univariate benchmarks, highlighting where joint modeling could improve forecasts.

Instructions

N5/N6 — Jenkins and Alavi's comparison of forecast uncertainty (their Table VIII): the standard deviation of the forecast errors by lead time, V(l) = SUM psi_j Sigma psi_j', of the last estimated model against the univariate models, per cent for series in logs. In sample, parameters taken as known: a quick reading of where the multivariate model could help; evaluate is the test. horizon defaults to the frequency (4 for annual data).

estwin (optional): re-estimate the model AND the univariate models on the first estwin observations of the first series, as Jenkins and Alavi did for their Table VIII (§6.3: the muskrat-mink models refitted on 48 observations, the rest withheld). 0 = the last fit, on all the data.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "name": "<name>"
    },
    "name": "forecast_uncertainty"
  }
}

Input Schema

NameRequiredDescriptionDefault
nameYes
estwinNo
horizonNo

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does disclose non-obvious behavior: results are in-sample with parameters taken as known, `horizon` falls back to the data frequency (4 for annual), and `estwin` triggers re-estimation of both the model and the univariate models on a truncated sample — a state-changing side effect an agent must know about. It stops short of saying whether that re-estimation persists or how it interacts with the previously fitted model.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Roughly 90 words of dense econometric prose; every sentence carries content, but the front-loading is poor — it opens with an opaque 'N5/N6' label and a matrix formula, while the decisive guidance ('`evaluate` is the test') and the parameter defaults are pushed into the middle and end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and the in-sample caveat plus both non-trivial parameter defaults are covered. Gaps remain for a 3-parameter tool with zero schema documentation: the required `name` argument is undefined and the persistence/state impact of `estwin` re-estimation is unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It does well for `horizon` (default = frequency, 4 for annual) and `estwin` (0 = last fit on all data; N = re-estimate on the first N observations of the first series), but the required `name` parameter is never explained at all, and 'first series' remains ambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

It names the specific quantity computed — the standard deviation of forecast errors by lead time, comparing the multivariate model against univariate models — which is a real verb+resource statement, not a restatement of the name. However the leading 'N5/N6 —' tag and the inline matrix formula obscure the purpose for anyone without the Jenkins/Alavi reference, and no sibling is distinguished by name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames the tool as 'a quick reading of where the multivariate model could help' and points to `evaluate` as the formal test, which is a genuine when-to-use/when-to-use-something-else contrast. There is no explicit exclusion beyond that single alternative, and the guidance is buried mid-paragraph rather than front-loaded.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.