noisefloor

by beepboop2025

Overview Schema Source Related Servers Discussions Admin

Replay forecast performance

score_forecasts

Read-onlyIdempotent

Backtest NoiseFloor's one-step forecast on chronological historical prefixes to evaluate coverage and misses without hindsight. Requires 20+ values; returns scores, misses, baseline comparison.

Instructions

Backtest NoiseFloor's own one-step forecast on chronological historical prefixes, issuing before observing and then updating without hindsight. Use to evaluate forecast_next's coverage and misses; this tool does not accept externally supplied predictions. At least 20 readings are required; shorter valid histories return ok=false. Successful results contain n_forecasts, empirical_coverage, nominal_coverage, score, baseline_score, beats_baseline, records, misses and worst_misses. Scores are mean weighted interval scores in input units (lower is better), compared with fixed historical-error quantiles. Records use zero-based input indices and retain actual, forecast, interval and missed_by. within_coverage_tolerance and its calibrated alias mean observed coverage is within 0.05 of target; they are descriptive, not a certification or significance test. Coverage and favorable scores do not establish future market performance or profitability.

Input Schema

TableJSON Schema

Name	Required	Description	Default
`values`	Yes	One to 2048 chronological realized observations with consistent units/cadence; at least 20 to score. Predictions begin after ten one-step errors, so the first scored record has zero-based index 11.
`nominal`	No	Target interval coverage fraction, default 0.80. Use the same value passed to forecast_next for like-for-like replay; not a confidence level for the backtest result.

Output Schema

TableJSON Schema

Name	Required	Description
`ok`	No	False denotes insufficient history; forecast or scoring fields are then absent.
`error`	No	Validation or computation failure message; inspect MCP isError or the HTTP status.
`score`	No	Mean adaptive weighted interval score.
`method`	No	Shared forecast issuance and scoring method.
`misses`	No	Every uncovered observation.
`caveats`	No	Ordered statements or identifiers.
`reading`	No	Result interpretation or minimum-history requirement.
`records`	No	Every scored observation.
`n_misses`	No	Number of uncovered observations.
`validity`	No	Coverage is empirical only.
`guarantee`	No	No statistical or market-performance guarantee is made.
`calibrated`	No	Compatibility alias of within_coverage_tolerance; descriptive only.
`n_forecasts`	No	Number of issued forecasts scored.
`worst_misses`	No	At most five misses sorted by distance outside the interval.
`baseline_score`	No	Mean baseline weighted interval score.
`beats_baseline`	No	Adaptive mean score is strictly below baseline.
`nominal_coverage`	No	Target coverage fraction.
`records_complete`	No	All scored observations are included.
`skill_vs_baseline`	No	One minus adaptive/baseline score; null when baseline score is zero. Null means unavailable or not applicable, never zero.
`empirical_coverage`	No	Observed covered fraction.
`calibration_criterion`	No	Descriptive tolerance rule, not a significance test.
`within_coverage_tolerance`	No	Observed coverage is within 0.05 of the target.

Schema Changelog

Changes observed during successful MCP inspections.

Changed4 schema fields changedv0.4.3Oct 18, 2026

removedOutput schema / additionalProperties
```
Removed value: -true
```

addedOutput schema / anyOf

Added value: +[
+  {
+    "additionalProperties": false,
+    "allOf": [
+      {
+        "if": {
+          "properties": {
+            "ok": {
+              "const": true
+            }
+          }
+        },
+        "then": {
+          "required": [
+            "n_forecasts",
+            "nominal_coverage",
+            "empirical_coverage",
+            "calibrated",
+            "within_coverage_tolerance",
+            "calibration_criterion",
+            "score",
+            "baseline_score",
+            "beats_baseline",
+            "skill_vs_baseline",
+            "n_misses",
+            "records",
+            "misses",
+            "worst_misses",
+            "records_complete"
+          ]
+        }
+      }
+    ],
+    "properties": {
+      "baseline_score": {},
+      "beats_baseline": {},
+      "calibrated": {},
+      "calibration_criterion": {},
+      "caveats": {},
+      "empirical_coverage": {},
+      "guarantee": {},
+      "method": {},
+      "misses": {},
+      "n_forecasts": {},
+      "n_misses": {},
+      "nominal_coverage": {},
+      "ok": {},
+      "reading": {},
+      "records": {},
+      "records_complete": {},
+      "score": {},
+      "skill_vs_baseline": {},
+      "validity": {},
+      "within_coverage_tolerance": {},
+      "worst_misses": {}
+    },
+    "required": [
+      "ok",
+      "validity",
+      "guarantee",
+      "method",
+      "caveats",
+      "reading"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "description": "Existing tool-error payload; no successful result fields are implied.",
+    "properties": {
+      "error": {
+        "description": "Validation or computation failure message; inspect MCP isError or the HTTP status.",
+        "type": "string"
+      }
+    },
+    "required": [
+      "error"
+    ],
+    "type": "object"
+  }
+]

changedOutput schema / description

Previous value: -"Tool-specific result object described in each tool's description. Check status/ok/state, reading, validity and limits before using numbers. MCP returns this object in structuredContent and JSON text; invalid input returns isError=true with an error field. No tool fetches sources or executes trades."New value: +"Forecast replay scores or insufficient-history result. Tool errors retain the separate error-message shape."

addedOutput schema / properties

Added value: +{
+  "baseline_score": {
+    "description": "Mean baseline weighted interval score.",
+    "type": "number"
+  },
+  "beats_baseline": {
+    "description": "Adaptive mean score is strictly below baseline.",
+    "type": "boolean"
+  },
+  "calibrated": {
+    "description": "Compatibility alias of within_coverage_tolerance; descriptive only.",
+    "type": "boolean"
+  },
+  "calibration_criterion": {
+    "description": "Descriptive tolerance rule, not a significance test.",
+    "type": "string"
+  },
+  "caveats": {
+    "description": "Ordered statements or identifiers.",
+    "items": {
+      "description": "One retained statement or identifier.",
+      "type": "string"
+    },
+    "type": "array"
+  },
+  "empirical_coverage": {
+    "description": "Observed covered fraction.",
+    "type": "number"
+  },
+  "error": {
+    "description": "Validation or computation failure message; inspect MCP isError or the HTTP status.",
+    "type": "string"
+  },
+  "guarantee": {
+    "description": "No statistical or market-performance guarantee is made.",
+    "type": "null"
+  },
+  "method": {
+    "description": "Shared forecast issuance and scoring method.",
+    "type": "string"
+  },
+  "misses": {
+    "description": "Every uncovered observation.",
+    "items": {
+      "additionalProperties": false,
+      "description": "One prequential issue/observe/score record; future observations do not alter it.",
+      "properties": {
+        "actual": {
+          "description": "Observed value in input units.",
+          "type": "number"
+        },
+        "adaptive_alpha": {
+          "description": "Clipped adaptive miscoverage parameter as a fraction.",
+          "type": "number"
+        },
+        "baseline_score": {
+          "description": "Historical-quantile baseline weighted interval score in input units.",
+          "type": "number"
+        },
+        "covered": {
+          "description": "Actual value lies inside the issued interval.",
+          "type": "boolean"
+        },
+        "error_history_size": {
+          "description": "Historical absolute errors used before issuing this forecast.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "forecast": {
+          "description": "One-step random-walk point forecast in input units.",
+          "type": "number"
+        },
+        "index": {
+          "description": "Zero-based index of the observation scored after forecast issuance.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "interval": {
+          "description": "Lower and upper bounds, in that order.",
+          "items": {
+            "description": "Bound in the original input units.",
+            "type": "number"
+          },
+          "maxItems": 2,
+          "minItems": 2,
+          "type": "array"
+        },
+        "missed_by": {
+          "description": "Distance outside the interval in input units, zero when covered.",
+          "type": "number"
+        },
+        "nominal_coverage": {
+          "description": "Requested target coverage as a fraction, not a guarantee.",
+          "type": "number"
+        },
+        "score": {
+          "description": "Adaptive weighted interval score in input units; lower is better.",
+          "type": "number"
+        }
+      },
+      "required": [
+        "index",
+        "forecast",
+        "interval",
+        "nominal_coverage",
+        "adaptive_alpha",
+        "error_history_size",
+        "actual",
+        "covered",
+        "missed_by",
+        "score",
+        "baseline_score"
+      ],
+      "type": "object"
+    },
+    "type": "array"
+  },
+  "n_forecasts": {
+    "description": "Number of issued forecasts scored.",
+    "minimum": 0,
+    "type": "integer"
+  },
+  "n_misses": {
+    "description": "Number of uncovered observations.",
+    "minimum": 0,
+    "type": "integer"
+  },
+  "nominal_coverage": {
+    "description": "Target coverage fraction.",
+    "type": "number"
+  },
+  "ok": {
+    "description": "False denotes insufficient history; forecast or scoring fields are then absent.",
+    "type": "boolean"
+  },
+  "reading": {
+    "description": "Result interpretation or minimum-history requirement.",
+    "type": "string"
+  },
+  "records": {
+    "description": "Every scored observation.",
+    "items": {
+      "additionalProperties": false,
+      "description": "One prequential issue/observe/score record; future observations do not alter it.",
+      "properties": {
+        "actual": {
+          "description": "Observed value in input units.",
+          "type": "number"
+        },
+        "adaptive_alpha": {
+          "description": "Clipped adaptive miscoverage parameter as a fraction.",
+          "type": "number"
+        },
+        "baseline_score": {
+          "description": "Historical-quantile baseline weighted interval score in input units.",
+          "type": "number"
+        },
+        "covered": {
+          "description": "Actual value lies inside the issued interval.",
+          "type": "boolean"
+        },
+        "error_history_size": {
+          "description": "Historical absolute errors used before issuing this forecast.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "forecast": {
+          "description": "One-step random-walk point forecast in input units.",
+          "type": "number"
+        },
+        "index": {
+          "description": "Zero-based index of the observation scored after forecast issuance.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "interval": {
+          "description": "Lower and upper bounds, in that order.",
+          "items": {
+            "description": "Bound in the original input units.",
+            "type": "number"
+          },
+          "maxItems": 2,
+          "minItems": 2,
+          "type": "array"
+        },
+        "missed_by": {
+          "description": "Distance outside the interval in input units, zero when covered.",
+          "type": "number"
+        },
+        "nominal_coverage": {
+          "description": "Requested target coverage as a fraction, not a guarantee.",
+          "type": "number"
+        },
+        "score": {
+          "description": "Adaptive weighted interval score in input units; lower is better.",
+          "type": "number"
+        }
+      },
+      "required": [
+        "index",
+        "forecast",
+        "interval",
+        "nominal_coverage",
+        "adaptive_alpha",
+        "error_history_size",
+        "actual",
+        "covered",
+        "missed_by",
+        "score",
+        "baseline_score"
+      ],
+      "type": "object"
+    },
+    "type": "array"
+  },
+  "records_complete": {
+    "const": true,
+    "description": "All scored observations are included.",
+    "type": "boolean"
+  },
+  "score": {
+    "description": "Mean adaptive weighted interval score.",
+    "type": "number"
+  },
+  "skill_vs_baseline": {
+    "anyOf": [
+      {
+        "description": "One minus adaptive/baseline score; null when baseline score is zero.",
+        "type": "number"
+      },
+      {
+        "type": "null"
+      }
+    ],
+    "description": "One minus adaptive/baseline score; null when baseline score is zero. Null means unavailable or not applicable, never zero."
+  },
+  "validity": {
+    "const": "empirical_only",
+    "description": "Coverage is empirical only.",
+    "type": "string"
+  },
+  "within_coverage_tolerance": {
+    "description": "Observed coverage is within 0.05 of the target.",
+    "type": "boolean"
+  },
+  "worst_misses": {
+    "description": "At most five misses sorted by distance outside the interval.",
+    "items": {
+      "additionalProperties": false,
+      "description": "One prequential issue/observe/score record; future observations do not alter it.",
+      "properties": {
+        "actual": {
+          "description": "Observed value in input units.",
+          "type": "number"
+        },
+        "adaptive_alpha": {
+          "description": "Clipped adaptive miscoverage parameter as a fraction.",
+          "type": "number"
+        },
+        "baseline_score": {
+          "description": "Historical-quantile baseline weighted interval score in input units.",
+          "type": "number"
+        },
+        "covered": {
+          "description": "Actual value lies inside the issued interval.",
+          "type": "boolean"
+        },
+        "error_history_size": {
+          "description": "Historical absolute errors used before issuing this forecast.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "forecast": {
+          "description": "One-step random-walk point forecast in input units.",
+          "type": "number"
+        },
+        "index": {
+          "description": "Zero-based index of the observation scored after forecast issuance.",
+          "minimum": 0,
+          "type": "integer"
+        },
+        "interval": {
+          "description": "Lower and upper bounds, in that order.",
+          "items": {
+            "description": "Bound in the original input units.",
+            "type": "number"
+          },
+          "maxItems": 2,
+          "minItems": 2,
+          "type": "array"
+        },
+        "missed_by": {
+          "description": "Distance outside the interval in input units, zero when covered.",
+          "type": "number"
+        },
+        "nominal_coverage": {
+          "description": "Requested target coverage as a fraction, not a guarantee.",
+          "type": "number"
+        },
+        "score": {
+          "description": "Adaptive weighted interval score in input units; lower is better.",
+          "type": "number"
+        }
+      },
+      "required": [
+        "index",
+        "forecast",
+        "interval",
+        "nominal_coverage",
+        "adaptive_alpha",
+        "error_history_size",
+        "actual",
+        "covered",
+        "missed_by",
+        "score",
+        "baseline_score"
+      ],
+      "type": "object"
+    },
+    "maxItems": 5,
+    "type": "array"
+  }
+}

Changed4 schema fields changedv0.4.3-glama.1Oct 17, 2026

addedInput schema / properties / nominal / default
```
Added value: +0.8
```

addedInput schema / properties / nominal / description

Added value: +"Target interval coverage fraction, default 0.80. Use the same value passed to forecast_next for like-for-like replay; not a confidence level for the backtest result."

addedInput schema / properties / values / description

Added value: +"One to 2048 chronological realized observations with consistent units/cadence; at least 20 to score. Predictions begin after ten one-step errors, so the first scored record has zero-based index 11."

addedOutput schema / description

Added value: +"Tool-specific result object described in each tool's description. Check status/ok/state, reading, validity and limits before using numbers. MCP returns this object in structuredContent and JSON text; invalid input returns isError=true with an error field. No tool fetches sources or executes trades."

Changed7 schema fields changedv0.4.1Oct 17, 2026
- removedInput schema / properties / nominal / description
```
Removed value: -"range coverage, default 0.8"
```
- addedInput schema / properties / nominal / exclusiveMaximum
```
Added value: +1
```
- addedInput schema / properties / nominal / exclusiveMinimum
```
Added value: +0
```
- removedInput schema / properties / values / description
```
Removed value: -"the metric's history, oldest first"
```
- addedInput schema / properties / values / maxItems
```
Added value: +2048
```
- addedInput schema / properties / values / minItems
```
Added value: +1
```
- changedOutput schema / (root)
```
Previous value: -nullNew value: +{
+  "additionalProperties": true,
+  "type": "object"
+}
```
First observedv0.1.2Jul 28, 2026

TDQS

A4.6/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint, idempotentHint, destructiveHint=false), but the description goes well beyond them: it discloses the ≥20-reading threshold and the ok=false failure mode, explains the scoring direction (mean weighted interval scores, lower is better, compared against fixed historical-error quantiles), and warns that coverage flags are descriptive rather than a significance test and do not imply future performance. That is substantial behavioral context for a read-only tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and exclusions are front-loaded in the first two sentences, and the caveats about tolerance flags and future performance are valuable. The enumeration of return fields (n_forecasts, empirical_coverage, score, ...) is somewhat redundant given an output schema exists, making it slightly longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the analytical complexity, the description covers prerequisites, failure behavior, scoring semantics, index conventions, and interpretation caveats, while the output schema handles return structure. An agent has everything needed to decide to call it and to interpret the result responsibly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description's parameter-related details (values needing at least 20 readings, first scored record at index 11, nominal needing the same value passed to forecast_next) largely restate what the schema already documents. Baseline 3 is appropriate when the schema does the heavy lifting and the description adds little new parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (backtest/replay) on a specific resource (NoiseFloor's own one-step forecast) and immediately distinguishes itself from forecast_next by declaring it will not accept externally supplied predictions. An agent can tell exactly what this tool does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the intended use case ('Use to evaluate forecast_next's coverage and misses'), states a hard exclusion ('does not accept externally supplied predictions'), and gives a prerequisite ('at least 20 readings are required; shorter valid histories return ok=false'). The when-to-use and when-not-to-use conditions are both present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server