tradefloor

Overview Schema Source Related Servers Discussions Admin

Rank strategies across many seeds

rank_strategies

Read-onlyIdempotent

Score strategies across multiple seeds against baseline agents, then rank pairs with a paired sign test to avoid single-seed luck after evaluating candidates.

Instructions

Score strategies across MANY seeds, beside the baseline agents, and rank them with a paired sign test on each pair. Use it after evaluate_strategies, because one seed's ordering is often luck. Costs about one evaluate_strategies call per seed: 2 to 12 seeds (default six), days 1 to 60 here, up to 252 through start_job. Returns each entrant's record across the seeds (median P&L, seeds ahead of buy-and-hold) and each pair's sign test. Takes preset as evaluate_strategies does. Runs isolated only and refuses population, because the paired test needs every strategy on the same market. Deterministic.

Input Schema

Name	Required	Description
`days`	No	Trading days to run: 1 to 60 in a direct call, up to 252 (the certified horizon) through start_job.
`seeds`	No	Simulation seeds, 2 to 12 of them. Every entrant trades the same market on each seed. Omit for [1, 2, 3, 4, 5, 6].
`preset`	No	A shipped model preset to run, by name: pt-v1, pt-v2, pt-v3, pt-v4, pt-v5, pt-v6, pt-v7, pt-v8, pt-v9, pt-v10, pt-v11, pt-v12, pt-v13, pt-v14, pt-v15, pt-v16, pt-v18, pt-v19, pt-v20, pt-v21. Omit for the shipped default, pt-v21, the only preset the realism envelope certifies; a result under any other carries a caveat saying it is outside the certification, with that preset's own measured record. describe_simulator lists every preset's record.
`universe`	No	A roster document, usually the `universe` field of a build_universe result. Either {"size": n, "seed": s, "sectors": [...]} or {"instruments": [...]}. When given it replaces universe_size, universe_seed and universe_sectors.
`population`	No	rank_strategies refuses any population, because its paired test needs every strategy on the same market. Pass population to evaluate_strategies or run_stress_scenario instead.
`strategies`	Yes	Strategies to run, keyed by a name you choose. Each value is a strategy spec, for example {"signal": {"kind": "momentum", "lookback_days": 1.0}, "portfolio": {"top_k": 5, "gross": 1.0}}. Signal kinds: hold, random, momentum, mean_reversion, oracle, blend. At most 8. The baseline names (buy_and_hold, random, momentum, mean_reversion, oracle) are taken. Check a spec with validate_strategy before running it.
`max_leverage`	No	Cap on gross exposure as a multiple of net worth. null removes the cap, and the result then warns that trading size alone can win.
`steps_per_day`	No	Decision points per trading day, 1 to 22. Each entrant is asked for orders at each one. A step is 65 minutes, so 6 cover the trading session, and days times steps may be at most 360 in a direct call.
`universe_seed`	No	Seed that generates the roster, separate from the simulation seed. Ignored when `universe` is given.
`universe_size`	No	Names in a generated roster, 2 to 120. Ignored when `universe` is given.
`universe_sectors`	No	Lowercase sector ids to concentrate a generated roster on, for example ["technology", "energy"]. The ids: technology, financial_services, healthcare, energy, consumer_discretionary, consumer_staples, industrials, materials, real_estate, utilities, telecommunications, transportation. A concentrated roster is a named envelope gap, and the result says so.

Output Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

Changed2 schema fields changedv0.10.1Oct 7, 2026

addedInput schema / properties / population

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "description": "rank_strategies refuses any population, because its paired test needs every strategy on the same market. Pass population to evaluate_strategies or run_stress_scenario instead.",
+  "title": "Population"
+}

addedInput schema / properties / preset

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "description": "A shipped model preset to run, by name: pt-v1, pt-v2, pt-v3, pt-v4, pt-v5, pt-v6, pt-v7, pt-v8, pt-v9, pt-v10, pt-v11, pt-v12, pt-v13, pt-v14, pt-v15, pt-v16, pt-v18, pt-v19, pt-v20, pt-v21. Omit for the shipped default, pt-v21, the only preset the realism envelope certifies; a result under any other carries a caveat saying it is outside the certification, with that preset's own measured record. describe_simulator lists every preset's record.",
+  "title": "Preset"
+}

First observedv0.1.0Oct 4, 2026

TDQS

A4.6/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only/idempotent/non-destructive, and the description adds genuinely new context: cost is roughly one evaluate_strategies call per seed, execution is isolated-only, it refuses population, it is deterministic, and non-certified presets carry a caveat. That is substantial behavioral disclosure beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and method, then constraints and cost, with no filler sentences. It is dense and slightly repetitive of the schema on seed/day limits, but every sentence carries real information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 11-parameter tool with nested strategy specs, the description covers the key operating envelope: seed count, horizon limits, cost, isolation constraint, determinism, and refusal behavior. An output schema exists, so the brief note on returns is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 11 parameters including seeds, days, preset, and the refusal on population. The description adds only marginal meaning, mainly cross-referencing 'preset as evaluate_strategies does', so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (rank strategies) plus the method (paired sign test across many seeds beside baseline agents), which is a distinct operation from evaluate_strategies. An agent can immediately tell this is the multi-seed ranking step, not a single-run evaluation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use it after evaluate_strategies and why (one seed's ordering is often luck), and it names the refused path (population) with the redirect to evaluate_strategies or run_stress_scenario. When-to-use, when-not, and alternatives are all present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.