tradefloor

Overview Schema Source Related Servers Discussions Admin

Evaluate strategies on one market

evaluate_strategies

Read-onlyIdempotent

Run and score trading strategies on a simulated market alongside baselines, measuring return, P&L, trading costs, turnover, and errors.

Instructions

Run strategies on one simulated market, beside the baseline agents on the same market, and score each one: return, P&L, the cost of its own trading in basis points, turnover and errors. Use it first, but it runs one seed, so use rank_strategies before believing an ordering. A strategy is data, for example {"signal": {"kind": "momentum", "lookback_days": 1.0}, "portfolio": {"top_k": 5, "gross": 1.0}}, and validate_strategy checks one without running it. days 1 to 60 here (a few seconds), up to 252 through start_job; roster 2 to 120 names. preset runs another shipped model preset, outside the certification, and population adds background traders that react to each strategy. Deterministic: the same arguments give the same scores.

Input Schema

Name	Required	Description
`cash`	No	Starting cash for each entrant, in currency.
`days`	No	Trading days to run: 1 to 60 in a direct call, up to 252 (the certified horizon) through start_job.
`seed`	No	Simulation seed, an integer from 0 to 2**64 - 1. The same seed and arguments give the same result.
`preset`	No	A shipped model preset to run, by name: pt-v1, pt-v2, pt-v3, pt-v4, pt-v5, pt-v6, pt-v7, pt-v8, pt-v9, pt-v10, pt-v11, pt-v12, pt-v13, pt-v14, pt-v15, pt-v16, pt-v18, pt-v19, pt-v20, pt-v21. Omit for the shipped default, pt-v21, the only preset the realism envelope certifies; a result under any other carries a caveat saying it is outside the certification, with that preset's own measured record. describe_simulator lists every preset's record.
`universe`	No	A roster document, usually the `universe` field of a build_universe result. Either {"size": n, "seed": s, "sectors": [...]} or {"instruments": [...]}. When given it replaces universe_size, universe_seed and universe_sectors.
`population`	No	A shipped population of background traders to run beside, by name: standard, crowded. They share each strategy's market and react to its trading (populated mode), so a result says whether an edge survives other traders, and strategies no longer meet identical markets. Omit for isolated mode, the default. describe_simulator lists each population's traders.
`strategies`	Yes	Strategies to run, keyed by a name you choose. Each value is a strategy spec, for example {"signal": {"kind": "momentum", "lookback_days": 1.0}, "portfolio": {"top_k": 5, "gross": 1.0}}. Signal kinds: hold, random, momentum, mean_reversion, oracle, blend. At most 8. The baseline names (buy_and_hold, random, momentum, mean_reversion, oracle) are taken. Check a spec with validate_strategy before running it.
`max_leverage`	No	Cap on gross exposure as a multiple of net worth. null removes the cap, and the result then warns that trading size alone can win.
`steps_per_day`	No	Decision points per trading day, 1 to 22. Each entrant is asked for orders at each one. A step is 65 minutes, so 6 cover the trading session, and days times steps may be at most 360 in a direct call.
`universe_seed`	No	Seed that generates the roster, separate from the simulation seed. Ignored when `universe` is given.
`universe_size`	No	Names in a generated roster, 2 to 120. Ignored when `universe` is given.
`universe_sectors`	No	Lowercase sector ids to concentrate a generated roster on, for example ["technology", "energy"]. The ids: technology, financial_services, healthcare, energy, consumer_discretionary, consumer_staples, industrials, materials, real_estate, utilities, telecommunications, transportation. A concentrated roster is a named envelope gap, and the result says so.
`include_baselines`	No	Add the baseline agents (buy_and_hold, random, momentum, mean_reversion, oracle) to the same market. On by default, because a return means little without buy-and-hold's beside it.

Output Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

Changed2 schema fields changedv0.10.1Oct 7, 2026

addedInput schema / properties / population

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "description": "A shipped population of background traders to run beside, by name: standard, crowded. They share each strategy's market and react to its trading (populated mode), so a result says whether an edge survives other traders, and strategies no longer meet identical markets. Omit for isolated mode, the default. describe_simulator lists each population's traders.",
+  "title": "Population"
+}

addedInput schema / properties / preset

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "description": "A shipped model preset to run, by name: pt-v1, pt-v2, pt-v3, pt-v4, pt-v5, pt-v6, pt-v7, pt-v8, pt-v9, pt-v10, pt-v11, pt-v12, pt-v13, pt-v14, pt-v15, pt-v16, pt-v18, pt-v19, pt-v20, pt-v21. Omit for the shipped default, pt-v21, the only preset the realism envelope certifies; a result under any other carries a caveat saying it is outside the certification, with that preset's own measured record. describe_simulator lists every preset's record.",
+  "title": "Preset"
+}

First observedv0.1.0Oct 4, 2026

TDQS

A4.6/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly, idempotent, non-destructive), it discloses determinism ('the same arguments give the same scores'), the direct-call day cap (1-60) versus 252 via start_job, the 8-strategy ceiling, that baseline names are reserved, that populated mode makes strategies no longer share identical markets, and that a null leverage cap triggers a warning. This is unusually rich behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and scoring outputs, then constraints and routing in descending priority. It is dense and mostly earns its sentences, though a few details (day/roster bounds, preset caveat) duplicate the schema and could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not explain return values, and it covers the remaining gaps: determinism, run-size limits, baseline inclusion rationale, and the certification caveat for non-default presets. An agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter including presets, populations, roster bounds and reserved baseline names. The description mostly restates those facts (days 1-60, roster 2-120, preset certification caveat) rather than adding new parameter-level meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource ('Run strategies on one simulated market... and score each one: return, P&L, cost of its own trading in basis points, turnover and errors'), which is far more specific than the title. It also implicitly distinguishes itself from validate_strategy (checks without running) and rank_strategies (orders across seeds).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit sequencing ('Use it first, but it runs one seed, so use rank_strategies before believing an ordering'), names validate_strategy for spec checking, and routes longer horizons to start_job. When-to-use and alternatives are both stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.