court_leaderboard
Live ranking of decision models by Brier score on resolved real-world questions. Free. Returns empty lists when nothing has resolved.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| task_type | No |
Live ranking of decision models by Brier score on resolved real-world questions. Free. Returns empty lists when nothing has resolved.
| Name | Required | Description | Default |
|---|---|---|---|
| task_type | No |
Changes observed during successful MCP inspections.
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so safety is covered; the description still adds two things annotations cannot: the call costs nothing and it returns empty lists when nothing has resolved, sparing the agent from treating an empty result as an error. It leaves out ranking direction (whether lower Brier ranks higher) and result-size limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the resource and metric, with the cost and empty-result caveats following. Nothing is padded or repeated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only list tool with no output schema, it covers what the ranking is and the degenerate empty case, but it omits the meaning of the only parameter and the ordering/return shape, leaving gaps an agent must guess at.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the single parameter task_type is never mentioned, so an agent cannot tell whether it filters the leaderboard by task category or is ignored. With low coverage the description should compensate, and it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific resource (ranking of decision models) plus the ranking metric (Brier score) and population (resolved real-world questions), so an agent can distinguish it from court_model_report, which profiles one model rather than ranking all of them. It stops short of naming those siblings explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to call this versus court_model_report or court_calibrate_labels, and no mention of prerequisites. 'Free' and the empty-list note describe cost and output behavior, not when-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add one secure layer between your agents and this server.