Skip to main content
Glama
Sign In
Polydev

Polydev

Official
by polydev-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Hybrid

Rank answers

rank_perspectives

Rank ask_model answers best first when a user requests or agrees to a comparison, then log model scores and feedback for the PolyDev leaderboard.

Instructions

Rank the answers you got from ask_model, best first, when the user asks for it or agrees. Rankings feed the public leaderboard at https://www.polydev.ai/leaderboard. Pass your own model ID as base_model so the ranking records which model judged.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "ranked_models": "<ranked_models>"
    },
    "name": "rank_perspectives"
  }
}

Input Schema

NameRequiredDescriptionDefault
base_modelNoYour own model ID (the model making this ranking)
model_scoresNoOptional: a score from 1 to 10 per model ID
feedback_textNoOptional: why you ranked them this way
ranked_modelsYesModel IDs ranked best to worst, exactly as shown in the ask_model output

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by disclosing a real side effect the annotations only imply: rankings feed a public leaderboard at a named URL, which is important because the data becomes externally visible. That public-exposure disclosure is valuable context not present in readOnlyHint=false / openWorldHint=true, though it doesn't cover whether rankings are reversible or how failures are reported.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and ordering, then the consent condition, then the recording requirement. The leaderboard URL is a slight detour but earns its place as justification for the call; little else could be cut.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the essential pieces for a 4-parameter write tool with no output schema: the trigger, the required model-ID provenance, and the public consequence of ranking. Missing only edge-case behavior (what happens with partial or malformed ranked_models), which the fully described schema largely mitigates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine intent for the non-required base_model parameter ('so the ranking records which model judged'), signalling it is effectively expected even though optional. It does not add anything for model_scores or feedback_text beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('rank the answers you got from ask_model, best first') and pins the exact ordering convention. The reference to ask_model makes the relationship to the sibling production tool unambiguous, so an agent can tell this is the post-comparison step rather than a query tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger condition: use it 'when the user asks for it or agrees,' which correctly gates the call behind user consent. It lacks an explicit when-not clause (e.g., do not rank before answers are collected, do not rank without consent), so it falls just short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools