Analyze top-k stability

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered; the description goes further by naming the data source (mcpg_rag.rerank_events), the emitted flag (topk_stable at WARNING severity), and the degraded path — zero counts when the table is absent or the window is empty. That empty-table fallback is exactly the kind of behavior an agent cannot infer from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Leads with the metric definition, then interpretation, then the emitted signal, then data-source and edge-case handling — a sensible front-loaded ordering with no filler. Four sentences is proportionate, though the double-backtick markup around topk_stable is noise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't explain the report shape and correctly focuses on method, interpretation, and failure mode. For a 5-parameter analytical tool with 0 required parameters and near-zero schema coverage, however, the absence of any guidance on k, days, model, or retrieval_index leaves the caller guessing at scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Five parameters with only 20% schema description coverage, and the description explains none of them beyond the incidental phrases 'at this K' and 'the window'. It never says what k selects, what unit days represents, or how model and retrieval_index scope the query, so it does not compensate for the low coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Names the exact computation (Jaccard overlap between top-K-by-bi-rank and top-K-by-cross-rank, aggregated per query) and the meaning of the score, which is specific enough to separate it from NDCG/lift siblings in the rerank family. It never names a sibling explicitly, so differentiation is implicit rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'High mean Jaccard means the reranker isn't actually changing the top-K membership' and 'when the rerank is barely earning its place at this K', which tells an agent the diagnostic situation. There is no explicit when-to-use / when-not-to-use statement and no routing to analyze_rerank_ndcg or analyze_reranker_lift for the adjacent questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.