permutation_importance

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavior disclosure. It explains model-agnostic nature, measurement on test data, and that it shows error bars from multiple random shuffles, indicating stochastic behavior. It does not mention potential computational cost or return format, but adds meaningful context beyond the bare name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief, front-loaded with the core definition, and each sentence adds value. The example is compact and directly illustrates usage without wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity, no annotations, and no output schema, the description covers the what and why but omits the return value or how results are presented. It also does not specify prerequisites or error scenarios, leaving some gaps for a tool with 6 optional parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for 6 undocumented parameters. It only mentions 'model_name' and 'test_df_name' in the example and hints at 'n_repeats' via 'multiple random shuffles', but leaves 'top_n', 'save_path', and 'target_column' unexplained. This is insufficient for proper parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'model-agnostic feature importance measured on test data' and contrasts with 'built-in feature_importances_', which aligns with the sibling tool 'feature_importance'. This distinguishes the tool's purpose and resource clearly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells the agent when to prefer this tool ('More reliable than built-in feature_importances_') and implies it should be used on test data. It provides a concrete example invocation, but lacks explicit exclusions or prerequisites like requiring a trained model or existing dataframes.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.