Server Quality Checklist

Tool Scores

• gemini_code_golfC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden, but it only discloses the API key requirement and the return type. It does not mention that code is sent to an external service, potential rate limits/costs, or what 'golfed' output looks like, leaving significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description has a clear Args/Returns structure and is reasonably concise, but it repeats the Google/Gemini dependency in consecutive sentences and uses awkward phrasing. It would be tighter with the redundancy removed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential parameters, return type, and the prerequisite, but misses important context such as the meaning of 'golfed,' whether the code is transformed in place, and external API implications. It is adequate but not complete for an external-call tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists all three parameters with meanings and defaults, compensating for the schema's 0% description coverage. However, it erroneously marks lean_code as 'optional' while the schema marks it as required, which is a dangerous contradiction that could mislead an agent into omitting a required argument.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'uses the Google Gemini model to simplify Lean code,' distinguishing it from sibling tools that inspect or interact with Lean files. However, the phrase 'compiled by the compiler' is confusing and it doesn't explicitly contrast with the other Gemini-based tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to choose this tool over alternatives such as gemini_informal_prover or discussion_partner. The only usage hint is the GOOGLE_API_KEY prerequisite, which is operational, not selectional.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gpt_informal_proverB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It mentions the return format (List[str] with solution and verification_result) but does not disclose limitations, potential for incorrect outputs, or randomness effects despite a temperature parameter. The claim 'GPT's math skills are outstanding; you can trust the answers he gives you' is an unsupported assertion rather than a factual behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is verbose and includes extraneous content: the trust claim about GPT, and a lengthy workflow involving Gemini and leandex that is tangential to the tool's own behavior. The key information is buried within a long paragraph, making it less scannable. A shorter, focused description would be more effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is part of a complex pipeline, but the description leaves critical ambiguity: it tells the user to ask Gemini for a solution while this is a GPT tool, and the 'switch to informal_prover' line is unclear. It does document the return value, but the overall workflow instructions are confusing and incomplete, making the tool difficult to invoke correctly in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description explains all three parameters: math_problem as 'the detailed text description', model as optional with default 'gpt-5.2-pro', and temperature as controlling randomness with default 0.7. This adds meaningful semantics beyond the bare schema, though it could further clarify model choices or temperature range.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Use OpenAI GPT model to solve math problems and provide detailed solution' and 'takes a raw math problem string, solves it, and explains the reasoning step-by-step.' This distinguishes it from sibling tools like gemini_informal_prover by explicitly naming the GPT model and the informal proving task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers some explicit guidance: 'Use this tool frequently for natural language math problems.' It also outlines a workflow involving leandex and switching to informal_prover when mathlib infrastructure is missing. However, the instruction to 'ask Gemini for a detailed solution' is confusing for a tool named gpt_informal_prover, and the reference to switching to informal_prover is ambiguous, undermining clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_leandexA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explains leandex is a 'semantic search engine' and that it returns 'Search results or error msg.' It also hints at possible failure by advising query refinement. However, it does not disclose limitations like potential approximate matches or response structure details, so it is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear opening sentence and organized into sections. It includes extensive query pattern examples, which are valuable but make it longer than necessary. Each example earns its place, so the verbosity is justified, but it could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters and no annotations, the description is comprehensive. It covers purpose, usage context, query patterns, arguments, and return type. The presence of an output schema (though not shown) means return values need not be detailed, and the description provides sufficient guidance for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no property descriptions, so coverage is 0%. The description compensates with an Args section: 'query (str): Search query' and 'num_results (int, optional): Max results. Defaults to 5.' This adds meaningful semantics beyond the schema, though it could be more detailed about query formatting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search for theorems and definitions using leandex.' It identifies the resource (theorems/definitions) and the action (search). However, it does not distinguish this search tool from siblings like lean_loogle or lean_local_search, so it loses a point for missing explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: 'It's recommended to use leandex to search whether there exist relevant results before you start to prove a somewhat classic goal.' It also advises refining queries. However, it does not explicitly state when *not* to use leandex or mention alternatives, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_diagnostic_messagesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does add useful context, such as interpreting 'no goals to be solved' as a sign that code may need removal, and it specifies the return type (List[str] or str). However, it does not disclose whether the tool runs Lean on the file, requires the file to exist, or has any side effects. It is reasonably transparent but leaves some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and well-structured. It front-loads the main purpose in the first sentence, includes a single clarifying note, and then provides clean Args and Returns sections. Every sentence earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description is fairly complete. It covers the purpose, parameter meaning, and return type, and adds an interpretive note. The only minor gap is that it does not explain when a string error message is returned versus a list of diagnostics, but the output schema likely covers that (and the description does mention 'or error msg').

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'file_path' as a string with no description. The tool description adds critical semantic value by specifying 'Abs path to Lean file', clarifying that the path must be absolute. This fully compensates for the 0% schema description coverage for the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get all diagnostic msgs (errors, warnings, infos) for a Lean file.' This is a specific verb+resource combination that distinguishes it from sibling tools like lean_goal or lean_file_contents. The added note about 'no goals to be solved' further clarifies the purpose by explaining a common diagnostic outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by describing what the tool returns, but it does not explicitly state when to use this tool versus alternatives (e.g., 'use this to debug compilation errors'). There is no mention of when not to use it or which sibling tools might be preferred. The usage is inferred from the purpose, but explicit guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_file_contentsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the deprecation status, the context-bloating cost, the absolute path requirement, and that it returns an error message on failure. This provides meaningful behavioral context beyond the input schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with sections for deprecation, purpose, usage, arguments, and returns. It is slightly padded with indentation and blank lines, but each sentence provides necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers the key aspects: what it does, when to use it, parameters, return type, and deprecation. An output schema exists, so return value explanation is not strictly needed, but it is included. It could be slightly more complete by naming alternatives for when not to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description's Args block compensates by explaining file_path as an absolute path and annotate_lines as enabling line numbers with a default of True. It adds useful meaning beyond the bare schema property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets the text contents of a Lean file with optional line annotation, which is specific and useful. However, it does not explicitly differentiate itself from sibling tools like lean_file_outline, though the phrase 'text contents' implies full raw content rather than structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear situational guidance: 'Use sparingly (bloats context). Mainly when unsure about line numbers.' This tells an agent when to use the tool, though it does not explicitly mention when not to use it or reference alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_local_searchA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the index scope (theorems, lemmas, defs, etc.), the return format (list of dicts with name/kind/file or error message), and the fact that it's fast. Since there are no annotations, this is valuable, though it does not address edge cases like no matches or whether the operation is read-only, which is implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections and is mostly concise. The exclamation 'VERY USEFUL AND FAST!' is redundant filler, and the sentence could be trimmed, but overall it is appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, and return values, and the output schema provides additional structure. However, the missing project_root explanation and lack of explicit distinction from global search tools make it slightly incomplete for a tool with 3 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains 'query' (declaration name or prefix) and 'limit' (max matches, default 10), which is helpful given no schema descriptions. However, it omits the 'project_root' parameter entirely, leaving a meaningful gap in parameter coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific purpose: 'Confirm declarations exist in the current workspace to prevent hallucinating APIs.' It clearly identifies the resource (workspace declarations) and the verb (confirm), and distinguishes itself from sibling tools by emphasizing 'current workspace' and the local search scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use: to confirm declaration existence before use, and even gives a shortcut example ('Pass a short prefix'). However, it does not explicitly mention when not to use it or name alternatives like global search tools, so it lacks exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_file_outlineA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool is 'Slow-ish' and returns 'Markdown formatted outline or error msg', which adds behavioral context. However, it does not explicitly state the operation is read-only or describe any side effects, though the name implies a read.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured with clear Args and Returns sections. Each sentence adds value: the purpose, the token-efficiency note, the speed warning, and the input/output spec. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool, the description is complete: it explains what the tool does, what input it takes, and what output to expect. The return type is explicitly stated as Markdown formatted outline or error msg, so no output schema detail is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description compensates by documenting 'file_path (str): Abs path to Lean file' in its Args section. This adds the critical requirement of an absolute path, which is not present in the schema. The baseline for 0 coverage is 4, and the description meets it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Get a concise outline' of imports and declarations with type signatures, covering theorems, defs, classes, and structures. This distinguishes it from sibling tools like lean_file_contents by emphasizing 'concise' and 'outline'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Highly useful and token-efficient' implies this is for when you want a quick overview rather than full file contents, but it does not explicitly name alternatives or state when not to use it. The 'Slow-ish' warning is a usage consideration but not a clear guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_declaration_fileA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses important behavioral traits: the symbol must already be present in the file, the tool returns file contents or an error message, and files can be large. However, it does not elaborate on potential side effects (e.g., no edits), error conditions beyond symbol absence, or how it resolves the file path. It adds some value but is not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence summary, a short note block with warnings, and a clear Args/Returns layout. Every sentence contributes value—the note about using hover info is especially useful. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter tool with a string-or-error return, the description covers the key aspects: what it does, how to use it safely (hover first), required argument semantics, and return type. It could mention why to prefer this over lean_file_contents or how it handles the file lookup when the symbol appears in multiple files, but overall it is complete enough for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning the description must compensate. It does: file_path is specified as an absolute path, and symbol is described as case-sensitive and the symbol to look up. This adds practical meaning beyond the bare schema types and titles, though it could go further with path format examples or handling of multiple declarations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource: 'Get the file contents where a symbol/lemma/class/structure is declared.' This clearly distinguishes it from siblings like lean_file_contents (which likely just reads any file) and lean_hover_info (which gives hover details). The scope is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage note explicitly recommends using lean_hover_info before this tool and warns about large files, providing clear context. It doesn't explicitly state when NOT to use it or compare to alternatives, but the prerequisite guidance is actionable and helps select this tool correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_term_goalA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of disclosing behavior. It states the return type and that errors produce messages, but it does not mention potential prerequisites (e.g., a running Lean server), side effects, or failure modes beyond a generic error string. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized with Args/Returns sections. Every sentence adds value, and the information is front-loaded with a one-sentence purpose statement followed by concise parameter details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 flat params, no nesting), the description covers all necessary aspects: purpose, parameters with defaults, and return behavior. An output schema exists, so the return format is additionally structured, making this sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains every parameter in detail, including path type, 1-indexing for line/column, and the column default behavior (None => end of line). Since schema description coverage is 0%, this fully compensates and adds critical meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('expected type (term goal) at a specific location in a Lean file'). It clearly differentiates from siblings like lean_goal by naming the term goal concept and location-based query, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying it operates at a file/line/column location, but it does not explicitly state when to choose this tool over similar siblings (e.g., lean_goal, lean_hover_info). No exclusions or alternative guidance is provided, so usage context is only implied, not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discussion_partnerA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It explains that the tool sends questions to external models and returns a string, and it documents model choices. However, it omits caveats about response variability, latency, or the potential for inaccurate answers, which are relevant when invoking external LLMs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: it opens with a purpose statement, uses a bulleted list for use cases, and clearly labels Args and Returns. It is somewhat verbose but every section adds value, and the formatting improves scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and a string output, the description is complete. It covers purpose, use cases, parameter semantics, model options, and return type, leaving no major gaps. The explicit return description is a nice touch.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent (0% coverage), but the description fully explains both parameters: question (with examples of acceptable content) and model (with allowed values and default). This adds crucial meaning beyond the raw schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function ('interact with a specialized partner model') and specifies the domain (proof strategies, reasoning, formalization). It lists concrete use cases, including comparing suggestions from different models, which distinguishes it from sibling Lean-specific and model-specific tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear list of when the tool is useful (discussing strategies, getting alternative paths, comparing models, debugging stuck proofs). However, it does not explicitly mention alternatives or exclusions relative to sibling tools like gemini_informal_prover or gpt_informal_prover.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gemini_informal_proverA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the burden. It discloses the tool's process (solving, step-by-step explanation), the return format as a list of solution and verification_result, and instructs to mention Lean 4 formalization. It also makes a trust claim about Gemini's math skills, though this could be seen as overstatement. Missing disclosures include failure modes, token limits, and what exactly verification_result contains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections, but includes incidental content such as 'Gemini's math skills are outstanding; you can trust the answers he gives you' and an ambiguous reference to 'informal_prover' rather than the full tool name. The length is justified by the workflow, but some sentences could be tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a full workflow, input specification, and return type, making it reasonably complete for the tool's complexity. However, it leaves ambiguity around the 'switch to informal_prover' instruction and doesn't elaborate on the verification_result semantics or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain each parameter. It does so clearly: math_problem as 'detailed text description', model as 'Gemini model to use' with default, temperature as 'generated temperature, controlling randomness' with default. This adds essential meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool solves math problems using Google Gemini and provides step-by-step detailed solutions. It clearly distinguishes itself from sibling tools like gemini_code_golf and gpt_informal_prover by focusing on natural language math problem solving with Lean 4 formalization context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance to use frequently for natural language math problems, and specifies a workflow involving leandex for search and falling back to informal_prover when mathlib infrastructure is missing. However, it lacks explicit when-not-to-use scenarios or direct comparison to sibling tools like discussion_partner.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_goalA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It discloses the 'no goals' return, default column behavior, and how to target 'sorry'. This is solid transparency, though it does not explicitly state read-only safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with args and returns sections, but the 'VERY USEFUL!' exclamation is unnecessary. Most sentences provide value, though the editorializing could be trimmed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no annotations, the description covers purpose, parameters, return behavior, and a nuanced column tip. Given the tool's simplicity and the presence of an output schema, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, but the description fully explains every parameter: absolute file path, 1-indexed line, and optional column with clear default behavior. This exceeds the schema's bare structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets proof goals at a specific location in a Lean file, using a specific verb and resource. It does not explicitly differentiate from siblings like lean_term_goal, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool ('main tool to understand the proof state') and gives practical guidance on column usage. It does not explicitly mention alternatives or when not to use it, so it misses the top score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_hammer_premiseA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the rate limit (3req/30s), input requirements, and return type/error behavior ('List[str] | str: List of relevant premises or error message'). This is useful behavioral context beyond what the schema provides, though it does not explicitly state side-effect-free behavior or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a rate limit note, a one-sentence purpose, a clear args list, and a return type. Every sentence provides necessary information without redundancy or fluff, making it appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all parameters, return values, and rate limits. The optional output schema is described via the Returns line, so no additional return-value explanation is needed. However, it lacks explicit error scenarios beyond 'error message' and does not mention any project or file prerequisites, leaving minor gaps for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It provides detailed explanations for all parameters: 'file_path' as absolute path, 'line' and 'column' as 1-indexed positions, and 'num_results' with default value 32. This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for premises based on proof state using the lean hammer premise search.' This uses a specific verb ('Search') and resource ('premises based on proof state'), and the mechanism ('hammer premise search') distinguishes it from sibling search tools like lean_local_search or lean_loogle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need premises for a proof state at a given location) but does not explicitly explain when to prefer it over alternatives or provide exclusion criteria. The rate limit ('Limit: 3req/30s') is a usage constraint but not a comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_run_codeA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must clarify behavior. It discloses the need for self-contained code and that output is diagnostics or an error message. However, it does not mention runtime side effects, execution context, or persistence, leaving some ambiguity about what executing the snippet might do.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: a single purpose sentence, two imperative constraint/guideline sentences, and a minimal Args/Returns block. Every sentence contributes meaning without redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple code runner, and the description covers purpose, usage constraints, and parameter requirements. An output schema exists, so return-value details are unnecessary. It could mention side effects or execution environment, but for this straightforward tool the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only lists 'code' as a string. The description supplements this with essential semantics: the snippet must be complete, self-contained, and include all imports and definitions. This compensates for the 0% schema description coverage and guides the agent in constructing a valid parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action and resource: 'Run a complete, self-contained code snippet and return diagnostics.' This distinguishes it from sibling tools like lean_build and lean_diagnostic_messages, which handle file-based operations rather than ad-hoc snippet execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Only use for testing outside open files!' and an alternative is suggested by 'Keep the user in the loop by editing files instead.' The requirement 'Has to include all imports and definitions!' adds a clear precondition for correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_completionsA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the types of completions and the return format ('List of possible completions or error msg'), which adds meaningful behavioral context. However, it does not explicitly state that the operation is read-only or does not modify files, which could be inferred but is not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a brief purpose, bullet-pointed completion types, and a clean 'Args' section. Every sentence provides value without redundancy. It is front-loaded with the key usage constraint and efficiently conveys all necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters and an output schema, the description covers purpose, usage, parameters, and return type. It could be more explicit about error scenarios or prerequisites, but the given details are sufficient for an agent to use the tool correctly in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description compensates fully. It explains that file_path is an absolute path, line and column are 1-indexed, and max_completions defaults to 32. These details are not present in the schema and are essential for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get code completions at a location in a Lean file.' It further breaks down into three specific completion types (Dot, Identifier, Import), making it distinct from sibling tools like lean_hover_info or lean_goal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use only on INCOMPLETE lines/statements, providing a clear when-not scenario. It also details three specific use cases (dot completion, identifier completion, import completion), which serves as concrete guidance for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_hover_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the return is a string containing hover info or an error message, and it adds a behavioral nuance about column placement. It does not detail preconditions like file state, but for a read-only hover tool, this is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized with Args and Returns sections. Every sentence contributes new information, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple hover tool, the description covers all necessary inputs, the return type, and a practical positional caveat. The output schema is minimal (just a string), so the description adequately explains the return. It lacks explicit comparison to other tools, but that is addressed in the guidance dimension.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameter descriptions, so the description fully compensates. It explains each parameter with type and meaning, and specifically adds valuable guidance on the column parameter to avoid pointing at the end of a term. This is essential for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Get hover info' with the resource being a specific location in a Lean file, and augments it with 'docs for syntax, variables, functions, etc.' This clearly distinguishes it from sibling tools like lean_goal or lean_completions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use it at a specific file location to obtain hover documentation. It also gives a practical guideline about column position (start or within the term, not the end). It does not explicitly name alternatives, but the intended use is obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_leanfinderA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden of disclosure. It mentions a rate limit (10req/30s) and clearly states the return format (List[Dict] | str) with error messages. It does not explicitly state that the tool is read-only, which would be helpful, but the search action implies it. This is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured but somewhat lengthy due to detailed query types and tips. However, every segment adds value, and the information is front-loaded with purpose and limits. It earns a 4 rather than a 5 because it could be tightened without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (semantic search with multiple query forms) and lack of annotations, the description is remarkably complete. It explains query types, parameters, return values, and rate limits. The output schema is not provided, but the description lists the returned object fields, making it self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for parameters, and the description fully compensates by defining 'query' as 'Mathematical concept or proof state' and 'num_results' as 'Max results. Defaults to 5.' This adds meaning beyond the schema and covers both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Mathlib theorems/definitions semantically by mathematical concept or proof state. It distinguishes itself from sibling search tools by emphasizing semantic search, and the verb 'search' is specific to the resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool, including effective query types and tips like 'Multiple targeted queries beat one complex query.' However, it does not explicitly mention when not to use it or name alternative search tools (e.g., lean_loogle or lean_leandex), so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_state_searchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses a rate limit ('Limit: 3req/30s'), a behavioral nuance ('Only uses first goal if multiple'), and the return type ('List | str: Search results or error msg'). This provides useful transparency beyond the bare schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized. It leads with the rate limit, states the core purpose, notes a behavioral caveat, then lists parameters and return type. Every sentence adds value, and the structure follows a logical docstring format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, parameters, return type, and a key limitation. It is adequate for the tool's complexity, but it could provide more context about how line/column map to a proof state or what the search results contain. The output schema is not shown, but the return type is described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates. Each parameter is explicitly defined: file_path ('Abs path to Lean file'), line and column ('1-indexed'), and num_results ('Max results. Defaults to 5.'). This leaves no ambiguity about parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search for theorems based on proof state using premise-search.com.' The verb 'search' and resource 'theorems based on proof state' are specific, and the mention of the external service distinguishes it from sibling tools like lean_local_search or lean_loogle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: given a file path, line, and column, it searches for theorems. It also notes a limitation ('Only uses first goal if multiple'), which is implicit usage guidance. However, it does not explicitly compare against alternatives or state when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_buildA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It mentions side effects (restarting LSP Server), the potentially long execution time for 'clean', and the path inference behavior from previous calls. The warning about 'clean' addresses safety and resource usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with purpose, usage, Args, and Returns sections. Every sentence adds value, including the important warning about clean. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers purpose, parameters, return type, and an important caution. With an output schema present, it need not detail the return structure further. The description is complete for an AI agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully explains both parameters: lean_project_path is optional and inferred, clean runs 'lake clean' with an explicit warning and default. This adds meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Build the Lean project and restart the LSP Server', using specific verbs and the resource affected. This distinguishes it from sibling tools, which are query/analysis tools without build or restart functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use only if needed (e.g. new imports)' and warns about the clean flag being time-consuming. It does not explicitly mention alternatives, but the context of sibling tools makes it clear this is the only build-related tool, so the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_loogleA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It mentions a rate limit ('3req/30s') and the return type ('List[dict] | str'), which are useful operational details. It does not explicitly state it is read-only, but the search nature makes that obvious. It could elaborate on error behavior, but the provided information is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a leading purpose, then query patterns, args, and returns. Every sentence adds value, and the length is justified by the complexity of the query syntax. It is front-loaded with the key information (limit and purpose).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex due to its flexible query language, and the description covers all essential aspects: purpose, rate limit, query patterns, argument semantics, and return type. The presence of an output schema covers the detailed return structure, so the high-level return description is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so extensively by providing seven query patterns that describe the expected format for the query parameter, plus a clear explanation for num_results. This goes far beyond the bare schema and gives the agent a strong understanding of how to construct valid queries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for definitions and theorems using loogle.' This is a specific verb+resource combination that distinguishes it from sibling search tools like lean_local_search or lean_leandex. The query patterns further reinforce what the tool is for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool with detailed query patterns and argument explanations. However, it does not explicitly mention when not to use this tool or compare it to alternatives, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lean_multi_attemptA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It discloses key constraints: 'Only single-line, fully-indented snippets are supported' and 'Avoid comments for best results.' It also notes the recommendation of '3+ snippets' and describes the return type. However, it does not explicitly state whether the tool modifies the file or is purely read-only, leaving a minor gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a summary, usage guidelines, note, args, and returns. It is concise and every sentence adds value—no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 required parameters, no annotations, but has an output schema. The description covers purpose, usage, constraints, parameter details, and return type. It is comprehensive enough for an agent to decide when and how to invoke it, and it explicitly references the output as 'List[str] | str'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining each parameter: file_path as 'Abs path to Lean file', line as 'Line number (1-indexed)', and snippets as 'List of snippets (3+ are recommended)'. This adds meaning beyond the basic schema titles, though it could be more detailed about the exact role of 'line'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Try multiple Lean code snippets at a line and get the goal state and diagnostics for each.' This specific verb+resource+outcome clearly distinguishes it from siblings like `lean_diagnostic_messages` and `lean_goal`, which handle single snippets or goal states.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use to compare tactics or approaches. Use rarely-prefer direct file edits to keep users involved. For a single snippet, edit the file and run `lean_diagnostic_messages` instead.' This tells the agent when to use this tool and when to use an alternative, and even names the alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.