Skip to main content
Glama
  • :rocket: Serena is a powerful coding agent toolkit capable of turning an LLM into a fully-featured agent that works directly on your codebase. Unlike most other tools, it is not tied to an LLM, framework or an interface, making it easy to use it in a variety of ways.

  • :wrench: Serena provides essential semantic code retrieval and editing tools that are akin to an IDE's capabilities, extracting code entities at the symbol level and exploiting relational structure. When combined with an existing coding agent, these tools greatly enhance (token) efficiency.

  • :free: Serena is free & open-source, enhancing the capabilities of LLMs you already have access to free of charge.

You can think of Serena as providing IDE-like tools to your LLM/coding agent. With it, the agent no longer needs to read entire files, perform grep-like searches or basic string replacements to find the right parts of the code and to edit code. Instead, it can use code-centric tools like find_symbol, find_referencing_symbols and insert_after_symbol.

TIP

TheSerena JetBrains plugin has been released!

LLM Integration

Serena provides the necessary tools for coding workflows, but an LLM is required to do the actual work, orchestrating tool use.

In general, Serena can be integrated with an LLM in several ways:

  • by using the model context protocol (MCP). Serena provides an MCP server which integrates with

    • Claude Code and Claude Desktop,

    • terminal-based clients like Codex, Gemini-CLI, Qwen3-Coder, rovodev, OpenHands CLI and others,

    • IDEs like VSCode, Cursor or IntelliJ,

    • Extensions like Cline or Roo Code

    • Local clients like OpenWebUI, Jan, Agno and others

  • by using mcpo to connect it to ChatGPT or other clients that don't support MCP but do support tool calling via OpenAPI.

  • by incorporating Serena's tools into an agent framework of your choice, as illustrated here. Serena's tool implementation is decoupled from the framework-specific code and can thus easily be adapted to any agent framework.

Related MCP server: MCP Files

Serena in Action

Demonstration 1: Efficient Operation in Claude Code

A demonstration of Serena efficiently retrieving and editing code within Claude Code, thereby saving tokens and time. Efficient operations are not only useful for saving costs, but also for generally improving the generated code's quality. This effect may be less pronounced in very small projects, but often becomes of crucial importance in larger ones.

https://github.com/user-attachments/assets/ab78ebe0-f77d-43cc-879a-cc399efefd87

Demonstration 2: Serena in Claude Desktop

A demonstration of Serena implementing a small feature for itself (a better log GUI) with Claude Desktop. Note how Serena's tools enable Claude to find and edit the right symbols.

https://github.com/user-attachments/assets/6eaa9aa1-610d-4723-a2d6-bf1e487ba753

Programming Language Support & Semantic Analysis Capabilities

Serena provides a set of versatile code querying and editing functionalities based on symbolic understanding of the code. Equipped with these capabilities, Serena discovers and edits code just like a seasoned developer making use of an IDE's capabilities would. Serena can efficiently find the right context and do the right thing even in very large and complex projects!

There are two alternative technologies powering these capabilities:

  • Language servers implementing the language server Protocol (LSP) — the free/open-source alternative.

  • The Serena JetBrains Plugin, which leverages the powerful code analysis and editing capabilities of your JetBrains IDE.

You can choose either of these backends depending on your preferences and requirements.

Language Servers

Serena incorporates a powerful abstraction layer for the integration of language servers that implement the language server protocol (LSP). The underlying language servers are typically open-source projects (like Serena) or at least freely available for use.

With Serena's LSP library, we provide support for over 30 programming languages, including AL, Bash, C#, C/C++, Clojure, Dart, Elixir, Elm, Erlang, Fortran, Go, Groovy (partial support), Haskell, Java, Javascript, Julia, Kotlin, Lua, Markdown, MATLAB, Nix, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Swift, TOML, TypeScript, YAML, and Zig.

IMPORTANT

Some language servers require additional dependencies to be installed; see theLanguage Support page for details.

The Serena JetBrains Plugin

As an alternative to language servers, the Serena JetBrains Plugin leverages the powerful code analysis capabilities of your JetBrains IDE. The plugin naturally supports all programming languages and frameworks that are supported by JetBrains IDEs, including IntelliJ IDEA, PyCharm, Android Studio, WebStorm, PhpStorm, RubyMine, GoLand, CLion, and others. Only Rider is not supported.

The plugin offers the most robust and most powerful Serena experience.
See our documentation page for further details and instructions.

Quick Start

Prerequisites. Serena is managed by uv. If you don’t already have it, you need to install uv before proceeding.

Starting the MCP Server. The easiest way to start the Serena MCP server is by running the latest version from GitHub using uvx. Issue this command to see available options:

uvx --from git+https://github.com/oraios/serena serena start-mcp-server --help

Configuring Your Client. To connect Serena to your preferred MCP client, you typically need to configure a launch command in your client. Follow the link for specific instructions on how to set up Serena for Claude Code, Codex, Claude Desktop, MCP-enabled IDEs and other clients (such as local and web-based GUIs).

TIP

While getting started quickly is easy, Serena is a powerful toolkit with many configuration options. We highly recommend reading through theuser guide to get the most out of Serena.

Specifically, we recommend to read about ...

User Guide

Please refer to the user guide for detailed instructions on how to use Serena effectively.

Community Feedback

Most users report that Serena has strong positive effects on the results of their coding agents, even when used within very capable agents like Claude Code. Serena is often described to be a game changer, providing an enormous productivity boost.

Serena excels at navigating and manipulating complex codebases, providing tools that support precise code retrieval and editing in the presence of large, strongly structured codebases. However, when dealing with tasks that involve only very few/small files, you may not benefit from including Serena on top of your existing coding agent. In particular, when writing code from scratch, Serena will not provide much value initially, as the more complex structures that Serena handles more gracefully than simplistic, file-based approaches are yet to be created.

Several videos and blog posts have talked about Serena:

Acknowledgements

Sponsors

We are very grateful to our sponsors who help us drive Serena's development. The core team (the founders of Oraios AI) put in a lot of work in order to turn Serena into a useful open source project. So far, there is no business model behind this project, and sponsors are our only source of income from it.

Sponsors help us dedicating more time to the project, managing contributions, and working on larger features (like better tooling based on more advanced LSP features, VSCode integration, debugging via the DAP, and several others). If you find this project useful to your work, or would like to accelerate the development of Serena, consider becoming a sponsor.

We are proud to announce that the Visual Studio Code team, together with Microsoft’s Open Source Programs Office and GitHub Open Source have decided to sponsor Serena with a one-time contribution!

Community Contributions

A significant part of Serena, especially support for various languages, was contributed by the open source community. We are very grateful for the many contributors who made this possible and who played an important role in making Serena what it is today.

Technologies

We built Serena on top of multiple existing open-source technologies, the most important ones being:

  1. multilspy. A library which wraps language server implementations and adapts them for interaction via Python. It provided the basis for our library Solid-LSP (src/solidlsp). Solid-LSP provides pure synchronous LSP calls and extends the original library with the symbolic logic that Serena required.

  2. Python MCP SDK

  3. All the language servers that we use through Solid-LSP.

Without these projects, Serena would not have been possible (or would have been significantly more difficult to build).

Customizing and Extending Serena

It is straightforward to extend Serena's AI functionality with your own ideas. Simply implement a new tool by subclassing serena.agent.Tool and implement the apply method with a signature that matches the tool's requirements. Once implemented, SerenaAgent will automatically have access to the new tool.

It is also relatively straightforward to add support for a new programming language.

We look forward to seeing what the community will come up with! For details on contributing, see contributing guidelines.

Available Tools

29 tools
activate_projectActivate ProjectB
Read-only

Activates the project with the given name or path.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectYesThe name of a registered project to activate or a path to a project directory.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, so the agent knows it's a safe, non-destructive operation. The description adds that it activates a project, implying a state change, but doesn't disclose behavioral details like side effects, permissions needed, or what 'activation' entails. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action and parameter. There is no wasted verbiage, and it directly communicates the tool's function without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given one parameter with full schema coverage, annotations covering safety, and an output schema (implied by context signals), the description is reasonably complete. It could benefit from more behavioral context (e.g., what activation does), but the structured data reduces the burden. For a simple activation tool, it's adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with one parameter ('project') fully documented in the schema. The description adds that the project can be a 'name' or 'path', which slightly clarifies the schema's 'name of a registered project or path', but doesn't provide additional syntax or format details. Baseline 3 is appropriate as the schema handles most documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('activates') and the resource ('project'), specifying it uses a name or path. It distinguishes from siblings like 'get_current_config' or 'switch_modes' by focusing on project activation, though it doesn't explicitly differentiate from similar tools (none listed).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., project registration), exclusions, or related tools like 'switch_modes' that might handle mode changes. The description only states what it does, not when it's appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_onboarding_performedCheck Onboarding PerformedA
Read-only

Checks whether project onboarding was already performed. You should always call this tool before beginning to actually work on the project/after activating a project.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, which the description aligns with by describing a check operation. The description adds valuable context beyond annotations by specifying the timing ('before beginning to actually work on the project/after activating a project'), which helps the agent understand when to invoke it. However, it doesn't detail output behavior (e.g., what the check returns), though an output schema exists to cover that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with zero waste: the first states the purpose, and the second provides usage guidelines. It is front-loaded with the core function and efficiently adds necessary context, making it easy for an agent to parse and apply.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (0 parameters, annotations covering safety, and an output schema to handle return values), the description is complete. It clearly explains what the tool does, when to use it, and aligns with annotations, leaving no gaps for the agent to misinterpret its role among siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on usage context. A baseline of 4 is applied since it compensates well for the lack of parameters by providing clear operational guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('checks whether project onboarding was already performed') and the resource ('project onboarding'). It distinguishes this tool from siblings like 'onboarding' (which likely performs onboarding) and 'activate_project' (which likely activates a project), making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool: 'You should always call this tool before beginning to actually work on the project/after activating a project.' It specifies the exact context (pre-work/post-activation) and implies an alternative (if onboarding was performed, proceed; if not, use another tool like 'onboarding'), offering clear usage rules.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_text_fileCreate Text FileA
Destructive

Write a new file or overwrite an existing file. Returns a message indicating success or failure.

ParametersJSON Schema
NameRequiredDescriptionDefault
relative_pathYesThe relative path to the file to create.
contentYesThe (appropriately encoded) content to write to the file.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond annotations: it clarifies that the tool can overwrite existing files and returns success/failure messages. Annotations already indicate it's not read-only and is destructive, but the description enhances this by specifying the overwrite behavior and output format, though it could mention more about error handling or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded, consisting of just two sentences that directly convey the core functionality and outcome. Every word serves a purpose, with no redundant or extraneous information, making it highly efficient for agent comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (a destructive write operation with 2 parameters), annotations covering safety aspects, and an output schema (implied by context signals), the description is reasonably complete. It explains the action and result, though it could benefit from more usage context or error details to be fully comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the input schema fully documents both parameters ('relative_path' and 'content'). The description doesn't add any parameter-specific details beyond what's in the schema, such as path formatting or content encoding examples, so it meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Write a new file or overwrite an existing file') and resource ('file'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'replace_content' or 'write_memory', which could have overlapping functionality, so it doesn't reach the highest score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'replace_content' for modifying existing files or 'write_memory' for other storage, nor does it specify prerequisites or exclusions, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_memoryDelete MemoryA
Destructive

Delete a memory file. Should only happen if a user asks for it explicitly, for example by saying that the information retrieved from a memory file is no longer correct or no longer relevant for the project.

ParametersJSON Schema
NameRequiredDescriptionDefault
memory_file_nameYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate destructiveHint=true and readOnlyHint=false, which already signal a non-read-only, destructive operation. The description adds valuable context by specifying that deletion should be user-initiated and justified, which goes beyond the annotations. It doesn't detail permissions, side effects, or confirmation steps, but the added guidance is meaningful for safe usage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences that are front-loaded with the core action and followed by essential usage constraints. Every word earns its place—no redundancy or fluff—making it highly efficient and easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's destructive nature (annotations show destructiveHint=true), the description adequately covers the critical aspect of when to use it. With an output schema present, return values don't need explanation. However, it could mention error cases or confirmation behaviors, leaving minor gaps for a high-stakes deletion tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, with only one parameter ('memory_file_name') documented structurally. The description doesn't explicitly mention parameters, but it implies the parameter's role by referring to 'a memory file', adding semantic context about what the file represents. This compensates somewhat for the low schema coverage, though it lacks format or validation details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Delete') and resource ('a memory file'), making the purpose unambiguous. It distinguishes from siblings like 'edit_memory', 'read_memory', and 'write_memory' by specifying deletion rather than modification or access. However, it doesn't explicitly contrast with 'list_memories' or other file-related tools, keeping it from a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool: 'only if a user asks for it explicitly', with concrete examples ('information retrieved from a memory file is no longer correct or no longer relevant'). This clearly distinguishes it from alternatives like editing or reading memories, ensuring the agent understands the restrictive context for invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

edit_memoryEdit MemoryA
Destructive

Replaces content matching a regular expression in a memory.

ParametersJSON Schema
NameRequiredDescriptionDefault
memory_file_nameYesThe name of the memory.
needleYesThe string or regex pattern to search for. If `mode` is "literal", this string will be matched exactly. If `mode` is "regex", this string will be treated as a regular expression (syntax of Python's `re` module, with flags DOTALL and MULTILINE enabled).
replYesThe replacement string (verbatim).
modeYesEither "literal" or "regex", specifying how the `needle` parameter is to be interpreted.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide readOnlyHint=false and destructiveHint=true, indicating a mutation operation with destructive potential. The description adds valuable context by specifying it's a replacement operation (not just editing) and mentions regex matching, which helps the agent understand the tool's behavior beyond the basic safety profile. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the core functionality without unnecessary words. Every element ('Replaces content matching a regular expression in a memory') serves a clear purpose, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (regex operations with destructive potential), the description provides adequate context when combined with annotations and schema. However, it doesn't mention side effects like what happens if no match is found or whether changes are reversible, which could be helpful. The presence of an output schema reduces the need to describe return values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each parameter well-documented in the schema (e.g., 'needle' explains literal vs regex modes with Python re module details). The description doesn't add significant semantic value beyond the schema, as it only mentions 'regular expression' generally without elaborating on parameter interactions or edge cases.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Replaces content') on a specific resource ('in a memory') using a specific method ('matching a regular expression'). It distinguishes from siblings like 'replace_content' (general file replacement) and 'write_memory' (full overwrite) by specifying regex-based partial replacement within memories.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for regex-based editing of memory content, suggesting when to use it (for pattern-based replacements in memories). However, it doesn't explicitly state when NOT to use it or name specific alternatives like 'write_memory' for complete overwrites or 'replace_content' for non-memory files, leaving some ambiguity about tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

execute_shell_commandExecute Shell CommandA
Destructive

Execute a shell command and return its output. If there is a memory about suggested commands, read that first. Never execute unsafe shell commands! IMPORTANT: Do not use this tool to start

  • long-running processes (e.g. servers) that are not intended to terminate quickly,

  • processes that require user interaction. Returns a JSON object containing the command's stdout and optionally stderr output.

ParametersJSON Schema
NameRequiredDescriptionDefault
commandYesThe shell command to execute.
cwdNoThe working directory to execute the command in. If None, the project root will be used.
capture_stderrNoWhether to capture and return stderr output.
max_answer_charsNoIf the output is longer than this number of characters, no content will be returned. -1 means using the default value, don't adjust unless there is no other way to get the content required for the task.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already indicate destructiveHint=true and readOnlyHint=false, which the description aligns with by describing command execution. The description adds valuable behavioral context beyond annotations: safety warnings about unsafe commands, constraints on process types, and output format details. However, it doesn't mention rate limits, authentication needs, or specific error handling behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded with the core purpose. The safety warnings and usage constraints are efficiently presented. However, the formatting with asterisks and some redundancy ('Returns a JSON object...' could be integrated more smoothly) prevents a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the high-risk nature of shell command execution, the description provides excellent contextual completeness. It covers safety constraints, usage boundaries, output format, and references to memory checks. With annotations covering destructive/read-only aspects and an output schema presumably detailing the JSON response, no significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the input schema already documents all 4 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. The baseline score of 3 reflects adequate coverage through structured data alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Execute a shell command and return its output') and distinguishes it from all sibling tools, which are focused on file operations, memory management, and project configuration rather than shell command execution. The verb+resource combination is precise and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when NOT to use this tool ('Never execute unsafe shell commands!', 'Do not use this tool to start long-running processes...', 'processes that require user interaction') and references an alternative approach ('If there is a memory about suggested commands, read that first'). This gives clear boundaries for appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_fileFind FileA
Read-only

Finds non-gitignored files matching the given file mask within the given relative path. Returns a JSON object with the list of matching files.

ParametersJSON Schema
NameRequiredDescriptionDefault
file_maskYesThe filename or file mask (using the wildcards * or ?) to search for.
relative_pathYesThe relative path to the directory to search in; pass "." to scan the project root.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds valuable behavioral context beyond this: it specifies that gitignored files are excluded from results, describes the return format ('JSON object with the list of matching files'), and mentions the wildcard syntax (* and ?) for file masks. This provides useful operational details not covered by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise with two sentences that each earn their place: the first states the core functionality with key constraints (non-gitignored, file mask matching), and the second specifies the return format. No wasted words, and the most important information (what it does) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, 100% schema coverage, comprehensive annotations (readOnlyHint, destructiveHint), and the presence of an output schema (which handles return value documentation), the description is complete. It covers the core functionality, key behavioral constraints (gitignore exclusion), and return format, providing everything needed for an agent to understand when and how to use this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters well-documented in the schema. The description adds minimal additional parameter semantics beyond what's in the schema - it mentions wildcard usage (* and ?) which is already in the schema's description for file_mask, and provides the example '.' for project root scanning which is already in the schema for relative_path. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Finds non-gitignored files matching the given file mask'), the resource ('files'), and scope ('within the given relative path'). It distinguishes from sibling tools like 'list_dir' (which lists directory contents) and 'search_for_pattern' (which searches file content) by focusing on filename pattern matching while excluding gitignored files.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context about when to use this tool (searching for files by name/mask pattern while respecting gitignore). However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, though the distinction from 'list_dir' and 'search_for_pattern' is implied by the functionality described.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_referencing_symbolsFind Referencing SymbolsA
Read-only

Finds references to the symbol at the given name_path. The result will contain metadata about the referencing symbols as well as a short code snippet around the reference. Returns a list of JSON objects with the symbols referencing the requested symbol.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_pathYesFor finding the symbol to find references for, same logic as in the `find_symbol` tool.
relative_pathYesThe relative path to the file containing the symbol for which to find references. Note that here you can't pass a directory but must pass a file.
include_kindsNoSame as in the `find_symbol` tool.
exclude_kindsNoSame as in the `find_symbol` tool.
max_answer_charsNoSame as in the `find_symbol` tool.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds valuable behavioral context beyond annotations: it specifies the result format ('metadata about the referencing symbols as well as a short code snippet around the reference') and output structure ('Returns a list of JSON objects'). This enhances transparency about what the tool returns, which annotations don't cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded: it states the core purpose in the first sentence, adds details about the result in the second, and specifies the return format in the third. Each sentence adds value without redundancy, making it efficient and well-structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (finding symbol references), the description is complete enough: it explains the purpose, result format, and output structure. With annotations covering safety (read-only, non-destructive), 100% schema coverage for parameters, and an output schema present (implied by context signals), no additional details are necessary. The description effectively complements the structured data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with all parameters well-documented in the schema itself (e.g., 'name_path' description references 'find_symbol' tool logic). The description doesn't add significant semantic details beyond the schema, as it only mentions 'name_path' briefly. Baseline 3 is appropriate since the schema does the heavy lifting for parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Finds references to the symbol at the given `name_path`' with specific verb ('Finds') and resource ('references to the symbol'). It distinguishes from siblings like 'find_symbol' by focusing on references rather than the symbol itself. However, it doesn't explicitly contrast with all possible alternatives among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through parameter references (e.g., 'same logic as in the `find_symbol` tool'), suggesting it's used when you need to find where a symbol is referenced rather than the symbol definition. However, it lacks explicit guidance on when to use this tool versus alternatives like 'find_symbol' or 'search_for_pattern', and doesn't mention prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_symbolFind SymbolA
Read-only

Retrieves information on all symbols/code entities (classes, methods, etc.) based on the given name path pattern. The returned symbol information can be used for edits or further queries. Specify depth > 0 to also retrieve children/descendants (e.g., methods of a class).

A name path is a path in the symbol tree within a source file. For example, the method my_method defined in class MyClass would have the name path MyClass/my_method. If a symbol is overloaded (e.g., in Java), a 0-based index is appended (e.g. "MyClass/my_method[0]") to uniquely identify it.

To search for a symbol, you provide a name path pattern that is used to match against name paths. It can be

  • a simple name (e.g. "method"), which will match any symbol with that name

  • a relative path like "class/method", which will match any symbol with that name path suffix

  • an absolute name path "/class/method" (absolute name path), which requires an exact match of the full name path within the source file. Append an index [i] to match a specific overload only, e.g. "MyClass/my_method[1]". Returns a list of symbols (with locations) matching the name.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_path_patternYesThe name path matching pattern (see above).
depthNoDepth up to which descendants shall be retrieved (e.g. use 1 to also retrieve immediate children; for the case where the symbol is a class, this will return its methods). Default 0.
relative_pathNoOptional. Restrict search to this file or directory. If None, searches entire codebase. If a directory is passed, the search will be restricted to the files in that directory. If a file is passed, the search will be restricted to that file. If you have some knowledge about the codebase, you should use this parameter, as it will significantly speed up the search as well as reduce the number of results.
include_bodyNoIf True, include the symbol's source code. Use judiciously.
include_kindsNoOptional. List of LSP symbol kind integers to include. (e.g., 5 for Class, 12 for Function). Valid kinds: 1=file, 2=module, 3=namespace, 4=package, 5=class, 6=method, 7=property, 8=field, 9=constructor, 10=enum, 11=interface, 12=function, 13=variable, 14=constant, 15=string, 16=number, 17=boolean, 18=array, 19=object, 20=key, 21=null, 22=enum member, 23=struct, 24=event, 25=operator, 26=type parameter. If not provided, all kinds are included.
exclude_kindsNoOptional. List of LSP symbol kind integers to exclude. Takes precedence over `include_kinds`. If not provided, no kinds are excluded.
substring_matchingNoIf True, use substring matching for the last element of the pattern, such that "Foo/get" would match "Foo/getValue" and "Foo/getData".
max_answer_charsNoMax characters for the JSON result. If exceeded, no content is returned. -1 means the default value from the config will be used.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true and destructiveHint=false, which the description doesn't contradict. The description adds valuable behavioral context beyond annotations: it explains that results are returned as a list of symbols with locations, mentions that the tool can be used for subsequent edits, warns about using 'include_body' judiciously, and describes performance implications of the 'relative_path' parameter. This provides practical guidance that annotations alone don't cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and appropriately sized. It starts with the core purpose, then explains key parameters (name path patterns, depth), provides concrete examples, and ends with what's returned. While comprehensive, some sentences could be more concise (e.g., the name path pattern explanation spans multiple paragraphs). Overall, it's informative without being excessively verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 parameters, sophisticated matching logic), the description provides excellent context. It explains the core matching logic, provides practical examples, mentions performance considerations, and notes that results can be used for edits. With annotations covering safety, 100% schema coverage, and an output schema existing, the description focuses appropriately on the conceptual aspects that structured fields don't capture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already documents all 8 parameters thoroughly. The description adds some semantic context: it explains name path patterns with examples (simple name, relative path, absolute path), clarifies how overloaded symbols are handled with indexes, and mentions that 'depth > 0' retrieves children/descendants. However, most parameter details are already in the schema, so the description provides only moderate additional value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Retrieves information on all symbols/code entities (classes, methods, etc.) based on the given name path pattern.' It specifies the verb ('retrieves'), resource ('symbols/code entities'), and scope ('based on name path pattern'). It also distinguishes from siblings like 'find_file' (which finds files) and 'find_referencing_symbols' (which finds references rather than symbols themselves).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool: for retrieving symbol information that 'can be used for edits or further queries.' It mentions using 'depth > 0' for retrieving descendants and suggests using the 'relative_path' parameter to speed up searches. However, it doesn't explicitly state when NOT to use this tool or name specific alternatives among siblings (e.g., when to use 'find_file' instead).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_current_configGet Current ConfigA
Read-only

Print the current configuration of the agent, including the active and available projects, tools, contexts, and modes.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read operation. The description adds useful context about what configuration elements are included (projects, tools, contexts, modes), which goes beyond the annotations. However, it doesn't describe behavioral traits like whether this returns cached or live data, format details, or performance characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action and immediately specifies what's included. Every word earns its place—no redundancy, no unnecessary elaboration. It's appropriately sized for a zero-parameter read tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has zero parameters, rich annotations (readOnlyHint, destructiveHint), and an output schema exists, the description is reasonably complete. It specifies what configuration elements are returned, which complements the output schema. For a simple status-checking tool, this provides adequate context, though it could benefit from mentioning when this information is most useful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0 parameters and 100% schema description coverage, the baseline would be 4. The description correctly indicates this tool takes no parameters ('Print the current configuration' implies no inputs needed), which aligns perfectly with the empty schema. No additional parameter semantics are needed or provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Print' and the resource 'current configuration of the agent', specifying what information is included (active/available projects, tools, contexts, modes). It distinguishes from siblings like 'list_memories' or 'get_symbols_overview' by focusing on overall agent configuration rather than specific components. However, it doesn't explicitly differentiate from all siblings, so it's not a perfect 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, timing considerations, or suggest when this tool is appropriate versus other configuration or status-checking tools. The agent must infer usage from the purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_symbols_overviewGet Symbols OverviewA
Read-only

Use this tool to get a high-level understanding of the code symbols in a file. This should be the first tool to call when you want to understand a new file, unless you already know what you are looking for. Returns a JSON object containing info about top-level symbols in the file.

ParametersJSON Schema
NameRequiredDescriptionDefault
relative_pathYesThe relative path to the file to get the overview of.
depthNoDepth up to which descendants of top-level symbols shall be retrieved (e.g. 1 retrieves immediate children). Default 0.
max_answer_charsNoIf the overview is longer than this number of characters, no content will be returned. -1 means the default value from the config will be used. Don't adjust unless there is really no other way to get the content required for the task.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true and destructiveHint=false, which the description doesn't contradict. The description adds valuable behavioral context: it specifies this is for 'high-level understanding' and 'top-level symbols,' clarifies it should be called first for new files, and mentions the return format ('JSON object'). This goes beyond annotations by providing usage context and output expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three concise sentences with zero waste. The first sentence states the purpose, the second provides usage guidelines, and the third describes the return format. Each sentence earns its place by adding distinct value. It's appropriately sized and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, rich annotations (readOnlyHint, destructiveHint), 100% schema coverage, and presence of an output schema, the description is complete enough. It covers purpose, usage context, and return format without needing to explain parameters or output details that are already in structured fields. It provides exactly what an agent needs to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all three parameters. The description doesn't add any parameter-specific details beyond what's in the schema. It mentions 'top-level symbols' which relates to the 'depth' parameter's default behavior, but this is already covered in the schema description. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'get a high-level understanding of the code symbols in a file' with specific verb ('get') and resource ('code symbols in a file'). It distinguishes from siblings like 'find_symbol' (specific search) or 'read_file' (raw content) by focusing on symbol overview. The description explicitly positions it as the first tool for understanding new files.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'This should be the first tool to call when you want to understand a new file, unless you already know what you are looking for.' This gives clear when-to-use (initial exploration) and when-not-to-use (when specific symbols are already known), distinguishing it from alternatives like 'find_symbol' for targeted searches.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

initial_instructionsInitial InstructionsA
Read-only

Provides the 'Serena Instructions Manual', which contains essential information on how to use the Serena toolbox. IMPORTANT: If you have not yet read the manual, call this tool immediately after you are given your task by the user, as it will critically inform you!.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds behavioral context about the manual's critical importance and timing, but doesn't specify what the manual contains, its format, or how it's delivered. With annotations covering safety, this adds moderate value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero waste: the first states the purpose, the second provides critical usage guidance. It's appropriately sized and front-loaded with the essential information, though the second sentence is quite long.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 0 parameters, annotations covering safety, and an output schema exists (so return values are documented elsewhere), the description is reasonably complete. It covers purpose and critical usage timing, though it could better explain what the manual contains or how it differs from similar tools like 'onboarding'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description appropriately doesn't discuss parameters, maintaining focus on the tool's purpose and usage. Baseline for 0 parameters is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'Provides the Serena Instructions Manual' with the purpose of delivering essential usage information. It specifies the resource (manual) and its importance, though it doesn't explicitly differentiate from sibling tools like 'onboarding' or 'get_current_config' that might also provide guidance.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'If you have not yet read the manual, call this tool immediately after you are given your task by the user, as it will critically inform you!' This clearly states when to use it (immediately after task assignment if manual not read) and implies it's a prerequisite for effective tool use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

insert_after_symbolInsert After SymbolA
Destructive

Inserts the given body/content after the end of the definition of the given symbol (via the symbol's location). A typical use case is to insert a new class, function, method, field or variable assignment.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_pathYesName path of the symbol after which to insert content (definitions in the `find_symbol` tool apply).
relative_pathYesThe relative path to the file containing the symbol.
bodyYesThe body/content to be inserted. The inserted code shall begin with the next line after the symbol.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate 'destructiveHint: true' and 'readOnlyHint: false', which the description aligns with by describing an insertion operation. The description adds useful context about the insertion location ('after the end of the definition') and references 'find_symbol' for symbol definitions, but it does not disclose additional behavioral traits like error handling, formatting requirements, or side effects beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action in the first sentence and uses a second sentence for a typical use case, with no wasted words. Every sentence adds value by clarifying purpose and context, making it efficiently structured and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (destructive insertion with three parameters), annotations cover safety aspects, and an output schema exists (though not provided here), the description is reasonably complete. It explains the insertion logic and references related tools, but could benefit from more details on error cases or output format, though the output schema mitigates this gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by implying the relationship between 'name_path' and 'find_symbol' and noting that insertion occurs on the next line, but it does not provide significant additional semantic details. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Inserts') and target ('after the end of the definition of the given symbol'), distinguishing it from sibling tools like 'insert_before_symbol' and 'replace_symbol_body'. It provides concrete examples of typical use cases (inserting classes, functions, methods, etc.), making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool by mentioning 'after the end of the definition' and referencing the 'find_symbol' tool for symbol definitions, but it does not explicitly state when not to use it or name alternatives. It provides clear context for insertion scenarios without detailing exclusions or comparisons to siblings like 'insert_before_symbol'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

insert_before_symbolInsert Before SymbolA
Destructive

Inserts the given content before the beginning of the definition of the given symbol (via the symbol's location). A typical use case is to insert a new class, function, method, field or variable assignment; or a new import statement before the first symbol in the file.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_pathYesName path of the symbol before which to insert content (definitions in the `find_symbol` tool apply).
relative_pathYesThe relative path to the file containing the symbol.
bodyYesThe body/content to be inserted before the line in which the referenced symbol is defined.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, so the agent knows this is a mutating operation. The description adds useful context about 'via the symbol's location' and typical use cases, but doesn't elaborate on potential side effects, error conditions, or how it interacts with the file system beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately concise with two sentences: the first states the core functionality, the second provides typical use cases. Both sentences add value without redundancy. It could be slightly more front-loaded by integrating the use case into the main statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the destructive nature (annotations), 3 fully documented parameters (schema), and existence of an output schema, the description provides adequate context. It covers the core operation and typical use cases, though additional behavioral details about error handling or constraints would make it more complete for a destructive editing tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, all parameters are well-documented in the schema itself. The description mentions 'given content' and 'given symbol' which map to 'body' and 'name_path', but adds no additional semantic context beyond what the schema already provides about parameter meaning or usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('inserts') and target ('before the beginning of the definition of the given symbol'), with specific examples of typical use cases (classes, functions, imports). It distinguishes from sibling 'insert_after_symbol' by specifying 'before', but doesn't explicitly contrast with other editing tools like 'replace_content' or 'replace_symbol_body'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a typical use case context (inserting new code elements or imports) which implies when to use it, and the 'before' wording distinguishes it from 'insert_after_symbol'. However, it lacks explicit guidance on when NOT to use it or alternatives for different editing scenarios among the many sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_dirList DirA
Read-only

Lists files and directories in the given directory (optionally with recursion). Returns a JSON object with the names of directories and files within the given directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
relative_pathYesThe relative path to the directory to list; pass "." to scan the project root.
recursiveYesWhether to scan subdirectories recursively.
skip_ignored_filesNoWhether to skip files and directories that are ignored.
max_answer_charsNoIf the output is longer than this number of characters, no content will be returned. -1 means the default value from the config will be used. Don't adjust unless there is really no other way to get the content required for the task.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, establishing this as a safe read operation. The description adds useful behavioral context about recursion capability and the JSON return format, but doesn't mention potential limitations like performance implications of recursion, permission requirements, or rate limits. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured in two sentences that convey the core functionality and return format. It's appropriately sized for the tool's complexity, though it could be slightly more front-loaded by mentioning recursion earlier. No wasted words or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the existence of both comprehensive annotations (readOnlyHint, destructiveHint) and an output schema (implied by 'Has output schema: true'), the description provides adequate context. It covers the basic operation, recursion capability, and return format, though it could benefit from mentioning performance considerations or common use cases relative to sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, all parameters are already well-documented in the input schema. The description mentions recursion but doesn't add meaningful semantic context beyond what the schema provides about parameters. The baseline score of 3 is appropriate when the schema carries the full parameter documentation burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Lists files and directories'), identifies the target resource ('in the given directory'), and distinguishes from siblings by specifying the exact scope of listing operations. It explicitly mentions optional recursion, which differentiates it from simpler directory listing tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through 'optionally with recursion' and mentions what the tool returns, but provides no explicit guidance on when to use this tool versus alternatives like 'find_file' or 'search_for_pattern' among the sibling tools. No when-not-to-use scenarios or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_memoriesList MemoriesA
Read-only

List available memories. Any memory can be read using the read_memory tool.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read operation. The description adds no behavioral traits beyond this, such as pagination, sorting, or filtering behavior. It doesn't contradict annotations, but provides minimal additional context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with zero waste: the first states the purpose, and the second provides a useful follow-up action. It's front-loaded and efficiently structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 0 parameters, annotations covering safety, and an output schema (implying return values are documented elsewhere), the description is reasonably complete. It could benefit from more behavioral details (e.g., listing scope or format), but the essentials are covered for this simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing on tool purpose instead, which aligns with the schema's completeness.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose as 'List available memories' (verb+resource), which is specific and understandable. It distinguishes from sibling 'read_memory' by mentioning that any listed memory can be read with that tool, though it doesn't explicitly differentiate from other listing tools like 'list_dir' or 'get_symbols_overview'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by stating 'Any memory can be read using the `read_memory` tool,' suggesting this tool is for discovery before reading. However, it doesn't provide explicit guidance on when to use this vs. other listing tools (e.g., 'list_dir' for files) or alternatives for memory access, leaving some context gaps.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

onboardingOnboardingA
Read-only

Call this tool if onboarding was not performed yet. You will call this tool at most once per conversation. Returns instructions on how to create the onboarding information.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, which the description aligns with by implying a safe, informational operation. The description adds valuable context beyond annotations: it specifies that the tool is for one-time use per conversation and returns instructions, which are behavioral traits not covered by annotations. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise and front-loaded, consisting of only two sentences that efficiently convey key information: when to call the tool and what it returns. Every sentence earns its place by addressing critical aspects without any redundant or vague language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (0 parameters, read-only, non-destructive) and the presence of an output schema, the description is mostly complete. It covers purpose, usage guidelines, and behavioral context effectively. However, it could slightly enhance completeness by hinting at the output format or linking to sibling tools, but this is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters, and schema description coverage is 100%. With no parameters, the description doesn't need to add parameter semantics. It appropriately focuses on usage and behavior, so a baseline score of 4 is justified as it compensates well for the lack of parameters by providing clear context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: it returns instructions on how to create onboarding information. It specifies a verb ('returns instructions') and resource ('onboarding information'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'check_onboarding_performed' or 'initial_instructions', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines: 'Call this tool if onboarding was not performed yet' and 'You will call this tool at most once per conversation.' This clearly defines when to use it (when onboarding is incomplete) and includes a frequency constraint, which is helpful for agent decision-making without mentioning alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prepare_for_new_conversationPrepare For New ConversationB
Read-only

Instructions for preparing for a new conversation. This tool should only be called on explicit user request.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, indicating this is a safe, non-destructive operation. The description adds no behavioral context beyond what annotations provide, such as what 'preparing' entails, potential side effects, or performance characteristics. However, it doesn't contradict annotations, so it meets the lower bar with annotations present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with zero wasted words, making it appropriately sized. It's front-loaded with the purpose statement, though that statement is weak. The second sentence adds crucial usage guidance efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 0 parameters, annotations covering safety, and an output schema (which means return values are documented elsewhere), the description is minimally adequate. However, it fails to explain what 'preparing' actually does operationally, leaving a gap in understanding the tool's effect despite the structured data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the empty input. The description doesn't need to explain parameters, and it doesn't introduce any confusion about inputs. Baseline for 0 parameters is 4, as there's no parameter information to add beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Instructions for preparing for a new conversation' is tautological, essentially restating the tool name/title without specifying what actions or resources are involved. It lacks a clear verb+resource combination and doesn't distinguish this tool from siblings like 'initial_instructions' or 'switch_modes' that might also relate to conversation setup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states 'This tool should only be called on explicit user request,' providing clear when-to-use guidance. It establishes a strict prerequisite condition for invocation, which is precisely what usage guidelines should convey.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_fileRead FileA
Read-only

Reads the given file or a chunk of it. Generally, symbolic operations like find_symbol or find_referencing_symbols should be preferred if you know which symbols you are looking for. Returns the full text of the file at the given relative path.

ParametersJSON Schema
NameRequiredDescriptionDefault
relative_pathYesThe relative path to the file to read.
start_lineNoThe 0-based index of the first line to be retrieved.
end_lineNoThe 0-based index of the last line to be retrieved (inclusive). If None, read until the end of the file.
max_answer_charsNoIf the file (chunk) is longer than this number of characters, no content will be returned. Don't adjust unless there is really no other way to get the content required for the task.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read operation. The description adds valuable context beyond annotations: it explains that the tool can read 'a chunk of it' (implying partial file reading), mentions a constraint ('If the file (chunk) is longer than this number of characters, no content will be returned'), and hints at performance considerations ('Don't adjust unless there is really no other way'). This provides useful behavioral insights not covered by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured in three sentences: the first states the core functionality, the second provides crucial usage guidance, and the third clarifies the return value. Every sentence adds essential information with zero waste, making it appropriately sized and front-loaded with the most important information first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, rich annotations (readOnlyHint, destructiveHint), 100% schema coverage, and the presence of an output schema (which handles return value documentation), the description is complete enough. It covers purpose, usage guidelines, key behavioral aspects, and integrates well with the structured data, leaving no significant gaps for agent understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all four parameters. The description adds minimal parameter semantics beyond the schema: it mentions 'chunk' reading which relates to start_line/end_line parameters, and hints at max_answer_chars usage. However, it doesn't provide significant additional meaning beyond what's in the schema descriptions, warranting the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('Reads', 'Returns') and resources ('the given file or a chunk of it', 'full text of the file'). It distinguishes from siblings by explicitly mentioning alternatives ('symbolic operations like find_symbol or find_referencing_symbols should be preferred if you know which symbols you are looking for'), making the scope and differentiation explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool vs. alternatives: 'Generally, symbolic operations like find_symbol or find_referencing_symbols should be preferred if you know which symbols you are looking for.' This clearly indicates when not to use this tool and names specific sibling tools as better options in certain contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_memoryRead MemoryA
Read-only

Read the content of a memory file. This tool should only be used if the information is relevant to the current task. You can infer whether the information is relevant from the memory file name. You should not read the same memory file multiple times in the same conversation.

ParametersJSON Schema
NameRequiredDescriptionDefault
memory_file_nameYes
max_answer_charsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds valuable behavioral context beyond annotations: it specifies relevance-based usage constraints and prohibits redundant reads within a conversation. This enhances the agent's understanding of appropriate invocation patterns without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured in three sentences, each serving a distinct purpose: stating the core function, providing usage criteria, and setting a behavioral constraint. There is no redundant or verbose language, and the key information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (reading files with relevance checks), the description covers core behavioral guidelines well. Annotations handle safety aspects, and an output schema exists (though not provided here), reducing the need to describe return values. The main gap is parameter semantics, but overall, the description provides sufficient context for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the schema provides no parameter documentation. The description mentions 'memory file name' but doesn't explain what constitutes a valid name or format. It completely omits 'max_answer_chars', leaving its purpose and effect (e.g., truncation behavior) undocumented. The description fails to compensate for the schema's lack of coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Read') and resource ('content of a memory file'), making the purpose immediately understandable. It distinguishes this tool from siblings like 'list_memories' (which lists files) and 'write_memory' (which creates/updates). However, it doesn't explicitly differentiate from 'read_file' (which may read non-memory files), leaving some ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use ('only if the information is relevant to the current task') and when not to use ('should not read the same memory file multiple times in the same conversation'). It also offers a heuristic for relevance determination ('infer from the memory file name'), making it highly actionable for an AI agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rename_symbolRename SymbolA
Destructive

Renames the symbol with the given name_path to new_name throughout the entire codebase. Note: for languages with method overloading, like Java, name_path may have to include a method's signature to uniquely identify a method. Returns result summary indicating success or failure.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_pathYesName path of the symbol to rename (definitions in the `find_symbol` tool apply).
relative_pathYesThe relative path to the file containing the symbol to rename.
new_nameYesThe new name for the symbol.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and readOnlyHint=false, but the description adds valuable context: it specifies the scope ('throughout the entire codebase'), mentions language-specific considerations (Java method overloading), and notes the return format ('result summary indicating success or failure'). This goes beyond the annotations by providing implementation details and behavioral expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured in two sentences: the first states the core purpose and scope, the second adds important behavioral notes and return information. Every sentence adds value without redundancy, and it's front-loaded with the primary action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (destructive operation with codebase-wide impact), the description is complete: it covers purpose, scope, language considerations, and return format. With annotations providing safety context and an output schema presumably detailing the 'result summary', no essential information is missing for effective agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all three parameters. The description adds minimal extra meaning: it references 'name_path' and 'new_name' in context but doesn't provide additional syntax, format, or usage details beyond what the schema descriptions state. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Renames'), target resource ('the symbol with the given name_path'), and scope ('throughout the entire codebase'). It distinguishes this from sibling tools like 'find_symbol' (which only locates) and 'replace_symbol_body' (which modifies content rather than renaming).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool (renaming symbols across the codebase) and includes a helpful note about method overloading in languages like Java. However, it doesn't explicitly state when NOT to use it or name specific alternatives among siblings, though the context implies it's for renaming rather than finding or replacing content.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

replace_contentReplace ContentA
Destructive

Replaces one or more occurrences of a given pattern in a file with new content.

This is the preferred way to replace content in a file whenever the symbol-level tools are not appropriate.

VERY IMPORTANT: The "regex" mode allows very large sections of code to be replaced without fully quoting them! Use a regex of the form "beginning.*?end-of-text-to-be-replaced" to be faster and more economical! ALWAYS try to use wildcards to avoid specifying the exact content to be replaced, especially if it spans several lines. Note that you cannot make mistakes, because if the regex should match multiple occurrences while you disabled allow_multiple_occurrences, an error will be returned, and you can retry with a revised regex. Therefore, using regex mode with suitable wildcards is usually the best choice!.

ParametersJSON Schema
NameRequiredDescriptionDefault
relative_pathYesThe relative path to the file.
needleYesThe string or regex pattern to search for. If `mode` is "literal", this string will be matched exactly. If `mode` is "regex", this string will be treated as a regular expression (syntax of Python's `re` module, with flags DOTALL and MULTILINE enabled).
replYesThe replacement string (verbatim). If mode is "regex", the string can contain backreferences to matched groups in the needle regex, specified using the syntax $!1, $!2, etc. for groups 1, 2, etc.
modeYesEither "literal" or "regex", specifying how the `needle` parameter is to be interpreted.
allow_multiple_occurrencesNoIf True, the regex may match multiple occurrences in the file and all of them will be replaced. If this is set to False and the regex matches multiple occurrences, an error will be returned (and you may retry with a revised, more specific regex).

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate destructiveHint=true (mutation) and readOnlyHint=false (not read-only), which the description aligns with by describing a replacement operation. The description adds valuable context beyond annotations: it emphasizes regex efficiency ('very large sections of code'), safety mechanisms ('error will be returned' for mismatches), and practical advice ('ALWAYS try to use wildcards'). It doesn't mention rate limits or auth needs, but annotations cover the core safety profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose, but it becomes verbose with repetitive regex advice (e.g., multiple sentences urging regex use). Sentences like 'Therefore, using regex mode with suitable wildcards is usually the best choice!' could be condensed. It's informative but not maximally efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (destructive file editing), annotations cover mutation safety, and an output schema exists (so return values needn't be explained). The description adds practical guidance on regex usage and error handling. However, it could better address edge cases (e.g., file permissions, encoding issues) for a tool with destructive potential.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds high-level context about regex usage (e.g., 'wildcards to avoid specifying exact content') but doesn't provide additional semantic details beyond what's in the schema. This meets the baseline of 3 when schema coverage is high.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Replaces one or more occurrences of a given pattern in a file with new content.' It specifies the verb ('replaces'), resource ('a file'), and scope ('one or more occurrences'), distinguishing it from sibling tools like replace_symbol_body (symbol-specific) or create_text_file (file creation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: 'This is the preferred way to replace content in a file whenever the symbol-level tools are not appropriate.' This explicitly contrasts with symbol-level alternatives (e.g., replace_symbol_body). However, it doesn't specify when NOT to use it (e.g., for simple file creation or reading).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

replace_symbol_bodyReplace Symbol BodyA
Destructive

Replaces the body of the symbol with the given name_path.

The tool shall be used to replace symbol bodies that have been previously retrieved (e.g. via find_symbol). IMPORTANT: Do not use this tool if you do not know what exactly constitutes the body of the symbol.

ParametersJSON Schema
NameRequiredDescriptionDefault
name_pathYesFor finding the symbol to replace, same logic as in the `find_symbol` tool.
relative_pathYesThe relative path to the file containing the symbol.
bodyYesThe new symbol body. The symbol body is the definition of a symbol in the programming language, including e.g. the signature line for functions. IMPORTANT: The body does NOT include any preceding docstrings/comments or imports, in particular.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide readOnlyHint=false and destructiveHint=true, indicating this is a destructive mutation. The description adds valuable context beyond annotations: it clarifies the tool's intended workflow (use after retrieval via find_symbol) and provides a critical warning about understanding symbol body composition. However, it doesn't mention potential side effects, error conditions, or authentication requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly structured: a clear purpose statement followed by two focused guidance sentences. Every sentence earns its place - the first defines the action, the second provides usage context, and the third gives a critical warning. No wasted words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the destructive nature (destructiveHint=true), 3 required parameters, and presence of an output schema, the description provides good contextual coverage. It explains the tool's purpose, usage prerequisites, and includes an important warning. However, for a destructive mutation tool, it could benefit from more explicit information about what 'replaces' entails (overwrites vs merges) and potential consequences.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already documents all three parameters thoroughly. The description adds minimal parameter-specific information beyond what's in the schema - it only references 'name_path' in the first sentence. No additional syntax, format, or constraint details are provided for the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Replaces the body of the symbol') and identifies the target resource ('symbol with the given name_path'). It distinguishes from sibling tools like 'rename_symbol' or 'replace_content' by focusing specifically on symbol body replacement rather than other modifications.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'The tool shall be used to replace symbol bodies that have been previously retrieved (e.g. via find_symbol)' and 'IMPORTANT: Do not use this tool if you do not know what exactly constitutes the body of the symbol.' This gives clear when-to-use context and prerequisites, though it doesn't explicitly mention alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_for_patternSearch For PatternA
Read-only

Offers a flexible search for arbitrary patterns in the codebase, including the possibility to search in non-code files. Generally, symbolic operations like find_symbol or find_referencing_symbols should be preferred if you know which symbols you are looking for.

Pattern Matching Logic: For each match, the returned result will contain the full lines where the substring pattern is found, as well as optionally some lines before and after it. The pattern will be compiled with DOTALL, meaning that the dot will match all characters including newlines. This also means that it never makes sense to have .* at the beginning or end of the pattern, but it may make sense to have it in the middle for complex patterns. If a pattern matches multiple lines, all those lines will be part of the match. Be careful to not use greedy quantifiers unnecessarily, it is usually better to use non-greedy quantifiers like .*? to avoid matching too much content.

File Selection Logic: The files in which the search is performed can be restricted very flexibly. Using restrict_search_to_code_files is useful if you are only interested in code symbols (i.e., those symbols that can be manipulated with symbolic tools like find_symbol). You can also restrict the search to a specific file or directory, and provide glob patterns to include or exclude certain files on top of that. The globs are matched against relative file paths from the project root (not to the relative_path parameter that is used to further restrict the search). Smartly combining the various restrictions allows you to perform very targeted searches. Returns A mapping of file paths to lists of matched consecutive lines.

ParametersJSON Schema
NameRequiredDescriptionDefault
substring_patternYesRegular expression for a substring pattern to search for.
context_lines_beforeNoNumber of lines of context to include before each match.
context_lines_afterNoNumber of lines of context to include after each match.
paths_include_globNoOptional glob pattern specifying files to include in the search. Matches against relative file paths from the project root (e.g., "*.py", "src/**/*.ts"). Supports standard glob patterns (*, ?, [seq], **, etc.) and brace expansion {a,b,c}. Only matches files, not directories. If left empty, all non-ignored files will be included.
paths_exclude_globNoOptional glob pattern specifying files to exclude from the search. Matches against relative file paths from the project root (e.g., "*test*", "**/*_generated.py"). Supports standard glob patterns (*, ?, [seq], **, etc.) and brace expansion {a,b,c}. Takes precedence over paths_include_glob. Only matches files, not directories. If left empty, no files are excluded.
relative_pathNoOnly subpaths of this path (relative to the repo root) will be analyzed. If a path to a single file is passed, only that will be searched. The path must exist, otherwise a `FileNotFoundError` is raised.
restrict_search_to_code_filesNoWhether to restrict the search to only those files where analyzed code symbols can be found. Otherwise, will search all non-ignored files. Set this to True if your search is only meant to discover code that can be manipulated with symbolic tools. For example, for finding classes or methods from a name pattern. Setting to False is a better choice if you also want to search in non-code files, like in html or yaml files, which is why it is the default.
max_answer_charsNoIf the output is longer than this number of characters, no content will be returned. -1 means the default value from the config will be used. Don't adjust unless there is really no other way to get the content required for the task. Instead, if the output is too long, you should make a stricter query.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, which the description aligns with by describing a search operation. The description adds valuable behavioral context beyond annotations, such as pattern matching logic (e.g., DOTALL compilation, advice on greedy quantifiers), file selection restrictions, and output format details ('Returns A mapping of file paths to lists of matched consecutive lines'), enhancing the agent's understanding of how the tool behaves.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (e.g., 'Pattern Matching Logic', 'File Selection Logic') and front-loaded key information. It avoids unnecessary repetition, but could be slightly more concise by integrating some details more tightly. Overall, it's efficient and informative without wasted sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity, rich input schema (100% coverage), annotations, and presence of an output schema, the description is complete. It covers purpose, usage guidelines, behavioral details, and parameter context adequately, ensuring the agent has sufficient information to use the tool effectively without needing to explain return values (handled by output schema).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds some semantic context, like explaining the purpose of `restrict_search_to_code_files` and how glob patterns work relative to the project root, but it doesn't provide significant additional meaning beyond what the schema offers. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Offers a flexible search for arbitrary patterns in the codebase, including the possibility to search in non-code files.' It specifies the verb ('search'), resource ('patterns in the codebase'), and scope ('including non-code files'), and distinguishes it from sibling tools like find_symbol and find_referencing_symbols by advising when to prefer those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool versus alternatives: 'Generally, symbolic operations like find_symbol or find_referencing_symbols should be preferred if you know which symbols you are looking for.' It also offers context on file selection logic, such as using `restrict_search_to_code_files` for code symbols, making it clear when to apply specific parameters.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

switch_modesSwitch ModesB
Read-only

Activates the desired modes, like ["editing", "interactive"] or ["planning", "one-shot"].

ParametersJSON Schema
NameRequiredDescriptionDefault
modesYesThe names of the modes to activate.

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe, non-destructive operation. The description adds some context by specifying it 'activates' modes, but doesn't elaborate on what activation entails, whether it's persistent, or if there are side effects. It provides minimal behavioral insight beyond what annotations already cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise - a single sentence with two helpful examples. Every word earns its place, and the structure is front-loaded with the core purpose immediately stated. No wasted verbiage or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has annotations covering safety, 100% schema coverage, and an output schema exists, the description is minimally adequate. However, for a mode-switching operation, it should ideally explain what 'activation' means, whether modes are exclusive or combinable, and what the expected outcome is, despite the output schema existing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the input schema already fully documents the single 'modes' parameter. The description adds marginal value by providing example mode arrays (['editing', 'interactive'] and ['planning', 'one-shot']), which helps illustrate valid inputs but doesn't explain parameter semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('activates') and resource ('desired modes'), and provides concrete examples of mode combinations. However, it doesn't explicitly differentiate this tool from potential sibling tools like 'activate_project' or 'get_current_config' that might also involve mode management.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, when this should be called versus other configuration tools, or what happens if modes conflict. The examples suggest usage but don't provide explicit decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

think_about_collected_informationThink About Collected InformationA
Read-only

Think about the collected information and whether it is sufficient and relevant. This tool should ALWAYS be called after you have completed a non-trivial sequence of searching steps like find_symbol, find_referencing_symbols, search_files_for_pattern, read_file, etc.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe, non-destructive operation. The description adds context by specifying it's for post-search reflection, which is useful behavioral insight beyond the annotations. However, it doesn't detail aspects like response format or any limitations, keeping it at a moderate score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: two sentences that efficiently convey purpose and usage guidelines without any wasted words. It's front-loaded with the core action and follows with specific instructions, making it easy to parse and apply.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (low, with 0 parameters), rich annotations (readOnlyHint, destructiveHint), and the presence of an output schema, the description is complete. It adequately explains when and why to use the tool, and the output schema handles return values, so no additional details are necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here. A baseline of 4 is applied since there are no parameters to document, and the description focuses on usage context instead.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: to 'think about the collected information and whether it is sufficient and relevant.' It specifies a verb ('think about') and resource ('collected information'), making the action clear. However, it doesn't distinctly differentiate from sibling tools like 'think_about_task_adherence' or 'think_about_whether_you_are_done' beyond the focus on 'collected information,' which keeps it from a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines: it states 'This tool should ALWAYS be called after you have completed a non-trivial sequence of searching steps' and lists examples like 'find_symbol, find_referencing_symbols, search_files_for_pattern, read_file, etc.' This clearly defines when to use it, including context and prerequisites, with no misleading information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

think_about_task_adherenceThink About Task AdherenceA
Read-only

Think about the task at hand and whether you are still on track. Especially important if the conversation has been going on for a while and there has been a lot of back and forth.

This tool should ALWAYS be called before you insert, replace, or delete code.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, which the description does not contradict. The description adds behavioral context by emphasizing the tool's importance in specific scenarios (long conversations, back-and-forth) and its mandatory use before certain code operations. However, it does not disclose additional traits like rate limits, auth needs, or detailed behavioral outcomes beyond what annotations provide, resulting in moderate value addition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized with three sentences that are front-loaded: the first states the purpose, the second adds context, and the third provides usage guidelines. Each sentence adds value without redundancy, though it could be slightly more structured for optimal clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (low, with 0 parameters), rich annotations (readOnlyHint, destructiveHint), and presence of an output schema, the description is mostly complete. It covers purpose, usage guidelines, and context, but lacks details on output behavior or specific outcomes. However, with annotations and output schema handling safety and returns, the description is sufficient for the tool's role.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so no parameter information is needed. The description does not discuss parameters, which is appropriate. Baseline score is 4 for zero parameters, as the description does not need to compensate for any schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool is for 'thinking about the task at hand and whether you are still on track,' which provides a vague purpose. It mentions a specific context ('conversation has been going on for a while'), but does not clearly distinguish it from sibling tools like 'think_about_collected_information' or 'think_about_whether_you_are_done.' The purpose is understandable but lacks specificity in differentiating its role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'Especially important if the conversation has been going on for a while and there has been a lot of back and forth' and 'This tool should ALWAYS be called before you insert, replace, or delete code.' It provides clear context and exclusions, naming specific sibling tools (insert, replace, delete code) as alternatives or prerequisites, making usage guidelines highly explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

think_about_whether_you_are_doneThink About Whether You Are DoneB
Read-only

Whenever you feel that you are done with what the user has asked for, it is important to call this tool.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe, non-destructive operation. The description adds no behavioral traits beyond this (e.g., what 'thinking' entails, if it returns a result, or any side effects). With annotations covering safety, the description adds minimal value, meeting the baseline for annotations present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the usage condition. It is front-loaded with the key instruction and has no wasted words, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (low, with no parameters) and the presence of annotations and an output schema, the description is minimally adequate. It explains when to call the tool but lacks details on what the tool does internally or what the output might indicate. With structured data covering safety and output, the description is complete enough for basic use but could be more informative.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters, and schema description coverage is 100% (empty schema). The description does not need to explain parameters, so it naturally compensates for the lack of parameters. A baseline of 4 is appropriate as no parameter information is required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is tautological, essentially restating the tool name ('think about whether you are done') as an instruction to call it when done. It does not specify what the tool actually does (e.g., evaluates completion criteria, returns a decision, triggers a state change) or how it differs from sibling tools like 'think_about_collected_information' or 'think_about_task_adherence'. The purpose remains vague beyond the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the tool ('whenever you feel that you are done with what the user has asked for'), which is explicit guidance. However, it does not mention when not to use it (e.g., during intermediate steps) or alternatives (e.g., other 'think' tools), so it falls short of a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

write_memoryWrite MemoryA
Destructive

Write some information (utf-8-encoded) about this project that can be useful for future tasks to a memory in md format. The memory name should be meaningful.

ParametersJSON Schema
NameRequiredDescriptionDefault
memory_file_nameYes
contentYes
max_answer_charsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate destructiveHint=true and readOnlyHint=false, which the description aligns with by implying a write operation. The description adds valuable context beyond annotations: it specifies the content format (UTF-8, markdown) and purpose (for future tasks), and hints at persistence. However, it doesn't detail side effects like overwriting existing memories or authentication needs, which could be useful given the destructive nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences that are front-loaded with the core action and purpose. It avoids redundancy and wastes no words, though it could be slightly more structured by explicitly separating parameter guidance. Every sentence adds value, making it efficient but not perfectly optimized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (destructive write with 3 parameters), annotations cover safety, and an output schema exists (though not provided here), reducing the burden on the description. The description adequately explains the tool's purpose and key parameters but lacks details on error handling or output format. It's mostly complete for an agent to use correctly, with minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning for 'memory_file_name' by stating it 'should be meaningful' and for 'content' by specifying UTF-8 and markdown format. However, it doesn't explain 'max_answer_chars' or its default value of -1, leaving one parameter undocumented. This partial compensation justifies a baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'write' and resource 'memory' with specific details: writing UTF-8 encoded information in markdown format for future project tasks. It distinguishes from siblings like 'read_memory' and 'edit_memory' by focusing on creation rather than retrieval or modification, though it doesn't explicitly contrast with 'create_text_file' which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for storing project information for future tasks, suggesting when to use it. However, it lacks explicit guidance on when not to use it or alternatives, such as whether to choose 'create_text_file' for non-memory files or 'edit_memory' for updates. No prerequisites or exclusions are mentioned, leaving gaps in decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A3.9/5.0
Disambiguation4/5

Most tools have distinct purposes, but there is some overlap between find_symbol, get_symbols_overview, and search_for_pattern, which could cause confusion about when to use each. The descriptions help clarify, but an agent might still hesitate between these for certain tasks.

Naming Consistency5/5

Tool names consistently follow a verb_noun pattern (e.g., activate_project, create_text_file, execute_shell_command) with clear and descriptive verbs. There are no deviations in style, making it easy to predict tool names and their functions.

Tool Count3/5

With 29 tools, the count feels heavy for a code assistant server, as it includes many overlapping or specialized tools (e.g., multiple 'think_about' tools). This could overwhelm agents and lead to inefficiency, though it's not extreme.

Completeness5/5

The tool set provides comprehensive coverage for code editing, file management, symbol manipulation, memory handling, and project onboarding. It includes all necessary CRUD operations and lifecycle steps, with no obvious gaps for the domain.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    C
    maintenance
    A coding agent toolkit that transforms LLMs into coding assistants capable of working directly on your codebase with semantic code retrieval and editing tools, providing IDE-like capabilities without requiring API subscriptions.
    33
    1
    MIT
  • A
    license
    A
    quality
    Not graded
    maintenance
    Enables agents to quickly find and edit code in a codebase with surgical precision. Find symbols, edit them everywhere with tools for reading code blocks, searching/replacing text, and making precise line-based modifications.
    3
    11
  • A
    license
    B
    quality
    D
    maintenance
    A coding agent toolkit that provides IDE-like semantic code retrieval and editing tools, enabling LLMs to efficiently navigate and modify codebases using symbol-level operations instead of basic file reading and string replacements.
    19
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI coding agents to query a pre-built semantic knowledge graph of code, reducing token usage and tool calls. Supports 16 tools for code exploration, analysis, and context building.
    13
    7
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/chrisgreenx-ctrl/serena'

If you have feedback or need assistance with the MCP directory API, please join our Discord server