Compiler Explorer

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint: true annotation is contradicted by the description and schema since executing code is clearly a write/mutate operation. However, the description itself is transparent about what the tool does. The annotation contradiction is flagged separately, but the description accurately says 'compile and return assembly output, stdout, and stderr', and the schema details execution behavior. The annotation issue is a failing of the annotation, not the description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that precisely captures the core function. Every word is necessary, there is no fluff or repetition. It is front-loaded and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the comprehensive input schema (100% coverage, nested objects well documented) and the lack of an output schema, the description adequately states what is returned (assembly output, stdout, stderr). No additional detail is needed—the schema covers parameter behavior extensively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds no parameter details itself, but the schema descriptions are thorough, covering all 11 parameters including complex nested objects like filters and libraries. The short description does not need to repeat schema content.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compiles source code and returns assembly output, stdout, and stderr. This is a specific verb-resource combination that distinguishes it from siblings like list_compilers or lookup_asm_instruction. However, the description omits mention of the execution mode documented in the schema, which is a significant secondary capability.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives. However, given sibling tools like list_compilers, list_languages, and list_libraries, it is implied this tool is for actual compilation after selecting a compiler. There is no guidance on prerequisites (e.g., need to list compilers first) or when to use execution mode.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false. The description adds no new behavioral context beyond 'create'. It does not contradict annotations but also does not elaborate on side effects, authentication, or rate limits. With annotations present, a score of 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence (12 words) that is front-loaded with the verb 'Create'. It is concise and contains no filler or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description does not explain what the short URL looks like, how it is returned, or any prerequisites (e.g., valid compiler must be selected). For a 5-parameter tool with no output schema, the description is too brief to fully guide an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 5 parameters have descriptions in the input schema (100% coverage). The description does not add any parameter details beyond what the schema provides. The description's mention of 'compiler settings' vaguely references parameters but adds no specific semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a Compiler Explorer short URL for sharing code with compiler settings' clearly states the verb (create), resource (short URL), and purpose (sharing code with settings). It distinguishes from siblings like 'compile' (run code) and 'get_shortlink_info' (retrieve link info).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for sharing code but does not provide explicit guidance on when to use versus alternatives (e.g., 'compile' for running code, 'get_shortlink_info' for retrieving existing links). No exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only (readOnlyHint=true). The description adds valuable behavioral details beyond annotations: it specifies the output shape (matching 'compile' tool's structure) and the flattening of multi-pane shortlinks. This informs the agent of what data survives and how results are returned. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences and 44 words, with no superfluous information. The primary action is stated first, followed by important details about output shape and flattening behavior. Every sentence adds value, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, annotations present), the description covers the essential aspects: purpose, output structure, and a notable behavioral nuance (flattening). It does not address error handling or missing shortlinks, but the context is sufficient for basic usage. Sibling tools are listed externally, so no need to reference them here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear parameter description ('Short link ID or full URL'). The tool description restates the parameter's purpose in context ('Retrieve source and compiler config from a CE short URL') but adds no new semantic meaning beyond the schema. Baseline score of 3 is appropriate as the schema already conveys the necessary information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'source and compiler config from a CE short URL'. It is specific and distinguishable from sibling tools like 'compile' (which compiles code) and 'generate_short_url' (which creates shortlinks). The output shape is also explicitly described, leaving no ambiguity about the tool's function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implicitly advises that multi-pane shortlinks are flattened, hinting that the tool may not be suitable for recovering full multi-pane structures. However, it does not explicitly state when to use this tool versus alternatives (e.g., 'compile' for actual compilation, or 'list_compilers' for compiler discovery). No direct comparison or exclusion criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already marks this as read-only. The description provides some additional behavioral context (e.g., degrading to lean results beyond maxResults, refinement hints), but does not fully detail output structure or pagination limits, making it adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (single sentence) and front-loaded with the core action. However, it could be slightly expanded to include when to use this tool versus alternatives, but brevity is maintained without unnecessary fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters (all documented in schema), readOnlyHint annotation, and no output schema, the description sufficiently covers the tool's purpose and major usage patterns. It could benefit from mentioning output structure, but the parameter documentation compensates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the description does not need to add much, but it enriches the schema with usage notes (e.g., 'Case-insensitive AND-of-tokens filter', 'numeric tokens match whole-word') and relationships between parameters (e.g., 'Combine with latestPerMajor: true'). This goes beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies the action ('List') and resource ('compilers') with an optional filter by language, clearly distinguishing it from sibling tools like 'compile' or 'list_languages'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a browsing context but does not explicitly state when to use this tool versus alternatives like 'compile' for specific compilers. The rich parameter documentation guides usage but lacks direct comparison to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, so the agent knows it's a read-only operation. The description adds that each entry has 'defaultCompiler' and 'compilerCount', but does not disclose other behavioral aspects like the number of languages, pagination, or rate limits. For a simple list endpoint with no side effects, the combination of annotations and description is minimally adequate, earning a 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at two short sentences (25 words). The first sentence states the core purpose, and the second provides key structural details. Every word earns its place with no redundancy or extra fluff. This is an excellent length for such a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters, no output schema, and annotations covering read-only intent, the description provides just enough to understand its role: it lists languages and their associated compilers. The mention of 'defaultCompiler' and 'compilerCount' partially compensates for the missing output schema. The tool is simple, so completeness is high. For a more complex tool, this might be insufficient, but here it works well.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0 parameters and 100% schema description coverage, the schema already documents all necessary info. The description adds no parameter details because there are none. Per guidelines, 0 params baseline is 4, and the description does not detract from that. The mention of 'defaultCompiler' and 'compilerCount' is a bonus, hinting at the structure of the response, which is beyond the schema's scope.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the purpose: 'List supported languages'. It also specifies the shape of each entry ('defaultCompiler' and 'compilerCount'), which is specific and informative. The tool name is 'list_languages' and the title is 'List languages', aligning perfectly. No sibling tool has a similar purpose, so it is well-distinguished.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide guidance on when to use this tool versus alternatives. Given siblings like 'list_compilers' and 'list_libraries', a user might benefit from knowing if this tool is a prerequisite for others. However, the tool is simple enough that usage is implied: call it to see available languages before using other tools. The lack of explicit guidance keeps it at a baseline 3.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already supply readOnlyHint=true, so the description is not required to restate that. However, the description adds no behavioral context beyond the basic purpose, such as result format, pagination behavior, or what 'available' means. This is minimal disclosure even given the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It immediately conveys the core action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite 100% schema coverage, the description lacks context about return values, parameter interactions (e.g., how 'lean' affects output), and expected behavior. Without an output schema, the description should provide more guidance to make the tool fully understandable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All four parameters have descriptions in the input schema (100% coverage). The tool description itself does not add any parameter semantics beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List', the resource 'libraries', and the condition 'for a given programming language'. This directly differentiates it from sibling tools like list_compilers and list_languages, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool: when needing libraries for a specific language. However, it does not explicitly mention when not to use it or point to alternatives, though sibling tools are different enough to avoid confusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true and openWorldHint=false. The description does not add any new behavioral traits beyond what the annotations convey. It neither contradicts nor enhances the behavioral profile, so a baseline score of 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is front-loaded with the key action. Every word earns its place, and there is no unnecessary detail or verbosity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (two parameters, no output schema), the description is mostly complete. It lacks a hint about the return format (e.g., what the documentation looks like), but the annotations and schema cover the rest. Slightly incomplete but still adequate for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the existing schema descriptions for opcode and instruction_set are clear. The tool description does not add any additional meaning or context for the parameters, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Look up documentation for an assembly instruction' clearly states the verb ('look up') and resource ('documentation for an assembly instruction'), distinguishing it from sibling tools that deal with compilation or URL generation. It is not a tautology since it adds specificity over the tool name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. Usage is implied by the purpose, but no explicit 'when-to-use' or 'when-not-to-use' context is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.