Compiler Explorer

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint: true annotation is contradicted by the description and schema since executing code is clearly a write/mutate operation. However, the description itself is transparent about what the tool does. The annotation contradiction is flagged separately, but the description accurately says 'compile and return assembly output, stdout, and stderr', and the schema details execution behavior. The annotation issue is a failing of the annotation, not the description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that precisely captures the core function. Every word is necessary, there is no fluff or repetition. It is front-loaded and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the comprehensive input schema (100% coverage, nested objects well documented) and the lack of an output schema, the description adequately states what is returned (assembly output, stdout, stderr). No additional detail is needed—the schema covers parameter behavior extensively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds no parameter details itself, but the schema descriptions are thorough, covering all 11 parameters including complex nested objects like filters and libraries. The short description does not need to repeat schema content.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compiles source code and returns assembly output, stdout, and stderr. This is a specific verb-resource combination that distinguishes it from siblings like list_compilers or lookup_asm_instruction. However, the description omits mention of the execution mode documented in the schema, which is a significant secondary capability.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives. However, given sibling tools like list_compilers, list_languages, and list_libraries, it is implied this tool is for actual compilation after selecting a compiler. There is no guidance on prerequisites (e.g., need to list compilers first) or when to use execution mode.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false. The description adds no new behavioral context beyond 'create'. It does not contradict annotations but also does not elaborate on side effects, authentication, or rate limits. With annotations present, a score of 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence (12 words) that is front-loaded with the verb 'Create'. It is concise and contains no filler or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description does not explain what the short URL looks like, how it is returned, or any prerequisites (e.g., valid compiler must be selected). For a 5-parameter tool with no output schema, the description is too brief to fully guide an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 5 parameters have descriptions in the input schema (100% coverage). The description does not add any parameter details beyond what the schema provides. The description's mention of 'compiler settings' vaguely references parameters but adds no specific semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a Compiler Explorer short URL for sharing code with compiler settings' clearly states the verb (create), resource (short URL), and purpose (sharing code with settings). It distinguishes from siblings like 'compile' (run code) and 'get_shortlink_info' (retrieve link info).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for sharing code but does not provide explicit guidance on when to use versus alternatives (e.g., 'compile' for running code, 'get_shortlink_info' for retrieving existing links). No exclusions or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only (readOnlyHint=true). The description adds valuable behavioral details beyond annotations: it specifies the output shape (matching 'compile' tool's structure) and the flattening of multi-pane shortlinks. This informs the agent of what data survives and how results are returned. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences and 44 words, with no superfluous information. The primary action is stated first, followed by important details about output shape and flattening behavior. Every sentence adds value, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, annotations present), the description covers the essential aspects: purpose, output structure, and a notable behavioral nuance (flattening). It does not address error handling or missing shortlinks, but the context is sufficient for basic usage. Sibling tools are listed externally, so no need to reference them here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear parameter description ('Short link ID or full URL'). The tool description restates the parameter's purpose in context ('Retrieve source and compiler config from a CE short URL') but adds no new semantic meaning beyond the schema. Baseline score of 3 is appropriate as the schema already conveys the necessary information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'source and compiler config from a CE short URL'. It is specific and distinguishable from sibling tools like 'compile' (which compiles code) and 'generate_short_url' (which creates shortlinks). The output shape is also explicitly described, leaving no ambiguity about the tool's function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implicitly advises that multi-pane shortlinks are flattened, hinting that the tool may not be suitable for recovering full multi-pane structures. However, it does not explicitly state when to use this tool versus alternatives (e.g., 'compile' for actual compilation, or 'list_compilers' for compiler discovery). No direct comparison or exclusion criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description adds nothing beyond that: it does not mention the default maxResults cap of 25, the degradation to lean results beyond the cap, or any pagination/ordering behavior — all of which live only in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. It is efficient, though for a tool with seven filter parameters and several unusual semantics, the extreme brevity borders on under-specification rather than tight writing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The rich schema and existing annotations carry most of the burden, and no output schema exists so return values needn't be described. Still, for a catalog-listing tool with complex filtering semantics and adjacent sibling list tools, the description leaves the agent without any routing or behavior context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the seven parameters (including the non-obvious match token semantics, latestPerMajor, and includeExperimental interactions) are fully documented in the schema. The description only restates the language filter, adding no meaning beyond structured data; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("List available compilers") and mentions the optional language filter, so the agent knows what the tool returns. It does not, however, distinguish itself from siblings list_languages or list_libraries, which an agent could plausibly confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase "optionally filtered by language" implies one usage path but gives no guidance on when to reach for this tool versus list_languages/list_libraries/compile, nor when to use the advanced filters (match, latestPerMajor, lean) that the schema defines. No exclusions or prerequisites are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, so the agent knows it's a read-only operation. The description adds that each entry has 'defaultCompiler' and 'compilerCount', but does not disclose other behavioral aspects like the number of languages, pagination, or rate limits. For a simple list endpoint with no side effects, the combination of annotations and description is minimally adequate, earning a 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at two short sentences (25 words). The first sentence states the core purpose, and the second provides key structural details. Every word earns its place with no redundancy or extra fluff. This is an excellent length for such a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no parameters, no output schema, and annotations covering read-only intent, the description provides just enough to understand its role: it lists languages and their associated compilers. The mention of 'defaultCompiler' and 'compilerCount' partially compensates for the missing output schema. The tool is simple, so completeness is high. For a more complex tool, this might be insufficient, but here it works well.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0 parameters and 100% schema description coverage, the schema already documents all necessary info. The description adds no parameter details because there are none. Per guidelines, 0 params baseline is 4, and the description does not detract from that. The mention of 'defaultCompiler' and 'compilerCount' is a bonus, hinting at the structure of the response, which is beyond the schema's scope.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the purpose: 'List supported languages'. It also specifies the shape of each entry ('defaultCompiler' and 'compilerCount'), which is specific and informative. The tool name is 'list_languages' and the title is 'List languages', aligning perfectly. No sibling tool has a similar purpose, so it is well-distinguished.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide guidance on when to use this tool versus alternatives. Given siblings like 'list_compilers' and 'list_libraries', a user might benefit from knowing if this tool is a prerequisite for others. However, the tool is simple enough that usage is implied: call it to see available languages before using other tools. The lack of explicit guidance keeps it at a baseline 3.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already supply readOnlyHint=true, so the description is not required to restate that. However, the description adds no behavioral context beyond the basic purpose, such as result format, pagination behavior, or what 'available' means. This is minimal disclosure even given the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It immediately conveys the core action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite 100% schema coverage, the description lacks context about return values, parameter interactions (e.g., how 'lean' affects output), and expected behavior. Without an output schema, the description should provide more guidance to make the tool fully understandable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All four parameters have descriptions in the input schema (100% coverage). The tool description itself does not add any parameter semantics beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List', the resource 'libraries', and the condition 'for a given programming language'. This directly differentiates it from sibling tools like list_compilers and list_languages, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool: when needing libraries for a specific language. However, it does not explicitly mention when not to use it or point to alternatives, though sibling tools are different enough to avoid confusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that: it does not say what the lookup returns (operands, flags, encoding, examples) or what happens when an opcode is not found, so behavioral value beyond annotations is essentially zero.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. It is well structured, though it is arguably too terse given the tool's undocumented return shape.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter lookup with full schema coverage, the description is adequate. With no output schema, however, it should ideally hint at what documentation is returned (mnemonic details, operands, flags, encoding), which is a real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — both 'opcode' and 'instruction_set' are documented in the schema, including examples for opcode and a note that instruction_set has no default. The description adds no parameter meaning on top of that, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('look up documentation') and resource ('assembly instruction'), which is unambiguous against siblings like compile or list_compilers. It does not explicitly differentiate itself from any sibling, but its domain is distinct enough that confusion is unlikely.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the name and description — an agent can infer you call this when you need details about a mnemonic. However, there is no explicit when-to-use guidance, no mention of prerequisites, and no note about coverage limits of the instruction-set list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.