Skip to main content
Glama

Update custom benchmark

update_custom_benchmark
Destructive

Add/remove members or rename a custom benchmark. Creates a NEW immutable version (the previous version stays fully reproducible) and returns the exact change-impact on the median/quartiles/index. Pass benchmark_id.

Input Schema

NameRequiredDescriptionDefault
addNoProvider ids (handles) to add to the benchmark
removeNoProvider ids (handles) to remove from the benchmark
renameNoNew name for the benchmark
benchmark_idYesYour cb_ token

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changed
    • addedInput schema / properties / add / description
      Added value: +"Provider ids (handles) to add to the benchmark"
    • addedInput schema / properties / remove / description
      Added value: +"Provider ids (handles) to remove from the benchmark"
    • addedInput schema / properties / rename / description
      Added value: +"New name for the benchmark"
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "additionalProperties": true,
      +  "description": "JSON result; identical to the text content block.",
      +  "type": "object"
      +}
  2. Added

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing that a NEW immutable version is created, that the previous version remains fully reproducible, and that the response carries exact change-impact on median/quartiles/index. This adds meaningful mutation semantics (versioning/immutability) that destructiveHint alone does not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences with no waste; the operation and the required parameter are front-loaded before the versioning/return detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return-value explanation is unnecessary, and the description still notes the response carries change-impact. Versioning behavior, required param, and mutation nature are all covered; nothing critical for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and every parameter is already documented, so baseline 3 applies. The description only reinforces 'Pass benchmark_id' without adding format or interaction details (e.g., whether add and remove can be combined) beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States specific verbs (add/remove members, rename) on a specific resource (custom benchmark), which clearly separates it from create_custom_benchmark, delete_custom_benchmark, and get_custom_benchmark. An agent can identify the operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies use for modifying an existing benchmark and instructs passing benchmark_id, which distinguishes it from creation. It does not explicitly name sibling alternatives (create vs. delete) or state prerequisites such as who may mutate a benchmark, so it stops short of full when/when-not routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources