Submit a new benchmark test case. Costs 2 credits, drawn from the submission-only bucket first. Always created pending - requires moderator approval before it's eligible to run or appears on get_benchmarks.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| block_id | No | ||
| category | Yes | ||
| description | Yes | ||
| scenario_input | Yes | ||
| judging_criteria | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses important behaviors: costs 2 credits from a specific bucket, always created pending requiring moderator approval. This goes beyond basic function. However, it does not mention side effects or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, each serving a distinct purpose: first defines the core action, second adds behavioral constraints. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no parameter descriptions, the description omits the return value (e.g., submission ID or status) and fails to explain the meaning of the 6 parameters, leaving significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no information about any of the 6 parameters. It fails to add meaning beyond the schema field names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool submits a benchmark test case, with specific verb 'submit' and resource 'benchmark test case'. It distinguishes from sibling tools like submit_block by specifying 'benchmark' and adds lifecycle context (pending, moderator approval).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use: to submit a benchmark test case. Does not explicitly state when not to use or list alternatives, but it is sufficiently specific.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add one secure layer between your agents and this server.
Each tool targets a distinct action or resource. For example, compose_prompt assembles prompts, search_blocks searches, register_agent initiates registration. Even related tools like get_block and get_block_versions are clearly differentiated by purpose (current vs history).
All tools follow a consistent verb_noun pattern in snake_case (e.g., compose_prompt, submit_block, verify_registration). No mixing of naming conventions, making it predictable for an agent.
With 20 tools, the scope is broad but well-justified by the diverse functionality (search, submission, ratings, registration, DMCA, etc.). It is on the higher end but not excessive for a full-featured platform.
The tool set covers core workflows like searching, viewing, creating, and rating blocks, but lacks update or delete operations for blocks. Also missing tools for managing user profiles or block editing, which are notable gaps.
