Skip to main content
Glama
Sign In
OpenConstruction MCP

OpenConstruction MCP

by Ben11304
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

Benchmark Submission Prep

prepare_benchmark_submission

Create a checklist for benchmark submission, covering required metadata and linked resources such as dataset ID, model ID, metric, and task.

Instructions

Create a checklist for preparing benchmark metadata and linked resources.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "task": "<task>"
    },
    "name": "prepare_benchmark_submission"
  }
}

Input Schema

NameRequiredDescriptionDefault
taskYes
metricNo
model_idNo
dataset_idNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

D1.5/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must convey behavior. It only says 'create a checklist,' which implies some output or side effect, but there is no mention of what happens, whether it returns data, whether it has side effects, or whether it requires specific inputs or permissions. Completely opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, which is concise, but it lacks structure and specificity. It could be expanded to mention key parameters or expected behavior without becoming verbose. As is, it's minimal but does not fully utilize the format.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and zero parameter documentation, this description is severely under-specified. An agent cannot determine what the tool does, what inputs mean, what it returns, or how it relates to sibling tools. It is inadequate for reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description mentions none of the four parameters (task, metric, model_id, dataset_id). The description adds zero information about what these parameters mean, how they relate to the checklist, or how to populate them. Given high parameter count and zero coverage in schema, this is a critical gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a verb ('Create a checklist') and a resource ('benchmark metadata and linked resources'), but the exact function is vague. It doesn't clarify what the tool actually does—does it generate a to-do list, return a template, or validate something? The phrase 'checklist' is ambiguous and doesn't distinguish from siblings like validate_metadata_record or find_datasets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus any sibling. There is no mention of prerequisites, context, or what scenario this tool is designed for. Users are left to guess whether this is a planning step, a validation step, or something else.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools