Skip to main content
Glama

Create rubric

create_rubric

Store a reusable scoring rubric and return its ID for later evaluation runs, including criteria, weights, scales, aggregation, gates, grades, and pass thresholds.

Instructions

Store a rubric and get back its id for reuse. Criteria need id/name; optional weight, scale, anchors (= rubric levels for decision models), judge ({type: decision|human|benchmark|llm|panel}; default decision/typesafe). Aggregation supports weighted_mean, weighted_sum, mean, median, min, max, geometric_mean, formula; gates; grades; passThreshold.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "rubric": "<rubric>"
    },
    "name": "create_rubric"
  }
}

Input Schema

NameRequiredDescriptionDefault
rubricYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full behavioral burden. It does not say whether the rubric is validated on write, whether duplicates are rejected or overwritten, whether authorization is required, or what happens when the payload is malformed. The judge default note ('decision/typesafe') adds a little behavioral context but is about payload semantics, not the write's side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence, and the payload detail follows in a dense but scannable run of semicolon-separated clauses. The telegraphic style ('anchors (= rubric levels for decision models)') is efficient but occasionally cryptic; nothing is padding, though.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create tool whose real schema is a blank object, the description does the heavy lifting on payload shape and states what it returns (an id), which compensates for the missing output schema. It is still incomplete on validation behavior, error conditions, and how the stored rubric relates to the other rubric tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the schema itself is almost useless (a single 'rubric' that is any string or arbitrary object), so the description must compensate. It does: criteria require id/name, and weight, scale, anchors, judge type, and the aggregation/gates/grades/passThreshold options are all named, giving an agent real structure to build the payload from. It stops short of full field-level typing, so not a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The definition opens with a specific verb+resource and an outcome: 'Store a rubric and get back its id for reuse.' An agent immediately knows this persists a rubric and yields an identifier. It does not distinguish itself from siblings like validate_rubric or fork_rubric, which is the only gap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance. 'For reuse' hints that the returned id feeds later calls, but nothing states whether a rubric should be validated first, whether fork_rubric is the alternative for deriving from an existing one, or what preconditions exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.