Skip to main content
Glama

§7: ¿Cómo saber si funciona bien?

disenar_evaluacion

Evalúa el plan de evaluación del agente (o ayuda a crearlo): 3 dimensiones (capacidades/trayectoria/resultado), métricas (tasa de éxito, consistencia, coste por tarea, adopción), golden set de 20-50 tareas, monitoreo de drift y self-consistency para alto stake.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
languageNoIdioma de la respuesta (default "es"). / Response language: pass "en" for English output.
agent_definitionYesDefinición completa del agente a mejorar: system prompt, frontmatter, configuración, descripción de tools y cualquier doc de diseño. Cuanto más completa, mejor la evaluación.

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility. It lacks disclosure of side effects (e.g., whether the tool modifies anything), authentication needs, rate limits, or return behavior. The description focuses on what the tool evaluates but not on its operational traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense sentence listing multiple aspects. While it packs information, it is not front-loaded with a clear primary action and could be more structured for readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description does not specify what the tool returns (e.g., a report, score, or feedback). Given the absence of an output schema, the agent lacks understanding of the invocation outcome. For a tool dealing with evaluation plans, this is a significant gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters documented. The tool description adds context about the agent_definition parameter ('the more complete, the better the evaluation') and the language enum, but overall adds limited additional meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool evaluates or helps create an agent's evaluation plan, listing specific dimensions, metrics, and other components. This provides a clear verb and resource, distinguishing it from siblings like 'evaluar_sistema' which likely evaluate the agent directly. However, the dual role (evaluate vs. create) introduces slight ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for evaluation plan assessment or creation but does not explicitly state when to use this tool versus alternatives. No exclusions or specific context are given, leaving the agent to infer from the name and sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.8/5.0
Disambiguation4/5

Tools are mostly distinct with clear purposes, but some overlap exists between auditing and reviewing tools (e.g., auditar_contexto vs. revisar_rol_y_frontera). The meta-tool recomendar_flujo might cause initial confusion but descriptions clarify its role.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in Spanish (e.g., aplicar_challenger, auditar_contexto). The naming is uniform and predictable across all 18 tools.

Tool Count5/5

18 tools is well-scoped for the server's purpose of comprehensive agent governance. Each tool covers a distinct aspect of the agent lifecycle without being overwhelming.

Completeness5/5

The tool surface covers the full lifecycle from evaluation, design, auditing, contract generation, to final construction. No obvious gaps; the set is self-contained for agent governance.