Skip to main content
Glama
Sign In

Relay — agent service discovery

by site.chatgpt.orellise.relay-agent-services
OverviewDiscussionsAdmin

Compare task readiness and evidence gaps

plan_benchmark
Read-onlyIdempotent

Plan a predefined benchmark case without running it. Read get_benchmark_suite for a caseId, then send requestJson containing {caseId,constraints?:{maxTaskCostUsd?,maxDurationMs?,requireMeasuredQuality?}}. Preserve every intended constraint inside requestJson; unknown and duplicate inner fields are rejected. Budget is a planning ceiling, never payment permission. Uses current published listings and exactly bound setup guides. Documented anonymous access is not a live health check, account check or guarantee that ToolSkipper has an executable adapter. Host accounts, quotas, task quality, duration and provider-billed cost remain unknown; constrained fit is unknown when measurements are absent. No ranking by invented performance, account connection, external call, automatic fallback or saved report.

Input Schema

NameRequiredDescriptionDefault
requestJsonYesComplete JSON planning request: {"caseId":"search-payment-flow-en","constraints":{"maxTaskCostUsd":0,"maxDurationMs":25000,"requireMeasuredQuality":true}}. Select a caseId from get_benchmark_suite. Optional constraints are hard requirements; unknown measurements cannot establish fit. maxTaskCostUsd: 0–1000; maxDurationMs: integer 1–300000. No execution, credentials, custom URLs, payment or account checks.

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "additionalProperties": false,
-    "properties": {
-      "canExecute": {
-        "const": false,
-        "type": "boolean"
-      },
-      "candidates": {
-        "items": {
-          "additionalProperties": false,
-          "properties": {
-            "accountStatus": {
-              "const": "unknown",
-              "type": "string"
-            },
-            "constraintChecks": {
-              "items": {
-                "additionalProperties": false,
-                "properties": {
-                  "constraint": {
-                    "enum": [
-                      "maxTaskCostUsd",
-                      "maxDurationMs",
-                      "requireMeasuredQuality"
-                    ],
-                    "type": "string"
-                  },
-                  "reason": {
-                    "type": "string"
-                  },
-                  "status": {
-                    "const": "unknown",
-                    "type": "string"
-                  }
-                },
-                "required": [
-                  "constraint",
-                  "status",
-                  "reason"
-                ],
-                "type": "object"
-              },
-              "maxItems": 3,
-              "type": "array"
-            },
-            "constraintFit": {
-              "enum": [
-                "not_requested",
-                "unknown",
-                "unavailable"
-              ],
-              "type": "string"
-            },
-            "liveAvailability": {
-              "const": "not_checked",
-              "type": "string"
-            },
-            "measurements": {
-              "additionalProperties": false,
-              "properties": {
-                "durationMs": {
-                  "type": "null"
-                },
-                "qualityScore": {
-                  "type": "null"
-                },
-                "sampleCount": {
-                  "const": 0,
-                  "type": "number"
-                },
-                "taskCostUsd": {
-                  "type": "null"
-                }
-              },
-              "required": [
-                "taskCostUsd",
-                "durationMs",
-                "qualityScore",
-                "sampleCount"
-              ],
-              "type": "object"
-            },
-            "name": {
-              "type": "string"
-            },
-            "nextSteps": {
-              "items": {
-                "type": "string"
-              },
-              "type": "array"
-            },
-            "provider": {
-              "type": "string"
-            },
-            "quotaStatus": {
-              "const": "unknown",
-              "type": "string"
-            },
-            "readiness": {
-              "enum": [
-                "anonymous_attempt_available",
-                "host_connection_required",
-                "unavailable"
-              ],
-              "type": "string"
-            },
-            "reasonCode": {
-              "type": "string"
-            },
-            "reasons": {
-              "items": {
-                "type": "string"
-              },
-              "type": "array"
-            },
-            "review": {
-              "additionalProperties": false,
-              "properties": {
-                "listingSnapshotMatched": {
-                  "type": "boolean"
-                },
-                "reviewedAt": {
-                  "type": [
-                    "string",
-                    "null"
-                  ]
-                }
-              },
-              "required": [
-                "reviewedAt",
-                "listingSnapshotMatched"
-              ],
-              "type": "object"
-            },
-            "serviceUrl": {
-              "type": "string"
-            },
-            "sources": {
-              "items": {
-                "additionalProperties": false,
-                "properties": {
-                  "checkedAt": {
-                    "type": "string"
-                  },
-                  "title": {
-                    "type": "string"
-                  },
-                  "url": {
-                    "format": "uri",
-                    "type": "string"
-                  }
-                },
-                "required": [
-                  "url",
-                  "title",
-                  "checkedAt"
-                ],
-                "type": "object"
-              },
-              "type": "array"
-            }
-          },
-          "required": [
-            "provider",
-            "name",
-            "serviceUrl",
-            "readiness",
-            "reasonCode",
-            "reasons",
-            "accountStatus",
-            "quotaStatus",
-            "liveAvailability",
-            "constraintFit",
-            "constraintChecks",
-            "measurements",
-            "review",
-            "sources",
-            "nextSteps"
-          ],
-          "type": "object"
-        },
-        "maxItems": 3,
-        "type": "array"
-      },
-      "case": {
-        "additionalProperties": false,
-        "properties": {
-          "cohortKey": {
-            "type": "string"
-          },
-          "family": {
-            "type": "string"
-          },
-          "id": {
-            "type": "string"
-          },
-          "input": {
-            "type": "string"
-          },
-          "instruction": {
-            "type": "string"
-          },
-          "language": {
-            "enum": [
-              "en",
-              "tr",
-              "de",
-              "es"
-            ],
-            "type": "string"
-          },
-          "rubric": {
-            "items": {
-              "additionalProperties": false,
-              "properties": {
-                "description": {
-                  "type": "string"
-                },
-                "id": {
-                  "type": "string"
-                }
-              },
-              "required": [
-                "id",
-                "description"
-              ],
-              "type": "object"
-            },
-            "minItems": 1,
-            "type": "array"
-          },
-          "sourceUrls": {
-            "items": {
-              "format": "uri",
-              "type": "string"
-            },
-            "minItems": 1,
-            "type": "array"
-          },
-          "task": {
-            "enum": [
-              "web_search",
-              "read_page"
-            ],
-            "type": "string"
-          },
-          "title": {
-            "type": "string"
-          }
-        },
-        "required": [
-          "id",
-          "family",
-          "language",
-          "task",
-          "title",
-          "instruction",
-          "input",
-          "sourceUrls",
-          "rubric",
-          "cohortKey"
-        ],
-        "type": "object"
-      },
-      "constraints": {
-        "additionalProperties": false,
-        "properties": {
-          "maxDurationMs": {
-            "maximum": 300000,
-            "minimum": 1,
-            "type": [
-              "integer",
-              "null"
-            ]
-          },
-          "maxTaskCostUsd": {
-            "maximum": 1000,
-            "minimum": 0,
-            "type": [
-              "number",
-              "null"
-            ]
-          },
-          "requireMeasuredQuality": {
-            "type": "boolean"
-          }
-        },
-        "required": [
-          "maxTaskCostUsd",
-          "maxDurationMs",
-          "requireMeasuredQuality"
-        ],
-        "type": "object"
-      },
-      "limitations": {
-        "items": {
-          "type": "string"
-        },
-        "type": "array"
-      },
-      "mode": {
-        "const": "planning_only",
-        "type": "string"
-      },
-      "schemaVersion": {
-        "const": "1",
-        "type": "string"
-      },
-      "suiteVersion": {
-        "const": "2026-10-12.1",
-        "type": "string"
-      }
-    },
-    "required": [
-      "schemaVersion",
-      "suiteVersion",
-      "mode",
-      "case",
-      "constraints",
-      "canExecute",
-      "candidates",
-      "limitations"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": false,
-    "properties": {
-      "error": {
-        "additionalProperties": false,
-        "properties": {
-          "code": {
-            "minLength": 1,
-            "type": "string"
-          },
-          "message": {
-            "type": "string"
-          }
-        },
-        "required": [
-          "code",
-          "message"
-        ],
-        "type": "object"
-      },
-      "error_code": {
-        "minLength": 1,
-        "type": "string"
-      }
-    },
-    "required": [
-      "error_code",
-      "error"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "canExecute": {
+        "const": false,
+        "type": "boolean"
+      },
+      "candidates": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "accountStatus": {
+              "const": "unknown",
+              "type": "string"
+            },
+            "constraintChecks": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "constraint": {
+                    "enum": [
+                      "maxTaskCostUsd",
+                      "maxDurationMs",
+                      "requireMeasuredQuality"
+                    ],
+                    "type": "string"
+                  },
+                  "reason": {
+                    "type": "string"
+                  },
+                  "status": {
+                    "const": "unknown",
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "constraint",
+                  "status",
+                  "reason"
+                ],
+                "type": "object"
+              },
+              "maxItems": 3,
+              "type": "array"
+            },
+            "constraintFit": {
+              "enum": [
+                "not_requested",
+                "unknown",
+                "unavailable"
+              ],
+              "type": "string"
+            },
+            "liveAvailability": {
+              "const": "not_checked",
+              "type": "string"
+            },
+            "measurements": {
+              "additionalProperties": false,
+              "properties": {
+                "durationMs": {
+                  "type": "null"
+                },
+                "qualityScore": {
+                  "type": "null"
+                },
+                "sampleCount": {
+                  "const": 0,
+                  "type": "number"
+                },
+                "taskCostUsd": {
+                  "type": "null"
+                }
+              },
+              "required": [
+                "taskCostUsd",
+                "durationMs",
+                "qualityScore",
+                "sampleCount"
+              ],
+              "type": "object"
+            },
+            "name": {
+              "type": "string"
+            },
+            "nextSteps": {
+              "items": {
+                "type": "string"
+              },
+              "type": "array"
+            },
+            "provider": {
+              "type": "string"
+            },
+            "quotaStatus": {
+              "const": "unknown",
+              "type": "string"
+            },
+            "readiness": {
+              "enum": [
+                "anonymous_attempt_available",
+                "host_connection_required",
+                "unavailable"
+              ],
+              "type": "string"
+            },
+            "reasonCode": {
+              "type": "string"
+            },
+            "reasons": {
+              "items": {
+                "type": "string"
+              },
+              "type": "array"
+            },
+            "review": {
+              "additionalProperties": false,
+              "properties": {
+                "listingSnapshotMatched": {
+                  "type": "boolean"
+                },
+                "reviewedAt": {
+                  "type": [
+                    "string",
+                    "null"
+                  ]
+                }
+              },
+              "required": [
+                "reviewedAt",
+                "listingSnapshotMatched"
+              ],
+              "type": "object"
+            },
+            "serviceUrl": {
+              "type": "string"
+            },
+            "sources": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "checkedAt": {
+                    "type": "string"
+                  },
+                  "title": {
+                    "type": "string"
+                  },
+                  "url": {
+                    "format": "uri",
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "url",
+                  "title",
+                  "checkedAt"
+                ],
+                "type": "object"
+              },
+              "type": "array"
+            }
+          },
+          "required": [
+            "provider",
+            "name",
+            "serviceUrl",
+            "readiness",
+            "reasonCode",
+            "reasons",
+            "accountStatus",
+            "quotaStatus",
+            "liveAvailability",
+            "constraintFit",
+            "constraintChecks",
+            "measurements",
+            "review",
+            "sources",
+            "nextSteps"
+          ],
+          "type": "object"
+        },
+        "maxItems": 3,
+        "type": "array"
+      },
+      "case": {
+        "additionalProperties": false,
+        "properties": {
+          "cohortKey": {
+            "type": "string"
+          },
+          "family": {
+            "type": "string"
+          },
+          "id": {
+            "type": "string"
+          },
+          "input": {
+            "type": "string"
+          },
+          "instruction": {
+            "type": "string"
+          },
+          "language": {
+            "enum": [
+              "en",
+              "tr",
+              "de",
+              "es"
+            ],
+            "type": "string"
+          },
+          "rubric": {
+            "items": {
+              "additionalProperties": false,
+              "properties": {
+                "description": {
+                  "type": "string"
+                },
+                "id": {
+                  "type": "string"
+                }
+              },
+              "required": [
+                "id",
+                "description"
+              ],
+              "type": "object"
+            },
+            "minItems": 1,
+            "type": "array"
+          },
+          "sourceUrls": {
+            "items": {
+              "format": "uri",
+              "type": "string"
+            },
+            "minItems": 1,
+            "type": "array"
+          },
+          "task": {
+            "enum": [
+              "web_search",
+              "read_page"
+            ],
+            "type": "string"
+          },
+          "title": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "id",
+          "family",
+          "language",
+          "task",
+          "title",
+          "instruction",
+          "input",
+          "sourceUrls",
+          "rubric",
+          "cohortKey"
+        ],
+        "type": "object"
+      },
+      "constraints": {
+        "additionalProperties": false,
+        "properties": {
+          "maxDurationMs": {
+            "maximum": 300000,
+            "minimum": 1,
+            "type": [
+              "integer",
+              "null"
+            ]
+          },
+          "maxTaskCostUsd": {
+            "maximum": 1000,
+            "minimum": 0,
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "requireMeasuredQuality": {
+            "type": "boolean"
+          }
+        },
+        "required": [
+          "maxTaskCostUsd",
+          "maxDurationMs",
+          "requireMeasuredQuality"
+        ],
+        "type": "object"
+      },
+      "limitations": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "mode": {
+        "const": "planning_only",
+        "type": "string"
+      },
+      "schemaVersion": {
+        "const": "1",
+        "type": "string"
+      },
+      "suiteVersion": {
+        "const": "2026-10-12.1",
+        "type": "string"
+      }
+    },
+    "required": [
+      "schemaVersion",
+      "suiteVersion",
+      "mode",
+      "case",
+      "constraints",
+      "canExecute",
+      "candidates",
+      "limitations"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "error": {
+        "additionalProperties": false,
+        "properties": {
+          "code": {
+            "minLength": 1,
+            "type": "string"
+          },
+          "message": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "code",
+          "message"
+        ],
+        "type": "object"
+      },
+      "error_code": {
+        "minLength": 1,
+        "type": "string"
+      },
+      "recovery": {
+        "additionalProperties": false,
+        "properties": {
+          "automaticRetryAllowed": {
+            "const": false
+          },
+          "guidance": {
+            "type": "string"
+          },
+          "requiredArguments": {
+            "items": {
+              "type": "string"
+            },
+            "type": "array"
+          },
+          "tool": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "tool",
+          "requiredArguments",
+          "guidance",
+          "automaticRetryAllowed"
+        ],
+        "type": "object"
+      }
+    },
+    "required": [
+      "error_code",
+      "error"
+    ],
+    "type": "object"
+  }
+]
  2. Changed4 schema fields changed
    • removedOutput schema / additionalProperties
      Removed value: -false
    • addedOutput schema / anyOf
      Added value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "canExecute": {
+        "const": false,
+        "type": "boolean"
+      },
+      "candidates": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "accountStatus": {
+              "const": "unknown",
+              "type": "string"
+            },
+            "constraintChecks": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "constraint": {
+                    "enum": [
+                      "maxTaskCostUsd",
+                      "maxDurationMs",
+                      "requireMeasuredQuality"
+                    ],
+                    "type": "string"
+                  },
+                  "reason": {
+                    "type": "string"
+                  },
+                  "status": {
+                    "const": "unknown",
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "constraint",
+                  "status",
+                  "reason"
+                ],
+                "type": "object"
+              },
+              "maxItems": 3,
+              "type": "array"
+            },
+            "constraintFit": {
+              "enum": [
+                "not_requested",
+                "unknown",
+                "unavailable"
+              ],
+              "type": "string"
+            },
+            "liveAvailability": {
+              "const": "not_checked",
+              "type": "string"
+            },
+            "measurements": {
+              "additionalProperties": false,
+              "properties": {
+                "durationMs": {
+                  "type": "null"
+                },
+                "qualityScore": {
+                  "type": "null"
+                },
+                "sampleCount": {
+                  "const": 0,
+                  "type": "number"
+                },
+                "taskCostUsd": {
+                  "type": "null"
+                }
+              },
+              "required": [
+                "taskCostUsd",
+                "durationMs",
+                "qualityScore",
+                "sampleCount"
+              ],
+              "type": "object"
+            },
+            "name": {
+              "type": "string"
+            },
+            "nextSteps": {
+              "items": {
+                "type": "string"
+              },
+              "type": "array"
+            },
+            "provider": {
+              "type": "string"
+            },
+            "quotaStatus": {
+              "const": "unknown",
+              "type": "string"
+            },
+            "readiness": {
+              "enum": [
+                "anonymous_attempt_available",
+                "host_connection_required",
+                "unavailable"
+              ],
+              "type": "string"
+            },
+            "reasonCode": {
+              "type": "string"
+            },
+            "reasons": {
+              "items": {
+                "type": "string"
+              },
+              "type": "array"
+            },
+            "review": {
+              "additionalProperties": false,
+              "properties": {
+                "listingSnapshotMatched": {
+                  "type": "boolean"
+                },
+                "reviewedAt": {
+                  "type": [
+                    "string",
+                    "null"
+                  ]
+                }
+              },
+              "required": [
+                "reviewedAt",
+                "listingSnapshotMatched"
+              ],
+              "type": "object"
+            },
+            "serviceUrl": {
+              "type": "string"
+            },
+            "sources": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "checkedAt": {
+                    "type": "string"
+                  },
+                  "title": {
+                    "type": "string"
+                  },
+                  "url": {
+                    "format": "uri",
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "url",
+                  "title",
+                  "checkedAt"
+                ],
+                "type": "object"
+              },
+              "type": "array"
+            }
+          },
+          "required": [
+            "provider",
+            "name",
+            "serviceUrl",
+            "readiness",
+            "reasonCode",
+            "reasons",
+            "accountStatus",
+            "quotaStatus",
+            "liveAvailability",
+            "constraintFit",
+            "constraintChecks",
+            "measurements",
+            "review",
+            "sources",
+            "nextSteps"
+          ],
+          "type": "object"
+        },
+        "maxItems": 3,
+        "type": "array"
+      },
+      "case": {
+        "additionalProperties": false,
+        "properties": {
+          "cohortKey": {
+            "type": "string"
+          },
+          "family": {
+            "type": "string"
+          },
+          "id": {
+            "type": "string"
+          },
+          "input": {
+            "type": "string"
+          },
+          "instruction": {
+            "type": "string"
+          },
+          "language": {
+            "enum": [
+              "en",
+              "tr",
+              "de",
+              "es"
+            ],
+            "type": "string"
+          },
+          "rubric": {
+            "items": {
+              "additionalProperties": false,
+              "properties": {
+                "description": {
+                  "type": "string"
+                },
+                "id": {
+                  "type": "string"
+                }
+              },
+              "required": [
+                "id",
+                "description"
+              ],
+              "type": "object"
+            },
+            "minItems": 1,
+            "type": "array"
+          },
+          "sourceUrls": {
+            "items": {
+              "format": "uri",
+              "type": "string"
+            },
+            "minItems": 1,
+            "type": "array"
+          },
+          "task": {
+            "enum": [
+              "web_search",
+              "read_page"
+            ],
+            "type": "string"
+          },
+          "title": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "id",
+          "family",
+          "language",
+          "task",
+          "title",
+          "instruction",
+          "input",
+          "sourceUrls",
+          "rubric",
+          "cohortKey"
+        ],
+        "type": "object"
+      },
+      "constraints": {
+        "additionalProperties": false,
+        "properties": {
+          "maxDurationMs": {
+            "maximum": 300000,
+            "minimum": 1,
+            "type": [
+              "integer",
+              "null"
+            ]
+          },
+          "maxTaskCostUsd": {
+            "maximum": 1000,
+            "minimum": 0,
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "requireMeasuredQuality": {
+            "type": "boolean"
+          }
+        },
+        "required": [
+          "maxTaskCostUsd",
+          "maxDurationMs",
+          "requireMeasuredQuality"
+        ],
+        "type": "object"
+      },
+      "limitations": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "mode": {
+        "const": "planning_only",
+        "type": "string"
+      },
+      "schemaVersion": {
+        "const": "1",
+        "type": "string"
+      },
+      "suiteVersion": {
+        "const": "2026-10-12.1",
+        "type": "string"
+      }
+    },
+    "required": [
+      "schemaVersion",
+      "suiteVersion",
+      "mode",
+      "case",
+      "constraints",
+      "canExecute",
+      "candidates",
+      "limitations"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "error": {
+        "additionalProperties": false,
+        "properties": {
+          "code": {
+            "minLength": 1,
+            "type": "string"
+          },
+          "message": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "code",
+          "message"
+        ],
+        "type": "object"
+      },
+      "error_code": {
+        "minLength": 1,
+        "type": "string"
+      }
+    },
+    "required": [
+      "error_code",
+      "error"
+    ],
+    "type": "object"
+  }
+]
    • removedOutput schema / properties
      Removed value: -{
-  "canExecute": {
-    "const": false,
-    "type": "boolean"
-  },
-  "candidates": {
-    "items": {
-      "additionalProperties": false,
-      "properties": {
-        "accountStatus": {
-          "const": "unknown",
-          "type": "string"
-        },
-        "constraintChecks": {
-          "items": {
-            "additionalProperties": false,
-            "properties": {
-              "constraint": {
-                "enum": [
-                  "maxTaskCostUsd",
-                  "maxDurationMs",
-                  "requireMeasuredQuality"
-                ],
-                "type": "string"
-              },
-              "reason": {
-                "type": "string"
-              },
-              "status": {
-                "const": "unknown",
-                "type": "string"
-              }
-            },
-            "required": [
-              "constraint",
-              "status",
-              "reason"
-            ],
-            "type": "object"
-          },
-          "maxItems": 3,
-          "type": "array"
-        },
-        "constraintFit": {
-          "enum": [
-            "not_requested",
-            "unknown",
-            "unavailable"
-          ],
-          "type": "string"
-        },
-        "liveAvailability": {
-          "const": "not_checked",
-          "type": "string"
-        },
-        "measurements": {
-          "additionalProperties": false,
-          "properties": {
-            "durationMs": {
-              "type": "null"
-            },
-            "qualityScore": {
-              "type": "null"
-            },
-            "sampleCount": {
-              "const": 0,
-              "type": "number"
-            },
-            "taskCostUsd": {
-              "type": "null"
-            }
-          },
-          "required": [
-            "taskCostUsd",
-            "durationMs",
-            "qualityScore",
-            "sampleCount"
-          ],
-          "type": "object"
-        },
-        "name": {
-          "type": "string"
-        },
-        "nextSteps": {
-          "items": {
-            "type": "string"
-          },
-          "type": "array"
-        },
-        "provider": {
-          "type": "string"
-        },
-        "quotaStatus": {
-          "const": "unknown",
-          "type": "string"
-        },
-        "readiness": {
-          "enum": [
-            "anonymous_attempt_available",
-            "host_connection_required",
-            "unavailable"
-          ],
-          "type": "string"
-        },
-        "reasonCode": {
-          "type": "string"
-        },
-        "reasons": {
-          "items": {
-            "type": "string"
-          },
-          "type": "array"
-        },
-        "review": {
-          "additionalProperties": false,
-          "properties": {
-            "listingSnapshotMatched": {
-              "type": "boolean"
-            },
-            "reviewedAt": {
-              "type": [
-                "string",
-                "null"
-              ]
-            }
-          },
-          "required": [
-            "reviewedAt",
-            "listingSnapshotMatched"
-          ],
-          "type": "object"
-        },
-        "serviceUrl": {
-          "type": "string"
-        },
-        "sources": {
-          "items": {
-            "additionalProperties": false,
-            "properties": {
-              "checkedAt": {
-                "type": "string"
-              },
-              "title": {
-                "type": "string"
-              },
-              "url": {
-                "format": "uri",
-                "type": "string"
-              }
-            },
-            "required": [
-              "url",
-              "title",
-              "checkedAt"
-            ],
-            "type": "object"
-          },
-          "type": "array"
-        }
-      },
-      "required": [
-        "provider",
-        "name",
-        "serviceUrl",
-        "readiness",
-        "reasonCode",
-        "reasons",
-        "accountStatus",
-        "quotaStatus",
-        "liveAvailability",
-        "constraintFit",
-        "constraintChecks",
-        "measurements",
-        "review",
-        "sources",
-        "nextSteps"
-      ],
-      "type": "object"
-    },
-    "maxItems": 3,
-    "type": "array"
-  },
-  "case": {
-    "additionalProperties": false,
-    "properties": {
-      "cohortKey": {
-        "type": "string"
-      },
-      "family": {
-        "type": "string"
-      },
-      "id": {
-        "type": "string"
-      },
-      "input": {
-        "type": "string"
-      },
-      "instruction": {
-        "type": "string"
-      },
-      "language": {
-        "enum": [
-          "en",
-          "tr",
-          "de",
-          "es"
-        ],
-        "type": "string"
-      },
-      "rubric": {
-        "items": {
-          "additionalProperties": false,
-          "properties": {
-            "description": {
-              "type": "string"
-            },
-            "id": {
-              "type": "string"
-            }
-          },
-          "required": [
-            "id",
-            "description"
-          ],
-          "type": "object"
-        },
-        "minItems": 1,
-        "type": "array"
-      },
-      "sourceUrls": {
-        "items": {
-          "format": "uri",
-          "type": "string"
-        },
-        "minItems": 1,
-        "type": "array"
-      },
-      "task": {
-        "enum": [
-          "web_search",
-          "read_page"
-        ],
-        "type": "string"
-      },
-      "title": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "id",
-      "family",
-      "language",
-      "task",
-      "title",
-      "instruction",
-      "input",
-      "sourceUrls",
-      "rubric",
-      "cohortKey"
-    ],
-    "type": "object"
-  },
-  "constraints": {
-    "additionalProperties": false,
-    "properties": {
-      "maxDurationMs": {
-        "maximum": 300000,
-        "minimum": 1,
-        "type": [
-          "integer",
-          "null"
-        ]
-      },
-      "maxTaskCostUsd": {
-        "maximum": 1000,
-        "minimum": 0,
-        "type": [
-          "number",
-          "null"
-        ]
-      },
-      "requireMeasuredQuality": {
-        "type": "boolean"
-      }
-    },
-    "required": [
-      "maxTaskCostUsd",
-      "maxDurationMs",
-      "requireMeasuredQuality"
-    ],
-    "type": "object"
-  },
-  "limitations": {
-    "items": {
-      "type": "string"
-    },
-    "type": "array"
-  },
-  "mode": {
-    "const": "planning_only",
-    "type": "string"
-  },
-  "schemaVersion": {
-    "const": "1",
-    "type": "string"
-  },
-  "suiteVersion": {
-    "const": "2026-10-12.1",
-    "type": "string"
-  }
-}
    • removedOutput schema / required
      Removed value: -[
-  "schemaVersion",
-  "suiteVersion",
-  "mode",
-  "case",
-  "constraints",
-  "canExecute",
-  "candidates",
-  "limitations"
-]
  3. First observed

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: budget is a planning ceiling and never payment permission, unknown/duplicate inner fields are rejected, unspecified measurements make constrained fit unknown, and nothing is persisted. These are the exact behavioral facts an agent needs before trusting the result, and they aren't in the readOnly/idempotent hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and workflow are front-loaded, which is good, but the back half is a dense run of defensive negations ("not a live health check, account check or guarantee...", "no ranking by invented performance, account connection, external call...") that partly restate the schema's no-execution clause and the openWorldHint annotation. It remains readable but several clauses could be compressed without losing meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with an output schema, the description covers the workflow, the strict JSON contract, the meaning of constraints, and the known unknowns (cost, duration, quotas, adapter existence). Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so the baseline is 3, but the description adds real semantics: constraints inside requestJson must be preserved verbatim, unknown and duplicate inner fields are rejected, and the top-level shape mirrors the schema example. That strictness contract is not stated in the schema itself.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Plan a predefined benchmark case") and immediately bounds scope with "without running it," which separates it from execution-style siblings. It also points at get_benchmark_suite as the source of the caseId, so an agent can place it in the tool graph without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit prerequisite workflow: read get_benchmark_suite for a caseId, then call this with requestJson. It also states the negative boundary (planning, not running, not ranking, not an account/health check). It stops short of naming a concrete alternative tool for the cases where you'd want comparison instead (e.g. compare_services).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Relay — agent service discovery toolsInspect
Try in Browser
Context Every Agent Can ShareAgents share what they learn, hand off work, and keep it when you switch models. Start with 5 GB free.fulcradynamics.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources