Skip to main content
Glama
Sign In

Relay — agent service discovery

by site.chatgpt.orellise.relay-agent-services
OverviewDiscussionsAdmin

Read the task benchmark suite

get_benchmark_suite
Read-onlyIdempotent

Read 20 predefined public-documentation task cases across English, Turkish, German and Spanish, with source URLs and review rubrics. Five task families have four language variants; this is a test design, not 20 independently tested capabilities. The suite publishes no shared live evidence, quality rankings or measured costs. Private receipts are available separately through the authenticated benchmark run tools. Use plan_benchmark to inspect candidate access and hard-constraint gaps. No providers, accounts, payments or writes.

Input Schema

NameRequiredDescriptionDefault

No arguments

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "additionalProperties": false,
-    "properties": {
-      "cases": {
-        "items": {
-          "additionalProperties": false,
-          "properties": {
-            "cohortKey": {
-              "type": "string"
-            },
-            "family": {
-              "type": "string"
-            },
-            "id": {
-              "type": "string"
-            },
-            "input": {
-              "type": "string"
-            },
-            "instruction": {
-              "type": "string"
-            },
-            "language": {
-              "enum": [
-                "en",
-                "tr",
-                "de",
-                "es"
-              ],
-              "type": "string"
-            },
-            "rubric": {
-              "items": {
-                "additionalProperties": false,
-                "properties": {
-                  "description": {
-                    "type": "string"
-                  },
-                  "id": {
-                    "type": "string"
-                  }
-                },
-                "required": [
-                  "id",
-                  "description"
-                ],
-                "type": "object"
-              },
-              "minItems": 1,
-              "type": "array"
-            },
-            "sourceUrls": {
-              "items": {
-                "format": "uri",
-                "type": "string"
-              },
-              "minItems": 1,
-              "type": "array"
-            },
-            "task": {
-              "enum": [
-                "web_search",
-                "read_page"
-              ],
-              "type": "string"
-            },
-            "title": {
-              "type": "string"
-            }
-          },
-          "required": [
-            "id",
-            "family",
-            "language",
-            "task",
-            "title",
-            "instruction",
-            "input",
-            "sourceUrls",
-            "rubric",
-            "cohortKey"
-          ],
-          "type": "object"
-        },
-        "maxItems": 20,
-        "minItems": 20,
-        "type": "array"
-      },
-      "description": {
-        "type": "string"
-      },
-      "evidence": {
-        "additionalProperties": false,
-        "properties": {
-          "independentlyReviewedCount": {
-            "const": 0,
-            "type": "number"
-          },
-          "liveRunCount": {
-            "const": 0,
-            "type": "number"
-          },
-          "measuredCostUsd": {
-            "type": "null"
-          }
-        },
-        "required": [
-          "liveRunCount",
-          "independentlyReviewedCount",
-          "measuredCostUsd"
-        ],
-        "type": "object"
-      },
-      "languages": {
-        "items": {
-          "enum": [
-            "en",
-            "tr",
-            "de",
-            "es"
-          ],
-          "type": "string"
-        },
-        "maxItems": 4,
-        "minItems": 4,
-        "type": "array",
-        "uniqueItems": true
-      },
-      "limitations": {
-        "items": {
-          "type": "string"
-        },
-        "type": "array"
-      },
-      "schemaVersion": {
-        "const": "1",
-        "type": "string"
-      },
-      "suiteVersion": {
-        "const": "2026-10-12.1",
-        "type": "string"
-      },
-      "title": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "schemaVersion",
-      "suiteVersion",
-      "title",
-      "description",
-      "languages",
-      "cases",
-      "evidence",
-      "limitations"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": false,
-    "properties": {
-      "error": {
-        "additionalProperties": false,
-        "properties": {
-          "code": {
-            "minLength": 1,
-            "type": "string"
-          },
-          "message": {
-            "type": "string"
-          }
-        },
-        "required": [
-          "code",
-          "message"
-        ],
-        "type": "object"
-      },
-      "error_code": {
-        "minLength": 1,
-        "type": "string"
-      }
-    },
-    "required": [
-      "error_code",
-      "error"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "cases": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "cohortKey": {
+              "type": "string"
+            },
+            "family": {
+              "type": "string"
+            },
+            "id": {
+              "type": "string"
+            },
+            "input": {
+              "type": "string"
+            },
+            "instruction": {
+              "type": "string"
+            },
+            "language": {
+              "enum": [
+                "en",
+                "tr",
+                "de",
+                "es"
+              ],
+              "type": "string"
+            },
+            "rubric": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "description": {
+                    "type": "string"
+                  },
+                  "id": {
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "id",
+                  "description"
+                ],
+                "type": "object"
+              },
+              "minItems": 1,
+              "type": "array"
+            },
+            "sourceUrls": {
+              "items": {
+                "format": "uri",
+                "type": "string"
+              },
+              "minItems": 1,
+              "type": "array"
+            },
+            "task": {
+              "enum": [
+                "web_search",
+                "read_page"
+              ],
+              "type": "string"
+            },
+            "title": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "id",
+            "family",
+            "language",
+            "task",
+            "title",
+            "instruction",
+            "input",
+            "sourceUrls",
+            "rubric",
+            "cohortKey"
+          ],
+          "type": "object"
+        },
+        "maxItems": 20,
+        "minItems": 20,
+        "type": "array"
+      },
+      "description": {
+        "type": "string"
+      },
+      "evidence": {
+        "additionalProperties": false,
+        "properties": {
+          "independentlyReviewedCount": {
+            "const": 0,
+            "type": "number"
+          },
+          "liveRunCount": {
+            "const": 0,
+            "type": "number"
+          },
+          "measuredCostUsd": {
+            "type": "null"
+          }
+        },
+        "required": [
+          "liveRunCount",
+          "independentlyReviewedCount",
+          "measuredCostUsd"
+        ],
+        "type": "object"
+      },
+      "languages": {
+        "items": {
+          "enum": [
+            "en",
+            "tr",
+            "de",
+            "es"
+          ],
+          "type": "string"
+        },
+        "maxItems": 4,
+        "minItems": 4,
+        "type": "array",
+        "uniqueItems": true
+      },
+      "limitations": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "schemaVersion": {
+        "const": "1",
+        "type": "string"
+      },
+      "suiteVersion": {
+        "const": "2026-10-12.1",
+        "type": "string"
+      },
+      "title": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "schemaVersion",
+      "suiteVersion",
+      "title",
+      "description",
+      "languages",
+      "cases",
+      "evidence",
+      "limitations"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "error": {
+        "additionalProperties": false,
+        "properties": {
+          "code": {
+            "minLength": 1,
+            "type": "string"
+          },
+          "message": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "code",
+          "message"
+        ],
+        "type": "object"
+      },
+      "error_code": {
+        "minLength": 1,
+        "type": "string"
+      },
+      "recovery": {
+        "additionalProperties": false,
+        "properties": {
+          "automaticRetryAllowed": {
+            "const": false
+          },
+          "guidance": {
+            "type": "string"
+          },
+          "requiredArguments": {
+            "items": {
+              "type": "string"
+            },
+            "type": "array"
+          },
+          "tool": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "tool",
+          "requiredArguments",
+          "guidance",
+          "automaticRetryAllowed"
+        ],
+        "type": "object"
+      }
+    },
+    "required": [
+      "error_code",
+      "error"
+    ],
+    "type": "object"
+  }
+]
  2. Changed4 schema fields changed
    • removedOutput schema / additionalProperties
      Removed value: -false
    • addedOutput schema / anyOf
      Added value: +[
+  {
+    "additionalProperties": false,
+    "properties": {
+      "cases": {
+        "items": {
+          "additionalProperties": false,
+          "properties": {
+            "cohortKey": {
+              "type": "string"
+            },
+            "family": {
+              "type": "string"
+            },
+            "id": {
+              "type": "string"
+            },
+            "input": {
+              "type": "string"
+            },
+            "instruction": {
+              "type": "string"
+            },
+            "language": {
+              "enum": [
+                "en",
+                "tr",
+                "de",
+                "es"
+              ],
+              "type": "string"
+            },
+            "rubric": {
+              "items": {
+                "additionalProperties": false,
+                "properties": {
+                  "description": {
+                    "type": "string"
+                  },
+                  "id": {
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "id",
+                  "description"
+                ],
+                "type": "object"
+              },
+              "minItems": 1,
+              "type": "array"
+            },
+            "sourceUrls": {
+              "items": {
+                "format": "uri",
+                "type": "string"
+              },
+              "minItems": 1,
+              "type": "array"
+            },
+            "task": {
+              "enum": [
+                "web_search",
+                "read_page"
+              ],
+              "type": "string"
+            },
+            "title": {
+              "type": "string"
+            }
+          },
+          "required": [
+            "id",
+            "family",
+            "language",
+            "task",
+            "title",
+            "instruction",
+            "input",
+            "sourceUrls",
+            "rubric",
+            "cohortKey"
+          ],
+          "type": "object"
+        },
+        "maxItems": 20,
+        "minItems": 20,
+        "type": "array"
+      },
+      "description": {
+        "type": "string"
+      },
+      "evidence": {
+        "additionalProperties": false,
+        "properties": {
+          "independentlyReviewedCount": {
+            "const": 0,
+            "type": "number"
+          },
+          "liveRunCount": {
+            "const": 0,
+            "type": "number"
+          },
+          "measuredCostUsd": {
+            "type": "null"
+          }
+        },
+        "required": [
+          "liveRunCount",
+          "independentlyReviewedCount",
+          "measuredCostUsd"
+        ],
+        "type": "object"
+      },
+      "languages": {
+        "items": {
+          "enum": [
+            "en",
+            "tr",
+            "de",
+            "es"
+          ],
+          "type": "string"
+        },
+        "maxItems": 4,
+        "minItems": 4,
+        "type": "array",
+        "uniqueItems": true
+      },
+      "limitations": {
+        "items": {
+          "type": "string"
+        },
+        "type": "array"
+      },
+      "schemaVersion": {
+        "const": "1",
+        "type": "string"
+      },
+      "suiteVersion": {
+        "const": "2026-10-12.1",
+        "type": "string"
+      },
+      "title": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "schemaVersion",
+      "suiteVersion",
+      "title",
+      "description",
+      "languages",
+      "cases",
+      "evidence",
+      "limitations"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": false,
+    "properties": {
+      "error": {
+        "additionalProperties": false,
+        "properties": {
+          "code": {
+            "minLength": 1,
+            "type": "string"
+          },
+          "message": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "code",
+          "message"
+        ],
+        "type": "object"
+      },
+      "error_code": {
+        "minLength": 1,
+        "type": "string"
+      }
+    },
+    "required": [
+      "error_code",
+      "error"
+    ],
+    "type": "object"
+  }
+]
    • removedOutput schema / properties
      Removed value: -{
-  "cases": {
-    "items": {
-      "additionalProperties": false,
-      "properties": {
-        "cohortKey": {
-          "type": "string"
-        },
-        "family": {
-          "type": "string"
-        },
-        "id": {
-          "type": "string"
-        },
-        "input": {
-          "type": "string"
-        },
-        "instruction": {
-          "type": "string"
-        },
-        "language": {
-          "enum": [
-            "en",
-            "tr",
-            "de",
-            "es"
-          ],
-          "type": "string"
-        },
-        "rubric": {
-          "items": {
-            "additionalProperties": false,
-            "properties": {
-              "description": {
-                "type": "string"
-              },
-              "id": {
-                "type": "string"
-              }
-            },
-            "required": [
-              "id",
-              "description"
-            ],
-            "type": "object"
-          },
-          "minItems": 1,
-          "type": "array"
-        },
-        "sourceUrls": {
-          "items": {
-            "format": "uri",
-            "type": "string"
-          },
-          "minItems": 1,
-          "type": "array"
-        },
-        "task": {
-          "enum": [
-            "web_search",
-            "read_page"
-          ],
-          "type": "string"
-        },
-        "title": {
-          "type": "string"
-        }
-      },
-      "required": [
-        "id",
-        "family",
-        "language",
-        "task",
-        "title",
-        "instruction",
-        "input",
-        "sourceUrls",
-        "rubric",
-        "cohortKey"
-      ],
-      "type": "object"
-    },
-    "maxItems": 20,
-    "minItems": 20,
-    "type": "array"
-  },
-  "description": {
-    "type": "string"
-  },
-  "evidence": {
-    "additionalProperties": false,
-    "properties": {
-      "independentlyReviewedCount": {
-        "const": 0,
-        "type": "number"
-      },
-      "liveRunCount": {
-        "const": 0,
-        "type": "number"
-      },
-      "measuredCostUsd": {
-        "type": "null"
-      }
-    },
-    "required": [
-      "liveRunCount",
-      "independentlyReviewedCount",
-      "measuredCostUsd"
-    ],
-    "type": "object"
-  },
-  "languages": {
-    "items": {
-      "enum": [
-        "en",
-        "tr",
-        "de",
-        "es"
-      ],
-      "type": "string"
-    },
-    "maxItems": 4,
-    "minItems": 4,
-    "type": "array",
-    "uniqueItems": true
-  },
-  "limitations": {
-    "items": {
-      "type": "string"
-    },
-    "type": "array"
-  },
-  "schemaVersion": {
-    "const": "1",
-    "type": "string"
-  },
-  "suiteVersion": {
-    "const": "2026-10-12.1",
-    "type": "string"
-  },
-  "title": {
-    "type": "string"
-  }
-}
    • removedOutput schema / required
      Removed value: -[
-  "schemaVersion",
-  "suiteVersion",
-  "title",
-  "description",
-  "languages",
-  "cases",
-  "evidence",
-  "limitations"
-]
  3. First observed

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description adds substantial new context: what the suite deliberately does not publish (shared live evidence, quality rankings, measured costs), that no providers/accounts/payments/writes are touched, and that the 20 cases are a design rather than independent tests. That negative-space disclosure directly prevents agents from over-claiming results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Five tight sentences, front-loaded with what is returned and immediately followed by the scope caveat. No filler, no restatement of the title, and each clause (languages, families, exclusions, alternatives) carries distinct decision-relevant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation; what remains — scope, non-coverage, and where adjacent needs live — is fully covered. For a parameterless read of a static fixture, nothing an agent needs before calling is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so there is nothing for the description to disambiguate and the baseline applies. The description spends its words on scope and non-coverage instead, which is the right allocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Read 20 predefined public-documentation task cases') plus the exact contents: languages, source URLs, review rubrics. It also pre-empts a likely misreading ('this is a test design, not 20 independently tested capabilities'), which no sibling name could convey.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes two adjacent needs elsewhere: plan_benchmark for candidate access and hard-constraint gaps, and the authenticated benchmark run tools for private receipts. Clear context with exclusions, though it never states the positive trigger condition (e.g. 'call this when the user asks what the benchmark measures').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Relay — agent service discovery toolsInspect
Try in Browser
Context Every Agent Can ShareAgents share what they learn, hand off work, and keep it when you switch models. Start with 5 GB free.fulcradynamics.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources