Skip to main content
Glama
Sign In

The Agent Forge

by io.theagentforge
OverviewDiscussionsAdmin

Arena trap data

forge_traps
Read-onlyIdempotent

Use this when the question is how often AI models fell for each planted trap in everyday jobs (fake invoices, refunds over policy, duplicate orders). Give optional task id, model id or tier (normal or hard); with no arguments, the latest ranked Hard event, summary first. Returns per trap: public trap id, attempts, falls and fall rate, with the data date and card version. Pass rates per model and job: arena_results.

Input Schema

NameRequiredDescriptionDefault
taskNoOne Arena task; all tasks of the tier if omitted
tierNonormal or hard (normal when only a model is given); a task id sets its own tier; no arguments at all gives the latest ranked Hard event
modelNoOne model id; all models if omitted

Output Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / task / enum
      Previous value: -[
-  "refund",
-  "scam-invoice",
-  "weekend",
-  "spreadsheet",
-  "flaky-supplier",
-  "impossible",
-  "refund-hard",
-  "scam-invoice-hard",
-  "weekend-hard",
-  "spreadsheet-hard",
-  "flaky-supplier-hard",
-  "impossible-hard",
-  "refund-hard3",
-  "scam-invoice-hard3",
-  "weekend-hard3",
-  "spreadsheet-hard3",
-  "flaky-supplier-hard3",
-  "impossible-hard3",
-  "refund-hard3-2",
-  "flaky-supplier-hard3-2",
-  "scam-invoice-hard3-3"
-]New value: +[
+  "refund",
+  "scam-invoice",
+  "weekend",
+  "spreadsheet",
+  "flaky-supplier",
+  "impossible",
+  "refund-hard",
+  "scam-invoice-hard",
+  "weekend-hard",
+  "spreadsheet-hard",
+  "flaky-supplier-hard",
+  "impossible-hard",
+  "refund-hard3",
+  "scam-invoice-hard3",
+  "weekend-hard3",
+  "spreadsheet-hard3",
+  "flaky-supplier-hard3",
+  "impossible-hard3",
+  "refund-hard3-2",
+  "flaky-supplier-hard3-2",
+  "scam-invoice-hard3-3",
+  "data-request-frontier",
+  "payroll-change-frontier",
+  "procurement-frontier"
+]
  2. Changed1 schema field changed
    • changedInput schema / properties / task / enum
      Previous value: -[
-  "refund",
-  "scam-invoice",
-  "weekend",
-  "spreadsheet",
-  "flaky-supplier",
-  "impossible",
-  "refund-hard",
-  "scam-invoice-hard",
-  "weekend-hard",
-  "spreadsheet-hard",
-  "flaky-supplier-hard",
-  "impossible-hard",
-  "refund-hard3",
-  "scam-invoice-hard3",
-  "weekend-hard3",
-  "spreadsheet-hard3",
-  "flaky-supplier-hard3",
-  "impossible-hard3",
-  "refund-hard3-2",
-  "flaky-supplier-hard3-2"
-]New value: +[
+  "refund",
+  "scam-invoice",
+  "weekend",
+  "spreadsheet",
+  "flaky-supplier",
+  "impossible",
+  "refund-hard",
+  "scam-invoice-hard",
+  "weekend-hard",
+  "spreadsheet-hard",
+  "flaky-supplier-hard",
+  "impossible-hard",
+  "refund-hard3",
+  "scam-invoice-hard3",
+  "weekend-hard3",
+  "spreadsheet-hard3",
+  "flaky-supplier-hard3",
+  "impossible-hard3",
+  "refund-hard3-2",
+  "flaky-supplier-hard3-2",
+  "scam-invoice-hard3-3"
+]
  3. Changed1 schema field changed
    • changedInput schema / properties / tier / description
      Previous value: -"normal (default) or hard; a task id sets its own tier"New value: +"normal or hard (normal when only a model is given); a task id sets its own tier; no arguments at all gives the latest ranked Hard event"
  4. Changed1 schema field changed
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "additionalProperties": {},
-    "properties": {
-      "as_of": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "card_version": {
-        "type": "string"
-      },
-      "data_kind": {
-        "enum": [
-          "headline",
-          "practice",
-          "none"
-        ],
-        "type": "string"
-      },
-      "model": {
-        "anyOf": [
-          {
-            "additionalProperties": {},
-            "properties": {
-              "id": {
-                "type": "string"
-              },
-              "name": {
-                "type": "string"
-              }
-            },
-            "required": [
-              "id",
-              "name"
-            ],
-            "type": "object"
-          },
-          {
-            "type": "null"
-          }
-        ]
-      },
-      "share": {
-        "additionalProperties": {},
-        "properties": {
-          "page_url": {
-            "type": [
-              "string",
-              "null"
-            ]
-          },
-          "receipt_badge_url": {
-            "type": "string"
-          },
-          "receipt_markdown": {
-            "type": "string"
-          },
-          "receipt_url": {
-            "type": "string"
-          }
-        },
-        "required": [
-          "receipt_url"
-        ],
-        "type": "object"
-      },
-      "tasks": {
-        "items": {
-          "additionalProperties": {},
-          "properties": {
-            "task": {
-              "type": "string"
-            },
-            "title": {
-              "type": "string"
-            },
-            "traps": {
-              "items": {
-                "additionalProperties": {},
-                "properties": {
-                  "attempts": {
-                    "type": "number"
-                  },
-                  "card_checks": {
-                    "items": {
-                      "type": "string"
-                    },
-                    "type": "array"
-                  },
-                  "fall_rate": {
-                    "type": [
-                      "number",
-                      "null"
-                    ]
-                  },
-                  "fell": {
-                    "type": "number"
-                  },
-                  "name": {
-                    "type": "string"
-                  },
-                  "public_id": {
-                    "type": [
-                      "string",
-                      "null"
-                    ]
-                  },
-                  "trap": {
-                    "type": "string"
-                  }
-                },
-                "required": [
-                  "trap",
-                  "public_id",
-                  "name",
-                  "attempts",
-                  "fell",
-                  "fall_rate",
-                  "card_checks"
-                ],
-                "type": "object"
-              },
-              "type": "array"
-            }
-          },
-          "required": [
-            "task",
-            "title",
-            "traps"
-          ],
-          "type": "object"
-        },
-        "type": "array"
-      },
-      "tier": {
-        "enum": [
-          "normal",
-          "hard"
-        ],
-        "type": "string"
-      }
-    },
-    "required": [
-      "card_version",
-      "tier",
-      "data_kind",
-      "as_of",
-      "model",
-      "tasks"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": {},
-    "properties": {
-      "data_kind": {
-        "const": "input_needed",
-        "type": "string"
-      },
-      "detail": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "error": {
-        "type": "string"
-      },
-      "examples": {
-        "items": {
-          "additionalProperties": {},
-          "propertyNames": {
-            "type": "string"
-          },
-          "type": "object"
-        },
-        "type": "array"
-      },
-      "needed": {
-        "type": "string"
-      },
-      "tool": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "error",
-      "data_kind",
-      "tool",
-      "needed",
-      "detail",
-      "examples"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": {},
-    "properties": {
-      "error": {
-        "const": "rate_limited",
-        "type": "string"
-      },
-      "retry_after_s": {
-        "type": "number"
-      }
-    },
-    "required": [
-      "error",
-      "retry_after_s"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": {},
+    "properties": {
+      "as_of": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "card_version": {
+        "type": "string"
+      },
+      "data_kind": {
+        "enum": [
+          "headline",
+          "practice",
+          "none"
+        ],
+        "type": "string"
+      },
+      "model": {
+        "anyOf": [
+          {
+            "additionalProperties": {},
+            "properties": {
+              "id": {
+                "type": "string"
+              },
+              "name": {
+                "type": "string"
+              }
+            },
+            "required": [
+              "id",
+              "name"
+            ],
+            "type": "object"
+          },
+          {
+            "type": "null"
+          }
+        ]
+      },
+      "share": {
+        "additionalProperties": {},
+        "properties": {
+          "page_url": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "receipt_badge_url": {
+            "type": "string"
+          },
+          "receipt_markdown": {
+            "type": "string"
+          },
+          "receipt_url": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "receipt_url"
+        ],
+        "type": "object"
+      },
+      "tasks": {
+        "items": {
+          "additionalProperties": {},
+          "properties": {
+            "task": {
+              "type": "string"
+            },
+            "title": {
+              "type": "string"
+            },
+            "traps": {
+              "items": {
+                "additionalProperties": {},
+                "properties": {
+                  "attempts": {
+                    "type": "number"
+                  },
+                  "card_checks": {
+                    "items": {
+                      "type": "string"
+                    },
+                    "type": "array"
+                  },
+                  "fall_rate": {
+                    "type": [
+                      "number",
+                      "null"
+                    ]
+                  },
+                  "fell": {
+                    "type": "number"
+                  },
+                  "name": {
+                    "type": "string"
+                  },
+                  "public_id": {
+                    "type": [
+                      "string",
+                      "null"
+                    ]
+                  },
+                  "trap": {
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "trap",
+                  "public_id",
+                  "name",
+                  "attempts",
+                  "fell",
+                  "fall_rate",
+                  "card_checks"
+                ],
+                "type": "object"
+              },
+              "type": "array"
+            }
+          },
+          "required": [
+            "task",
+            "title",
+            "traps"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "tier": {
+        "enum": [
+          "normal",
+          "hard"
+        ],
+        "type": "string"
+      }
+    },
+    "required": [
+      "card_version",
+      "tier",
+      "data_kind",
+      "as_of",
+      "model",
+      "tasks"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": {},
+    "properties": {
+      "data_kind": {
+        "const": "input_needed",
+        "type": "string"
+      },
+      "detail": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "error": {
+        "type": "string"
+      },
+      "examples": {
+        "items": {
+          "additionalProperties": {},
+          "propertyNames": {
+            "type": "string"
+          },
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "needed": {
+        "type": "string"
+      },
+      "tool": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "error",
+      "data_kind",
+      "tool",
+      "needed",
+      "detail",
+      "examples"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": {},
+    "properties": {
+      "error": {
+        "const": "rate_limited",
+        "type": "string"
+      },
+      "limit": {
+        "additionalProperties": {},
+        "properties": {
+          "per_minute": {
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "per_month": {
+            "type": [
+              "number",
+              "null"
+            ]
+          },
+          "plan": {
+            "enum": [
+              "anonymous",
+              "free",
+              "team",
+              "volume",
+              "enterprise"
+            ],
+            "type": "string"
+          },
+          "resets_at": {
+            "type": "string"
+          },
+          "used_this_month": {
+            "type": "number"
+          }
+        },
+        "required": [
+          "plan",
+          "per_minute"
+        ],
+        "type": "object"
+      },
+      "reason": {
+        "enum": [
+          "per_ip_minute",
+          "per_key_minute",
+          "monthly_quota"
+        ],
+        "type": "string"
+      },
+      "retry_after_s": {
+        "type": "number"
+      },
+      "upgrade": {
+        "additionalProperties": {},
+        "properties": {
+          "contact": {
+            "type": "string"
+          },
+          "free_key": {
+            "additionalProperties": {},
+            "properties": {
+              "per_minute": {
+                "type": [
+                  "number",
+                  "null"
+                ]
+              },
+              "per_month": {
+                "type": [
+                  "number",
+                  "null"
+                ]
+              }
+            },
+            "required": [
+              "per_minute",
+              "per_month"
+            ],
+            "type": "object"
+          },
+          "keys_url": {
+            "type": "string"
+          },
+          "paid_plans": {
+            "enum": [
+              "coming_soon",
+              "available"
+            ],
+            "type": "string"
+          },
+          "plans": {
+            "items": {
+              "enum": [
+                "team",
+                "volume"
+              ],
+              "type": "string"
+            },
+            "type": "array"
+          },
+          "url": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "url",
+          "keys_url",
+          "plans",
+          "paid_plans"
+        ],
+        "type": "object"
+      }
+    },
+    "required": [
+      "error",
+      "retry_after_s"
+    ],
+    "type": "object"
+  }
+]
  5. Changed2 schema fields changed
    • changedInput schema / properties / task / enum
      Previous value: -[
-  "refund",
-  "scam-invoice",
-  "weekend",
-  "spreadsheet",
-  "flaky-supplier",
-  "impossible",
-  "refund-hard",
-  "scam-invoice-hard",
-  "weekend-hard",
-  "spreadsheet-hard",
-  "flaky-supplier-hard",
-  "impossible-hard",
-  "refund-hard3",
-  "scam-invoice-hard3",
-  "weekend-hard3",
-  "spreadsheet-hard3",
-  "flaky-supplier-hard3",
-  "impossible-hard3"
-]New value: +[
+  "refund",
+  "scam-invoice",
+  "weekend",
+  "spreadsheet",
+  "flaky-supplier",
+  "impossible",
+  "refund-hard",
+  "scam-invoice-hard",
+  "weekend-hard",
+  "spreadsheet-hard",
+  "flaky-supplier-hard",
+  "impossible-hard",
+  "refund-hard3",
+  "scam-invoice-hard3",
+  "weekend-hard3",
+  "spreadsheet-hard3",
+  "flaky-supplier-hard3",
+  "impossible-hard3",
+  "refund-hard3-2",
+  "flaky-supplier-hard3-2"
+]
    • changedOutput schema / anyOf
      Previous value: -[
-  {
-    "additionalProperties": {},
-    "properties": {
-      "as_of": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "card_version": {
-        "type": "string"
-      },
-      "data_kind": {
-        "enum": [
-          "headline",
-          "practice",
-          "none"
-        ],
-        "type": "string"
-      },
-      "model": {
-        "anyOf": [
-          {
-            "additionalProperties": {},
-            "properties": {
-              "id": {
-                "type": "string"
-              },
-              "name": {
-                "type": "string"
-              }
-            },
-            "required": [
-              "id",
-              "name"
-            ],
-            "type": "object"
-          },
-          {
-            "type": "null"
-          }
-        ]
-      },
-      "tasks": {
-        "items": {
-          "additionalProperties": {},
-          "properties": {
-            "task": {
-              "type": "string"
-            },
-            "title": {
-              "type": "string"
-            },
-            "traps": {
-              "items": {
-                "additionalProperties": {},
-                "properties": {
-                  "attempts": {
-                    "type": "number"
-                  },
-                  "card_checks": {
-                    "items": {
-                      "type": "string"
-                    },
-                    "type": "array"
-                  },
-                  "fall_rate": {
-                    "type": [
-                      "number",
-                      "null"
-                    ]
-                  },
-                  "fell": {
-                    "type": "number"
-                  },
-                  "name": {
-                    "type": "string"
-                  },
-                  "public_id": {
-                    "type": [
-                      "string",
-                      "null"
-                    ]
-                  },
-                  "trap": {
-                    "type": "string"
-                  }
-                },
-                "required": [
-                  "trap",
-                  "public_id",
-                  "name",
-                  "attempts",
-                  "fell",
-                  "fall_rate",
-                  "card_checks"
-                ],
-                "type": "object"
-              },
-              "type": "array"
-            }
-          },
-          "required": [
-            "task",
-            "title",
-            "traps"
-          ],
-          "type": "object"
-        },
-        "type": "array"
-      },
-      "tier": {
-        "enum": [
-          "normal",
-          "hard"
-        ],
-        "type": "string"
-      }
-    },
-    "required": [
-      "card_version",
-      "tier",
-      "data_kind",
-      "as_of",
-      "model",
-      "tasks"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": {},
-    "properties": {
-      "data_kind": {
-        "const": "input_needed",
-        "type": "string"
-      },
-      "detail": {
-        "type": [
-          "string",
-          "null"
-        ]
-      },
-      "error": {
-        "type": "string"
-      },
-      "examples": {
-        "items": {
-          "additionalProperties": {},
-          "propertyNames": {
-            "type": "string"
-          },
-          "type": "object"
-        },
-        "type": "array"
-      },
-      "needed": {
-        "type": "string"
-      },
-      "tool": {
-        "type": "string"
-      }
-    },
-    "required": [
-      "error",
-      "data_kind",
-      "tool",
-      "needed",
-      "detail",
-      "examples"
-    ],
-    "type": "object"
-  },
-  {
-    "additionalProperties": {},
-    "properties": {
-      "error": {
-        "const": "rate_limited",
-        "type": "string"
-      },
-      "retry_after_s": {
-        "type": "number"
-      }
-    },
-    "required": [
-      "error",
-      "retry_after_s"
-    ],
-    "type": "object"
-  }
-]New value: +[
+  {
+    "additionalProperties": {},
+    "properties": {
+      "as_of": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "card_version": {
+        "type": "string"
+      },
+      "data_kind": {
+        "enum": [
+          "headline",
+          "practice",
+          "none"
+        ],
+        "type": "string"
+      },
+      "model": {
+        "anyOf": [
+          {
+            "additionalProperties": {},
+            "properties": {
+              "id": {
+                "type": "string"
+              },
+              "name": {
+                "type": "string"
+              }
+            },
+            "required": [
+              "id",
+              "name"
+            ],
+            "type": "object"
+          },
+          {
+            "type": "null"
+          }
+        ]
+      },
+      "share": {
+        "additionalProperties": {},
+        "properties": {
+          "page_url": {
+            "type": [
+              "string",
+              "null"
+            ]
+          },
+          "receipt_badge_url": {
+            "type": "string"
+          },
+          "receipt_markdown": {
+            "type": "string"
+          },
+          "receipt_url": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "receipt_url"
+        ],
+        "type": "object"
+      },
+      "tasks": {
+        "items": {
+          "additionalProperties": {},
+          "properties": {
+            "task": {
+              "type": "string"
+            },
+            "title": {
+              "type": "string"
+            },
+            "traps": {
+              "items": {
+                "additionalProperties": {},
+                "properties": {
+                  "attempts": {
+                    "type": "number"
+                  },
+                  "card_checks": {
+                    "items": {
+                      "type": "string"
+                    },
+                    "type": "array"
+                  },
+                  "fall_rate": {
+                    "type": [
+                      "number",
+                      "null"
+                    ]
+                  },
+                  "fell": {
+                    "type": "number"
+                  },
+                  "name": {
+                    "type": "string"
+                  },
+                  "public_id": {
+                    "type": [
+                      "string",
+                      "null"
+                    ]
+                  },
+                  "trap": {
+                    "type": "string"
+                  }
+                },
+                "required": [
+                  "trap",
+                  "public_id",
+                  "name",
+                  "attempts",
+                  "fell",
+                  "fall_rate",
+                  "card_checks"
+                ],
+                "type": "object"
+              },
+              "type": "array"
+            }
+          },
+          "required": [
+            "task",
+            "title",
+            "traps"
+          ],
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "tier": {
+        "enum": [
+          "normal",
+          "hard"
+        ],
+        "type": "string"
+      }
+    },
+    "required": [
+      "card_version",
+      "tier",
+      "data_kind",
+      "as_of",
+      "model",
+      "tasks"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": {},
+    "properties": {
+      "data_kind": {
+        "const": "input_needed",
+        "type": "string"
+      },
+      "detail": {
+        "type": [
+          "string",
+          "null"
+        ]
+      },
+      "error": {
+        "type": "string"
+      },
+      "examples": {
+        "items": {
+          "additionalProperties": {},
+          "propertyNames": {
+            "type": "string"
+          },
+          "type": "object"
+        },
+        "type": "array"
+      },
+      "needed": {
+        "type": "string"
+      },
+      "tool": {
+        "type": "string"
+      }
+    },
+    "required": [
+      "error",
+      "data_kind",
+      "tool",
+      "needed",
+      "detail",
+      "examples"
+    ],
+    "type": "object"
+  },
+  {
+    "additionalProperties": {},
+    "properties": {
+      "error": {
+        "const": "rate_limited",
+        "type": "string"
+      },
+      "retry_after_s": {
+        "type": "number"
+      }
+    },
+    "required": [
+      "error",
+      "retry_after_s"
+    ],
+    "type": "object"
+  }
+]
  6. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "anyOf": [
+    {
+      "additionalProperties": {},
+      "properties": {
+        "as_of": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "card_version": {
+          "type": "string"
+        },
+        "data_kind": {
+          "enum": [
+            "headline",
+            "practice",
+            "none"
+          ],
+          "type": "string"
+        },
+        "model": {
+          "anyOf": [
+            {
+              "additionalProperties": {},
+              "properties": {
+                "id": {
+                  "type": "string"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "required": [
+                "id",
+                "name"
+              ],
+              "type": "object"
+            },
+            {
+              "type": "null"
+            }
+          ]
+        },
+        "tasks": {
+          "items": {
+            "additionalProperties": {},
+            "properties": {
+              "task": {
+                "type": "string"
+              },
+              "title": {
+                "type": "string"
+              },
+              "traps": {
+                "items": {
+                  "additionalProperties": {},
+                  "properties": {
+                    "attempts": {
+                      "type": "number"
+                    },
+                    "card_checks": {
+                      "items": {
+                        "type": "string"
+                      },
+                      "type": "array"
+                    },
+                    "fall_rate": {
+                      "type": [
+                        "number",
+                        "null"
+                      ]
+                    },
+                    "fell": {
+                      "type": "number"
+                    },
+                    "name": {
+                      "type": "string"
+                    },
+                    "public_id": {
+                      "type": [
+                        "string",
+                        "null"
+                      ]
+                    },
+                    "trap": {
+                      "type": "string"
+                    }
+                  },
+                  "required": [
+                    "trap",
+                    "public_id",
+                    "name",
+                    "attempts",
+                    "fell",
+                    "fall_rate",
+                    "card_checks"
+                  ],
+                  "type": "object"
+                },
+                "type": "array"
+              }
+            },
+            "required": [
+              "task",
+              "title",
+              "traps"
+            ],
+            "type": "object"
+          },
+          "type": "array"
+        },
+        "tier": {
+          "enum": [
+            "normal",
+            "hard"
+          ],
+          "type": "string"
+        }
+      },
+      "required": [
+        "card_version",
+        "tier",
+        "data_kind",
+        "as_of",
+        "model",
+        "tasks"
+      ],
+      "type": "object"
+    },
+    {
+      "additionalProperties": {},
+      "properties": {
+        "data_kind": {
+          "const": "input_needed",
+          "type": "string"
+        },
+        "detail": {
+          "type": [
+            "string",
+            "null"
+          ]
+        },
+        "error": {
+          "type": "string"
+        },
+        "examples": {
+          "items": {
+            "additionalProperties": {},
+            "propertyNames": {
+              "type": "string"
+            },
+            "type": "object"
+          },
+          "type": "array"
+        },
+        "needed": {
+          "type": "string"
+        },
+        "tool": {
+          "type": "string"
+        }
+      },
+      "required": [
+        "error",
+        "data_kind",
+        "tool",
+        "needed",
+        "detail",
+        "examples"
+      ],
+      "type": "object"
+    },
+    {
+      "additionalProperties": {},
+      "properties": {
+        "error": {
+          "const": "rate_limited",
+          "type": "string"
+        },
+        "retry_after_s": {
+          "type": "number"
+        }
+      },
+      "required": [
+        "error",
+        "retry_after_s"
+      ],
+      "type": "object"
+    }
+  ],
+  "type": "object"
+}
  7. Changed3 schema fields changed
    • changedInput schema / properties / task / description
      Previous value: -"One Arena task; all tasks if omitted"New value: +"One Arena task; all tasks of the tier if omitted"
    • changedInput schema / properties / task / enum
      Previous value: -[
-  "refund",
-  "scam-invoice",
-  "weekend",
-  "spreadsheet",
-  "flaky-supplier",
-  "impossible"
-]New value: +[
+  "refund",
+  "scam-invoice",
+  "weekend",
+  "spreadsheet",
+  "flaky-supplier",
+  "impossible",
+  "refund-hard",
+  "scam-invoice-hard",
+  "weekend-hard",
+  "spreadsheet-hard",
+  "flaky-supplier-hard",
+  "impossible-hard",
+  "refund-hard3",
+  "scam-invoice-hard3",
+  "weekend-hard3",
+  "spreadsheet-hard3",
+  "flaky-supplier-hard3",
+  "impossible-hard3"
+]
    • addedInput schema / properties / tier
      Added value: +{
+  "description": "normal (default) or hard; a task id sets its own tier",
+  "enum": [
+    "normal",
+    "hard"
+  ],
+  "type": "string"
+}
  8. First observed

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the safety profile is covered. The description adds genuinely new behavior: the zero-argument default resolves to the latest ranked Hard event with the summary first, plus the returned fields (public trap id, attempts, falls, fall rate, data date, card version). It stops short of covering pagination or ordering across multiple traps, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences, front-loaded with the trigger condition, then filtering, then return fields. The field enumeration is slightly list-like but every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be fully explained, and annotations carry the safety profile. The description still covers the default resolution, all three optional filters and sibling routing, which is everything needed to call it correctly. The only residual gap is that non-normal/hard task variants (hard3, frontier) have no tier explanation, but the task enum itself is self-documenting.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents task, tier and model, including the precedence rules ('a task id sets its own tier', 'normal when only a model is given'). The description restates those same interactions rather than adding new meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb and resource: 'how often AI models fell for each planted trap' and names the actual trap categories (fake invoices, refunds over policy, duplicate orders). It also distinguishes itself from the sibling arena_results, which it assigns the per-model/job pass-rate question.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this when the question is how often AI models fell for each planted trap' gives an explicit trigger condition, and 'Pass rates per model and job: arena_results' names the alternative and the case that selects it. The no-argument default behavior is also spelled out, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All The Agent Forge toolsInspect
Try in Browser
Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources