Skip to main content
Glama

Vidova

by ai.vidova
OverviewDiscussionsAdmin

Track a flat surface

track_plane

Track a flat surface in perspective as its four corners on every frame: a shipping label or a sheet in a hand, a phone or laptop screen, a sign, a box side, a poster, a wall or a floor. Seed it at source time at with corners (four source pixels in order top left, top right, bottom right, bottom left as the surface shows them there) or a box around it ([x0, y0, x1, y1], x1>x0, y1>y0) whose corners it finds when all four edges show there, or with fromProposal, a surface propose_surfaces found (its corners and seed time); the others null. Seed at a moment where the whole surface is in view, and check the corners on the sheet. Specify the source-time range, fps (null samples every frame of the footage on its own frame times, for a pin or a cover; 8 to look) and label. The corners keep their order however the surface turns; frames where it is hidden or out of the frame are marked lost, and it is taken again when it comes back. Bind a picture, a video, a component or a black-box region to it with . Coordinates are source pixels and times source seconds unless you say otherwise: pass points read off a preview with space preview and its previewId (the seed's time is then the preview's), film pixels with space film, and a film second with atSpace timeline. The answer ends with a json block giving the seed and the result in source, film and film_norm, with the source and timeline seconds, and warns about coordinates that look wrong. Every frame is checked: the answer gives a quality verdict, the flagged ranges (a corner drifting off the surface, a jump, the tracker unsure) and amber tiles on the sheet for them. Fix a flagged track with autoReseed true (it tracks those frames again from the last good frame), more seeds where the object looks different, or edit_track (free: it fixes a frame and fades the fix into its neighbours). read_track reads every frame's corners and quality. Uses about 38 credits a minute of the range, billed by the second. Read builtin-object-tracking first.

Input Schema

NameRequiredDescriptionDefault
atYes
boxYesA box around the surface, [x0, y0, x1, y1] in source pixels; null when corners are given.
fpsYesSamples a second, 1 to 30. null samples every frame of the footage on its own frame times (every second frame of 50 or 60 fps footage): use it for a pin, a cover or a follow. 8 is enough to look. An fps above the footage's repeats frames.
snapYestrue: move the seed onto the picture first (corners onto the surface's real corners, a box onto the object's edges, points onto the nearest corner), as the snap tool does; each move is in the answer, and a point with weak evidence stays put. Use it on corners read off a preview. null or false: the seed as given.
labelYes
seedsYesMore seeds besides at's, up to 8, in the call's space and atSpace: where the object looks different (it turns, comes back after leaving, changes size). Each seed is tracked to its neighbours and the walks are joined into one track where they meet, keeping the better frame. null tracks from at's seed alone.
spaceYesThe space of this call's coordinates. null or source: pixels of the asset file. preview: pixels of a preview image you were shown, with previewId; its time is that preview's time. film: scene pixels from the frame's centre; film_norm: 0 to 1 of the frame; clip: the clip's own picture (with clipId). The tool converts them to the asset's pixels through the film's real transforms.
clipIdYesThe film clip that plays this asset, when space or atSpace needs one and several clips play it. null picks the clip that shows the asset at that time.
detailYesbrief: the track id, its verdict and the flagged ranges, with no contact sheet. normal (null): the answer and its contact sheet. full: the same, as normal for now.
assetIdYes
atSpaceYesWhat this call's times are. null or source: seconds into the asset file. timeline: seconds of the film, read through the clip that plays the asset then (clipId to pick one).
cornersYesThe surface's four corners in source pixels, top left, top right, bottom right, bottom left as it shows them at at; null when a box or fromProposal is given.
endTimeYes
previewIdYesWith space preview or preview_norm: the previewId the preview_frame, preview_range or preview_timeline answer printed. Else null.
projectIdYesThe project this call runs on without changing the open project. null (or leaving it out) uses the open project. wait_for_jobs takes the projectId of the calls that started the jobs.
startTimeYes
autoReseedYestrue: where the quality check flags frames (a corner drifting, a jump, the tracker unsure), track them again from the last good frame before them, and keep the better frames. The answer says where it reseeded. Each reseed uses credits for the frames it reads. null or false: no reseeding.
snapRadiusYesHow far snap may move a point, in source pixels, up to 64. null: 12.
fromProposalYesA proposal id from propose_surfaces ("sf_..."): the seed is its corners at its seed time, and at, corners and box are not read for it (pass null for corners and box). null: seed with corners or box.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed17 schema fields changed
    • addedInput schema / properties / atSpace
      Added value: +{
+  "anyOf": [
+    {
+      "enum": [
+        "source",
+        "timeline"
+      ],
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "What this call's times are. null or source: seconds into the asset file. timeline: seconds of the film, read through the clip that plays the asset then (clipId to pick one)."
+}
    • addedInput schema / properties / autoReseed
      Added value: +{
+  "anyOf": [
+    {
+      "type": "boolean"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "true: where the quality check flags frames (a corner drifting, a jump, the tracker unsure), track them again from the last good frame before them, and keep the better frames. The answer says where it reseeded. Each reseed uses credits for the frames it reads. null or false: no reseeding."
+}
    • addedInput schema / properties / clipId
      Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "The film clip that plays this asset, when space or atSpace needs one and several clips play it. null picks the clip that shows the asset at that time."
+}
    • changedInput schema / properties / corners / description
      Previous value: -"The surface's four corners in source pixels, top left, top right, bottom right, bottom left as it shows them at at; null when a box is given."New value: +"The surface's four corners in source pixels, top left, top right, bottom right, bottom left as it shows them at at; null when a box or fromProposal is given."
    • addedInput schema / properties / detail
      Added value: +{
+  "anyOf": [
+    {
+      "enum": [
+        "brief",
+        "normal",
+        "full"
+      ],
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "brief: the track id, its verdict and the flagged ranges, with no contact sheet. normal (null): the answer and its contact sheet. full: the same, as normal for now."
+}
    • addedInput schema / properties / fps / anyOf
      Added value: +[
+  {
+    "maximum": 30,
+    "minimum": 1,
+    "type": "integer"
+  },
+  {
+    "type": "null"
+  }
+]
    • addedInput schema / properties / fps / description
      Added value: +"Samples a second, 1 to 30. null samples every frame of the footage on its own frame times (every second frame of 50 or 60 fps footage): use it for a pin, a cover or a follow. 8 is enough to look. An fps above the footage's repeats frames."
    • removedInput schema / properties / fps / maximum
      Removed value: -30
    • removedInput schema / properties / fps / minimum
      Removed value: -1
    • removedInput schema / properties / fps / type
      Removed value: -"integer"
    • addedInput schema / properties / fromProposal
      Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "A proposal id from propose_surfaces (\"sf_...\"): the seed is its corners at its seed time, and at, corners and box are not read for it (pass null for corners and box). null: seed with corners or box."
+}
    • addedInput schema / properties / previewId
      Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "With space preview or preview_norm: the previewId the preview_frame, preview_range or preview_timeline answer printed. Else null."
+}
    • addedInput schema / properties / seeds
      Added value: +{
+  "anyOf": [
+    {
+      "items": {
+        "additionalProperties": false,
+        "properties": {
+          "at": {
+            "description": "The seed's time, in the call's atSpace (null: source seconds).",
+            "type": "number"
+          },
+          "box": {
+            "anyOf": [
+              {
+                "items": {
+                  "type": "number"
+                },
+                "type": "array"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "description": "A box around the object or surface at at, [x0, y0, x1, y1] (track_box, track_pattern, or track_plane without corners). Else null."
+          },
+          "corners": {
+            "anyOf": [
+              {
+                "items": {
+                  "additionalProperties": false,
+                  "properties": {
+                    "x": {
+                      "type": "number"
+                    },
+                    "y": {
+                      "type": "number"
+                    }
+                  },
+                  "required": [
+                    "x",
+                    "y"
+                  ],
+                  "type": "object"
+                },
+                "type": "array"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "description": "track_plane: the surface's four corners at at, top left, top right, bottom right, bottom left. Else null."
+          },
+          "points": {
+            "anyOf": [
+              {
+                "items": {
+                  "additionalProperties": false,
+                  "properties": {
+                    "x": {
+                      "type": "number"
+                    },
+                    "y": {
+                      "type": "number"
+                    }
+                  },
+                  "required": [
+                    "x",
+                    "y"
+                  ],
+                  "type": "object"
+                },
+                "type": "array"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "description": "track_points: points on the object at at. Else null."
+          }
+        },
+        "required": [
+          "at",
+          "corners",
+          "box",
+          "points"
+        ],
+        "type": "object"
+      },
+      "maxItems": 8,
+      "type": "array"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "More seeds besides at's, up to 8, in the call's space and atSpace: where the object looks different (it turns, comes back after leaving, changes size). Each seed is tracked to its neighbours and the walks are joined into one track where they meet, keeping the better frame. null tracks from at's seed alone."
+}
    • addedInput schema / properties / snap
      Added value: +{
+  "anyOf": [
+    {
+      "type": "boolean"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "true: move the seed onto the picture first (corners onto the surface's real corners, a box onto the object's edges, points onto the nearest corner), as the snap tool does; each move is in the answer, and a point with weak evidence stays put. Use it on corners read off a preview. null or false: the seed as given."
+}
    • addedInput schema / properties / snapRadius
      Added value: +{
+  "anyOf": [
+    {
+      "type": "number"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "How far snap may move a point, in source pixels, up to 64. null: 12."
+}
    • addedInput schema / properties / space
      Added value: +{
+  "anyOf": [
+    {
+      "enum": [
+        "source",
+        "source_norm",
+        "clip",
+        "film",
+        "film_tl",
+        "film_norm",
+        "preview",
+        "preview_norm"
+      ],
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "The space of this call's coordinates. null or source: pixels of the asset file. preview: pixels of a preview image you were shown, with previewId; its time is that preview's time. film: scene pixels from the frame's centre; film_norm: 0 to 1 of the frame; clip: the clip's own picture (with clipId). The tool converts them to the asset's pixels through the film's real transforms."
+}
    • changedInput schema / required
      Previous value: -[
-  "assetId",
-  "startTime",
-  "endTime",
-  "fps",
-  "label",
-  "at",
-  "box",
-  "corners",
-  "projectId"
-]New value: +[
+  "assetId",
+  "startTime",
+  "endTime",
+  "fps",
+  "label",
+  "detail",
+  "at",
+  "box",
+  "corners",
+  "fromProposal",
+  "space",
+  "previewId",
+  "clipId",
+  "atSpace",
+  "seeds",
+  "autoReseed",
+  "snap",
+  "snapRadius",
+  "projectId"
+]
  2. Added

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, destructiveHint=false) are consistent, and the description layers on the substantive behavior: ~38 credits per minute billed by the second, reseed credits, lost/occluded frames marked lost, corner ordering preserved through rotation, per-frame quality verdicts with flagged ranges, and a json result block in source/film/film_norm. That is well beyond what the annotations cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, which is good, but the body is a dense run-on with typos and garbled clauses ('Seed it at source time at with corners', 'pass points read off a preview with space preview'). The information mostly earns its place, but readability and fidelity suffer.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 19-parameter tracking tool with no output schema, the description covers seeding modes, coordinate/time spaces, cost, failure handling and recovery alternatives, and even sketches the returned json block and quality verdicts. An agent has what it needs to call it correctly and to recover from a flagged track.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 74% schema coverage the schema already documents most params, but the description adds real meaning: fps null samples every frame ('8 is enough to look'), the four-corner ordering convention, box constraints, the space/previewId/atSpace coordinate conventions, and the up-to-8-seeds merge behavior. Some of this restates schema text, so it is additive but not uniformly so.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Track a flat surface in perspective as its four corners on every frame' — and immediately scopes what counts as a surface (label, screen, sign, box side, wall). This distinguishes it from corner-less siblings like track_points or track_box without the agent needing to open schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete when-to-use guidance: seed with corners, a box, or fromProposal; 'Seed at a moment where the whole surface is in view'; and it routes recovery to autoReseed, more seeds, or edit_track and reading to read_track. It stops short of explicitly naming track_points/track_box as the alternatives it is not, so the boundary is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Vidova toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources