Skip to main content
Glama

Server Details

Run data annotation and evaluation tasks with vetted domain experts: create, publish, get results.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-06-18
URL
ChatGPT
–
Claude
–

TDQS

A3.7/5.0

Scored across 10 tools

Disambiguation5/5

Each tool targets a distinct action: task lifecycle (create/update/publish/close), reads (get_task/list_tasks/get_results), item ingestion (add_items), and pool/dashboard access. The only mild adjacency is add_items vs update_task (both touch task content), but descriptions clearly separate item ingestion from settings changes.

Naming Consistency5/5

All ten tools follow a consistent snake_case verb_noun pattern (add_items, create_task, get_results, list_pools, publish_task, etc.). No mixed conventions or vague standalone verbs.

Tool Count5/5

Ten tools is well-scoped for a labeling/evaluation task management server, and each tool maps to a necessary step in the create-add-publish-review workflow. Nothing feels redundant or padded.

Completeness4/5

The surface covers the full task lifecycle (create, update, publish, close, get, list), item ingestion, results retrieval, and pool listing. Minor gaps exist: no delete/archive task, no item removal, and no pool/team management, but core workflows have no dead ends.

Available Tools

10 tools
add_itemsAdd items to an AI taskA
Idempotent
Inspect

Add items by public URL; Pathwize copies the files into its own storage. reference: true adds them to the qualification set (optionally with gold_label for automatic scoring). Idempotent per id, up to 1000 per call.

Inputs: items (required), taskId (required), reference.

close_taskClose or pause an AI taskA
Idempotent
Inspect

closed ends the task, draft pauses it. Experts are not emailed.

Inputs: taskId (required), status.

create_taskCreate an AI task (draft)AInspect

Create a labeling or evaluation task as a draft; nobody is notified until publish_task. Typical order: list_pools, create_task with instructions, classes, rate, pool_ids and one example image per class, add_items, then publish_task after the user confirms.

Inputs: title (required), rate, type, domain, +14 more.

get_resultsGet labeled resultsC
Read-only
Inspect

Finished items with final label, agreement, notes and flags. Page with starting_after (the last item id).

Inputs: taskId (required), limit, since, starting_after.

get_taskGet an AI taskB
Read-only
Inspect

One task with settings, classes, examples, qualification, reference item count and item counts by status.

Inputs: taskId (required).

list_poolsList expert poolsA
Read-only
Inspect

Pools a task can run on: active curated pools plus your own, with member counts.

Inputs: no named parameters.

list_tasksList AI tasksA
Read-only
Inspect

Your AI tasks, newest first, with status and item counts.

Inputs: no named parameters.

open_dashboardOpen the business dashboard, signed inAInspect

One-time link that opens the Pathwize business dashboard signed in as this business (optionally one task). Single use, one hour; give it only to the user.

Inputs: taskId.

publish_taskPublish an AI taskAInspect

Open a task for experts. This emails every expert in the task's pools once and cannot be undone, so call it only after the user explicitly confirmed, with confirm: true. Fails with a list of anything missing.

Inputs: taskId (required), confirm (required).

update_taskUpdate an AI taskB
Idempotent
Inspect

Change a task's settings; only the fields you pass change, examples are added. Does not publish.

Inputs: taskId (required), rate, title, domain, +14 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      C
      maintenance
      Real human judgment as agent tools -- an AI agent can ask a question and get back a structured, schema-validated JSON answer from a real quality-scored human. 16 response types (yes/no, ratings, rankings, A/B tests, sentiment, image/video/audio review, voice/video/photo capture). Fully programmatic signup with a $5 free trial credit, no card required.
      7
      47 npm
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to hire real human operators for tasks requiring physical presence, human perception, or judgment, such as verification, testing, data collection, and physical-world tasks.
      4
      59 npm
      1
      MIT
    • F
      license
      Not graded
      quality
      C
      maintenance
      Enables AI agents to price and open calls that send short tasks to people, retrieve individual responses with page-observation evidence, and cancel calls or inspect panels.
      -
    • A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to recruit real humans for evaluation tasks like surveys, A/B tests, and ratings on text, images, audio, and video, returning aggregated results directly into the conversation.
      13
      7
      MIT
    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources