score-studio-mcp
OfficialProvides a Score Studio MCP integration for GitHub Copilot and VS Code through a local or remote MCP endpoint, giving agents access to dataset/model inspection, training job tracking, evaluations, deployment verification, and workflow execution.
Provides a Score Studio MCP plugin for ChatGPT and OpenAI Codex, enabling agents to inspect datasets and model lineage, watch training jobs, read evaluations and reports, run evaluations against exact versions, verify deployments, and execute typed workflows.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@score-studio-mcp show me the latest evaluation report for dataset v2 and model lineage"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Score Studio agent plugins
Give your agents sight.
Score Studio is the computer vision layer for agents, automating data generation, annotation, training, evaluation, workflows, and deployment in one evidence-backed system.
This repository brings that complete vision loop to the major agent ecosystems through one portable MCP server, without publishing private source identifiers or internal implementation details.
Related MCP server: production-grade-mcp-agentic-system
What an agent can do
identify the signed-in user and organization memberships;
inspect versioned datasets and model lineage;
start tracked training jobs with explicit compute and dataset versions;
list measured evaluations and read immutable reports;
start evaluations against exact model/dataset version IDs;
inspect and verify deployments;
list and run typed workflows while reporting deferred blocks truthfully.
The MCP tools are intentionally narrower than the full Score Studio API. Delete, billing, member administration, provider credentials, and release switching are not exposed in the first version.
Local server
Requires Node.js 20+ and a Score Studio bearer token or API key.
npm install
npm run build
SCORESTUDIO_TOKEN=your-token npm startThe API defaults to https://api.scorestudio.ai. Set SCORESTUDIO_URL only to
target another Score Studio environment.
For local Streamable HTTP development:
npm run start:httpThe endpoint is http://127.0.0.1:3000/mcp; each request must include its own
Authorization: Bearer … header. /healthz does not require authentication.
Container deployment
The production image listens on 0.0.0.0:3000 and uses the production Score
Studio API by default; callers provide their own bearer token on each request.
docker build -t score-studio-mcp .
docker run --rm -p 3000:3000 score-studio-mcpPackages
Host | Package | Discovery route |
ChatGPT + Codex |
| OpenAI universal plugin directory |
Claude Code |
| Anthropic marketplace or this repo's marketplace |
Gemini CLI |
| Git install and Gemini extension gallery |
Cursor |
| This repository's Cursor marketplace; standalone marketplace repo remains supported |
Hermes Agent |
| Hermes GitHub skill tap plus native MCP client |
Lovable | Shared remote MCP endpoint | Custom chat connector; no equivalent public plugin submission flow yet |
GitHub Copilot / VS Code | Shared local or remote MCP endpoint | Manual MCP config, then GitHub MCP Registry when eligible |
Windsurf | Shared local or remote MCP endpoint | MCP Marketplace submission or manual Cascade config |
Other MCP hosts | Shared remote MCP endpoint | Streamable HTTP |
See distribution.md for installation, deployment, and marketplace requirements.
Cursor
Import this repository as a Cursor marketplace and install score-studio. The
plugin declares SCORESTUDIO_TOKEN as a required secret variable and starts the
shared MCP package through npx.
Hermes Agent
hermes skills tap add score-technologies/score-studio-agent-plugins
hermes skills install score-technologies/score-studio-agent-plugins/score-studioThen merge
plugins/hermes/score-studio/config.example.yaml
into ~/.hermes/config.yaml and run hermes mcp test score-studio.
Authentication and release boundary
Local packages read SCORESTUDIO_TOKEN from the host's secret configuration
and optionally accept a SCORESTUDIO_URL override. They do not persist or print
secrets.
A public ChatGPT listing cannot use this manual-token flow. It requires a stable public HTTPS MCP endpoint plus OAuth 2.1 discovery/authorization. That hosted OAuth boundary is deliberately left as a release gate; the repository does not claim the package is publicly installable before it exists.
License
Apache License 2.0.
Available Tools
12 toolsscorestudio_get_datasetGet datasetARead-only
Get a dataset and its version lineage by organization and slug.
Inputs: org_slug (required), dataset_slug (required).
scorestudio_get_evaluation_reportGet evaluation reportARead-only
Get metrics, slices, intervals, and evidence for an evaluation run.
Inputs: run_id (required), org_slug (required).
scorestudio_list_datasetsList datasetsCRead-only
List versioned datasets in a Score Studio organization.
Inputs: org_slug (required), limit, offset, include_archived.
scorestudio_list_deploymentsList deploymentsBRead-only
List deployments, revisions, and runtime state in an organization.
Inputs: org_slug (required), limit, offset.
scorestudio_list_evaluationsList evaluationsARead-only
List measured evaluation runs and their current states.
Inputs: org_slug (required), limit.
scorestudio_list_modelsList modelsARead-only
List models and versions in a Score Studio organization.
Inputs: org_slug (required), limit, offset.
scorestudio_list_workflowsList workflowsBRead-only
List typed Score Studio workflow definitions and status.
Inputs: org_slug (required).
scorestudio_run_workflowRun workflow previewC
Run a workflow against an existing Score Studio image object. The result may report deferred blocks.
Inputs: org_slug (required), workflow_slug (required), image_object_key (required).
scorestudio_start_evaluationStart evaluationA
Create an immutable measured evaluation run for a model version and dataset version.
Inputs: org_slug (required), model_version_id (required), dataset_version_id (required), task_type, +3 more.
scorestudio_start_trainingStart trainingA
Start a tracked training job against an immutable dataset version. This may consume compute credits.
Inputs: org_slug (required), dataset_slug (required), name, epochs, +5 more.
scorestudio_verify_deploymentVerify deploymentA
Run and retain deployment health verification evidence without changing the active revision.
Inputs: org_slug (required), deployment_slug (required).
scorestudio_whoamiGet Score Studio identityARead-only
Return the authenticated Score Studio user and organization memberships.
Inputs: no named parameters.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 12 tools
Each tool targets a distinct resource or action: auth, workflows, datasets, models, training, evaluations, and deployments. The similar-looking start/run tools are clearly separated by their object types.
Tools consistently use the scorestudio_ prefix with snake_case verb_noun names. Minor deviations like whoami and run_workflow vs start_training are easy to predict but not perfectly uniform.
With 12 tools, the server is well-scoped for an MLOps platform covering auth, workflows, datasets, models, training, evaluations, and deployments. Each tool has a clear purpose and none feel redundant.
Core read and start operations are present, but there are gaps: models only support listing, training jobs cannot be fetched after starting, and deployment management is limited to listing and verification. Agents may hit dead ends when trying to check training results or detailed model state.
Maintenance
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
Guarded MCP server for agent-readable business truth, provenance, readiness, and discovery.
MCP server for mandates, delegation, policy-gated execution, credential grants, and audit.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceMCP Server for AI agent identity and authorization. Create, verify, and manage agent identities with trust scores and scoped authorization tokens.31 PyPIMIT
- AlicenseNot gradedqualityDmaintenanceA production-grade MCP server designed for multi-tenant, authenticated, and observable AI agent systems, enabling secure tool execution across heterogeneous data sources.68MIT

AgentMark MCP Serverofficial
AlicenseNot gradedqualityAmaintenanceMCP server for AgentMark. Exposes the full AgentMark API to AI editors: list traces and spans, manage datasets, write scores, run experiments, manage apps, deployments, environments, and annotation queues.352AGPL 3.0- AlicenseBqualityBmaintenanceMCP server that initializes, manages, and operates traceable agentic infrastructure across your digital ecosystem, serving as a control plane to know project states, health, and initialize missing setups.371MIT