Skip to main content
Glama

Prepare an evaluation dataset your team can review

Arize AX + your AI assistant · powered by PipMCP

You want to prepare an evaluation dataset your team can review, using the account your team has actually connected.

Made for: Teams evaluating an AI feature

For real use, you will need PipMCP and access to Arize AX.

Ask your AI to set this up

Paste this into a coding assistant that can open project files and run commands, such as Codex, Claude Code or Grok Build. You do not need to understand the code first.

Help me use this project: https://github.com/jannikco/arize-ax-mcp

I'm not a programmer. My goal is: Prepare an evaluation dataset your team can review.

Read the project instructions and set it up in a new folder without overwriting my work.
Guide me through PipMCP and its private key setup; never ask me to paste secrets into this
chat. Run a documented sample-data demo if available, then help me get a first small
result from my connected app. Explain the result in plain English. Ask before changing
records, sending messages or publishing anything.

Related MCP server: Gate22

Three useful things to ask for

List available evaluation datasets and inspect the one I choose.
Prepare a dataset from example records I own. Show what will be included and wait for
approval before creating it.
Prepare a dataset-selection note for our AI evaluation review, using the dataset details
we inspected.

Your path to a real result

flowchart LR
  A["Ask your AI"] --> B["Connect PipMCP"] --> C["Prepare an evaluation<br/>dataset your team…"] --> D["Review together"]
  1. Open PipMCP setup and sign in or create your account.

  2. Connect the app in PipMCP with the permissions you want to grant.

  3. Create the product API key through the official setup, add it privately to your AI project's environment, and ask your AI to try a small read.

What to expect: Connect Arize AX in your PipMCP account, then use your PipMCP key for the supported actions. Review proposed changes before they are sent. Results depend on the connected account and its permissions; check the returned outcome before relying on it.

Add your PipMCP key privately in your project settings. Keep keys out of chat and public repositories.

Keep going

Arize AX MCP server

Use Arize AX from Claude Code, Codex, Grok Build and MCP clients through PipMCP. One PipMCP API key connects this package to private execution.

Activate with PipMCP · Integration directory

Build something useful

Review evaluation datasets and create a dataset with user-owned example records in an existing space.

The package is an authenticated client: discovery, schemas, policy decisions and execution come from PipMCP. Provider implementation remains on the server.

Quick start

Requires Node.js 22+. Clone this repository, then:

npm ci --ignore-scripts
npm test
npm start
# After configuring your provider in PipMCP and setting PIPMCP_API_KEY:
npm start -- --live
# Launch the stdio MCP bridge:
npm run server

Create a PipMCP API key with explicit connectors:read and connectors:call scopes. Set PIPMCP_API_KEY in the calling process environment. Provider credentials are configured in PipMCP. Multiple accounts require explicit PIPMCP_INTEGRATION_ID and PIPMCP_CONNECTION_ID from discovery. Existing keys do not gain scopes automatically.

Built for coding agents

Install this repository's skill into your project:

node lib/cli.mjs agents install --client codex --scope project --project /path/to/project
# Use claude or grok for those clients.

Connect AI agents to Arize AX using one PipMCP API key and actual hosted tool discovery. Native AGENTS.md, CLAUDE.md and project skills support the current coding session. No additional AI model key is required. capabilities.json describes the package; llms.txt indexes its instructions.

Inspect actual tools before calling

import {PipMCPGatewayClient} from './lib/index.mjs';
const client = new PipMCPGatewayClient(); // PIPMCP_API_KEY
const inventory = await client.getTools("arize-ax");
console.log(inventory.data.tools);
// Select a real tool and validate your arguments against its returned schema.
// client.callTool(provider, actualToolName, reviewedArguments, {idempotencyKey});

PipMCP enforces workspace, role, connection, policy/approval and usage controls. Preserve the original idempotency key when retrieving/resuming a receipt. Pending and confirmation_required are not completed provider results; uncertain writes are held for reconciliation. The bridge never auto-approves a hosted action.

Version 1 migration

This release uses the authenticated PipMCP gateway for real discovery/execution. Upgrade your MCP launch command and configure PIPMCP_API_KEY before reconnecting. Historical local runtimes remain available under their original MIT license in Git history.

Evidence and support

Transport, package and agent fixtures are tested. Authentication/provider access must be checked in your own configured workspace; synthetic fixtures do not establish live provider execution. Do not include credentials or customer records in public issues. MIT covers this public client.

Provider setup reference

Official provider documentation. Credentials are entered in PipMCP; this package does not accept or store provider secrets.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      A secure MCP gateway for enterprise AI tool execution, enabling governed invocation of business tools with authentication, RBAC, audit logging, PII redaction, and async processing.
      Apache 2.0
    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables organizations to centrally govern agent tool access by exposing authorized MCP tools through a unified endpoint with search and execute functions, while enforcing function-level permissions, credential modes, and auditability.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to discover and invoke versioned external tools through an MCP or REST gateway with authentication, authorization, mutation safety, reliability controls, audit logging, and replay.
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to discover and invoke tools generated from existing OpenAPI-described REST APIs through a governed MCP gateway with centralized authentication, policy-based access control, rate limiting, PII redaction, auditing, and token-efficient progressive discovery.
      MIT