How do I use ai-usage-metrics-mcp?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ai-usage-metrics-mcp show me the total token usage for last month" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

ai-usage-metrics-mcp

by charlie-818

Overview Schema Source Related Servers Discussions Admin

Remote

AI Usage Metrics MCP Server

Track AI usage metrics and structured logs across all your applications

Quick Start • One-Click Install • Documentation • Contributing

A Model Context Protocol (MCP) server for tracking AI usage metrics and structured logs across your applications. Monitor model calls, analyze usage patterns, track costs, and debug AI interactions with a clean, extensible architecture.

Related MCP server: runmeter

Overview

This MCP server provides a centralized way to track and analyze AI model usage across your applications. Whether you're building chatbots, RAG systems, or autonomous agents, this server helps you:

Track every model call with full context (inputs, outputs, metadata)
Analyze usage patterns across projects, environments, and users
Monitor costs via token counting and aggregation
Debug interactions by replaying sessions and conversations
Ensure safety by logging safety check results

The server implements the Model Context Protocol specification, making it compatible with Claude, Cursor, Windsurf, and any MCP-enabled AI assistant or agent framework.

Features

Feature	Description
Comprehensive Logging	Log model calls with full message history, token counts, latency, and custom metrics
Session Tracking	Group related calls into sessions for conversation replay and analysis
Multi-Environment	Track usage across dev, staging, and production environments
Flexible Filtering	Search and filter by project, environment, user, model, and date range
Real-time Aggregation	Get instant metrics on call counts, token usage, and latency
Safety Monitoring	Track safety check results (passed, flagged, blocked)
RAG Support	Log retrieved context with source attribution
Extensible Storage	Clean interface for swapping storage backends (in-memory, PostgreSQL, etc.)

🚀 Quick Start

No installation required! Just add the server to your AI platform config:

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

That's it! The package will be automatically downloaded and run when your AI platform starts.

📦 One-Click Install

Choose your AI platform and copy the configuration.

Claude Desktop

Config file: ~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

One-liner setup:

mkdir -p ~/Library/Application\ Support/Claude

cat > ~/Library/Application\ Support/Claude/claude_desktop_config.json << 'EOF'
{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}
EOF

Config file: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

PowerShell one-liner:

New-Item -ItemType Directory -Force -Path "$env:APPDATA\Claude"

@'
{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}
'@ | Out-File -FilePath "$env:APPDATA\Claude\claude_desktop_config.json" -Encoding UTF8

Config file: ~/.config/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

One-liner setup:

mkdir -p ~/.config/Claude
cat > ~/.config/Claude/claude_desktop_config.json << 'EOF'
{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}
EOF

Claude Code CLI

Recommended: Use the Claude Code command:

claude mcp add ai-usage-metrics -- npx -y ai-usage-metrics-mcp

Or manually edit ~/.claude/settings.json:

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

Cursor

Config file: ~/.cursor/mcp.json

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

One-liner setup (macOS/Linux):

mkdir -p ~/.cursor
cat > ~/.cursor/mcp.json << 'EOF'
{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}
EOF

Windsurf

Config file: ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}

One-liner setup (macOS/Linux):

mkdir -p ~/.codeium/windsurf
cat > ~/.codeium/windsurf/mcp_config.json << 'EOF'
{
  "mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"]
    }
  }
}
EOF

VS Code + Continue

Config file: ~/.continue/config.json

{
  "experimental": {
    "modelContextProtocolServers": [
      {
        "transport": {
          "type": "stdio",
          "command": "npx",
          "args": ["-y", "ai-usage-metrics-mcp"]
        }
      }
    ]
  }
}

Cline

Config file: VS Code Settings (settings.json)

{
  "cline.mcpServers": {
    "ai-usage-metrics": {
      "command": "npx",
      "args": ["-y", "ai-usage-metrics-mcp"],
      "disabled": false
    }
  }
}

Zed

Config file: ~/.config/zed/settings.json

{
  "context_servers": {
    "ai-usage-metrics": {
      "command": {
        "path": "npx",
        "args": ["-y", "ai-usage-metrics-mcp"]
      }
    }
  }
}

Other MCP-Compatible Platforms

For any MCP-compatible platform, use these standard connection details:

Setting	Value
Transport	`stdio`
Command	`npx`
Arguments	`["-y", "ai-usage-metrics-mcp"]`
Server Name	`ai-usage-metrics`

Generic MCP Configuration:

{
  "name": "ai-usage-metrics",
  "transport": "stdio",
  "command": "npx",
  "args": ["-y", "ai-usage-metrics-mcp"]
}

Manual Installation

For most users, the npx method above is recommended. Manual installation is useful for development or if you prefer a global install.

Global Install via npm

npm install -g ai-usage-metrics-mcp

Then use ai-usage-metrics-mcp as the command in your MCP config instead of npx.

From Source (for development)

# Clone the repository
git clone https://github.com/charlie-818/ai-history-mcp.git
cd ai-history-mcp

# Install dependencies
npm install

# Build the project
npm run build

# Run the server
npm start

Package Manager Scripts

Script	Description
`npm run build`	Compile TypeScript to JavaScript
`npm start`	Run the compiled server
`npm run dev`	Watch mode for development
`npm test`	Run the test suite
`npm run test:coverage`	Run tests with coverage report

Verifying Installation

After installation, verify the server works:

# Test the server starts correctly
echo '{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"capabilities": {}}}' | node dist/index.js

You should see a JSON response with server capabilities.

Usage

Logging Model Calls

After each AI model invocation in your application, log the call:

// Using MCP client
await mcpClient.callTool("log_model_call", {
  project: "my-chatbot",
  environment: "prod",
  sessionId: "session-abc-123",
  modelName: "claude-3-opus",
  inputMessages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is the capital of France?" }
  ],
  outputMessages: [
    { role: "assistant", content: "The capital of France is Paris." }
  ],
  tokensIn: 45,
  tokensOut: 12,
  latencyMs: 234
});

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "stored": true
}

Searching Calls

Find specific calls with flexible filtering:

// Search by project and date range
const calls = await mcpClient.callTool("search_model_calls", {
  project: "my-chatbot",
  environment: "prod",
  from: "2024-01-01T00:00:00Z",
  to: "2024-01-31T23:59:59Z",
  limit: 100
});

// Search by user
const userCalls = await mcpClient.callTool("search_model_calls", {
  userId: "user-12345",
  modelName: "gpt-4"
});

Session Management

Track conversations by grouping calls into sessions:

// List all sessions for a project
const sessions = await mcpClient.callTool("list_sessions", {
  project: "my-chatbot",
  environment: "prod"
});

// Get all calls in a specific session
const sessionCalls = await mcpClient.callTool("get_session_calls", {
  sessionId: "session-abc-123"
});

Session Summary Response:

{
  "sessionId": "session-abc-123",
  "project": "my-chatbot",
  "environment": "prod",
  "firstCallAt": "2024-01-15T10:30:00Z",
  "lastCallAt": "2024-01-15T10:45:00Z",
  "callCount": 8,
  "totalTokensIn": 1250,
  "totalTokensOut": 890,
  "avgLatencyMs": 245
}

Aggregate Metrics

Get high-level usage statistics:

// Get metrics for a project
const metrics = await mcpClient.callTool("get_aggregate_metrics", {
  project: "my-chatbot",
  environment: "prod",
  from: "2024-01-01T00:00:00Z",
  to: "2024-01-31T23:59:59Z"
});

Response:

{
  "callCount": 15420,
  "totalTokensIn": 2450000,
  "totalTokensOut": 1890000,
  "avgLatencyMs": 312
}

📚 API Reference

Tools

`log_model_call`

Log a model call for tracking.

Parameter	Type	Required	Description
`project`	string	Yes	Project identifier
`environment`	string	No	Environment (default: "dev")
`userId`	string	No	User identifier
`sessionId`	string	No	Session identifier for grouping calls
`modelName`	string	Yes	Model name (e.g., "gpt-4", "claude-3-opus")
`modelVersion`	string	No	Model version
`promptType`	string	No	Type: "chat", "rag", "tool", "agent"
`inputMessages`	array	Yes	Input messages `[{role, content}]`
`outputMessages`	array	Yes	Output messages `[{role, content}]`
`retrievedContext`	array	No	RAG context `[{source, docId, hash?}]`
`latencyMs`	number	No	Call latency in milliseconds
`tokensIn`	number	No	Input token count
`tokensOut`	number	No	Output token count
`safety`	object	No	Safety result `{status, details?}`
`metrics`	object	No	Custom metrics key-value pairs
`traceId`	string	No	Distributed tracing ID
`requestId`	string	No	Provider request ID

Returns: { id: string, stored: boolean }

`search_model_calls`

Search logged calls with filters.

Parameter	Type	Required	Description
`project`	string	No	Filter by project
`environment`	string	No	Filter by environment
`userId`	string	No	Filter by user
`modelName`	string	No	Filter by model
`from`	string	No	Start date (ISO format)
`to`	string	No	End date (ISO format)
`limit`	number	No	Max results (default: 50, max: 100)

Returns: Array of ModelCallLog objects (message content truncated for safety)

`list_sessions`

List session summaries.

Parameter	Type	Required	Description
`project`	string	No	Filter by project
`environment`	string	No	Filter by environment
`limit`	number	No	Max results (default: 50, max: 100)

Returns: Array of SessionSummary objects

`get_session_calls`

Get all calls for a session.

Parameter	Type	Required	Description
`sessionId`	string	Yes	Session identifier

Returns: Array of ModelCallLog objects (chronological order)

`get_aggregate_metrics`

Get aggregate metrics across calls.

Parameter	Type	Required	Description
`project`	string	No	Filter by project
`environment`	string	No	Filter by environment
`from`	string	No	Start date (ISO format)
`to`	string	No	End date (ISO format)

Returns: { callCount, totalTokensIn, totalTokensOut, avgLatencyMs? }

Resources

Resources provide read-only access via URI patterns:

URI Pattern	Description
`ai-usage://calls/{id}`	Get a specific call by ID
`ai-usage://sessions/{sessionId}`	Get session summary and all calls
`ai-usage://metrics/aggregate?project=...&environment=...`	Get aggregate metrics

Example resource access:

// Get a specific call
const call = await mcpClient.readResource("ai-usage://calls/550e8400-e29b-41d4-a716-446655440000");

// Get session details
const session = await mcpClient.readResource("ai-usage://sessions/session-abc-123");

// Get filtered metrics
const metrics = await mcpClient.readResource(
  "ai-usage://metrics/aggregate?project=my-chatbot&environment=prod"
);

Data Model

ModelCallLog

interface ModelCallLog {
  id: string;                    // Unique identifier (UUID)
  timestamp: string;             // ISO timestamp
  project: string;               // Project identifier
  environment: string;           // "dev" | "staging" | "prod" | custom
  userId?: string;               // Optional user identifier
  sessionId?: string;            // Optional session identifier
  modelName: string;             // Model name
  modelVersion?: string;         // Model version
  promptType?: string;           // "chat" | "rag" | "tool" | "agent" | custom
  inputMessages: Message[];      // Input messages
  outputMessages: OutputMessage[]; // Output messages
  retrievedContext?: RetrievedContext[]; // RAG context
  latencyMs?: number;            // Latency in milliseconds
  tokensIn?: number;             // Input tokens
  tokensOut?: number;            // Output tokens
  safety?: SafetyResult;         // Safety check result
  metrics?: Record<string, number | string>; // Custom metrics
  traceId?: string;              // Distributed tracing ID
  requestId?: string;            // Provider request ID
}

SessionSummary

interface SessionSummary {
  sessionId: string;
  project: string;
  environment: string;
  firstCallAt: string;           // ISO timestamp
  lastCallAt: string;            // ISO timestamp
  callCount: number;
  totalTokensIn: number;
  totalTokensOut: number;
  avgLatencyMs?: number;
}

Message Types

interface Message {
  role: "system" | "user" | "assistant" | "tool";
  content: string;
}

interface OutputMessage {
  role: "assistant" | "tool";
  content: string;
}

interface RetrievedContext {
  source: string;
  docId: string;
  hash?: string;
}

interface SafetyResult {
  status: "passed" | "flagged" | "blocked";
  details?: string;
}

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      MCP Server                             │
│  ┌─────────────────────────────────────────────────────┐   │
│  │                    src/index.ts                      │   │
│  │              Server wiring & handlers                │   │
│  └─────────────────────────────────────────────────────┘   │
│                            │                                │
│         ┌──────────────────┼──────────────────┐            │
│         ▼                  ▼                  ▼            │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐      │
│  │   Tools     │   │  Resources  │   │   Schema    │      │
│  │ src/tools/* │   │src/resources│   │ src/schema  │      │
│  └─────────────┘   └─────────────┘   └─────────────┘      │
│         │                  │                  │            │
│         └──────────────────┼──────────────────┘            │
│                            ▼                                │
│              ┌─────────────────────────┐                   │
│              │    MetricsStore         │                   │
│              │    Interface            │                   │
│              │    src/store.ts         │                   │
│              └─────────────────────────┘                   │
│                            │                                │
│              ┌─────────────┴─────────────┐                 │
│              ▼                           ▼                 │
│    ┌──────────────────┐      ┌──────────────────┐         │
│    │ InMemoryStore    │      │ PostgresStore    │         │
│    │ (included)       │      │ (extend)         │         │
│    └──────────────────┘      └──────────────────┘         │
└─────────────────────────────────────────────────────────────┘

Project Structure

ai-usage-metrics-mcp/
├── src/
│   ├── index.ts           # MCP server entry point
│   ├── schema.ts          # TypeScript type definitions
│   ├── store.ts           # Storage interface & in-memory implementation
│   ├── tools/
│   │   ├── index.ts       # Tool exports
│   │   ├── log-model-call.ts
│   │   ├── search-model-calls.ts
│   │   ├── list-sessions.ts
│   │   ├── get-session-calls.ts
│   │   └── get-aggregate-metrics.ts
│   └── resources/
│       └── index.ts       # Resource handlers
├── tests/
│   ├── fixtures.ts        # Test data factories
│   ├── store.test.ts
│   ├── resources.test.ts
│   ├── integration.test.ts
│   └── tools/
│       └── *.test.ts
├── package.json
├── tsconfig.json
└── vitest.config.ts

Extending the Server

Adding PostgreSQL Support

The MetricsStore interface makes it straightforward to add database support:

// src/postgres-store.ts
import { Pool } from 'pg';
import { MetricsStore, ModelCallLog, SessionSummary, ... } from './schema.js';

export class PostgresMetricsStore implements MetricsStore {
  private pool: Pool;

  constructor(connectionString: string) {
    this.pool = new Pool({ connectionString });
  }

  async logCall(input: LogCallInput): Promise<ModelCallLog> {
    const id = crypto.randomUUID();
    const timestamp = new Date().toISOString();

    await this.pool.query(
      `INSERT INTO model_calls (id, timestamp, project, ...) VALUES ($1, $2, $3, ...)`,
      [id, timestamp, input.project, ...]
    );

    return { id, timestamp, ...input };
  }

  async getCall(id: string): Promise<ModelCallLog | null> {
    const result = await this.pool.query(
      'SELECT * FROM model_calls WHERE id = $1',
      [id]
    );
    return result.rows[0] || null;
  }

  // Implement remaining methods...
}

Database Schema (PostgreSQL)

CREATE TABLE model_calls (
  id UUID PRIMARY KEY,
  timestamp TIMESTAMPTZ NOT NULL,
  project VARCHAR(255) NOT NULL,
  environment VARCHAR(50) NOT NULL,
  user_id VARCHAR(255),
  session_id VARCHAR(255),
  model_name VARCHAR(255) NOT NULL,
  model_version VARCHAR(50),
  prompt_type VARCHAR(50),
  input_messages JSONB NOT NULL,
  output_messages JSONB NOT NULL,
  retrieved_context JSONB,
  latency_ms INTEGER,
  tokens_in INTEGER,
  tokens_out INTEGER,
  safety JSONB,
  metrics JSONB,
  trace_id VARCHAR(255),
  request_id VARCHAR(255),
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Indexes for common queries
CREATE INDEX idx_model_calls_project ON model_calls(project);
CREATE INDEX idx_model_calls_environment ON model_calls(environment);
CREATE INDEX idx_model_calls_session_id ON model_calls(session_id);
CREATE INDEX idx_model_calls_user_id ON model_calls(user_id);
CREATE INDEX idx_model_calls_timestamp ON model_calls(timestamp);
CREATE INDEX idx_model_calls_model_name ON model_calls(model_name);

Adding Custom Metrics

Log custom metrics with any model call:

await mcpClient.callTool("log_model_call", {
  project: "my-app",
  modelName: "gpt-4",
  inputMessages: [...],
  outputMessages: [...],
  metrics: {
    // Custom numeric metrics
    confidence_score: 0.95,
    relevance_score: 0.87,
    response_quality: 4.5,

    // Custom string metrics
    intent_category: "information_query",
    sentiment: "neutral",
    language: "en"
  }
});

Development

Prerequisites

Node.js 18+
npm or pnpm

Setup

# Install dependencies
npm install

# Build
npm run build

# Run in development mode (watch)
npm run dev

Code Style

The project uses TypeScript with strict mode enabled. Key conventions:

All types defined in src/schema.ts
Tool implementations in separate files under src/tools/
Input validation using Zod schemas
Async/await for all asynchronous operations

Testing

The project includes a comprehensive test suite with 220+ tests:

# Run all tests
npm test

# Run tests in watch mode
npm run test:watch

# Run with coverage report
npm run test:coverage

Test Coverage

Category	Coverage
Statements	97%+
Branches	98%+
Functions	95%+
Lines	97%+

Test Structure

Unit Tests: Individual components (store, tools, resources)
Integration Tests: End-to-end workflows
Edge Cases: Error handling, boundary conditions, concurrent operations

Troubleshooting

Common Issues

Server not connecting:

Verify the path in your MCP client configuration is absolute
Check that the server is built (npm run build)
Ensure Node.js 18+ is installed

Data not persisting:

The default in-memory store loses data on restart
Implement a database-backed store for persistence

High memory usage:

The in-memory store grows unbounded
Implement pagination or data expiration for production use

Debug Mode

Enable debug logging by setting the DEBUG environment variable:

DEBUG=mcp:* node dist/index.js

Platform-Specific Issues

Ensure the config file is valid JSON (no trailing commas)
Use absolute paths, not relative paths
Restart Claude Desktop after config changes
Check the Claude Desktop logs for errors

Verify the config file location (~/.cursor/mcp.json)
Check Cursor's MCP status in the command palette
Ensure the server process can be executed by Cursor

Open Windsurf's Cascade settings
Verify the MCP server is listed and enabled
Check for any error messages in the Cascade panel

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please:

Fork the repository
Create a feature branch
Add tests for new functionality
Ensure all tests pass
Submit a pull request

Support

Issues: Report bugs and request features via GitHub Issues
Documentation: See the MCP specification
Discussions: Join the conversation on GitHub Discussions

⬆ Back to Top

Made with ❤️ for the AI developer community

Available Tools

5 tools

get_aggregate_metricsB

Get aggregate metrics (call count, total tokens, average latency) across model calls with optional filters.

ParametersJSON Schema

Name	Required	Description
`to`	No	Filter calls up to this ISO date
`from`	No	Filter calls from this ISO date onwards
`project`	No	Filter by project
`environment`	No	Filter by environment (dev, staging, prod)

TDQS

B3.4/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. The verb 'Get' implies a read-only, non-mutating operation, and 'aggregate metrics' suggests a summarized response rather than raw calls, but there is no explicit disclosure of permissions, rate limits, or behavior when no filters are supplied. This is minimally adequate for a read-only aggregation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the operation, the metrics returned, and the filtering scope without any wasted words. It is appropriately sized for a simple aggregation tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given low complexity (four optional filter parameters, no output schema, no annotations), the description provides the essential information: what is computed and that filtering is optional. It omits default behavior when filters are absent and does not describe the response shape, but the lack of an output schema means return values need not be explained in detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all four optional filter parameters (to, from, project, environment). The description adds only a generic reference to 'optional filters' and does not provide additional parameter semantics beyond what the schema provides, establishing the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (Get), resource (aggregate metrics), and enumerates the metrics returned (call count, total tokens, average latency). It clearly distinguishes itself from siblings like search_model_calls and log_model_call through the aggregating nature, but does not explicitly name or contrast those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions optional filters but gives no guidance on when to use this tool versus alternatives such as search_model_calls or get_session_calls. There are no when-to-use or when-not-to-use conditions, leaving the agent to infer context from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_session_callsB

Get all model calls for a specific session, sorted by timestamp (oldest first).

ParametersJSON Schema

Name	Required	Description	Default
`sessionId`	Yes	The session ID to retrieve calls for

TDQS

B3.3/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses the sort order (oldest first), which is real behavior beyond the schema, but says nothing about result size, pagination, or whether the list can be large.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that conveys the operation, the scope, and the ordering with zero wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with no output schema, this is nearly sufficient, but the absence of any note on result volume or pagination leaves a gap for a potentially large per-session call list.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single sessionId parameter is fully documented in the schema, so the description adds no parameter-level meaning beyond it. Baseline 3 is appropriate when the schema already does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get all model calls') scoped to a session, plus the ordering guarantee. It is largely distinguishable from siblings like search_model_calls, but the description never explicitly differentiates itself from them or from log_model_call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this versus search_model_calls, list_sessions, or log_model_call, and no prerequisites or exclusions stated. The agent must infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sessionsB

List session summaries showing aggregated metrics for each session. Sessions are groups of related model calls.

ParametersJSON Schema

Name	Required	Description
`limit`	No	Maximum number of results (default 50, max 100)
`project`	No	Filter by project
`environment`	No	Filter by environment (dev, staging, prod)

TDQS

B3.2/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden, and it does convey that results are read-only summaries containing aggregated per-session metrics. However, it says nothing about pagination behavior, result ordering, or what happens when no sessions match, all of which matter for a list endpoint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler; the purpose leads and the domain definition follows. Efficient, though the definition sentence, while useful, is the only extra content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-risk, no-required-param list tool with full schema coverage and no output schema, the description is close to adequate. The remaining gap is routing guidance versus get_session_calls and get_aggregate_metrics, which an agent needs to pick the right list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% — limit, project, and environment are all documented in the schema with defaults and constraints — so the description adds no parameter meaning beyond it. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (session summaries), and the second sentence clarifies that sessions are groups of related model calls. It implicitly distinguishes itself from get_session_calls by specifying summary/aggregated metrics, though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of the alternatives (get_session_calls for detail, get_aggregate_metrics for cross-session rollups) despite five sibling tools. Usage is only implied by the verb 'List'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

log_model_callA

Log a model call for tracking AI usage metrics. Call this after each model invocation to record the interaction.

ParametersJSON Schema

Name	Required	Description	Default
`safety`	No	Safety check results
`userId`	No	Optional user identifier
`metrics`	No	Custom metrics (key-value pairs)
`project`	Yes	Project identifier
`traceId`	No	Distributed tracing ID
`tokensIn`	No	Number of input tokens
`latencyMs`	No	Latency of the call in milliseconds
`modelName`	Yes	Name of the model used (e.g., gpt-4, claude-3-opus)
`requestId`	No	Request ID from the model provider
`sessionId`	No	Optional session identifier for grouping related calls
`tokensOut`	No	Number of output tokens
`promptType`	No	Type of prompt (chat, rag, tool, agent)
`environment`	No	Environment (dev, staging, prod)	dev
`modelVersion`	No	Version of the model
`inputMessages`	Yes	Input messages sent to the model
`outputMessages`	Yes	Output messages from the model
`retrievedContext`	No	Retrieved context for RAG prompts

TDQS

A3.7/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It adds genuine value by specifying invocation timing (after every model invocation), but says nothing about idempotency, whether duplicate calls are deduplicated, permission requirements, or what happens if logging fails.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with the purpose front-loaded and the usage trigger immediately after. Zero filler; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (17 params, nested objects) with no output schema and no annotations, so the description should carry behavioral context. It states purpose and timing but omits idempotency, failure semantics, and how logged data is consumed, leaving meaningful gaps despite the thorough schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% across all 17 parameters, so the schema fully documents each field including nested message structures. The description adds no parameter-level meaning beyond that, making the baseline 3 correct.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Log a model call for tracking AI usage metrics'), which is unambiguous and clearly a write operation in contrast to the read-oriented siblings (search_model_calls, get_aggregate_metrics). It does not explicitly name or differentiate from those siblings, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Call this after each model invocation to record the interaction' gives a clear, actionable trigger condition for the agent. There is no mention of when NOT to call it, no alternative tooling, and no note on failure handling, so it lacks the exclusions a 5 would require.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_model_callsA

Search for logged model calls with optional filters. Returns calls sorted by timestamp (most recent first).

ParametersJSON Schema

Name	Required	Description
`to`	No	Filter calls up to this ISO date
`from`	No	Filter calls from this ISO date onwards
`limit`	No	Maximum number of results (default 50, max 100)
`userId`	No	Filter by user ID
`project`	No	Filter by project
`modelName`	No	Filter by model name
`environment`	No	Filter by environment (dev, staging, prod)

TDQS

A3.6/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses the result ordering ('most recent first'), which is real information the agent cannot get elsewhere since there is no output schema. It does not disclose read-only nature, permissions/auth requirements, or pagination behavior beyond the schema's limit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action and followed by the return behavior. Every clause earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, all-optional read tool, the description covers what it does, that filters are optional, and the ordering of results. It stops short of describing the shape of a call record (no output schema exists) or pagination beyond the limit parameter, but nothing critical to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (from, to, limit, userId, project, modelName, environment) is already documented in the schema. The description adds only the general notion of 'optional filters' and no syntax, defaults, or combination semantics beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb ('Search') and resource ('logged model calls') and states the scope ('optional filters'). It is distinguishable from siblings conceptually, but it never explicitly contrasts itself with get_session_calls or get_aggregate_metrics, so the differentiation is left to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The word 'optional' implies all filters are optional and omitting them returns a broad result set, which is useful implied guidance. However, there is no explicit statement of when to prefer this tool over get_session_calls (session-scoped) or get_aggregate_metrics (aggregates), and no exclusions or prerequisites are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

5 tool updatesv1.0.0Sep 13, 2026
- First observedget_aggregate_metrics
- First observedget_session_calls
- First observedlist_sessions
- First observedlog_model_call
- First observedsearch_model_calls

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.7/5.0

Scored 2026-09-13 10:05 across 5 tools

Disambiguation4/5

Each tool has a distinct primary purpose: write (log_model_call), search raw calls (search_model_calls), list session summaries (list_sessions), fetch session-scoped calls (get_session_calls), and aggregate stats (get_aggregate_metrics). However, get_session_calls overlaps with search_model_calls since a session filter could achieve the same result, and search/aggregate both operate on the same call set with filters.

Naming Consistency5/5

All names follow a consistent snake_case verb_noun pattern (log_, search_, list_, get_, get_). Verbs vary appropriately by operation while the structure stays predictable and readable.

Tool Count5/5

Five tools are well-scoped for a focused usage-metrics server: one write path, three read/query paths, and one aggregation path. Each tool earns its place without redundancy or gaps in count.

Completeness4/5

The core lifecycle is covered: logging calls, retrieving them, session grouping, and aggregation. Minor gaps exist around deletion/retention and explicit session creation, but agents can work around these for typical metrics workflows.

Maintenance

ActivityInactive

ResponsivenessNo issues

Resources

Need Help?

Related Servers

Related MCP Connectors

anotherai
dev.anotherai
MCP server for building and testing AI agents with multi-model experimentation and insights.
Agent Orchestration Autonomous Agents Testing & QA Tools
2
SpanlyOAuth
com.spanly
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
Observability Monitoring
3
TokenOracle
com.guffeyholdings
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
Finance Monitoring
2
mcpulse AOAuth
com.getmcpulse
Analytics for MCP servers. Query your tool calls, first-call success, retries and schema cost.
Data Visualization Monitoring Observability
331

Related MCP Servers

Cost Management MCP
Cloud Platforms Finance Monitoring
knishioka
A
license
B
quality
C
maintenance
An MCP server for unified cost tracking and analysis across AWS, OpenAI, and Anthropic. It enables users to query expenditures, compare costs across providers, and analyze usage trends through natural language.
Updated 15 days ago (2026-09-01 04:15 UTC)
10
MIT
runmeter
Observability Monitoring
Aptica-Solutions
A
license
A
quality
B
maintenance
An MCP server that provides cost and reliability observability for LLM and agent workflows. It records model calls and allows querying and aggregating telemetry data through MCP tools.
Updated 7 days ago (2026-09-09 00:34 UTC)
6
MIT
mcp-audit
Monitoring Observability
nyx-builds
A
license
Not graded
quality
A
maintenance
An MCP server that gives AI agents observability over their own tool calls, enabling auditing, cost tracking, latency analysis, and alerting.
Updated a month ago (2026-08-06 09:10 UTC)
MIT
agent-observability-mcp
Monitoring Observability AI & Machine Learning
AgentIA1984-cmd
A
license
A
quality
B
maintenance
MCP server for AI agent observability, providing trace and span logging, search, latency/tokens/cost metrics, and anomaly detection using an in-memory buffer.
Updated 2 months ago (2026-07-28 06:55 UTC)
6
13 npm
MIT

AI Usage Metrics MCP Server

Table of Contents

Overview

Features

🚀 Quick Start

📦 One-Click Install

Claude Desktop

Claude Code CLI

Cursor

Windsurf

VS Code + Continue

Cline

Zed

Other MCP-Compatible Platforms

Manual Installation

Global Install via npm

From Source (for development)

Package Manager Scripts

Verifying Installation

Usage

Logging Model Calls

Searching Calls

Session Management

Aggregate Metrics

📚 API Reference

Tools

log_model_call

search_model_calls

list_sessions

get_session_calls

get_aggregate_metrics

Resources

Data Model

ModelCallLog

SessionSummary

Message Types

Architecture

Project Structure

Extending the Server

Adding PostgreSQL Support

Database Schema (PostgreSQL)

Adding Custom Metrics

Development

Prerequisites

Setup

Code Style

Testing

Test Coverage

Test Structure

Troubleshooting

Common Issues

Debug Mode

Platform-Specific Issues

License

Contributing

Support

Available Tools

Tool Schema Changelog

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

`log_model_call`

`search_model_calls`

`list_sessions`

`get_session_calls`

`get_aggregate_metrics`