Skip to main content
Glama

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
ZEP_API_KEYNoAPI key for Zep (required to run the zep backend).
MEM0_API_KEYNoAPI key for the Mem0 hosted platform (required to run the mem0 backend).
OPENAI_API_KEYNoOpenAI API key used by mem0_direct for embeddings and extraction.
OPENVIKING_API_KEYNoAPI key for OpenViking (required to run the openviking backend).
MEMPALACE_STORAGE_PATHNoPath to the MemPalace storage (required to run the mempalace backend).
MEMTRUST_JUDGE_API_KEYNoAPI key for the LLM judge used in LongMemEval/LoCoMo evals (needed for LLM-judged evals).
MEM0_DIRECT_VECTOR_STORE_URLNoURL for the vector store, default is http://localhost:6333.
MEM0_DIRECT_EMBEDDER_PROVIDERNoEmbedder provider for the mem0_direct backend (e.g., openai).
MEM0_DIRECT_VECTOR_STORE_PROVIDERNoVector store provider for the mem0_direct backend (e.g., qdrant).

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}
prompts
{
  "listChanged": false
}
resources
{
  "subscribe": false,
  "listChanged": false
}
experimental
{}

Tools

Functions exposed to the LLM to take actions

NameDescription
runA

Runs memtrust's agent-memory benchmark suite (LongMemEval, LoCoMo, contradiction-detection, and a dozen other evals) against one or more configured memory backends -- MemPalace, Mem0, Zep/Graphiti, OpenViking -- and returns the resulting JSON report. Call this when an agent needs a fresh, reproducible comparison of memory-backend quality or performance rather than trusting a vendor's self-reported numbers; do not call it just to read documentation (pass ["--help"] for that instead) or to inspect a report that already exists on disk (use the CLI's report subcommand directly for that). A backend is only evaluated if its credential env var is set in the server's environment (e.g. MEM0_SELFHOSTED_API_KEY, GRAPHITI_GEMINI_API_KEY); backends missing credentials are reported as skipped rather than causing an error, so no prerequisite check is required before calling.

Behavior: this shells out to the real memtrust run CLI as a subprocess (network calls happen for whichever backends are configured; evals also read/write a private temp file for the JSON report, cleaned up automatically). It is read-only with respect to the backends themselves -- it queries and scores them, it does not modify backend state -- and each call is idempotent: running the same args twice re-runs the full suite and produces a fresh, independent report rather than mutating prior results. Runs can take several minutes (subprocess timeout is 600s) depending on which evals and backends are selected. On a non-zero CLI exit, the tool does not raise; it returns {"error": ..., "returncode": ..., "stderr": ...} so the calling agent can inspect what failed.

Parameter: args is a list[str] of argv tokens appended to memtrust run (NOT a shell string, and do not include "run" itself or an --output flag -- the wrapper supplies its own temp output path). Real examples pulled from memtrust run --help:

  • [] -- run every eval against every configured backend (both flags default to "all")

  • ["--backends", "mempalace", "--eval", "contradiction"] -- one backend, one eval

  • ["--backends", "mem0,zep", "--eval", "longmemeval,locomo", "--scale-stress-n-records", "10000"] -- multiple backends and evals, plus a numeric tuning flag

  • ["--help"] -- exits before writing a report, so it surfaces as an error dict whose stdout field holds the CLI's own live help text; useful for discovering flags this description doesn't enumerate, such as --sign or --locomo-dataset-path

Return shape: on success, the parsed contents of memtrust's JSON report file (per-backend, per-eval results and scores -- see memtrust report --help for how a human would render the same file). On failure, a dict with an error key plus whichever of returncode / stderr / stdout is available.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

TDQS

A4.7/5.0

Scored across 1 tool

Disambiguation5/5

With only a single tool, there is no possibility of ambiguity or misselection. The tool's purpose is clearly defined and distinct from any other tool because none exist.

Naming Consistency5/5

The lone tool name 'run' is a clear verb that matches its action. While there is no pattern to compare against, the naming is consistent and intuitive for the single operation it performs.

Tool Count4/5

The server has exactly one tool, which is appropriate for its focused purpose of running a benchmark suite. Although the count is minimal, it fully covers the server's intended functionality without unnecessary bloat.

Completeness3/5

The tool covers the core operation of running benchmarks, but lacks separate tools for inspecting existing reports or listing backends/evals. These gaps are partially mitigated by the ability to pass '--help' and use the CLI's subcommands, but the MCP surface itself is limited to a single action.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive