Skip to main content
Glama
Sign In
total-agent-memory

total-agent-memory

by vbcherepanov
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

memory_eval_locomo

Read-onlyIdempotent

Runs recall and prevention scenario tests against the live memory store, measuring accuracy and latency. Supports configurable modes and scenario limits.

Instructions

v11.0 Phase 8: run the LongMemEval-style recall+prevention scenario suite (loaded from evals/scenarios/) against the live store. Forces MEMORY_MODE=fast by default. Returns {scenarios_total, scenarios_passed, recall_at_5, recall_at_10, latency_ms, mode, llm_calls_during_eval, network_calls_during_eval}.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "memory_eval_locomo"
  }
}

Input Schema

NameRequiredDescriptionDefault
modeNofast
limitNoCap how many scenarios to run.
top_kNo
scenarios_pathNoOptional override path.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the key behavioral detail of forcing MEMORY_MODE=fast, which is not covered by the annotations. Since the annotations already indicate read-only, idempotent, and non-destructive behavior, the description adds useful context without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that concisely conveys the action, input source, default behavior, and return fields. There is no extraneous information, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (4 parameters, no output schema), the description provides the return fields and default mode, which is helpful. However, it omits details about scenario format or output interpretation, leaving minor gaps for a fully autonomous agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 50% (only limit and scenarios_path have descriptions). The tool description does not clarify the meaning of mode or top_k, nor does it explain the return field semantics beyond listing them. This leaves significant ambiguity for the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: running a LongMemEval-style recall+prevention scenario suite against the live store. It differentiates from sibling eval tools by specifying the scenario type, though it does not explicitly name the alternative tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance on when to use this tool versus the many sibling eval tools (e.g., memory_eval_recall, memory_eval_temporal). It implies usage for LongMemEval-style scenarios but doesn't state conditions or alternatives directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools