Skip to main content
Glama
Alpha-Park

genpark-semantic-entropy-hallucination-estimator-skill

Official
by Alpha-Park

GenPark AI Agent Skill - Semantic Entropy Hallucination Estimator

A zero-dependency Python standard library skill for estimating LLM hallucination and confabulation via semantic entropy (Kuhn et al.). Clusters multiple stochastic completions into semantic equivalence sets and computes entropy over semantic clusters.

Architecture

graph TD
    A[Stochastic LLM Samples] --> B[N-gram Semantic Equivalence Grouping]
    B --> C[Cluster Assignment]
    C --> D[Compute Cluster Probability P_C]
    D --> E[Semantic Entropy -sum P_C log P_C]
    E --> F{Entropy >= Threshold?}
    F -->|Yes| G[Flag Hallucination / Confabulation]
    F -->|No| H[Verified Factual Alignment]

Related MCP server: groundlens-deprecated

Features

  • Semantic Equivalence Clustering: Group responses by semantic meaning rather than exact token matches.

  • Pure Python 3.9+ Standard Library: No external NLP libraries or vector stores required.

  • Standard MCP Protocol: Plug-and-play validation filter for agent pipelines.

Citations & Ecosystem

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    C
    maintenance
    AI evaluation toolkit that measures inter-rater agreement (Fleiss' κ, Kendall's W) across multiple LLM providers. Evaluate prompt reliability, detect contested outputs, and track consensus trends over time.
    -
  • A
    license
    A
    quality
    A
    maintenance
    Enables hallucination detection for AI assistants by providing tools to assess whether responses are grounded in source material or follow grounded patterns.
    3
    Apache 2.0
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables comparison of responses from multiple LLMs (OpenAI, Anthropic, Gemini) to the same prompt, returning a validated divergence score based on sentence embeddings.
    36 npm
    MIT