rag-pipeline-utils

Retrieval-augmented generation for Node.js, with answers you can check. Build a pipeline in one call, then check every answer against its sources: which sentences are supported, with numbers and negation checked, and no model calls needed.

npm install @devilsdev/rag-pipeline-utils

Node.js 22 or later. CommonJS, ESM and TypeScript declarations included.

Check an answer

From this library's pipeline, or from LangChain, Mastra, the AI SDK or LlamaIndex: pass the answer and the sources it was given.

import { checkAnswer } from "@devilsdev/rag-pipeline-utils";

const check = await checkAnswer({
  answer: "Full-time employees receive thirty days of paid vacation per year.",
  sources: [
    {
      pageContent:
        "Full-time employees receive twenty days of paid vacation per year.",
      metadata: { source: "handbook.md" },
    },
  ],
});

console.log(check.grounded); // false
console.log(check.unsupported.length); // 1

Try it on StackBlitz with an answer of your own. No install or API key.

Related MCP server: rag_mcp

Build a pipeline

This runs offline, with the built-in TF-IDF embedder and an in-memory store. Add an LLM, such as OpenAIConnector or any AI SDK model, to generate answers.

import {
  createRagPipeline,
  LocalEmbedder,
  MemoryRetriever,
} from "@devilsdev/rag-pipeline-utils";

const docs = [
  {
    id: "vacation",
    content:
      "Full-time employees receive twenty days of paid vacation per year.",
  },
  {
    id: "remote",
    content: "Remote work is allowed two days per week with manager approval.",
  },
];

const embedder = new LocalEmbedder();
embedder.train(docs.map((doc) => doc.content));
const retriever = new MemoryRetriever();
await retriever.store(
  docs.map((doc) => ({ ...doc, vector: embedder.embed(doc.content) })),
);

const pipeline = createRagPipeline({ embedder, retriever });
const result = await pipeline.run({
  query: "How many vacation days do I get?",
});

console.log(result.results[0].id); // "vacation"

What's in it

• Citations and grounding: per-sentence sources and a groundedness score, with numeric and negation checks. An optional LLM judge rules on paraphrases, and verifyCitations() checks the citations Claude and OpenAI return.

• Evaluation in CI: rag-pipeline eval scores a test set and fails the build when answers regress; rag-eval-action comments on pull requests.

• Framework integrations: grounding middleware for any Vercel AI SDK model, Mastra scorers and guardrails, and test matchers for Jest and Vitest.

• Guardrails: prompt-injection detection and PII redaction, with your own detectors and published precision and recall, and ACL filtering before generation.

• Production behaviour: hybrid search, reranking, token budgets, timeouts and cancellation, retries, caching, and OpenTelemetry spans with evaluation scores as events (or Langfuse scores).

• Connectors: OpenAI, Anthropic, Cohere, Ollama, pgvector, Qdrant and Pinecone (3.6.0); anything with embed() or generate() works too.

• A CLI: init, ingest, query, eval and serve against a local index, offline if you want.

• Starter templates (3.6.0): npm create rag-pipeline for Next.js with the AI SDK, Mastra or Cloudflare Workers.

Packages

Learn more

• Use cases with runnable code, most of them on StackBlitz

• Documentation: guides, API reference and the CLI

• How it compares with LangChain.js, Mastra and the AI SDK

• Examples and changelog

Published from GitHub Actions with npm provenance. Report vulnerabilities as described in SECURITY.md. MIT licensed.