@damoxing/datasource-manager

中文文档: README.zh-CN.md

HTTP-agnostic multi-datasource lifecycle manager.

It owns:

• datasource config loading

• datasource status tracking

• generic route-key resolution, with jgbh compatibility

• heartbeat

• pool recovery

• one-shot retry for connection errors

• graceful shutdown

The core manager is not bound to HTTP. It can run inside Express, a worker, a CLI, or another Node.js service.

Database adapters are included for Oracle, OceanBase (Oracle / MySQL Mode), MySQL, DM, PostgreSQL, GaussDB/openGauss, and Kingbase. Drivers are optional peer dependencies and are loaded only when the corresponding adapter is used.

Adapter Contract

An adapter instance should expose:

{
    isOracle: Boolean,
    adapterType: String,
    initialize: async () => {},
    close: async () => {},
    all: async (sql, params) => [],
    get: async (sql, params) => row,
    run: async (sql, params) => result,
    exec: async (sql) => {},
    transaction: async (work) => result
}

withConnection(callback) is optional.

Related MCP server: telemetry-mcp

Usage

CommonJS:

const {
    DataSourceManager,
    createDefaultAdapterFactories
} = require('@damoxing/datasource-manager');

const manager = new DataSourceManager({
    config,
    logger,
    shutdownSignals: ['SIGTERM'],
    adapterFactories: createDefaultAdapterFactories(logger),
});

await manager.initialize();

const db = manager.getByJgbh('1001');
const row = await db.get('SELECT 1 AS health_check');

await manager.closeAll();

ESM:

import {
    DataSourceManager,
    createDefaultAdapterFactories
} from '@damoxing/datasource-manager';

const manager = new DataSourceManager({
    config,
    logger,
    adapterFactories: createDefaultAdapterFactories(logger),
});

Use getByRouteKey() for new non-business-specific code:

const db = manager.getByRouteKey('tenant-a');

getByJgbh() remains as a compatibility wrapper.

MCP Server For Codex

This package includes a stdio MCP server so Codex can inspect configured datasources through the same manager and adapter layer:

npm run build
DAMOXING_MCP_CONFIG=/absolute/path/to/datasources.json node dist/cjs/mcp/server.js

After package installation, the bin entry is:

damoxing-datasource-mcp

The MCP server exposes health, read-only query, table/routine metadata, explain-plan tools, and MCP-only fixed physical sessions:

• damoxing_open_session

• damoxing_session_query

• damoxing_session_exec

• damoxing_session_commit

• damoxing_session_rollback

• damoxing_cancel_session_operation

• damoxing_get_notices

• damoxing_get_audit_events

• damoxing_close_session

All operations carrying the same sessionId use the same leased database connection and are serialized. PostgreSQL-compatible sessions run inside an explicit transaction; close and abnormal connection cleanup roll back before releasing the lease. A lost fixed connection is never retried on another connection.

Fixed sessions are read-only by default. Opening a read/write session requires DAMOXING_MCP_ALLOW_WRITE=true and confirmWrite=true. Datasource ids listed in DAMOXING_MCP_PRODUCTION_DATASOURCES are denied unless DAMOXING_MCP_ALLOW_PRODUCTION_DEBUG=true is set and the call passes confirmProduction=true. Destructive SQL additionally requires DAMOXING_MCP_ALLOW_DESTRUCTIVE=true and confirmDestructive=true.

Session policy can be bounded with:

• DAMOXING_MCP_MAX_SESSIONS (default 5)

• DAMOXING_MCP_MAX_SESSIONS_PER_DATASOURCE (default 2)

• DAMOXING_MCP_SESSION_IDLE_TIMEOUT_MS (default 300000)

• DAMOXING_MCP_SESSION_MAX_LIFETIME_MS (default 1800000)

• DAMOXING_MCP_SESSION_CLEANUP_INTERVAL_MS (default 30000)

• DAMOXING_MCP_AUDIT_MAX_EVENTS (default 1000)

The MCP session registry keeps a bounded in-memory audit trail. Audit events contain action, outcome, datasource/session identifiers, elapsed time, row counts, SQL kind, a normalized SQL fingerprint, and parameter counts/names. SQL text and parameter values are never stored. Use damoxing_get_audit_events to retrieve the redacted events.

The fixed-session manager, registry, timer, and reserved connections are created lazily by the MCP entry point. Importing @damoxing/datasource-manager directly does not load the MCP SDK or create MCP resources.

Fixed-session integration results from the local OrbStack lab on July 15, 2026:

Run npm run test:integration:session:pg, npm run test:integration:session:opengauss, npm run test:integration:session:oracle, and npm run test:integration:session:dm to repeat the verified matrix.

This phase is the fixed-session foundation, not complete stored-routine debugging. Structured OUT/INOUT values, cursor handles/fetch, routine profiling, and native breakpoints remain roadmap work.

The repository skill lives at skills/damoxing-database-mcp.

Health Output

getHealth() returns a stable, sanitized schema:

{
    generatedAt,
    initialized,
    strictRouting,
    defaultDatasourceId,
    routeCount,
    heartbeat: {
        enabled,
        running,
        intervalMs
    },
    shutdownHooks: {
        installed,
        signals
    },
    summary: {
        total,
        ready,
        failed,
        unhealthy,
        byStatus: {
            configured,
            initializing,
            ready,
            unhealthy,
            failed,
            recovering,
            closed
        }
    },
    datasources: [
        {
            id,
            type,
            status,
            jgbhCount,
            routeKeyCount,
            isDefault,
            lastHeartbeatAt,
            lastReadyAt,
            lastRecoverAt,
            lastStatusChangeAt,
            lastError
        }
    ]
}

Datasource config is never included in health output, so passwords and connect strings are not exposed.

Logging

Datasource errors are logged with structured context as the second logger argument:

{
    datasourceId,
    type,
    jgbh,
    jgbhList,
    routeKeys,
    status,
    lastError,
    error
}

SQL text logging is enabled by default. SQL parameter logging is disabled by default to avoid leaking production secrets.

Use adapter options to control parameter logging:

const adapterFactories = createDefaultAdapterFactories({
    logger,
    logSql: true,
    logParams: 'redacted',
    redactKeys: ['password', 'token', 'secret'],
    redactValue: '[REDACTED]'
});

logParams accepts:

• false or 'off': do not log SQL parameters

• true or 'redacted': log parameters with sensitive keys redacted

• 'raw': log raw parameters for local debugging only

Graceful Shutdown

Use shutdownSignals to close all pools when the process receives a signal:

const manager = new DataSourceManager({
    shutdownSignals: ['SIGTERM', 'SIGINT'],
    exitOnShutdownSignal: true,
    adapterFactories,
    config
});

closeAll() stops heartbeat timers, removes shutdown hooks, closes adapters, and marks datasources as closed.

Config Shape

{
    "strict_routing": true,
    "default_datasource": "oracle_main",
    "datasources": [
        {
            "id": "oracle_main",
            "type": "oracle",
            "jgbh_list": ["1001"],
            "route_keys": ["tenant-a"],
            "config": {}
        }
    ]
}

Pool Governance

Use normalized pool options at datasource level or under config.pool:

{
    "id": "pg_main",
    "type": "pg",
    "pool": {
        "minPoolSize": 1,
        "maxPoolSize": 10,
        "acquireTimeoutMs": 3000,
        "connectTimeoutMs": 3000,
        "idleTimeoutMs": 60000,
        "maxLifetimeMs": 1800000,
        "keepaliveMs": 30000
    },
    "config": {}
}

The manager maps supported options to each driver and logs unsupported options with datasource context. Invalid values, such as minPoolSize > maxPoolSize, fail during config build.

applyPoolGovernance(type, config, pool) is exported for tests and diagnostics.

Metrics

getMetrics() returns counters, gauges, and latency summaries without SQL text, parameters, passwords, or connection strings:

const metrics = manager.getMetrics();

Metrics currently track initialization, heartbeat, recovery, query success/failure, connection errors, retries, and transaction connection errors.

Runtime Governance

Datasources can be changed at runtime:

await manager.addDatasource({ id: 'tenant_a', type: 'pg', route_keys: ['TENANT_A'], config: {} });
await manager.updateDatasource('tenant_a', { id: 'tenant_a', type: 'pg', route_keys: ['TENANT_A'], config: {} });
await manager.removeDatasource('tenant_a');
await manager.reloadConfig(nextConfig);

updateDatasource() initializes and pings the replacement before closing the old pool. If replacement initialization fails, the old datasource remains active.

Read/Write Groups

Group routing is HTTP-agnostic:

{
    "groups": {
        "tenant_a": {
            "write": "tenant_a_master",
            "read": [
                "tenant_a_read_1",
                { "id": "tenant_a_read_2", "weight": 2 }
            ],
            "strategy": "round-robin",
            "fallbackToWrite": true
        }
    }
}

const readDb = manager.getReadHandle('tenant_a');
const writeDb = manager.getWriteHandle('tenant_a');

Read strategies are round-robin, random, weighted, and first-ready. Unhealthy read replicas are skipped.

Config Secrets

Config values support environment interpolation, environment references, secret resolver hooks, and encrypted values:

const manager = new DataSourceManager({
    env: process.env,
    secretResolver: async key => loadSecret(key),
    decryptor: async value => decrypt(value),
    config
});

{
    "password": "env:DB_PASSWORD",
    "token": "secret:database/token",
    "connectString": "postgres://app:${DB_PASSWORD}@localhost/db",
    "encrypted": "ENC(ciphertext)"
}

Resolved secrets are redacted from health output, metrics output, structured datasource error state, and manager logs.

Retry Rule

Connection-like errors may be retried once after pool recovery.

Transactions are not replayed automatically. If a transaction fails with a connection error, the manager rebuilds the pool and rethrows the original error.

npm Packaging

This package is authored in TypeScript under src/ and builds both module formats:

• CommonJS: dist/cjs/index.js

• ESM: dist/esm/index.mjs

• Types: dist/types/index.d.ts

Build locally before publishing:

npm install
npm run release:check

The ESM output is compiled from TypeScript with esbuild. Root and adapter aggregate imports keep database drivers lazy, so importing the package does not immediately load oracledb, dmdb, or pg.

Built-in database drivers are declared as optional peer dependencies:

• oracledb for Oracle

• mysql2 for MySQL and both OceanBase Oracle/MySQL tenant modes

• dmdb for DM

• pg for PostgreSQL, GaussDB/openGauss, and Kingbase

Importing the package root does not immediately load these drivers. Accessing adapter classes or creating an adapter through createDefaultAdapterFactories() loads only the driver needed by that datasource type.

See RELEASE.md for semver, registry, token, provenance, and final publish checklist guidance.

See ROADMAP.md for the enterprise datasource framework backlog and priority order.