failed-tests-summary

Instructions

Implementation Reference

• src/tools/aggregations.ts:17-43 (handler)

  Main handler function that aggregates dbt failed tests, DQ check failures by dataset, and most recent DQ failures into a single summary. Uses aggregate() to run all three data sources in parallel, returns combined result with caveats.

  export async function failedTestsSummary(
    args: z.infer<typeof failedTestsSummarySchema>,
  ): Promise<unknown> {
    const caveats: string[] = [];
    const { dbt, dqByDataset, dqLatest } = await aggregate(
      {
        dbt: () => dbtFailedTests({ recentRuns: args.recentRuns }),
        dqByDataset: () =>
          dqConfigured()
            ? dqFailedChecksByDataset({ sinceHours: args.sinceHours, topN: 20 })
            : Promise.resolve(null),
        dqLatest: () =>
          dqConfigured()
            ? dqListChecks({ sinceHours: args.sinceHours, status: "fail", limit: 50 })
            : Promise.resolve(null),
      },
      caveats,
    );
    if (!dqConfigured()) caveats.push("DQ_RESULTS_TABLE not configured — quality category skipped");
    return {
      window: { recentRuns: args.recentRuns, sinceHours: args.sinceHours },
      dbtFailures: dbt,
      dqFailuresByDataset: dqByDataset,
      dqRecentFailures: dqLatest,
      caveats,
    };
  }

• src/tools/aggregations.ts:12-15 (schema)

  Zod schema defining input parameters: recentRuns (N dbt runs, 1-20, default 3) and sinceHours (DQ window, 1-720, default 24).

  export const failedTestsSummarySchema = z.object({
    recentRuns: z.coerce.number().int().min(1).max(20).default(3).describe("Look at last N dbt runs"),
    sinceHours: z.coerce.number().int().min(1).max(720).default(24).describe("Recent window for DQ checks"),
  });

• src/index.ts:110-110 (registration)

  Registration call that binds the tool name 'failed-tests-summary' with its description, schema, and handler on the MCP server via the local tool() helper function.

  tool("failed-tests-summary", "Aggregated 24h-ish view: dbt failed tests + DQ checks failures grouped by dataset + most recent failing rows. Replaces 3+ tool calls (dbt-failed-tests + dq-failed-checks-by-dataset + dq-list-checks).", failedTestsSummarySchema.shape, wrapToolHandler(failedTestsSummary));

• src/index.ts:66-74 (registration)

  The tool() helper function that registers the tool with the registry (categorizing it under 'quality') and conditionally adds it to the MCP server if the category is enabled.

  let currentCategory: Category = "dbt";
  
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  function tool(name: string, description: string, schema: any, handler: any): void {
    registry.register(name, description, currentCategory);
    if (registry.isEnabled(currentCategory)) {
      server.tool(name, description, schema, handler);
    }
  }

• src/prompts/index.ts:29-43 (helper)

  Reference in the 'failed-tests-deep-dive' prompt that instructs the LLM to call failed-tests-summary as the first step of investigation.

            `1. Call \`failed-tests-summary\` with recentRuns=${runs}, sinceHours=${hours}. Capture dbt failures and dq failures by dataset.`,
            "2. For the top 5 failing datasets, call `dbt-list-models` filtered by schema=<dataset> and shortlist models with the highest failure count.",
            "3. For each top failing test, call `dbt-get-test` to read the test definition and `dbt-graph` (upstream depth=2) on the attached model — note any failed upstream models or stale sources.",
            "4. Cross-check: call `freshness-status` (failingOnly=true) to see if any of these tests sit downstream of a freshness violation.",
            "5. If @us-all/airflow-mcp is also installed, suggest the user call `airflow-list-runs` for the loading DAGs of the affected datasets to confirm scheduling/run-time issues.",
            "6. Produce a remediation report:",
            "   - Top failing datasets (with failure counts and severity).",
            "   - Failures classified as: 'upstream broken' / 'source stale' / 'schema drift' / 'data anomaly' / 'unknown'.",
            "   - Per-failure: test name, attached model, severity, last failure timestamp, message, suggested action.",
            "   - Owners-to-notify based on dbt model meta (if present).",
          ].join("\n"),
        },
      },
    ],
  };