mcp-server-circleci

Instructions

This tool retrieves information about flaky tests in a CircleCI project. 

The agent receiving this output MUST analyze the flaky test data and implement appropriate fixes based on the specific issues identified.

CRITICAL REQUIREMENTS:
1. Truncation Handling (HIGHEST PRIORITY):
   - ALWAYS check for <MCPTruncationWarning> in the output
   - When present, you MUST start your response with:
     "WARNING: The logs have been truncated. Only showing the most recent entries. Earlier build failures may not be visible."
   - Only proceed with log analysis after acknowledging the truncation

Input options (EXACTLY ONE of these THREE options must be used):

Option 1 - Project Slug:
- projectSlug: The project slug obtained from listFollowedProjects tool (e.g., "gh/organization/project")

Option 2 - Direct URL (provide ONE of these):
- projectURL: The URL of the CircleCI project in any of these formats:
  * Project URL: https://app.circleci.com/pipelines/gh/organization/project
  * Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123
  * Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def
  * Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/xyz

Option 3 - Project Detection (ALL of these must be provided together):
- workspaceRoot: The absolute path to the workspace root
- gitRemoteURL: The URL of the git remote repository

Additional Requirements:
- Never call this tool with incomplete parameters
- If using Option 1, make sure to extract the projectSlug exactly as provided by listFollowedProjects
- If using Option 2, the URLs MUST be provided by the user - do not attempt to construct or guess URLs
- If using Option 3, BOTH parameters (workspaceRoot, gitRemoteURL) must be provided
- If none of the options can be fully satisfied, ask the user for the missing information before making the tool call

Implementation Reference

• src/tools/getFlakyTests/handler.ts:18-67 (handler)

  Primary MCP tool handler function. Resolves project slug from inputs, fetches flaky tests data, handles file output if enabled, otherwise formats for MCP response.

  export const getFlakyTestLogs: ToolCallback<{
    params: typeof getFlakyTestLogsInputSchema;
  }> = async (args) => {
    const {
      workspaceRoot,
      gitRemoteURL,
      projectURL,
      projectSlug: inputProjectSlug,
    } = args.params ?? {};
  
    let projectSlug: string | null | undefined;
  
    if (inputProjectSlug) {
      projectSlug = inputProjectSlug;
    } else if (projectURL) {
      projectSlug = getProjectSlugFromURL(projectURL);
    } else if (workspaceRoot && gitRemoteURL) {
      projectSlug = await identifyProjectSlug({
        gitRemoteURL,
      });
    } else {
      return mcpErrorOutput(
        'Missing required inputs. Please provide either: 1) projectSlug, 2) projectURL, or 3) workspaceRoot with gitRemoteURL.',
      );
    }
  
    if (!projectSlug) {
      return mcpErrorOutput(`
            Project not found. Ask the user to provide the inputs user can provide based on the tool description.
  
            Project slug: ${projectSlug}
            Git remote URL: ${gitRemoteURL}
            `);
    }
  
    const tests = await getFlakyTests({
      projectSlug,
    });
  
    if (process.env.FILE_OUTPUT_DIRECTORY) {
      try {
        return await writeTestsToFiles({ tests });
      } catch (error) {
        console.error(error);
        return formatFlakyTests(tests);
      }
    }
  
    return formatFlakyTests(tests);
  };

• src/tools/getFlakyTests/inputSchema.ts:4-32 (schema)

  Zod schema defining the input parameters for the find_flaky_tests tool.

  export const getFlakyTestLogsInputSchema = z.object({
    projectSlug: z.string().describe(projectSlugDescriptionNoBranch).optional(),
    workspaceRoot: z
      .string()
      .describe(
        'The absolute path to the root directory of your project workspace. ' +
          'This should be the top-level folder containing your source code, configuration files, and dependencies. ' +
          'For example: "/home/user/my-project" or "C:\\Users\\user\\my-project"',
      )
      .optional(),
    gitRemoteURL: z
      .string()
      .describe(
        'The URL of the remote git repository. This should be the URL of the repository that you cloned to your local workspace. ' +
          'For example: "https://github.com/user/my-project.git"',
      )
      .optional(),
    projectURL: z
      .string()
      .describe(
        'The URL of the CircleCI project. Can be any of these formats:\n' +
          '- Project URL: https://app.circleci.com/pipelines/gh/organization/project\n' +
          '- Project URL with branch: https://app.circleci.com/pipelines/gh/organization/project?branch=feature-branch\n' +
          '- Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123\n' +
          '- Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def\n' +
          '- Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/xyz',
      )
      .optional(),
  });

• src/circleci-tools.ts:68-85 (registration)

  Registers the 'find_flaky_tests' tool name mapped to its handler function in the central CCI_HANDLERS object.

  export const CCI_HANDLERS = {
    get_build_failure_logs: getBuildFailureLogs,
    find_flaky_tests: getFlakyTestLogs,
    get_latest_pipeline_status: getLatestPipelineStatus,
    get_job_test_results: getJobTestResults,
    config_helper: configHelper,
    create_prompt_template: createPromptTemplate,
    recommend_prompt_template_tests: recommendPromptTemplateTests,
    run_pipeline: runPipeline,
    list_followed_projects: listFollowedProjects,
    run_evaluation_tests: runEvaluationTests,
    rerun_workflow: rerunWorkflow,
    download_usage_api_data: downloadUsageApiData,
    find_underused_resource_classes: findUnderusedResourceClasses,
    analyze_diff: analyzeDiff,
    run_rollback_pipeline: runRollbackPipeline,
    list_component_versions: listComponentVersions,
  } satisfies ToolHandlers;

• src/circleci-tools.ts:37-54 (registration)

  Includes the getFlakyTestLogsTool in the exported CCI_TOOLS array for tool registration.

  export const CCI_TOOLS = [
    getBuildFailureLogsTool,
    getFlakyTestLogsTool,
    getLatestPipelineStatusTool,
    getJobTestResultsTool,
    configHelperTool,
    createPromptTemplateTool,
    recommendPromptTemplateTestsTool,
    runPipelineTool,
    listFollowedProjectsTool,
    runEvaluationTestsTool,
    rerunWorkflowTool,
    downloadUsageApiDataTool,
    findUnderusedResourceClassesTool,
    analyzeDiffTool,
    runRollbackPipelineTool,
    listComponentVersionsTool,
  ];

• src/lib/flaky-tests/getFlakyTests.ts:6-56 (helper)

  Core helper function that fetches flaky tests from CircleCI API using insights.getProjectFlakyTests and retrieves detailed test data from jobs.

  const getFlakyTests = async ({ projectSlug }: { projectSlug: string }) => {
    const circleci = getCircleCIClient();
    const flakyTests = await circleci.insights.getProjectFlakyTests({
      projectSlug,
    });
  
    if (!flakyTests || !flakyTests.flaky_tests) {
      throw new Error('Flaky tests not found');
    }
  
    const flakyTestDetails = [
      ...new Set(
        flakyTests.flaky_tests.map((test) => ({
          jobNumber: test.job_number,
          test_name: test.test_name,
        })),
      ),
    ];
  
    const testsArrays = await rateLimitedRequests(
      flakyTestDetails.map(({ jobNumber, test_name }) => async () => {
        try {
          const tests = await circleci.tests.getJobTests({
            projectSlug,
            jobNumber,
          });
          const matchingTest = tests.find((test) => test.name === test_name);
          if (matchingTest) {
            return matchingTest;
          }
          console.error(`Test ${test_name} not found in job ${jobNumber}`);
          return tests.filter((test) => test.result === 'failure');
        } catch (error) {
          if (error instanceof Error && error.message.includes('404')) {
            console.error(`Job ${jobNumber} not found:`, error);
            return undefined;
          } else if (error instanceof Error && error.message.includes('429')) {
            console.error(`Rate limited for job request ${jobNumber}:`, error);
            return undefined;
          }
          throw error;
        }
      }),
    );
  
    const filteredTestsArrays = testsArrays
      .flat()
      .filter((test) => test !== undefined);
  
    return filteredTestsArrays;
  };