CircleCI MCP Server

Instructions

This tool retrieves test metadata for a CircleCI job.

PRIORITY USE CASE:
- When asked "are tests passing in CI?" or similar questions about test status
- When asked to "fix failed tests in CI" or help with CI test failures
- Use this tool to check if tests are passing in CircleCI and identify failed tests

Common use cases:
- Get test metadata for a specific job
- Get test metadata for all jobs in a project
- Get test metadata for a specific branch
- Get test metadata for a specific pipeline
- Get test metadata for a specific workflow
- Get test metadata for a specific job

CRITICAL REQUIREMENTS:
1. Truncation Handling (HIGHEST PRIORITY):
   - ALWAYS check for <MCPTruncationWarning> in the output
   - When present, you MUST start your response with:
     "WARNING: The test results have been truncated. Only showing the most recent entries. Some test data may not be visible."
   - Only proceed with test result analysis after acknowledging the truncation

2. Test Result Filtering:
   - Use filterByTestsResult parameter to filter test results:
     * filterByTestsResult: 'failure' - Show only failed tests
     * filterByTestsResult: 'success' - Show only successful tests
   - When looking for failed tests, ALWAYS set filterByTestsResult to 'failure'
   - When checking if tests are passing, set filterByTestsResult to 'success'

Input options (EXACTLY ONE of these THREE options must be used):

Option 1 - Project Slug and branch (BOTH required):
- projectSlug: The project slug obtained from listFollowedProjects tool (e.g., "gh/organization/project")
- branch: The name of the branch (required when using projectSlug)

Option 2 - Direct URL (provide ONE of these):
- projectURL: The URL of the CircleCI job in any of these formats:
  * Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/789
  * Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def
  * Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123

Option 3 - Project Detection (ALL of these must be provided together):
- workspaceRoot: The absolute path to the workspace root
- gitRemoteURL: The URL of the git remote repository
- branch: The name of the current branch

For simple test status checks (e.g., "are tests passing in CI?") or fixing failed tests, prefer Option 1 with a recent pipeline URL if available.

Additional Requirements:
- Never call this tool with incomplete parameters
- If using Option 1, make sure to extract the projectSlug exactly as provided by listFollowedProjects and include the branch parameter
- If using Option 2, the URL MUST be provided by the user - do not attempt to construct or guess URLs
- If using Option 3, ALL THREE parameters (workspaceRoot, gitRemoteURL, branch) must be provided
- If none of the options can be fully satisfied, ask the user for the missing information before making the tool call

Implementation Reference

• src/tools/getJobTestResults/handler.ts:14-72 (handler)

  The main handler function that implements the logic for retrieving and formatting job test results based on provided parameters like projectSlug, branch, projectURL, etc. It determines the project slug, fetches test results using getJobTests, and formats them.

  export const getJobTestResults: ToolCallback<{
    params: typeof getJobTestResultsInputSchema;
  }> = async (args) => {
    const {
      workspaceRoot,
      gitRemoteURL,
      branch,
      projectURL,
      filterByTestsResult,
      projectSlug: inputProjectSlug,
    } = args.params;
  
    let pipelineNumber: number | undefined;
    let projectSlug: string | undefined;
    let jobNumber: number | undefined;
    let branchFromURL: string | undefined;
  
    if (inputProjectSlug) {
      if (!branch) {
        return mcpErrorOutput(
          'Branch not provided. When using projectSlug, a branch must also be specified.',
        );
      }
      projectSlug = inputProjectSlug;
    } else if (projectURL) {
      pipelineNumber = getPipelineNumberFromURL(projectURL);
      projectSlug = getProjectSlugFromURL(projectURL);
      branchFromURL = getBranchFromURL(projectURL);
      jobNumber = getJobNumberFromURL(projectURL);
    } else if (workspaceRoot && gitRemoteURL && branch) {
      projectSlug = await identifyProjectSlug({
        gitRemoteURL,
      });
    } else {
      return mcpErrorOutput(
        'Missing required inputs. Please provide either: 1) projectSlug with branch, 2) projectURL, or 3) workspaceRoot with gitRemoteURL and branch.',
      );
    }
  
    if (!projectSlug) {
      return mcpErrorOutput(`
        Project not found. Please provide a valid project URL or project information.
  
        Project slug: ${projectSlug}
        Git remote URL: ${gitRemoteURL}
        Branch: ${branch}
      `);
    }
  
    const testResults = await getJobTests({
      projectSlug,
      pipelineNumber,
      branch: branchFromURL || branch,
      jobNumber,
      filterByTestsResult,
    });
  
    return formatJobTests(testResults);
  };

• src/tools/getJobTestResults/inputSchema.ts:7-45 (schema)

  Zod schema defining the input parameters for the get_job_test_results tool, including optional fields for projectSlug, branch, workspaceRoot, gitRemoteURL, projectURL, and filterByTestsResult.

  export const getJobTestResultsInputSchema = z.object({
    projectSlug: z.string().describe(projectSlugDescription).optional(),
    branch: z.string().describe(branchDescription).optional(),
    workspaceRoot: z
      .string()
      .describe(
        'The absolute path to the root directory of your project workspace. ' +
          'This should be the top-level folder containing your source code, configuration files, and dependencies. ' +
          'For example: "/home/user/my-project" or "C:\\Users\\user\\my-project"',
      )
      .optional(),
    gitRemoteURL: z
      .string()
      .describe(
        'The URL of the remote git repository. This should be the URL of the repository that you cloned to your local workspace. ' +
          'For example: "https://github.com/user/my-project.git"',
      )
      .optional(),
    projectURL: z
      .string()
      .describe(
        'The URL of the CircleCI project. Can be any of these formats:\n' +
          '- Project URL: https://app.circleci.com/pipelines/gh/organization/project\n' +
          '- Project URL with branch: https://app.circleci.com/pipelines/gh/organization/project?branch=feature-branch\n' +
          '- Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123\n' +
          '- Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def\n' +
          '- Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/123',
      )
      .optional(),
    filterByTestsResult: z
      .enum(['failure', 'success'])
      .describe(
        `Filter the tests by result.
        If "failure", only failed tests will be returned.
        If "success", only successful tests will be returned.
        `,
      )
      .optional(),
  });

• src/tools/getJobTestResults/tool.ts:4-61 (registration)

  Defines the tool object getJobTestResultsTool with name 'get_job_test_results', detailed description, and references the input schema.

  export const getJobTestResultsTool = {
    name: 'get_job_test_results' as const,
    description: `
      This tool retrieves test metadata for a CircleCI job.
  
      PRIORITY USE CASE:
      - When asked "are tests passing in CI?" or similar questions about test status
      - When asked to "fix failed tests in CI" or help with CI test failures
      - Use this tool to check if tests are passing in CircleCI and identify failed tests
      
      Common use cases:
      - Get test metadata for a specific job
      - Get test metadata for all jobs in a project
      - Get test metadata for a specific branch
      - Get test metadata for a specific pipeline
      - Get test metadata for a specific workflow
      - Get test metadata for a specific job
  
      CRITICAL REQUIREMENTS:
      1. Truncation Handling (HIGHEST PRIORITY):
         - ALWAYS check for <MCPTruncationWarning> in the output
         - When present, you MUST start your response with:
           "WARNING: The test results have been truncated. Only showing the most recent entries. Some test data may not be visible."
         - Only proceed with test result analysis after acknowledging the truncation
  
      2. Test Result Filtering:
         - Use filterByTestsResult parameter to filter test results:
           * filterByTestsResult: 'failure' - Show only failed tests
           * filterByTestsResult: 'success' - Show only successful tests
         - When looking for failed tests, ALWAYS set filterByTestsResult to 'failure'
         - When checking if tests are passing, set filterByTestsResult to 'success'
  
      Input options (EXACTLY ONE of these THREE options must be used):
  
      ${option1DescriptionBranchRequired}
  
      Option 2 - Direct URL (provide ONE of these):
      - projectURL: The URL of the CircleCI job in any of these formats:
        * Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/789
        * Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def
        * Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123
  
      Option 3 - Project Detection (ALL of these must be provided together):
      - workspaceRoot: The absolute path to the workspace root
      - gitRemoteURL: The URL of the git remote repository
      - branch: The name of the current branch
      
      For simple test status checks (e.g., "are tests passing in CI?") or fixing failed tests, prefer Option 1 with a recent pipeline URL if available.
  
      Additional Requirements:
      - Never call this tool with incomplete parameters
      - If using Option 1, make sure to extract the projectSlug exactly as provided by listFollowedProjects and include the branch parameter
      - If using Option 2, the URL MUST be provided by the user - do not attempt to construct or guess URLs
      - If using Option 3, ALL THREE parameters (workspaceRoot, gitRemoteURL, branch) must be provided
      - If none of the options can be fully satisfied, ask the user for the missing information before making the tool call
      `,
    inputSchema: getJobTestResultsInputSchema,
  };

• src/circleci-tools.ts:30-72 (registration)

  Registers the getJobTestResultsTool in the CCI_TOOLS array (line 34) and maps 'get_job_test_results' to getJobTestResults handler in CCI_HANDLERS (line 62). Includes imports at lines 8-9.

  export const CCI_TOOLS = [
    getBuildFailureLogsTool,
    getFlakyTestLogsTool,
    getLatestPipelineStatusTool,
    getJobTestResultsTool,
    configHelperTool,
    createPromptTemplateTool,
    recommendPromptTemplateTestsTool,
    runPipelineTool,
    listFollowedProjectsTool,
    runEvaluationTestsTool,
    rerunWorkflowTool,
    analyzeDiffTool,
    runRollbackPipelineTool,
  ];
  
  // Extract the tool names as a union type
  type CCIToolName = (typeof CCI_TOOLS)[number]['name'];
  
  export type ToolHandler<T extends CCIToolName> = ToolCallback<{
    params: Extract<(typeof CCI_TOOLS)[number], { name: T }>['inputSchema'];
  }>;
  
  // Create a type for the tool handlers that directly maps each tool to its appropriate input schema
  type ToolHandlers = {
    [K in CCIToolName]: ToolHandler<K>;
  };
  
  export const CCI_HANDLERS = {
    get_build_failure_logs: getBuildFailureLogs,
    find_flaky_tests: getFlakyTestLogs,
    get_latest_pipeline_status: getLatestPipelineStatus,
    get_job_test_results: getJobTestResults,
    config_helper: configHelper,
    create_prompt_template: createPromptTemplate,
    recommend_prompt_template_tests: recommendPromptTemplateTests,
    run_pipeline: runPipeline,
    list_followed_projects: listFollowedProjects,
    run_evaluation_tests: runEvaluationTests,
    rerun_workflow: rerunWorkflow,
    analyze_diff: analyzeDiff,
    run_rollback_pipeline: runRollbackPipeline,
  } satisfies ToolHandlers;