mcp-server-circleci

Instructions

This tool retrieves test metadata for a CircleCI job.

PRIORITY USE CASE:
- When asked "are tests passing in CI?" or similar questions about test status
- When asked to "fix failed tests in CI" or help with CI test failures
- Use this tool to check if tests are passing in CircleCI and identify failed tests

Common use cases:
- Get test metadata for a specific job
- Get test metadata for all jobs in a project
- Get test metadata for a specific branch
- Get test metadata for a specific pipeline
- Get test metadata for a specific workflow
- Get test metadata for a specific job

CRITICAL REQUIREMENTS:
1. Truncation Handling (HIGHEST PRIORITY):
   - ALWAYS check for <MCPTruncationWarning> in the output
   - When present, you MUST start your response with:
     "WARNING: The test results have been truncated. Only showing the most recent entries. Some test data may not be visible."
   - Only proceed with test result analysis after acknowledging the truncation

2. Test Result Filtering:
   - Use filterByTestsResult parameter to filter test results:
     * filterByTestsResult: 'failure' - Show only failed tests
     * filterByTestsResult: 'success' - Show only successful tests
   - When looking for failed tests, ALWAYS set filterByTestsResult to 'failure'
   - When checking if tests are passing, set filterByTestsResult to 'success'

Input options (EXACTLY ONE of these THREE options must be used):

Option 1 - Project Slug and branch (BOTH required):
- projectSlug: The project slug obtained from listFollowedProjects tool (e.g., "gh/organization/project")
- branch: The name of the branch (required when using projectSlug)

Option 2 - Direct URL (provide ONE of these):
- projectURL: The URL of the CircleCI job in any of these formats:
  * Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/789
  * Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def
  * Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123

Option 3 - Project Detection (ALL of these must be provided together):
- workspaceRoot: The absolute path to the workspace root
- gitRemoteURL: The URL of the git remote repository
- branch: The name of the current branch

For simple test status checks (e.g., "are tests passing in CI?") or fixing failed tests, prefer Option 1 with a recent pipeline URL if available.

Additional Requirements:
- Never call this tool with incomplete parameters
- If using Option 1, make sure to extract the projectSlug exactly as provided by listFollowedProjects and include the branch parameter
- If using Option 2, the URL MUST be provided by the user - do not attempt to construct or guess URLs
- If using Option 3, ALL THREE parameters (workspaceRoot, gitRemoteURL, branch) must be provided
- If none of the options can be fully satisfied, ask the user for the missing information before making the tool call

Implementation Reference

• src/tools/getJobTestResults/handler.ts:14-72 (handler)

  The main handler function that implements the logic for retrieving and formatting CircleCI job test results based on input parameters such as project slug, branch, URL, or workspace details.

  export const getJobTestResults: ToolCallback<{
    params: typeof getJobTestResultsInputSchema;
  }> = async (args) => {
    const {
      workspaceRoot,
      gitRemoteURL,
      branch,
      projectURL,
      filterByTestsResult,
      projectSlug: inputProjectSlug,
    } = args.params ?? {};
  
    let pipelineNumber: number | undefined;
    let projectSlug: string | undefined;
    let jobNumber: number | undefined;
    let branchFromURL: string | undefined;
  
    if (inputProjectSlug) {
      if (!branch) {
        return mcpErrorOutput(
          'Branch not provided. When using projectSlug, a branch must also be specified.',
        );
      }
      projectSlug = inputProjectSlug;
    } else if (projectURL) {
      pipelineNumber = getPipelineNumberFromURL(projectURL);
      projectSlug = getProjectSlugFromURL(projectURL);
      branchFromURL = getBranchFromURL(projectURL);
      jobNumber = getJobNumberFromURL(projectURL);
    } else if (workspaceRoot && gitRemoteURL && branch) {
      projectSlug = await identifyProjectSlug({
        gitRemoteURL,
      });
    } else {
      return mcpErrorOutput(
        'Missing required inputs. Please provide either: 1) projectSlug with branch, 2) projectURL, or 3) workspaceRoot with gitRemoteURL and branch.',
      );
    }
  
    if (!projectSlug) {
      return mcpErrorOutput(`
        Project not found. Please provide a valid project URL or project information.
  
        Project slug: ${projectSlug}
        Git remote URL: ${gitRemoteURL}
        Branch: ${branch}
      `);
    }
  
    const testResults = await getJobTests({
      projectSlug,
      pipelineNumber,
      branch: branchFromURL || branch,
      jobNumber,
      filterByTestsResult,
    });
  
    return formatJobTests(testResults);
  };

• src/tools/getJobTestResults/inputSchema.ts:7-45 (schema)

  Zod input schema defining parameters for the tool: projectSlug, branch, workspaceRoot, gitRemoteURL, projectURL, filterByTestsResult.

  export const getJobTestResultsInputSchema = z.object({
    projectSlug: z.string().describe(projectSlugDescription).optional(),
    branch: z.string().describe(branchDescription).optional(),
    workspaceRoot: z
      .string()
      .describe(
        'The absolute path to the root directory of your project workspace. ' +
          'This should be the top-level folder containing your source code, configuration files, and dependencies. ' +
          'For example: "/home/user/my-project" or "C:\\Users\\user\\my-project"',
      )
      .optional(),
    gitRemoteURL: z
      .string()
      .describe(
        'The URL of the remote git repository. This should be the URL of the repository that you cloned to your local workspace. ' +
          'For example: "https://github.com/user/my-project.git"',
      )
      .optional(),
    projectURL: z
      .string()
      .describe(
        'The URL of the CircleCI project. Can be any of these formats:\n' +
          '- Project URL: https://app.circleci.com/pipelines/gh/organization/project\n' +
          '- Project URL with branch: https://app.circleci.com/pipelines/gh/organization/project?branch=feature-branch\n' +
          '- Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123\n' +
          '- Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def\n' +
          '- Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/123',
      )
      .optional(),
    filterByTestsResult: z
      .enum(['failure', 'success'])
      .describe(
        `Filter the tests by result.
        If "failure", only failed tests will be returned.
        If "success", only successful tests will be returned.
        `,
      )
      .optional(),
  });

• src/tools/getJobTestResults/tool.ts:4-61 (registration)

  Tool definition object registering the tool with name 'get_job_test_results', detailed description, and input schema.

  export const getJobTestResultsTool = {
    name: 'get_job_test_results' as const,
    description: `
      This tool retrieves test metadata for a CircleCI job.
  
      PRIORITY USE CASE:
      - When asked "are tests passing in CI?" or similar questions about test status
      - When asked to "fix failed tests in CI" or help with CI test failures
      - Use this tool to check if tests are passing in CircleCI and identify failed tests
      
      Common use cases:
      - Get test metadata for a specific job
      - Get test metadata for all jobs in a project
      - Get test metadata for a specific branch
      - Get test metadata for a specific pipeline
      - Get test metadata for a specific workflow
      - Get test metadata for a specific job
  
      CRITICAL REQUIREMENTS:
      1. Truncation Handling (HIGHEST PRIORITY):
         - ALWAYS check for <MCPTruncationWarning> in the output
         - When present, you MUST start your response with:
           "WARNING: The test results have been truncated. Only showing the most recent entries. Some test data may not be visible."
         - Only proceed with test result analysis after acknowledging the truncation
  
      2. Test Result Filtering:
         - Use filterByTestsResult parameter to filter test results:
           * filterByTestsResult: 'failure' - Show only failed tests
           * filterByTestsResult: 'success' - Show only successful tests
         - When looking for failed tests, ALWAYS set filterByTestsResult to 'failure'
         - When checking if tests are passing, set filterByTestsResult to 'success'
  
      Input options (EXACTLY ONE of these THREE options must be used):
  
      ${option1DescriptionBranchRequired}
  
      Option 2 - Direct URL (provide ONE of these):
      - projectURL: The URL of the CircleCI job in any of these formats:
        * Job URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def/jobs/789
        * Workflow URL: https://app.circleci.com/pipelines/gh/organization/project/123/workflows/abc-def
        * Pipeline URL: https://app.circleci.com/pipelines/gh/organization/project/123
  
      Option 3 - Project Detection (ALL of these must be provided together):
      - workspaceRoot: The absolute path to the workspace root
      - gitRemoteURL: The URL of the git remote repository
      - branch: The name of the current branch
      
      For simple test status checks (e.g., "are tests passing in CI?") or fixing failed tests, prefer Option 1 with a recent pipeline URL if available.
  
      Additional Requirements:
      - Never call this tool with incomplete parameters
      - If using Option 1, make sure to extract the projectSlug exactly as provided by listFollowedProjects and include the branch parameter
      - If using Option 2, the URL MUST be provided by the user - do not attempt to construct or guess URLs
      - If using Option 3, ALL THREE parameters (workspaceRoot, gitRemoteURL, branch) must be provided
      - If none of the options can be fully satisfied, ask the user for the missing information before making the tool call
      `,
    inputSchema: getJobTestResultsInputSchema,
  };

• src/circleci-tools.ts:37-85 (registration)

  Aggregation and registration of all CircleCI tools, including getJobTestResultsTool in the tools array and mapping get_job_test_results to its handler in CCI_HANDLERS.

  export const CCI_TOOLS = [
    getBuildFailureLogsTool,
    getFlakyTestLogsTool,
    getLatestPipelineStatusTool,
    getJobTestResultsTool,
    configHelperTool,
    createPromptTemplateTool,
    recommendPromptTemplateTestsTool,
    runPipelineTool,
    listFollowedProjectsTool,
    runEvaluationTestsTool,
    rerunWorkflowTool,
    downloadUsageApiDataTool,
    findUnderusedResourceClassesTool,
    analyzeDiffTool,
    runRollbackPipelineTool,
    listComponentVersionsTool,
  ];
  
  // Extract the tool names as a union type
  type CCIToolName = (typeof CCI_TOOLS)[number]['name'];
  
  export type ToolHandler<T extends CCIToolName> = ToolCallback<{
    params: Extract<(typeof CCI_TOOLS)[number], { name: T }>['inputSchema'];
  }>;
  
  // Create a type for the tool handlers that directly maps each tool to its appropriate input schema
  type ToolHandlers = {
    [K in CCIToolName]: ToolHandler<K>;
  };
  
  export const CCI_HANDLERS = {
    get_build_failure_logs: getBuildFailureLogs,
    find_flaky_tests: getFlakyTestLogs,
    get_latest_pipeline_status: getLatestPipelineStatus,
    get_job_test_results: getJobTestResults,
    config_helper: configHelper,
    create_prompt_template: createPromptTemplate,
    recommend_prompt_template_tests: recommendPromptTemplateTests,
    run_pipeline: runPipeline,
    list_followed_projects: listFollowedProjects,
    run_evaluation_tests: runEvaluationTests,
    rerun_workflow: rerunWorkflow,
    download_usage_api_data: downloadUsageApiData,
    find_underused_resource_classes: findUnderusedResourceClasses,
    analyze_diff: analyzeDiff,
    run_rollback_pipeline: runRollbackPipeline,
    list_component_versions: listComponentVersions,
  } satisfies ToolHandlers;