Running Jest tests in a Docker container in an Azure Devops pipeline

There are some big advantages to running tests in a container within a CI pipeline, including:

  • Dependencies are within container - no need to install on the build agent
  • Local development envirionment same as build environment
  • Easier to debug build issues by running the container locally

The trick with running tests within a container is getting the test results out of the container and publishing them to the CI tool (in this case, Azure Devops).

Starting with a small JS app with Jest tests, with the following test command defined in the package.json to output the tests as an XML file:

1
2
3
"scripts": {
    "test": "jest --reporters=default --reporters=jest-junit"
}

We then create a simple docker file that will build the app and run tests. We define a label so we can pick the correct image later, and also output the status code of the yarn test command so we can verify the tests ran properly later on (credit to this StackOverflow question for this solution).

1
2
3
4
5
6
7
8
9
FROM node:15-buster

ARG BUILD_NUMBER

RUN yarn build
RUN yarn test; echo $? > /npm.exitcode

LABEL buildNumber=${BUILD_NUMBER}

We can then build the container in our pipeline:

1
2
3
4
5
6
7
8
9
10
- task: Docker@2
  displayName: Build JS app container image
  inputs:
    command: 'build'
    dockerFile: 'Dockerfile'
    buildContext: '.'
    arguments: >
      --build-arg BUILD_NUMBER=$(Build.BuildNumber)
    tags: |
      latest

Once built, we need to retrieve the test file from the image. To do this, we need to run the container and copy the file out. The container doesn’t have to be running wheh this happens - we can retrieve it from a container that is no longer running. We use the label we set in the Dockerfile to pick the correct container:

1
2
3
4
5
6
7
8
9
10
11
12
13
- displayName: Copy test results from container
  condition: succeededOrFailed()
  powershell: |
    $imageId = (docker images -a -f "label=buildNumber=$(Build.BuildNumber)" --format "{{.ID}})

    $containerId = docker run -d $imageId

    # I found that is was sometimes necessary to have a delay before attempting the copy
    # A more elegant retry solution would be better
    Start-Sleep -Seconds 2

    docker cp "$($containerId):/src/junit.xml" $(Pipeline.Workspace)/junit.xml
    docker cp "$($containerId):/npm.exitcode" $(Pipeline.Workspace)/npm.exitcode

We then publish the file to Azure Devops, so we can view the testing results in the UI. This step can also fail the build if there are failed tests.

1
2
3
4
5
6
7
8
9
- task: PublishTestResult@2
  condition: succeededOrFailed()
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'junit.xml'
    searchFolder: $(Pipeline.Workspace)
    mergeTestResults: true
    failTaskOnFailedTests: true
    restRunTitle: 'Jest Tests'

Finally, we must check the output of the yarn test command that ran in the container. This is because in some scenarios when the tests don’t run correctly (for example, an error starting any of the tests), the output file can be empty and not include any failures.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
- displayName: Verify tests passed
  condition: succeededOrFailed()
  powershell: |
    # Check exitcode file exists
    If("$(Test-Path $(Pipeline.Workspace)/npm.exitcode))" -eq "False") {
        Write-Host "Exit code for test run not found"
        exit 9
    }

    $exitCode = cat $(Pipeline.Workspace)/npm.exitcode

    If($exitCode -ne "0") {
        Write-Host "Tests failed"
        exit $exitCode
    }

And that’s it! For brevity I’ve not included some important aspects of a production-ready pipeline. Be sure to look into multi-stage docker builds to reduce image size and attack surface - they will still work with the approach above - the label will allow you to find the correct image.