Dynamic variable groups in Azure DevOps Pipelines

When building or deploying to multiple environments using Azure DevOps, it can be useful to have different variable groups per environment. As well as being easier to work with compared to combining everything in a single variable group, it also allows permissions to be adjusted per environment.

When running multiple builds/deployments for the same app in parallel (for example, in a multi-tenanted environment), we can usually use the strategy property to provide a matrix that pipelines will use to start up multiple jobs. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
jobs:
- job: Build
  strategy:
    matrix:
      Production:
        EnvironmentName: Production
      Staging:
        EnvironmentName: Staging
  steps:
  - task: Powershell@2
    inputs:
      targetType: 'inline'
      script: |
        Write-Host "Building $(EnvironmentName)"

Unfortunately we can’t dynamically construct a variable group name using strategy/matrix. The following would result in an “variable group not found” sort of error:

1
2
variables:
- group: $(EnvironmentName)_appsettings

Instead, we can use a more primitive each loop within a template that takes a list of environments (or whatever) as a parameter. The template is needed a we can’t define a complex object anywhere else but the parameter input.

The main pipeline looks like this:

1
2
3
4
5
6
jobs:
  - template: templates/build.yml
    parameters:
      environments:
      - Staging
      - Production

And the template, which constructs the variable group name, looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
parameters:
- name: environments
  type: object

jobs:
- ${{ each environment in parameters.environments }}:
  - job: Build_${{ environment }}
    displayName: Build ${{ environment }}
    variables:
    - group: ${{ environment }}_appsettings
    steps:
    - task: Powershell@2
      inputs:
        targetType: 'inline'
        script: |
          Write-Host "Building ${{ environment }}"
          Write-Host "Config: $(SomeConfig)"