How to test your Azure ARM Template with ARM Template toolkit (arm-ttk)

When we talk about Infrastructure as Code, we mean to build our services by using the same technics and logics used in Software Engineering. Software engineering is not only about code, but it's also about the way to collaborate, to find mistakes early and correct errors before going in production.

How can we do that with ARM Template? There is no tool like xUnit or Pester for ARM Template. What kind of test should we operate and how to do it?

Since de MS Ignite 2019, we have a new tool to automate ARM Template testing, ARM Template Tool Kit (arm-ttk).

ARM Template Tool Kit is a PowerShell module. Its principal intention is to control ARM template files before the deployment.

It’s still in preview and it cannot be, at this time, installed via the PowerShell Gallery. You will have to install it manually by downloading it from GitHub and import the module.

Open a PS Core console in the module directory

import-module .\arm-ttk.psd1

The main cmdlet of the module is Test-AzTemplate. The only required parameter is TemplatePath.
I created a simple template to deploy a storage account in a folder.

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
  },
  "variables": {
    "storageAccountName": "[concat('st', uniquestring(resourceGroup().id))]"
  },
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "apiVersion": "2016-05-01",
      "name": "[variables('storageAccountName')]",
      "location": "francecentral",
      "sku": {
        "name": "Standard_LRS"
      },
      "kind": "Storage",
      "properties": {}
    }
  ],
  "outputs": {
    "storageAccountName": {
      "type": "string",
      "value": "[variables('storageAccountName')]"
    }
  }
}

and test it with the cmdlet

Test-AzTemplate -TemplatePath samples\storage.json

The result is

Validating samples\storage.json
deploymentTemplate
[+] adminUsername Should Not Be A Literal (122 ms)

[-] apiVersions Should Be Recent (47 ms)
Api versions must be the latest or under 2 years old (730 days) - API version 2016-05-01 of Microsoft.Storage/storageAccounts is 1382 days old
Valid Api Versions:
2019-06-01
2019-04-01
2018-11-01
2018-07-01
2018-03-01-preview
[+] artifacts parameter (36 ms)
[+] DeploymentTemplate Schema Is Correct (30 ms)
[+] IDs Should Be Derived From ResourceIDs (56 ms)
[+] Location Should Not Be Hardcoded (54 ms)
[+] ManagedIdentityExtension must not be used (33 ms)
[+] Min And Max Value Are Numbers (71 ms)
[+] Outputs Must Not Contain Secrets (60 ms)
[+] Parameters Must Be Referenced (45 ms)
[+] Parameters Property Must Exist (30 ms)
[+] providers apiVersions Is Not Permitted (49 ms)
[+] ResourceIds should not contain (40 ms)
[+] Resources Should Have Location (47 ms)
[+] Secure String Parameters Cannot Have Default (111 ms)
[+] Template Should Not Contain Blanks (53 ms)
[+] Variables Must Be Referenced (79 ms)
[+] Virtual Machines Should Not Be Preview (59 ms)
[+] VM Images Should Use Latest Version (70 ms)
[+] VM Size Should Be A Parameter (42 ms)

For those who work with PowerShell and Pester it should be familiar. These are simply a list of test cases. Pester is used by default to evaluate the template against these tests. It provides the standard pester colorized output.

You can find them in the test cases folder in the module folder. A test case, in ARM-TTK, is just a script, for example, the ApiVersion test case (apiVersions-Should-Be-Recent.test.ps1), check if the age of an API version of a version. If the age is greater than 730 days, the script writes a PowerShell error therefore pester can evaluate the test as failed.

There are several test cases.

• adminUsername-Should-Not-Be-A-Literal
• apiVersions-Should-Be-Recent
• artifacts-parameter
• DeploymentTemplate-Schema-Is-Correct
• IDs-Should-Be-Derived-From-ResourceIDs
• Location-Should-Not-Be-Hardcoded
• ManagedIdentityExtension-must-not-be-used
• Min-And-Max-Value-Are-Numbers
• Outputs-Must-Not-Contain-Secrets
• Parameters-Must-Be-Referenced
• Parameters-Property-Must-Exist
• providers_apiVersions-Is-Not-Permitted
• ResourceIds-should-not-contain
• Resources-Should-Have-Location
• Secure-String-Parameters-Cannot-Have-Default
• Template-Should-Not-Contain-Blanks
• Variables-Must-Be-Referenced
• Virtual-Machines-Should-Not-Be-Preview
• VM-Images-Should-Use-Latest-Version
• VM-Size-Should-Be-A-Parameter

And it’s possible to add a test. Choose a test name, for example Template should have a correct version format, and create a file name by replacing space by – Template-should-have-a-correct-version-format.test.ps1. Finally, add your test script in deploymentTemplate directory.

To be able to test the template with the module, you should have at least one parameter [PSObject]$TemplateObject

The script itself looks like

<#
.Synopsis
    Ensures the contentVersion use the X.X.X.X format.
.Description
    Ensures the contentVersion use the X.X.X.X format.
#>
param(
[PSObject]$TemplateObject
)

$templateVersion = $TemplateObject.contentVersion

if (-not $templateVersion) {
    Write-Error 'DeploymentTemplate Missing contentVersion property' 
    return
}

if (!($templateVersion -match "^[0-9]+.[0-9]+.[0-9]+.[0-9]+$")) {
    Write-Error "DeploymentTemplate has an unexpected contentVersion. Format should be X.X.X.X"
    return
}

You can now re-run the previous test and you should see

[+] Template should have a correct version format (23 ms)

We can also use only a small number of tests. If for some reason, I want to make sure the template API Version and the contentVersion are correct, I can use the test parameter with a list of test with there -.

Test-AzTemplate -TemplatePath .\storage.json -Test DeploymentTemplate-Schema-Is-Correct, Template-should-have-a-correct-version-format

Now that we see how to use ARM-TTK for a small template, we need to see how to use it in a more real world.

In a more real-world, we rarely use a unique template, instead, we use separate JSON files linked to an azureDeploy.json or a mainTemplate.json file. This technique is called linked templates and it's perfect when you work with several people to create a complexes deployment.

In this case, you don't have to test JSON files one by one, simply provide the path of the folder where JSON files and the main template are located.

Test-AzTemplate -TemplatePath .\  

The cmdlet will test all the template in the folder

When you work on ARM templates in a team, you need versioning tools and a place where you can perform tests and deployments. Azure DevOps is one of the tools you can use, beside GitHub and GitLab.

You can run arm-ttk tests in Azure DevOps, thank Sam Cogan. You just need to install the ARM-TTK-EXTENSION from the marketplace.

After the installation, you can start to use it in a continuous integration pipeline. Creating an Azure DevOps pipeline is easy. You need to create an azure-pipelines.yaml file at the root of the repos.

name: ARM Deployment Pipeline

trigger:
- master

pool:
  vmImage: 'windows-latest'

stages:
  - stage: ARM_CI
    jobs:
    - job: ARM_Testing
      steps:
        - pwsh: ./InitializePipeline.ps1
          displayName: Prepare folders
          name: PsBuild


        - task: RunARMTTKTests@1
          displayName: Run ARM TTK tests 
          name: TTKtests
          inputs:
            templatelocation: '$(System.DefaultWorkingDirectory)\tpl'
            resultLocation: '$(System.DefaultWorkingDirectory)\results'

        - task: PublishTestResults@2
          displayName: Publish test restults
          name: PublishResults
          inputs:
            testResultsFormat: 'NUnit'
            testResultsFiles: '$(System.DefaultWorkingDirectory)\results\*-armttk.xml'
          condition: always()

The module is still in preview, and some modifications may happen like and new test cases may be added in the future. This is one of the tools you should use before deploying anything in production.