How to use Azure VM metadata service to automate post-provisioning metadata configuration in your IaC for VMSS

TL;DR: How to use cloud-init for Linux VMs and Azure Custom Script Extension for Windows VMs to create a .env file on the VM containing VM metadata from Azure VM metadata service when using Azure VM Scale Sets

When using Virtual Machines or Virtual Machine Scale Sets on Azure, it often becomes extremely useful to have certain VM metadata accessible to your applications. This type of metadata (like ID, name, private IP, etc.) gets normaly generated at the provisioning time, and having an automated way for applications to access these will come in handy.

Azure provides an amazing service called the Azure VM metadata service, which can be accessed from within a VM to retrieve a all VM specific information.

 curl -s -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2021-02-01" | jq

While this command is useful, integrating it into your Infrastructure as Code (IaC) can automate the process and ensure scalability.

In this blog, we'll explore how to package the VM metadata service call into a script, store the metadata in a file, and incorporate this process into both Windows and Linux VMs in a VMSS setup.

Creating a Generalized Metadata Retrieval Script

When looking at the VM metadata service endpoint from Azure, everything other than the IP appears to be generic. However, upon closer reading of the Azure documentation, it is mentioned that this "magic" IP is the same for all VMs.

"Azure's instance metadata service is a RESTful endpoint available to all IaaS VMs created via the new Azure Resource Manager. [..] The [VM metadata service] endpoint is available at a well-known non-routable IP address (169.254.169.254) that can be accessed only from within the VM."

This allows us to easily package the call up in a script and output the metadata in our needed format. For the sake of this blog, we will simply create a file that will contain the information we need.

Let's proceed with the implementation details for both Windows and Linux VMs. The full code can be found here.

Windows VMs: Utilizing Azure Custom Script Extension

For Windows VMs, the Azure Custom Script Extension is a powerful tool to execute post-provisioning scripts. Within the script, we can use the VM metadata service to retrieve the VM name and store it in a file under C:\ called vm-metadata.env.

# vm-metadata.ps1vm-metadata.ps1
$vmName = Invoke-RestMethod -Headers @{"Metadata"="true"} -Method GET -Uri "http://169.254.169.254/metadata/instance/compute/name?api-version=2021-02-01&format=text"
"VM_NAME=$vmName" | Out-File -FilePath C:\vm-metadata.env -Append

In the IaC definition, the above script can be passed either via an Azure storage account or from GitHub.

resource vmss 'Microsoft.Compute/virtualMachineScaleSets@2022-03-01' = {
  name: vmssName
  location: location
  ...
  properties: {
    singlePlacementGroup: null
    platformFaultDomainCount: 1
    virtualMachineProfile: {
      extensionProfile: {
        extensions: [ {
            name: 'CustomScriptExtension'
            properties: {
              publisher: 'Microsoft.Compute'
              type: 'CustomScriptExtension'
              typeHandlerVersion: '1.10'
              settings: {
                commandToExecute: 'powershell -ExecutionPolicy Unrestricted -File vm-metadata.ps1'
                fileUris: [ '<link-to-file>' ]
              }
            }
          } ]
      }
    }
    ...
  }
}

Linux VMs: Harnessing cloud-init

For Linux VMs, leveraging the native cloud-init tool simplifies the process.

Note: We could, however, also use the same Azure Custom Script Extension as we did for Windows here. Check out the docs for that here.

Amongst many other things, the cloud-init definition allows you to specify one or more commands in the runcmd section, which should run after the initial startup. Just like for the PowerShell script, the VM metadata is called and the extracted VM name is stored in the vm-metadata.env file.

#cloud-config
runcmd:
  -  vmName=$(curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance/compute/name?api-version=2021-02-01&format=text") && echo "VM_NAME=${vmName}" >> vm-metadata.env

Similar to regular VMs, the VMSS allows you to set the customData property when defining your OS profile. It behaves the same way as it does for a VM deployment with cloud-init, expecting the file to be passed as a base64-encoded string.

param cloudInitScript string = loadFileAsBase64('./cloud-init.yaml')

...

resource vmss 'Microsoft.Compute/virtualMachineScaleSets@2022-03-01' = {
  name: '${prefix}-vmss'
  location: location
  dependsOn: [
    vmssLB
    vmssNSG
  ]
  sku: {
    name: 'Standard_DS1_v2'
    capacity: 1
  }
  properties: {
    singlePlacementGroup: null
    platformFaultDomainCount: 1
    virtualMachineProfile: {
      osProfile: {
        computerNamePrefix: 'vmss'
        adminUsername: 'azureuser'
        adminPassword: adminPassword
        customData: cloudInitScript
      }
      ...

    }
    ...
  }
}

And with that, you know how to retrieve VM metadata values for your applications from a VM in your VMSS pool in an automatic fashion :)