Kenichiro Nakamura

Posted on Dec 4, 2022 • Edited on Dec 31, 2022

Bicep: Create IoT Hub and Storage routing with private endpoint

#azure #bicep #iothub #security

To further improve security from Bicep: Create IoT Hub and Storage routing with security in mind, I add private endpoint features to IoT Hub and storage account.

Private Endpoint for Azure Resources

By default, IoT Hub and Storage account has public IP address to expose their service endpoints. Some services can use firewall to restrict access to the resource, but we can secure it further by using private endpoint. Private Endpoint provides local IP address and disable public IP endpoint, therefore, to reach the service, you need to:

Resolve service URL to local IP address by using DNS
Need to be on the network which can reach to the local IP address.

Private Endpoint for IoT Hub

IoT Hub support for virtual networks with Private Link and Managed Identity explains how IoT Hub can integrate with VNET for enterprise scenario.

Private Endpoint for Storage account

Use private endpoints for Azure Storage explains how Storage account can integrate with VNET with private endpoint.

Overview

I use bicep to setup followings:

Virtual Network and Subnet
IoT Hub
- with managed identity enabled
- routing to storage account for all telemetry
- disable fallback for routing
- use private endpoint integrated with the subnet
Storage Account (v2)
- disable public access
- allow access only via the IoT Hub managed identity
- allow access specified subnet
- use private endpoint integrated with the subnet

bicep script

I have eight bicep scripts this time, as I need to setup extra resources and settings such as VNET, private endpoints and DNS zones.

main.bicep

The main.bicep calls modules by passing parameter. I use dependsOn to make sure the executing order.

param name string
param location string = resourceGroup().location

module vnet './vnet.bicep' = {
  name: 'vnetDeployment'
  params: {
    name: name
    location: location
  }
}

module iotHub './iotHub.bicep' = {
  name: 'iotHubDeployment'
  params: {
    name: name
    location: location
  }
}

module storage 'storage.bicep' = {
  name: 'storageDeployment'
  params: {
    name: name
    location: location
  }
}

module role './role.bicep' = {
  name: 'roleDeployment'
  params: {
    name: name
    iotHubPrincipalId: iotHub.outputs.iotPrincipalId
  }
  dependsOn: [
    iotHub 
    storage
  ]
}

module privateEndpointIoTHub './privateEndpointIoTHub.bicep' = {
  name: 'privateEndpointIoTHubDeployment'
  params: {
    name: name
    location: location
  }
  dependsOn: [
    vnet
    iotHub
  ]
}

module privateEndpointStorage './privateEndpointStorage.bicep' = {
  name: 'privateEndpointIoTStorageDeployment'
  params: {
    name: name
    location: location
  }
  dependsOn: [
    vnet
    iotHub
  ]
}

module iotHubRouting './iotHubRouting.bicep' = {
  name: 'iotHubRoutingDeployment'
  params: {
    name: name
    location: location
  }
  dependsOn: [
    privateEndpointIoTHub
  ]
}

vnet.bicep

Add one virtual network and one subnet.

param name string
param location string

resource vnet 'Microsoft.Network/virtualNetworks@2021-08-01' = {
  name: 'vnet-${name}-${uniqueString(resourceGroup().id)}'
  location: location
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/16'
      ]
    }
    subnets: [
      {
        name: 'snet-${name}-${uniqueString(resourceGroup().id)}'
        properties: {
          addressPrefix: '10.0.0.0/24'
        }
      }
    ]
  }
}

iotHub.bicep

This bicep creates IoT Hub with managed id and minimum settings. Returns the managed id information out.

param name string
param location string

resource iotHub 'Microsoft.Devices/IotHubs@2021-07-02' = {
  name: 'iot-${name}-${uniqueString(resourceGroup().id)}'
  location: location
  sku: {
    name: 'S1'
    capacity: 1
  }
  identity:{
     type: 'SystemAssigned'
  }
  properties: {
    publicNetworkAccess: 'Disabled'
  }
}

output iotPrincipalId string = iotHub.identity.principalId

storage.bicep

I create Azure Data Lake Gen2 type of storage with kind: 'StorageV2' and isHnsEnabled: true. Let only the IoT Hub managed Id access to the storage.

param name string
param location string

var storageAccountName = 'st${toLower(name)}${uniqueString(resourceGroup().id)}'
var storageContainerName = '${toLower(name)}results'
var iotHubName = 'iot-${name}-${uniqueString(resourceGroup().id)}'

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-08-01' = {
  name: storageAccountName
  location: location
  sku: {
    name: 'Standard_LRS'
  }
  kind: 'StorageV2'
  properties: {
    isHnsEnabled: true
    supportsHttpsTrafficOnly: true
    networkAcls: {
      bypass: 'None'
      defaultAction: 'Deny'
      resourceAccessRules: [ {
          resourceId: resourceId('Microsoft.Devices/IotHubs', iotHubName)
          tenantId: tenant().tenantId
        }
      ]
    }
  }
}

resource container 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-08-01' = {
  name: '${storageAccountName}/default/${storageContainerName}'
  properties: {
    publicAccess: 'None'
  }
  dependsOn: [
    storageAccount
  ]
}

role.bicep

Create IAM for IoT Hub managed to the storage so that IoT Hub can send telemetry via routing.

param name string
param iotHubPrincipalId string

var storageAccountName = 'st${toLower(name)}${uniqueString(resourceGroup().id)}'

resource storageAccount 'Microsoft.Storage/storageAccounts@2019-06-01' existing = {
  name: storageAccountName 
}

//https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles
resource roleDefinition 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {
  name:'ba92f5b4-2d11-453d-a403-e96b0029c9fe'
  scope: storageAccount
}

resource roleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
  scope: storageAccount
  name: guid(storageAccount.id, iotHubPrincipalId,  roleDefinition.id)
  properties: {
    roleDefinitionId: roleDefinition.id
    principalId: iotHubPrincipalId
    principalType: 'ServicePrincipal'
  }
}

privateEndpointIoTHub.bicep

This is most complex bicep file in this article. When we create a private endpoint form Azure Portal, it does so many things behind for us, but we have to explicitly declare them in bicep.

This creates:

Private endpoints for IoT Hub
Private DNS zones for IoT Hub
Links private DNS zones to VNET

By creating private endpoint, it creates NIC with local IP address for each private endpoint. To resolve the name, we need private DNS zones and map to VNET to name resolution.

param name string
param location string

var uniqueName = '${name}-${uniqueString(resourceGroup().id)}'
var vnetName = 'vnet-${uniqueName}'
var snetName = 'snet-${uniqueName}'
var endpointNameIoT = 'pep-iot-${uniqueName}'
// You can find private DNS for Azure resources here
// https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns
var privateDnsZoneNameAzureDevices = 'privatelink.azure-devices.net'
var privateDnsZoneNameServiceBus = 'privatelink.servicebus.windows.net'

resource privateEndpointIoT 'Microsoft.Network/privateEndpoints@2020-08-01' = {
  name: endpointNameIoT
  location: location
  properties: {
    subnet: {
      id: resourceId('Microsoft.Network/virtualNetworks/subnets', vnetName, snetName)
    }
    privateLinkServiceConnections: [
      {
        properties: {
          privateLinkServiceId: resourceId('Microsoft.Devices/IotHubs', 'iot-${uniqueName}')
          groupIds: [
            'iotHub'
          ]
        }
        name: 'sc-iot-${uniqueName}'
      }
    ]
  }
}

resource privateDnsZoneAzureDevices 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: privateDnsZoneNameAzureDevices
  location: 'global'
  properties: {}
}

resource privateDnsZoneServiceBus 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: privateDnsZoneNameServiceBus
  location: 'global'
  properties: {}
}

resource privateDnsZoneLinkAzureDevices 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  parent: privateDnsZoneAzureDevices
  name: '${privateDnsZoneNameAzureDevices}-link'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: resourceId('Microsoft.Network/virtualNetworks', vnetName)
    }
  }
}

resource privateDnsZoneLinkServiceBus 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  parent: privateDnsZoneServiceBus
  name: '${privateDnsZoneNameServiceBus}-link'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: resourceId('Microsoft.Network/virtualNetworks', vnetName)
    }
  }
}

resource pvtEndpointIoTDnsGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2021-05-01' = {
  name: '${endpointNameIoT}/default'
  properties: {
    privateDnsZoneConfigs: [
      {
        name: privateDnsZoneNameAzureDevices
        properties: {
          privateDnsZoneId: privateDnsZoneAzureDevices.id
        }
      }
      {
        name: privateDnsZoneNameServiceBus
        properties: {
          privateDnsZoneId: privateDnsZoneServiceBus.id
        }
      }
    ]
  }
  dependsOn: [
    privateEndpointIoT
  ]
}

privateEndpointStorage.bicep

Do the same but for Storage account.

param name string
param location string

var uniqueName = '${name}-${uniqueString(resourceGroup().id)}'
var vnetName = 'vnet-${uniqueName}'
var snetName = 'snet-${uniqueName}'
var storageAccountName = 'st${toLower(name)}${uniqueString(resourceGroup().id)}'
var endpointNameStorage = 'pep-st-${uniqueName}'
// You can find private DNS for Azure resources here
// https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns
var privateDnsZoneNameStorage = 'privatelink.blob.${environment().suffixes.storage}'

resource privateEndpointStorage 'Microsoft.Network/privateEndpoints@2020-08-01' = {
  name: endpointNameStorage
  location: location
  properties: {
    subnet: {
      id: resourceId('Microsoft.Network/virtualNetworks/subnets', vnetName, snetName)
    }
    privateLinkServiceConnections: [
      {
        properties: {
          privateLinkServiceId: resourceId('Microsoft.Storage/storageAccounts', storageAccountName)
          groupIds: [
            'Blob'
          ]
        }
        name: 'sc-st-${uniqueName}'
      }
    ]
  }
}

resource privateDnsZoneStorage 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: privateDnsZoneNameStorage
  location: 'global'
  properties: {}
}

resource privateDnsZoneLinkStorage 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  parent: privateDnsZoneStorage
  name: '${privateDnsZoneNameStorage}-link'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: resourceId('Microsoft.Network/virtualNetworks', vnetName)
    }
  }
}

resource pvtEndpointStoraeDnsGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2021-05-01' = {
  name: '${endpointNameStorage}/default'
  properties: {
    privateDnsZoneConfigs: [
      {
        name: privateDnsZoneNameStorage
        properties: {
          privateDnsZoneId: privateDnsZoneStorage.id
        }
      }
    ]
  }
  dependsOn: [
    privateEndpointStorage
  ]
}

iotHubRouting.bicep

Finally, finish by setting up routing which is same as before.

param name string
param location string
var storageEndpoint = '${name}StorageEndpont'
var storageAccountName = 'st${toLower(name)}${uniqueString(resourceGroup().id)}'
var storageContainerName = '${toLower(name)}results'

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-08-01'existing = {
  name: storageAccountName
}

resource iotHub 'Microsoft.Devices/IotHubs@2021-07-02' = {
  name: 'iot-${name}-${uniqueString(resourceGroup().id)}'
  location: location
  sku: {
    name: 'S1'
    capacity: 1
  }
  identity:{
     type: 'SystemAssigned'
  }
  properties: {
    publicNetworkAccess: 'Disabled'
    routing: {
      endpoints: {
        storageContainers: [
          {
            authenticationType: 'identityBased'
            endpointUri: storageAccount.properties.primaryEndpoints.blob
            containerName: storageContainerName
            fileNameFormat: '{iothub}/{partition}/{YYYY}/{MM}/{DD}/{HH}/{mm}'
            batchFrequencyInSeconds: 60
            maxChunkSizeInBytes: 104857600
            encoding: 'JSON'
            name: storageEndpoint
          }
        ]
      }
      routes: [
        {
          name: 'Route'
          source: 'DeviceMessages'
          condition: 'true'
          endpointNames: [
            storageEndpoint
          ]
          isEnabled: true
        }
      ]
    }
  }
}

Run bicep

az deployment group create -g <resource group name> --file-template main.bicep --parameters name=<any name>

Result

This is the result of run.

DNS and IP address

Let's take look into DNS for storage as an example. We can see IP address 10.0.0.6 registered as A record.

This IP address comes from private endpoint for storage. The private endpoint has link to both network interface as well as DNS zone.

You can compare privateEndpointStorage.bicep and this result to understand which resource maps to this screen.

IoT Hub Networking

Public access is disabled, and private access is established.

Storage Networking

Public network access is only limited to the IoT Hub managed id, which is used for routing.

To access storage from SDK or storage explorer, you need to do so from within VNET as it has private endpoint setup.

Summary

Using private endpoint is very strong, but we need to setup many dependent resources without mistake to make it work properly. Though it's worth doing!

DEV Community