Introduction
As part of my personal development, I've created a personal health platform that uses various different microservices (Built using Azure Functions) that extract data from my Fitbit account and store them in an Azure Cosmos DB database. I have other microservices that pass messages between different services via Azure Service Bus.
For this project, I use Azure DevOps to build my artifacts, run my unit tests and deploy my microservices to Azure. The great thing about DevOps is that we can do all of this with in the YAML pipeline.
Yes I said YAML. Honestly, I don't know what the fuss is all about ๐
In a previous post, I talked about how we can deploy NuGet packages to a private feed in Azure Artifacts using YAML pipelines. If you haven't read that post yet, you can check it out below!
In this article, we will turn our attention to building and deploying C# Azure Functions using a single build file.
What we'll cover
We've got quite a bit to cover, so I'll break down my YAML file and talk about each stage in the following order:
- Triggering a Build ๐ทโโ๏ธ๐ทโโ๏ธ
- Using User-Defined Variables in our pipelines ๐จโ๐ฌ๐ฉโ๐ฌ
- Defining Stages ๐ป
- Building our project ๐จ
- Running our tests ๐งช
- Getting code coverage ๐งพ
- Producing a Build Artifact ๐
- Using Secrets from Key Vault ๐
- Deploying our Function to Azure โก
- Running our build pipeline ๐
Triggering a Build ๐ทโโ๏ธ๐ทโโ๏ธ
In our build pipelines, we can use triggers to run our pipelines. We can use different types of triggers, ranging from triggering a build when we push our code to a defined branch, to scheduled triggers based on a CRON schedule.
For this project, I'm just triggering my build every time I push to the main branch. To do so, I define this in my YAML file like so:
trigger:
- main
Now if I wanted to do this properly, we could define multiple branch names to trigger builds from like so:
trigger:
- feature/*
- bugfix/*
This would trigger the builds when we push code to either our feature or bugfix branch. When we raise Pull Requests in DevOps, we can create policies on our PR to only be eligible to merge back into our main branch provided that the build was successful.
This helps us increase our confidence that the code we are merging (and deploying) is of high quality, particularly if we are running tests as part of our build pipeline.
To enable build validation on your branch policies, check out this documentation
Using User-Defined Variables in our pipelines ๐จโ๐ฌ๐ฉโ๐ฌ
We can use user-defined variables in our YAML files to use throughout our pipeline. Doing this gives us the benefit of re-using common values across various stages and tasks in our pipeline, and it allows us to track changes to them over time using version control.
We can define variables within out file with the following scope:
- At the root level, making it available to all jobs in our pipeline.
- At the stage level, making it available to a specific stage in our pipeline.
- At the job level, making it available to a specific job in our pipeline.
In my case, I'm going to be re-using these variables throughout my pipeline, so I'm defining them at the root level.
variables:
buildConfiguration: 'Release'
vmImageName: 'vs2017-win2016'
functionAppName: 'famyhealthfitbitbody'
azureSubscription: '<azureSubscription-id-or-name>'
workingDirectory: '$(System.DefaultWorkingDirectory)/MyHealth.Fitbit.Body'
projectName: 'MyHealth.Fitbit.Body'
We're not just limited to user-defined variables in YAML pipelines! We can use System and Environment variables as well. To learn more about defining variables in YAML pipelines, check out this article
Defining Stages ๐ป
Within our YAML pipelines, we can organize our jobs into Stages. These divide our pipeline into logical....well, stages that perform certain tasks, such as 'build the app', 'deploy to production'.
In this example, I just have a stage and deploy stage. Every YAML pipeline has at least one stage even if we don't define it. As I have done in my file, we can also run stages depending on whether the previous stage has succeeded or not, like so:
stages:
- stage: Build
displayName: Build Stage
jobs:
- job: Build
displayName: Build MyHealth.Fitbit.Body
pool:
vmImage: $(vmImageName)
Using YAML pipelines, we can define multiple stages of our pipeline and store it as code. Check out the documentation if you want to learn more about how Stages work in YAML pipelines.
Building our project ๐จ
Let's turn our attention to our Function. First up, I want to restore and build my .csproj files. In our YAML pipelines, we can se .NET Core tasks to run dotnet commands against our artifacts. To build my project, I'll restore and build.
I'm using a custom NuGet feed hosted on Azure Artifacts, so I will need to point to this feed when restoring.
For the build task, I want to build a release package, so I'll pass in the buildConfiguration variable as an argument when running the build command.
For both commands, I'm going to point to all .csproj files in my repository.
steps:
- task: DotNetCoreCLI@2
displayName: Restore
inputs:
command: 'restore'
feedsToUse: 'select'
vstsFeed: '<feed-id>'
projects: '**/*.csproj'
- task: DotNetCoreCLI@2
displayName: Build
inputs:
command: 'build'
projects: '**/*.csproj'
arguments: --configuration $(buildConfiguration)
Running our tests ๐งช
Alongside my function code, I have a project for my Unit Tests. As part of my build, I want to run my unit tests to make sure that my changes haven't broken any of our tests and that they still pass. If any of my tests fail, I want the build to fail.
To run our tests, we can use a .NET Core task that runs the test command and point the task at our Unit Test .csproj file.
I also want to collect code coverage metrics with Coverlet. Coverlet is .NET tool that allows us to get unit test coverage for line, branch and method coverage. In our arguments parameters, I'm telling my task to run Coverlet to get the code coverage and then publish the test results to the */TestResults/Coverage folder on my build agent:
- task: DotNetCoreCLI@2
displayName: Run Unit Tests
inputs:
command: 'test'
projects: '**/*UnitTests/*.csproj'
arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/'
publishTestResults: true
Getting code coverage ๐งพ
Once we've generated our coverage report, we can publish it to see the results. I've got two tasks within my pipeline to achieve this.
First, I'm using a bash script task to install the report generator onto the Build agent. Once that's installed, I'm creating the xml report into my TestResults/Coverage folder on the agent. We can also specify the report type that we want to generate, but for this tutorial I'm just generating an inline HTML report.
If you want to use a task for this instead of running an inline script, you can try installing the following task onto your DevOps organisation.
Once we have generated a report, I've got a second task that publishes the code coverage report.
- script: |
dotnet tool install -g dotnet-reportgenerator-globaltool
reportgenerator -reports:$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura
displayName: Create Code coverage report
- task: PublishCodeCoverageResults@1
displayName: 'Publish Code Coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)/**/coverage.cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)/TestResults/Coverage/'
When our build finishes, we can review our code coverage report within DevOps! It should look similar to this:
Producing a Build Artifact ๐
We've built our project and our tests pass! We now need to produce a build artifact that we will deploy as our Function.
Starting with publish, all we need here is a .NET Core task that runs the publish command, publishes our project as a 'Release' package, zip it and then publish it to our Artifact Staging Directory on our build agent.
We then use a Publish Build Artifact task to publish an artifact called 'drop' into our staging directory.
- task: DotNetCoreCLI@2
displayName: Publish
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/*.csproj'
arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: True
- task: PublishBuildArtifacts@1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
Just a mini-recap before we move onto deployment. In our Build Stage we have:
- Built our .NET Projects
- Run our Unit Tests to ensure our build is a quality build
- Generate and publish a code test coverage report.
- Publish a build artifact to deploy to Azure.
With that achieved, we are now ready to move onto our deployment stage!
Deploying our Function Stage
For our deployment stage, we'll need to create a new stage in our YAML file.
In this case, I've added a condition and dependsOn property to our Stage YAML block. In the YAML file, I'm saying that the 'Deploy' stage depends on our 'Build' stage and that the 'Deploy' stage will only run on the condition that the 'Build' Stage has succeeded.
- stage: Deploy
displayName: 'Deploy MyHealth.Fitbit.Body'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy MyHealth.Fitbit.Body
environment: Production
pool:
vmImage: $(vmImageName)
When we use multiple stages in a pipeline, they will run in the order that we define them in the YAML file. We can add dependencies to our stages using the dependsOn parameter.
We can control which stages run by specifying the condition of that Stage. In my case, I'm stating that the 'Deploy' stage will only run provided that the 'Build' stage has run successfully.
To learn more about how conditions work in YAML pipelines, check out the following documentation.
Using Secrets from Key Vault ๐
In my function, I'm doing performing a variety of tasks such as sending messages to Service Bus Queues and Topics, calling APIs using Access tokens etc. etc.
In order to make these operations, I need to use connection strings and secrets that I'm storing in a Azure Key Vault. Because these are secrets and I don't want to expose them, I need to use a Azure Key Vault task within my pipeline.
For this purpose, I can use the Azure Key Vault task to download the secrets I need.
As a pre-requisite for this task, I need to create a Azure Resource Manager service connection that's linked to my Azure Subscription and I need a Key Vault that contains my secrets.
Once I have this setup, I can use the following snippet in my pipeline:
steps:
- task: AzureKeyVault@1
inputs:
azureSubscription: '$(azureSubscription)'
KeyVaultName: '<key-vault-name>'
SecretsFilter: '*'
RunAsPreJob: false
Here, we are defining the subscription that contains our Key Vault with our secrets, and we are retrieving all of the secrets in our Key Vault. We could filter these out to use a comma-separated list of the secret names we want to download, but for now I'm just downloading everything.
Deploying our Function to Azure โก
We're finally ready to deploy our Function! โก
For this I'm going to use a Azure App Service Deploy Task
We can use this task to deploy a variety of different app service types, but in my YAML snippet, I'm specifying functionApp as our type.
As a pre-req for this task, you'll need an App Service instance to deploy your code to.
I'm also passing the package that we built earlier as our package, using the $functionAppName variable for the name of our Function and then passing in key-values for our App Settings. I can pass in the secrets I need for my Function since we downloaded them in our Key Vault task.
- task: AzureRmWebAppDeployment@4
displayName: 'Deploy Azure Function'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
WebAppName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/MyHealth.Fitbit.Body.zip'
appSettings: '-FUNCTIONS_WORKER_RUNTIME "dotnet" -FUNCTIONS_EXTENSION_VERSION "~3" -KeyVaultName "<key-vault-name>" -AccessTokenName "<secret-name>" -ServiceBusConnectionString "$(<secret-value>)" -APPINSIGHTS_INSTRUMENTATIONKEY "<some-key>" -ExceptionQueue "myhealthexceptionqueue" -BodyTopic "myhealthbodytopic" -WEBSITE_TIME_ZONE "New Zealand Standard Time"'
enableCustomDeployment: true
RemoveAdditionalFilesFlag: true
Our full YAML file
Before we finish, here's my full YAML build pipeline:
trigger:
- main
variables:
buildConfiguration: 'Release'
vmImageName: 'vs2017-win2016'
functionAppName: 'famyhealthfitbitbody'
azureSubscription: '<azureSubscription-id-or-name>'
workingDirectory: '$(System.DefaultWorkingDirectory)/MyHealth.Fitbit.Body'
projectName: 'MyHealth.Fitbit.Body'
stages:
- stage: Build
displayName: Build Stage
jobs:
- job: Build
displayName: Build MyHealth.Fitbit.Body
pool:
vmImage: $(vmImageName)
steps:
- task: DotNetCoreCLI@2
displayName: Restore
inputs:
command: 'restore'
feedsToUse: 'select'
vstsFeed: '<artifact-feed-id>'
projects: '**/*.csproj'
- task: DotNetCoreCLI@2
displayName: Build
inputs:
command: 'build'
projects: '**/*.csproj'
arguments: --configuration $(buildConfiguration)
- task: DotNetCoreCLI@2
displayName: Run Unit Tests
inputs:
command: 'test'
projects: '**/*UnitTests/*.csproj'
arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/'
publishTestResults: true
- script: |
dotnet tool install -g dotnet-reportgenerator-globaltool
reportgenerator -reports:$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura
displayName: Create Code coverage report
- task: PublishCodeCoverageResults@1
displayName: 'Publish Code Coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)/**/coverage.cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)/TestResults/Coverage/'
- task: DotNetCoreCLI@2
displayName: Publish
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/*.csproj'
arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: True
- task: PublishBuildArtifacts@1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
- stage: Deploy
displayName: 'Deploy MyHealth.Fitbit.Body'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy MyHealth.Fitbit.Body
environment: Production
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureKeyVault@1
inputs:
azureSubscription: '$(azureSubscription)'
KeyVaultName: '<key-vault-name>'
SecretsFilter: '*'
RunAsPreJob: false
- task: AzureRmWebAppDeployment@4
displayName: 'Deploy Azure Function'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
WebAppName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/MyHealth.Fitbit.Body.zip'
appSettings: '-FUNCTIONS_WORKER_RUNTIME "dotnet" -FUNCTIONS_EXTENSION_VERSION "~3" -KeyVaultName "<key-vault-name>" -AccessTokenName "<secret-name>" -ServiceBusConnectionString "$(<secret-value>)" -APPINSIGHTS_INSTRUMENTATIONKEY "<some-key>" -ExceptionQueue "myhealthexceptionqueue" -BodyTopic "myhealthbodytopic" -WEBSITE_TIME_ZONE "New Zealand Standard Time"'
enableCustomDeployment: true
RemoveAdditionalFilesFlag: true
Running our build pipeline ๐
We've successfully created a build pipeline in YAML that will be triggered every time we push to our main branch. We can view the status of our build in Azure DevOps, which looks like this:
As you can see at the bottom of the image, we can see the result of each stage within our pipeline. If a particular stage had failed, we could use this to re-run a particular stage instead of having to push our code again to main.
Wrapping Up
Multi-Stage Build pipelines are awesome for defining our build and release process all within the same file. By using source control, we can track changes to our build process over time. We can also do useful things like generate and publish code coverage reports, download secrets from key vault and deploy apps all within one file!
Hopefully you found this article useful! As always, if you have any questions, feel free to comment below or ask me on Twitter!
Happy Coding! ๐ป๐จโ๐ป๐ฉโ๐ป
Top comments (2)
Thanks for sharing. Food for thought, it might be worth doing a follow up post going through approvals and promotion to other environments.
Hi Kane!
That's a great idea! I'll add it to my Blogging to-do list.
Thanks ๐