Deploying AWS applications using Azure sounds insane , right ? However , that is pretty cool and interesting to do , and quite easy also . So , read on.
What am I trying to do ?
A few days back I created a serverless app on AWS powered by AWS SAM . It was a pretty simple app basically consisting of a single lambda function which you could use to validate whether a given AccessKeyID was valid or not . You can find all details about the app and the corresponding source code here https://dev.to/turjachaudhuri/a-serverless-api-to-validate-aws-access-keys-based-on-aws-sam-2l3d
Using a SAM template to control all the components of a serverless setup was pretty awesome , but I wanted to go a step further and deploy a CI/CD pipeline , so that whenever I made any changes to my source code on git , the pipeline would automatically build and upload my whole serverless application on to AWS in a controlled manner . I managed to achieve that quite easily using TravisCI as my continuous integration pipeline . Details on that can be found here -https://dev.to/turjachaudhuri/a-cicd-pipeline-using-git-and-travis-ci-for-a-serverless-app-based-on-sam-and-c-35n
Even though TravisCI was working even better than I expected it to , I still wanted to continue my DevOps journey and experiment with other tools and solutions . And who better to get your hands dirty with than Azure DevOps .
What is Azure DevOps?
Essentially Azure DevOps is a whole lot of things . Microsoft rebranded Visual Studio Team Services (VSTS), the DevOps offering that has been a part of the Visual Studio IDE for years, to be the cloud-hosted Azure DevOps.
You can create exhaustive build and release pipelines on a wide range of supported frameworks and runtimes using Azure DevOps . Also , Microsoft has off the shelf provided a lot of handy build and release tasks that you can use to push/deploy changes in a controlled but automatic fashion to a wide range of supported platforms like AWS and so on.
What is the basic premise of all this?
At the basic level what we are trying to achieve is - whenever someone pushes to our code repo in git , we want to build our project (written in dotnetcore 2.0) , publish the project (so that we can get the publish artifacts) , push the published code zip to a S3 bucket , transform our input SAM template file - template.json into a yaml file referencing the S3 code zip , and then deploy the whole setup to AWS using AWS Cloudformation.
The basic assumption here is that anyone reading about this blog knows about how to create and configure AWS serverless apps based on AWS SAM . If not , please read my blog for detailed steps :
https://dev.to/turjachaudhuri/my-first-application-in-aws-serverless-application-repository-1ahc
How does Azure DevOps help?
In any traditional CI/CD setup , one common pattern can be deciphered . There is most probably a source control repo that triggers everything . Then there is a build pipeline , which builds the project , runs unit test cases , and publishes artifacts . Then there are one or more release pipelines that uses those artifacts to finally deploy the solution into say a cloud environment , a docker image and so on.
Azure DevOps provides a seamless step by step approach to doing that quite easily . Here we will use Azure DevOps to do basically the following.
We have a git repo where we have the source code for the project including the SAM template file - template.json.
We will create a Build pipeline using the .NET Core Lambda deployment task to package our code into a zip and upload that into S3 , and transform the template.json into a serverless-output.yaml file.
We will then publish the created artifacts into a staging directory so that it can be referenced by the subsequent Release pipelines.
Then we will create a Release pipeline which will use the AWS Cloudformation Create/Update Stack task to deploy our serverless app using the serverless-output.yaml file and the code zip from S3.
Sounds good , let's start!
Step 1 : To connect from Azure DevOps to AWS , we need AWS Credentials . However , it is not a good practice to hardcode the AWS credentials like accessKey and secretKey in scripts . So , we will create a Service Endpoints in Azure DevOps . This is nothing but a connection that we create to AWS , and then we can reference this connection in our build/release pipelines via the service endpoint name rather than scripting them in the pipelines ourselves.
Step 2 : Setup a git repo to host the source code. This is pretty simple and easy to do.
Step 3 : Configure a Build Pipeline.
3.1 : I have created a build pipeline with the agent pool as Hosted Linux Preview. Think of the agent pool as the VM where your code will be downloaded and then the build process will run as per your specifications.
3.2 : The next step is to hook up the build process with a source control repo . The code in the repo will essentially be the source input for the build process . Select your project , repo and branch .
3.3 : Next you add the first task - *AWS Lambda .NET Core Deployment *. This task can actually be configured to both build and deploy to AWS in a single shot . However , we want separate build and release pipelines . As a result in the build pipeline ,we will only use a limited set of functionality to only create the packaged code zip and generate the CloudFormation output YAML file. A detailed description of the task and its associated attributes can be found here -
https://docs.aws.amazon.com/vsts/latest/userguide/lambda-netcore-deploy.html
The important portions to note here are -
- Choose the AWS credentials as per your configuration in Step 1
- Select the region where you want to deploy from the dropdown
- Select the type of deployment - Function ( if you want to deploy a single lambda function and nothing else) , Serverless Application ( if you want to deploy a complete serverless setup like the one we have here)
- Package-only output file - This signifies where the output yaml file which is an output of this step will be stored.
- Path to lambda project - Specify the serverless project/solution that we want to deploy
- Create Deployment Package only - Tick this as YES since we just want to package the serverless app and not deploy it . This is because we want to use separate build and deploy pipelines . The build pipeline will simply publish the artifacts , and not deploy them.
- Stack name - not needed as we are not doing any sort of deployment
- S3 Bucket - name of the bucket where the published solution will be stored as a zip .
- S3 Prefix - prefix to append to the file zip if any
- Additional Lambda Tools Command Line Arguments : By default the template file in these type of projects is named as serverless.template file . Since , my file is named template.json , I am specifying this additional command line argument : --template template.json
Step 3.4 : Add the Publish Build Artifacts step to the build process . In Step 3 , we generated the output serverless-output.yaml file . However , we need to publish it into an artifact that will be saved in Azure DevOps that we can later reference in the Release pipeline -
Important things to keep in mind here :
- Path to publish is the file/folder location that was the output of Step 3 and we want to publish it into an output artifact for referencing in the Release pipelines downstream.
- Artifact name is the name with which the published artifacts will be referenced both in the downstream pipelines or if we want to manually download them from the Azure DevOps console.
Step 4 : Add this stage , you can manually trigger a build action to confirm that everything is working as expected . If everything is okay , you should see a screen like this -
You can even check the logs / download the artifacts if you want to verify or investigate further.
Step 5 : Now we need to create the release pipeline . The input of the release pipeline will typically be the output of the build pipeline . So ,we first need to hook the release pipeline with the artifacts published as part of the build pipeline , which is pretty simple and just a matter of selecting the project , source (Build pipeline) and so on from the dropdowns.
Next , we need to add a stage . As of now , I have added a single stage called AWS-SAM-Deploy . Typically you might have one stage per environment , like say DEV,QA,Prod and variations of the same.
Within every stage , you can add tasks to configure and control your release stages. These tasks basically execute the crucial steps of deploying/infrastructure provisioning whenever the release pipeline runs . In our case , we have already packaged our code into S3 , and generated the serverless-output.yaml file as part of our build pipeline .So , we simply need to deploy our app using a Cloudformation stack based on the serverless-output.yaml file . So , we simply add a single task - AWS Cloudformation Create/Update Stack.
The important things to keep in mind here are-
- Stack Name : The name of the Cloudformation Stack within which the serverless resources will be deployed.
- Template Source : Local file (will input the file name and path rather than a S3 URI to access the template file)
- Template file : $(System.DefaultWorkingDirectory)/Azure-DecOps-AWS-SAM-ASP.NET Core-CI/SAMDeliverables/a/serverless-output.yaml Here we reference the SAMDeliverables artifact that we published as an artifact output in the Build pipeline.
- Create/Update the stack using a change set : This needs to be set for serverless app deployment using Cloudformation
Step 6 : Now , all configurations are done , you can manually trigger a Release pipeline . If all goes well , you should see a screen like this -
Also , if you see Cloudformation console in your AWS account , you should be able to see the created stack and all its associated resources .
All done , closing remarks
Azure DevOps is actually pretty cool , and I like it more than TravisCI . However , currently the first task that I used to build and package the code only support dotnetcore. So , it will not be possible to use it for NodeJS or Java based lambda functions and so on . Also, Azure DevOps is not open source . But , all said and done , it is pretty cool and I can't wait to do more interesting stuff with it.
Inspiration and help from -
Top comments (0)