TL;DR
acloudguru released a #CloudGuruChallenge on February 8th that requires a continuously integrated global app.
Use the following:
- Azure App Service for web
- Azure App Service Deployment Slots for staging
- Azure Cosmos DB for database
- Azure Front Door for routing
- Azure Pipelines for deployment
- Azure Resource Manager (ARM) for infrastructure
After a very bumpy ride I was able to complete this challenge. It's proven to be the most difficult challenge to date, probably due to my own choices, which explains why I'm posting a bit after the deadline that was on March 31st.
Due to the expensive nature of these services I have deleted the resources already. Considering, everything is scripted it's not terribly difficult to deploy the infrastructure with ARM and the code with Azure Pipelines.
Check it out! Let's connect?
GitHub: https://github.com/wheelerswebservices/cgc-azure-cicd
LinkedIn: https://www.linkedin.com/in/wheelerswebservices/
Naïve Beginning
The requirements mentioned a slew of programming language options that I could've used to complete the challenge.
- C#
- Java
- PHP
- Python
- Node.js
- Ruby (selected)
- VB
I chose Ruby since I have never used it before and I wanted to learn something new. From the list above I've used all of these languages before besides Ruby and VB.
I quickly realized that the only Ruby framework supported on app service was Rails. That put me off a bit since I had no prior Ruby experience and have heard about how large Rails truly is, yet I didn't turn back yet.
https://docs.microsoft.com/en-us/azure/app-service/quickstart-ruby
App Configuration
I tried to keep the app simple, but functional. I had to build a verification application that would capture user details like name, address, photo, and voice.
(Show User)
I would store the user details in Cosmos DB and store the photo and voice objects in Blob storage. I personally believe that binary objects like these should not be stored in traditional databases, even if they can be.
(Environment Variables)
I only used a single model, User, for this app. I decided to use the generated user id as a key in the Blob storage so I could correlate the binary objects back to the user they belong to. Since I needed the user id I moved the photo and voice capture to the Edit screen so the user would have to be created first and then updated.
Originally I was attempting the uploads on the user creation itself, but scraped that idea. I recalled that popular sites like GitHub and others require a user account before uploading an image so thought this implementation was fine.
(Cosmos Storage)
(Blob Storage)
Architecture
Azure
- App Service: used to host the app
- Blob Storage: used to store photo and voice objects
- Cosmos DB: used to store user data
- Front Door: used as entry point
- Resource Manager: used to deploy infrastructure (IaC)
- Virtual Network: used to protect the app
DevOps
- Disk: used to store local data for the agent
- Network Security Group (NSG): used to protect the agent
- Pipelines: used for CI/CD
- Public IP: used to access the agent
- Virtual Machine: used to run pipeline actions
(Deployment)
(Resource Group)
⚠️ ARM Trouble
The requirements outlined using ARM to deploy the infrastructure. This is the Infrastructure as Code (IaC) solution on Azure.
I created 2 ARM templates:
- verify-app: used to deploy the infrastructure
- ruby-agent: used to deploy the self-hosted agent
I'll be honest that I wasn't too familiar with ARM when I got started. Although, I'm very familiar with CloudFormation (AWS IaC), so I was excited to jump into this as it looked similar on the surface.
Let me tell you that I was mistaken. I love the ease of use that CloudFormation provides and didn't find that in ARM. The syntax was much harder to grasp and each resource had to define it's own API version that allowed Microsoft to make compatibility breaking changes. I received the tip early on to build my resources out using the Azure Portal and then export the template using the Export Template feature.
Okay yes, what a great tip. This helped a lot. Unfortunately, there were still complications.
- The portal did not export everything. Occasionally, I would get error messages saying some features were not supported.
- The API versions and fields exported from the portal didn't always match the ARM documentation. My templates would reference versions that were not listed in the documentation. Worse than that my templates had declared fields that were not listed in the documentation at all.
- The ARM documentation seemed like it needs work from a usability perspective. I would see resource fields declared that expected specific values that I could not find anywhere. I got to learn the expected values only when attempting to deploy that template with an invalid value in place.
- The portal does not seem to support ARM deployments the same way AWS does with CloudFormation. There is no service for using ARM. I had to leverage a Marketplace product that was in preview to do the deployments.
- There is no automatic rollback. I'm used to failures resulting in automated cleanup, but didn't get that in Azure. Failures would result in my infrastructure being partially deployed.
My overall feeling for ARM is that I'm disappointed.
⚠️ Database Trouble
The requirements outlined using Cosmos SQL API. If you didn't know Cosmos DB is capable of communicating over multiple APIs to ease use.
- Azure Tables
- Cassandra
- Core SQL
- Gremlin (Graph)
- MongoDB
Wait a second... you may have noticed that I mentioned I'm using the mongoid gem and the Cosmos MongoDB API. Well yes, I tried to use the SQL API like the requirements specified. However, my efforts were in vain. I didn't find any way to achieve that. I even posted on Stack Overflow once I felt lost who also suggested that I change paths.
https://stackoverflow.com/questions/66947849/how-to-connect-ruby-rails-6-app-to-azure-cosmos-sql-db
I truly believe that Ruby is not treated as a first-class citizen by Microsoft Azure. This was clear by the lack of documentation, examples, and overall help for Ruby apps. I would say that it's even more obvious since their flagship DB doesn't provide support for Ruby out-of-the-box.
I'm hoping this changes in the future.
⚠️ Ruby Version Trouble
I didn't spend sufficient time researching Ruby versions or Azure compatibility before diving in. I started with the RubyInstaller's recommendation of Ruby 2.7.2.
Once I had my app code written I tried to deploy the code to Azure App Service through the Azure Pipelines Microsoft-hosted agents. That's when I learned that the build agent was using Ruby 2.6.6.
I struggled to downgrade my Ruby version to match the build agent. I felt forced to uninstall Ruby and re-installing the proper version from scratch.
The next time I ran the pipeline it successfully deployed! Success? Not quite the App Service showed that the app failed to start. The logs informed me that App Service was using Ruby 2.6.2. 😕
I couldn't understand why Azure would run different versions of Ruby on their App Service and their build agents. I briefly tried to downgrade once again, but was unable since the RubyInstaller I've been using did not list 2.6.2 as an option.
I attempted to omit the minor version completely and deploy ~2.6, which failed.
App Service Log
2021-04-02T04:27:01.516359889Z _____
2021-04-02T04:27:01.516391591Z / _ \ __________ _________ ____
2021-04-02T04:27:01.516396291Z / /_\ \___ / | \_ __ \_/ __ \
2021-04-02T04:27:01.516400091Z / | \/ /| | /| | \/\ ___/
2021-04-02T04:27:01.516403592Z \____|__ /_____ \____/ |__| \___ >
2021-04-02T04:27:01.516407092Z \/ \/ \/
2021-04-02T04:27:01.516410492Z A P P S E R V I C E O N L I N U X
2021-04-02T04:27:01.516413692Z
2021-04-02T04:27:01.516416692Z Documentation: http://aka.ms/webapp-linux
2021-04-02T04:27:01.516419892Z Ruby quickstart: https://aka.ms/ruby-qs
2021-04-02T04:27:01.516423093Z Ruby version 2.6.2
2021-04-02T04:27:01.516426193Z Note: Any data outside '/home' is not persisted
2021-04-02T04:27:02.189573568Z Starting OpenBSD Secure Shell server: sshd.
2021-04-02T04:27:03.195641038Z Bundle install with no 'without' options
2021-04-02T04:27:03.202609004Z Defaulting gem installation directory to /tmp/bundle
2021-04-02T04:27:03.202897520Z Defaulting site config directory to /home/site/config
2021-04-02T04:27:03.204055381Z Generating a secret key base
2021-04-02T04:27:03.692631856Z RAILS_ENV not set, default to production
2021-04-02T04:27:03.692659357Z Removing any leftover pids if present
2021-04-02T04:27:03.692723961Z rbenv: version `ruby-2.6' is not installed (set by /home/site/wwwroot/.ruby-version)
2021-04-02T04:27:03.744540384Z Running bundle check
2021-04-02T04:27:04.261191335Z rbenv: version `ruby-2.6' is not installed (set by /home/site/wwwroot/.ruby-version)
2021-04-02T04:27:04.682632482Z rbenv: version `ruby-2.6' is not installed (set by /home/site/wwwroot/.ruby-version)
2021-04-02T04:27:04.692844319Z missing dependencies, try redeploying
2021-04-02T04:27:05.282227084Z rbenv: version `ruby-2.6' is not installed (set by /home/site/wwwroot/.ruby-version)
Pipeline Log
2021-04-01T03:26:44.8712433Z ##[section]Starting: UseRubyVersion
2021-04-01T03:26:44.8719527Z ==============================================================================
2021-04-01T03:26:44.8719967Z Task : Use Ruby version
2021-04-01T03:26:44.8720409Z Description : Use the specified version of Ruby from the tool cache, optionally adding it to the PATH
2021-04-01T03:26:44.8720799Z Version : 0.182.0
2021-04-01T03:26:44.8721102Z Author : Microsoft Corporation
2021-04-01T03:26:44.8721535Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/tool/use-ruby-version
2021-04-01T03:26:44.8722005Z ==============================================================================
2021-04-01T03:26:45.3163457Z ##[warning]It is not recommended to specify exact version on Microsoft-Hosted agents. Patch version of Ruby can be replaced by new one on Hosted agents without notice and build stops to work. it is recommended to specify only major or major and minor version (Example: `2` or `2.4`)
2021-04-01T03:26:45.3185530Z ##[error]Version spec 2.6.2 for architecture %25s did not match any version in Agent.ToolsDirectory.
Available versions: /opt/hostedtoolcache
2.5.8,2.6.6,2.7.2,3.0.0
If this is a Microsoft-hosted agent, check that this image supports side-by-side versions of Ruby at https://aka.ms/hosted-agent-software.
If this is a self-hosted agent, see how to configure side-by-side Ruby versions at https://go.microsoft.com/fwlink/?linkid=2005989.
2021-04-01T03:26:45.3229225Z ##[section]Finishing: UseRubyVersion
All this was incredibly frustrating and I reached out on Stack Overflow for guidance. The consensus was that I had to build a self-hosted agent to deploy this code to App Service.
Lesson learned here was that I struggled so much to switch between Ruby versions that I decided to install Ruby Version Manager (RVM) to simplify things. Since RVM is not supported on Windows at this time I decided to install Windows Subsystem for Linux (WSL) and Ubuntu on my Windows desktop. Before you ask, yes! That was easier than managing multiple Ruby versions on my Windows computer.
I'm thankful that my IDE, RubyMine, was able to utilize the Ruby SDK from the WSL system.
Pipeline Configuration
Speaking of Azure Pipelines I quickly understood the familiar yml syntax used. This acloudguru course, Build and Deploy Pipelines with Microsoft Azure, helped me close the gaps I did have so I didn't get stuck here for too long.
trigger:
- main
pool:
name: Default
demands:
- agent.name -equals wheeler-verify-app-agent
variables:
- name: appDir
value: app
steps:
- task: NodeTool@0
inputs:
versionSpec: '14.16.1'
displayName: 'Install Node'
- script: |
npm install --global yarn
yarn --version
displayName: 'Install Yarn'
workingDirectory: $(appDir)
- script: bundle install
displayName: 'Install Bundle Dependencies'
workingDirectory: $(appDir)
- script: yarn install --check-files
displayName: 'Install Yarn Dependencies'
workingDirectory: $(appDir)
- script: |
rm -rf bin/webpack*
rails webpacker:install
displayName: 'Install Webpacker Dependencies'
workingDirectory: $(appDir)
- task: AzureWebApp@1
inputs:
azureSubscription: 'WheelerLearning/wheeler146'
appName: 'wheeler-verify-app-site'
deploymentMethod: 'zipDeploy'
package: $(System.DefaultWorkingDirectory)/$(appDir)
slotName: 'stg'
- trigger: used to execute the pipeline when branches were changed
- pool: used to specify the build agent
- variables: used to reduce duplication
- steps: used to declare pipeline actions (build, test, deploy, etc.)
The steps were divided into script actions and task actions.
- script: used to execute bash scripts for manual actions
- task: used to execute pre-built logic provided by Microsoft
https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/?view=azure-devops
Conclusion
All considered I'm glad I participated in this challenge. I felt like I learned a tremendous amount in this exercise. I definitely believe that I learned a lot more than I would have if I simply decided to use a programming language that I am already proficient in like Java.
In hindsight it seems that if I spent more time at the start of the project researching I may have been fortunate enough to avoid some pitfalls and maybe even finish on time.
I will practice restraint to save heartache and headaches half-way through my future endeavors. Even though am eager to dive into the next adventure the #CloudGuruChallenge has in store for me.
Please don't hesitate to reach out to me on LinkedIn if you'd like to connect and let me know what you think about my experience.
GitHub: https://github.com/wheelerswebservices/cgc-azure-cicd
LinkedIn: https://www.linkedin.com/in/wheelerswebservices/
Did you also participate in #CloudGuruChallenge?
- I'd love to see your work.
Did you like this?
- I'd recommend that you check out my previous #CloudGuruChallenge accomplishments linked to this series.
Did you dislike this?
- Let me know what you didn't like so I can improve.
Top comments (0)