A New DevOps Workflow: On-Demand Environments

One of the unavoidable question when designing a development workflow in a company is the management of cloud environments. Environments are mostly needed in the development process to perform tests that are either too heavy to be run locally, or that use managed cloud services. Yet, they can also be required to perform demos to customers, for example.

Regarding cloud environments, most of the teams still work under the same paradigm as before the cloud era. They have static environments - a set of dedicated resources running continuously. However, thanks to the elasticity offered by the cloud, a new paradigm is getting adopted by top performers: on-demand cloud environments. It is time-consuming to set up the automation needed to reach that goal, but it allows teams to reduce significantly their cycle time (the time from the moment a new development branch is created, until it is released in production) and deploy safer code.

In this post, I will present and analyze these two different configurations adopted regarding cloud environments in companies, the usual static environments and the deployment of dynamic ones.

Static cloud environments

In companies that have static cloud environments, developers usually work on their machine — i.e. the local environment, and then make a pull / merge request when they feel ready to push it to the staging environment. The code must usually go through the CI and be reviewed before being deployed to staging, either automatically through a CD pipeline or manually by the DevOps team. The number of staging environments varies from one company to another, there can be from one staging for the whole tech team up to one per feature team. In this second case there is usually in addition a pre-prod environment which is the last step before the release to production.

In a classic organization, cloud environments are shared.

Because staging environments are shared, issues arise quickly:

• Every deployment on staging needs to be as foolproof as possible, because if the staging breaks it would slow down the whole team. Thus, it takes time for the code to go through the different stages and arrive to Production. It needs first to pass the CI and to be reviewed. Even for a functional preview with the product manager, one's code needs to pass those steps — slowing down the development process.
• Staging environments are rarely available. As you can't perform two tests simultaneously, there is usually a "waiting list" to test, slowing down the development process.

Since every developer is only given a specific slot on Staging, they only have time to perform a limited number of test, in order not to block the others. This in turn increases the risk of error in Production.

Thus, a bottleneck emerges. It's especially true if the number of developers per environment is important. But having multiple environments is a costly solution as more static environments means a bigger cloud bill.

Moreover, teams in this configuration usually have a low level of automation. Therefore, significant time — usually from the DevOps team, is dedicated to maintain those environments: clean the environments after the tests, provision them with recent data, and update features from every teams on each environment.

To sum up, having static environments is the easiest system to set up and works well for small teams that have a low level of automation. However, it can quickly create a bottleneck in the development process and be time-consuming for the DevOps team at scale.

Dynamic cloud environments

The best performers in deployment automation set up internal tools to allow developers to spin-up isolated cloud environments from scratch in just a few minutes. Team members can usually deploy a whole environment (containers and dependencies — databases, cache, etc.) on a Kubernetes namespace. This approach is becoming increasingly popular. See for example this article about how Dollar Shave Club implemented it in 2017.

Dynamic environments have a lot of benefits:

• The deployed environments are completely isolated — they are not linked to external components, making it risk-free to use them.
• Environments can be launched by anyone, and can be shared or personal — solving the issues cited above.
• Environments are deployed for a specific task and destroyed once achieved. They don't need to be cleaned and maintained.

How does it work? Usually, you deploy one environment per branch, on which you test before deploying to a shared staging environment. It's useful for functional previews with product managers, code reviews or complex automated testing (integration/load testing). There are a lot of possibilities, on-demand environments can even be use to set up demos.

In the most agile organizations, each developer deploys his own isolated environment.

On-demand environments encourage the autonomy of developers. They're the owner of the code at any time, from its writing, to its testing and finally its push to Production. This reduces the cycle time and helps ensure that the code deployed to Staging is bug-free. Finally, at some point it decreases the cloud bill compared to having one staging environment per team.

However, it's complex for companies to adopt on-demand environments. Indeed, implementing this kind of solution internally is time-consuming for the DevOps team — usually it's a several months project to set up this workflow. Moreover, it's not enough to be technically able to spin up environments in minutes, some other questions need to be answered to make the solution useful:

• How to test a feature which spans multiple repositories — i.e. how to deploy the right version of each service on your environment?
• How to provision these environments with data while ensuring that they are still isolated?
• How to ensure that the organization's members — developers but also product managers and sales — are autonomous on the deployment of environments?
• How to manage the cost of this approach? If environments aren't well sized, if some environments are "forgotten", or if the whole application is deployed each time, this approach becomes pretty expensive.

Answering all these questions through on-demand environments makes it a significant project for the DevOps to develop and maintain.

At Strio, we firmly believe in the benefits of on-demand environments for teams constantly wishing to improve their product at fast pace. However, we think it shouldn't be limited to the teams who have the resources to dedicate a whole team to set it up. That's why we decided to develop a turn-key solution that DevOps can configure and provide to the development team. Try it now!