In this article, I use the famous DevOps HandBook to regurgitate a concise history of DevOps, add my personal experience and opinion, and establish a link to Cloud Development Environments (CDEs), i.e. the practice of providing access to, and running, development environments online as a service for developers.
A Few Good Books on DevOps’ History
The history of DevOps is definitely worth reading in a few good books about it. On that topic, “The Phoenix Project”, self-characterized as “a novel of IT and DevOps”, is often mentioned as a must-read. Yet for practitioners like myself a more hands-on one is “The DevOps Handbook” (which shares Kim as author in addition to Debois, Willis, and Humble) that recounts some of the watershed moments around the evolution of software engineering and provides good references around implementation. This book actually describes how to replicate the transformation explained in the Phoenix Project and provides case studies.
In this brief article, I will use my notes on this great book to regurgitate a concise history of DevOps, add my personal experience and opinion, and establish a link to Cloud Development Environments (CDEs), i.e. the practice of providing access to, and running, development environments online as a service for developers.
In particular I explain how the use of CDEs concludes the effort of bringing DevOps “fully online”. Explaining the benefits of this shift in development practices, plus a few personal notes is my main contribution in this brief article. Before clarifying the link between DevOps and CDEs, let’s first dig into the chain of events and technical contributions that led to today’s main methodology to deliver software.
The Agile Manifesto
The creation of the Agile Manifesto in 2001 sets forth values and principles as a response to more cumbersome software development methodologies like Waterfall and the Rational Unified Process (RUP).
One of the manifesto's core principles emphasizes the importance of delivering working software frequently, ranging from a few weeks to a couple of months, with a preference for shorter timescales. The Agile movement's influence expanded in 2008 during the Agile Conference in Toronto, where Andrew Shafer suggested applying Agile principles to IT infrastructure rather than just to the application code.
This idea was further propelled by a 2009 presentation at the Velocity Conference, where a paper from Flickr demonstrated the impressive feat of "10 deployments a day" using Dev and Ops collaboration. Inspired by these developments, Patrick Debois organized the first DevOps Days in Belgium, effectively coining the term "DevOps." This marked a significant milestone in the evolution of software development and operational practices, blending Agile's swift adaptability with a more inclusive approach to the entire IT infrastructure.
The Three Ways of DevOps and the Principles of Flow
All the concepts that I discussed so far are today incarnated into the “Three Ways of DevOps”, i.e. the foundational principles that guide the practices and processes in DevOps. In brief, these principles focus on:
- improving the flow of work (First Way), i.e the elimination of bottlenecks, reduction of batch sizes, and acceleration of workflow from development to production,
- amplifying feedback loops (Second Way), i.e. quickly and accurately collect information about any issues or inefficiencies in the system, and
- fostering a culture of continuous learning and experimentation (Third Way), i.e encourage a culture of continuous learning and experimentation.
Following the leads from Lean Manufacturing and Agile, it is easy to understand what led to the definition of the above three principles. I delve more deeply into each of these principles in this conference presentation. For the current discussion though, i.e. how DevOps history leads to Cloud Development Environments, we just need to look at the First Way, the principle of flow, to understand the causative link.
Chapter 9 of the DevOps Handbook explains that the technologies of version control and containerization are central to implementing DevOps flows and establishing a reliable and consistent development process.
At the center of enabling the flow is the practice of incorporating all production artifacts into version control to serve as a single source of truth. This enables the recreation of the entire production environment in a repeatable and documented fashion. It ensures that production-like, code development environments can be automatically generated, entirely self-serviced without requiring manual intervention from Operations.
The significance of this approach becomes evident at release time, which is often the first time where an application's behavior is observed in a production-like setting, complete with realistic load and production data sets. To reduce the likelihood of issues, developers are encouraged to operate production-like environments on their workstations, created on-demand and self-serviced through mechanisms such as virtual images or containers, utilizing tools like Vagrant or Docker. Putting these environments under version control allows for the entire pre-production and build processes to be recreated. Note that, production-like environments really refer to environments that, in addition to having the same infrastructure and application configuration as the real production environments, also contain additional applications and layers necessary for development.
Figure: Developers are encouraged to operate production-like environments (Docker icon) on their workstations using mechanisms such as virtual images or containers to reduce the likelihood of execution issues in production.
From Developer Workstations to a CDE Platform
The notion of self-service is already emphasized in the DevOps Handbook as a key enabler to the principle of flow. Using 2016 technology, this is realized by downloading environments to the developers’ workstation from a registry (such as DockerHub) that provides pre-configured, production-like environments as files (dubbed as infrastructure-as-code). Docker is often a tool to implement this function.
Starting from this operation, developers create an application in effect as follows: (1) they access and copy files with development environment information to their machines, (2) add source code to it in the local storage and (3) build the application locally using their workstation computing resources. This is illustrated in the left part of the figure below.
Once the application works correctly, the source code is sent (“pushed) to a central code repository and the application is built and deployed online, i.e. using Cloud-based resources and applications such as CI/CD pipelines.
The three development steps listed above are in effect the only operations, in addition to the authoring of source code using an IDE, that are “local”, i.e. they use workstations’ physical storage and computing resources. All the rest of the DevOps operations are performed using web-based applications and used as-a-service by developers and operators (even when these applications are self-hosted by the organization.). The basic goal of Cloud Development Environments is to m_ove these development steps online as well_.
To do that, CDE platforms in essence provide the following basic services, illustrated in the right part of the figure below. :
- manage development environments online, as containers or virtual machines such that developers can access them fully built and configured, substituting step (1) above; then
- provide a mechanism for authoring source code online, i.e. inside the development environment using an IDE or a terminal, substituting step (2); and finally
- _provide a way to execute build commands _inside the development environment (via the IDE or terminal), substituting step (3).
Figure: (left) The classic development data flow requires the use of the local workstation resources. (right) The cloud development data flow replaced local storage and computing while keeping a similar developer experience. On each side, operations are (1) accessing environment information, (2) adding code, and (3) building the application.
Note that, the replacement of step (2) can be done in several ways: for example the IDE can be browser-based (aka a Cloud IDE), or a locally installed IDE can implement a way to remotely author the code in the remote environment. It is also possible to use a console text editor via a terminal such as vim.
I cannot conclude this discussion without mentioning that, often multiple containerized environments are used for testing on the workstation, in particular in combination with the main containerized development environment. Hence cloud IDE platforms need to reproduce the capability to run containerized environments inside the Cloud Development Environment (itself a containerized environment). If this recursive process becomes a bit complicated to grasp, don’t worry, we reached the end of the discussion and we can move to the conclusion.
What Comes Out of Using Cloud Development Environments in DevOps
A good way to conclude this discussion is to summarize the benefits of moving development environments from the developers’ workstation online using CDEs. As a result, the use for CDEs for DevOps lead the following advantages:
Streamlined Workflow: CDEs enhance the workflow by removing data from the developer's workstation, decoupling the hardware from the development process. This ensures that the development environment is consistent and not limited by local hardware constraints.
Environment Definition: With CDEs, version control becomes more robust as it can uniformize not only the environment definition but all the tools attached to the workflow, leading to a standardized development process across the organization and consistency across teams.
Centralized Environments: The self-service aspect is improved by centralizing the production, maintenance, and evolution of environments based on distributed development activities. This allows developers to quickly access and manage their environments without the need for Operations' manual work.
Asset Utilization: Migrating the consumption of computing resources from local hardware to centralized and shared cloud resources not only lightens the load on local machines but also leads to more efficient use of organizational resources and potential cost savings. You can test economic assumptions with this online calculator.
Improved Collaboration: Ubiquitous access to development environments, secured by embedded security measures in the access mechanisms, allows organizations to cater to a diverse group of developers, including internal, external, and temporary workers, fostering collaboration across various teams and geographies.
Scalability and Flexibility: CDEs offer scalable cloud resources that can be adjusted to project demands, facilitating the management of multiple containerized environments for testing and development, thus supporting the distributed nature of modern software development teams.
Enhanced Security and Observability: Centralizing development environments in the Cloud not only improves security (more about secure CDEs) but also provides immediate observability due to their online nature, allowing for real-time monitoring and management of development activities.
By integrating these aspects, CDEs become a solution for modern, im particular cloud-native software development, and aligns with the principles of DevOps to improve flow, but also feedback, and continuous learning. I will discuss in more detail contributions of CDEs across all three ways of DevOps in an upcoming article. In the meantime, you're welcome to share your feedback with me.
All material in this text can be shared and cited with appropriate credits. For more information about our platform, please contact us at hello@strong.network
Copyright © 2020-2024 Strong Network All rights reserved.
Top comments (0)