YAML Based VM Configuration With Cloud-init

Cloud-init is a Canonical project and describes itself as follows: " Cloud -init is the industry-standard multi-distribution method for the cross-platform initialization of cloud instances."

To put it more simply: Cloud-init enables the complete configuration of virtual machines (VMs) using a simple text file. Of course, cloud instances mean nothing other than VMs that run on cloud providers such as AWS, Azure or GCC. Basically only with one special feature, namely that the VMs are often fleeting (ephemeral), i.e. they only run for a short time and are then deleted again.

Initialization, in turn, simply means the configuration, i.e. everything apart from the hardware equipment: network connections, settings, installed software, user profiles and so on. And "multi-distribution" probably means nothing other than the fact that you can have several identically configured VMs created at the same time.

Cross-platform means pretty much all major cloud providers are supported; In addition to those already mentioned, Oracle, UpCloud, VMware, OpenNebula, among others, around 25 in total. And there are hardly any restrictions regarding the supported operating systems for the VMs; 26 distributions are listed, in addition to obvious ones such as Debian, Ubuntu , RHEL, Arch Linux and Fedora, also openEuler, Rocky, Virtuozzo and the common BSD systems.

Cloud-init explained in a very rudimentary way: "Dear IT department, we need several VMs with SSH access through Server X, Apache web server, mounted network drives, software packages according to the attached list, the user foobar and a RHEL registration." Cloud-init is nothing more than such a request - just formalized and sent to a supported system.

Cloud-init in Practice

The easiest way to play with cloud-init is probably Multipass , also a Canonical project and already presented here some time ago . With Multipass, "cloud instances" can be created locally, i.e. VMs, which can optionally be set up using cloud-init. This works on Windows as well as Linux.

By default, Multipass creates VMs with Ubuntu 22.04, hardware equipment and network connection are simply specified via options:

multipass launch --mem=2G --cpus=2 --name testvm1 --network name=LAN-Connection

So here an Ubuntu VM is created with 2 GB RAM, 2 CPUs and the (Windows) standard network connection "LAN connection".
If a configuration is to be carried out via cloud-config, the config file is also passed to "multipass launch":

multipass launch --mem=2G --cpus=2 --name testvm1 --network name=LAN-Connection -- cloud-init testconf.ini

All cloud-init work takes place in this "testconf.ini" - a YAML file. YAML is a language based on XML for the machine-readable representation of data structures.

This representation can sometimes be a bit fiddly: be sure to ensure correct indentation. Although it usually doesn't matter whether items are indented with two, four or even three spaces, it must be consistent across the entire document - indentation is part of the YAML syntax! Too many or too few spaces immediately lead to errors.

Cloud-config offers around 60 modules that manage general things like users, but also more specific aspects like the text-based window manager Byobu. A simple version could, for example, simply execute a command in the newly created Multipass VM:

#cloud-config
runcmd:
   - echo „Hello World!"
   - echo „End"

The shebang line is required. The module "runcmd" (run command) is then called, which simply executes the following commands.
Here is a practical example that should do the following tasks:

• Output "Hello, World!"
• Install the git and apache2 packages
• Set up standard user ("ubuntu") and user "peter".
• Enable passwordless access from the server "myserver" for user "talha".
• Mount a network drive (Samba share) "myserver/myshare".
• Write a message to the installation log

Formulated via YAML:

#cloud-config
runcmd:
   - echo 'Hello, World!'
packages:
   - git
   - apache2
users:
   - default
   - peter
ssh_authorized_keys:
   - ssh-rsa AAAAB3... talha@myserver
mounts:
   - [ //192.168.178.1/myshare, /media/myshare, cifs, "defaults,uid=1000,username=talha,password=34565234687gyft@" ]
final_message: |
   cloud-init done
   Version: $version
   Timestamp: $timestamp
   Datasource: $datasource
   Uptime: $uptime

The "packages" module is only used here to install packages, but can also update using the "package_update" and "package_upgrade" calls.

At this point the platform independence is also nice to see. In earlier versions it was called "apt_update", but now the appropriate package manager is targeted. Separate modules are then available for configuring the package managers, e.g. apt, yum/dnf or zypper.

Of course, the "users" module also has a lot more to offer. The full range of account settings can be used here, such as group memberships, user information, passwords, home directories, specific user shell, sudo membership, etc.

But the module "ssh_authorized_keys:" is really interesting. Here it is possible to store the contents of public SSH keys (admittedly in an abbreviated form above) in order to immediately grant password-free access for certain users. This would be practical, for example, for automatic configuration via Ansible , which by default connects to the hosts to be managed via SSH.

The "mounts" module also does exactly what is expected: it writes entries into the fstab - only the separation via comma differs from manual entries.

These examples clearly show what cloud-init essentially does: execute standard commands. Only, for example, it says abstractly "packages: apache2" instead of system-specifically "apt install apache2". You could say that cloud-init is ultimately "just" an abstraction layer for what admins otherwise do manually in the first half hour after setting up a VM… or even on the first day.

More Functions

If you’re interested, it’s best to look through the individual modules yourself; the Cloud-init module reference is very clear and, fortunately, mostly enriched with practical examples. To illustrate the diversity, here are a few highlights.

Using the Ansible module, playbooks can be executed via ansible pull — even without SSH access. There is a module “Byobu” that can be used to configure the window manager mentioned above — very special. The “Phone Home” module is powerful and can be used to send data to a URL.

Similarly powerful is “Scripts per Boot”, which, as expected, executes any script when an instance starts — every time, mind you! Most modules run once per instance, i.e. once during setup. Optionally, they can also be active (“Always”) at every start. For example, “Phone Home” could be used to send data to an API after every start .

Last but not least: Cloud-init also has a command line interface , for example to run individual modules, check cloud-config files, analyze logs and so on.

Cloud-init is a wonderful tool for understanding and implementing “Infrastructure as Code”. And not just on a large scale, by the way, the combination of cloud-init plus Multipass is unbeatable even for local test environments on the desktop.

This article was first published on my medium blog.