We ended the second article in this series about the configuration management tool Salt, with a problem. Throughout that post we defined roles in grains and configured what packages we wanted to install in the pillar. But at the end of the post, we were still installing packages by targeting the grains in the state.

To make the states really dynamic, we don't want to hard-code roles and packages in them. We want the state logic to be flexible. We want to configure our environment in salt's own key value store (the grains and the pillar). To do that, we need to go deeper into templating.

Renderers in Salt

By default, every salt state file is passed through a renderer before being processed by salt. The default renderer in salt is a Python templating engine called Jinja. The output of the Jinja process is then passed to a YAML processor which parses the data in the files into a data structure usable by salt. This rendering pipeline is commonly represented as jinja|yaml (if you know Unix pipes, you'll understand why).

These represent the two types of renderers in salt. Jinja is a text renderer, that takes in some text and returns some text. YAML is a data renderer, that takes in some text and returns a data structure.

There are other text renderers available such as genshi and mako, both of them templating engines. There are also other data renderers, such as json.

In this post we'll stick to the default jinja|yaml and focus on the text renderer, namely Jinja.

Jinja

As a templating language, Jinja allows us to generate output text files by processing a bit of logic placed in a source text file, a template file as it is called. Jinja even allows us to use a bit of Python for the logic.

For example let's say we would want to turn a Python list into a Markdown list we could have a template like this:

Here is a markdown list for you:
{% for word in ['apple', 'grapple', 'wroom'] %}
* {{ word }}{% endfor %}

This would produce the following output when we run it through the Jinja renderer:

Here is a markdown list for you:

* apple
* grapple
* wroom

Let's break this example down a bit. As you can see, Jinja's templating language is based on curly brackets {} accompanied by another sign denoting statements, expressions, or even comments.

Statements

Statements are wrapped in curly brackets accompanied by %. The for loop statement in the example above is written as:

{% for word in ['apple', 'grapple', 'wroom'] %}

This is resembles (and is) the Python way of defining a for loop (excluding the colon needed in Python). An if statement follows the same curly percentage format, so an if statement to check if the word contains the letter a, would be written as:

{% if 'a' in word %}

Python uses indents to close of blocks for statements like these. That would raise problems in templates because it wouldn't be clear when you want the template to output white space or when you're just indenting a statement. To get around this problem, Jinja closes blocks with a Jinja-specific statement.

To close off a for loop statement you would use {% endfor %} while closing an if statement requires {% endif %}. So when you find a for loop in a Jinja template you will see where the block ends by looking for the endfor statement.

Closing statements like this is only required for code blocks like loops or conditionals. There are statements that do not require it, such as the set statement which assigns a value to a variable. If we would want to assign the string 'snow' to a variable called weather we'd write that as a statement:

{% set weather = 'snow' %}

We do not close a statement like this with a endset because it isn't a code block. It's just a single line statement.

Expressions

Expressions to print out something like a variable are expressed with double curly brackets {{ }}. In the example we want to print out in each line in our list an asterisk followed by the variable word so we write that as:

* {{ word }}

The output will be an asterisk followed by white space, because Jinja prints out all non-Jinja symbols as is. Thereafter it would print out the variable word (the Jinja expression). If the value of the variable word is apple it would output:

* apple

White space removal

Jinja prints out all symbols as is. That means white space becomes slightly problematic, because white space and line breaks (the invisible \n in our templates) all get printed out too. The example with the apple, grapple, wroom may be difficult to parse as it is. If we would instead write it as:

Here is a markdown list for you:

{% for word in ['apple', 'grapple', 'wroom'] %}
  * {{ word }}
{% endfor %}

We'd end up with an output containing a lot more white space:

Here is a markdown list for you:


  * apple

  * grapple

  * wroom

Jinja gives us a nice way to remove white space. If instead of opening the statement with a {% you open it with a {%- (that is, with an appended minus sign), you remove all white space in front of the statement. If you prepend the minus sign to the closing bracket %} to get -%} you remove all white space after the statement.

So to make our example more readable, we could add a single minus sign when we close our for statement:

Here is a markdown list for you:

{% for word in ['apple', 'grapple', 'wroom'] -%}
* {{ word }}
{% endfor %}

This would give us the output we want, but in a more readable way.

Other nice features

There are plenty of other features in Jinja that I won't go into but I do recommend you read the official Jinja documentation. There are two honorable mentions that I recommend you to check out.

One is filters. Filters are functions you can use to modify variables. Filters are applied using the pipe and can be chained (similar to Unix pipes), i.e. variable|filter|filter.

If we would for example want to print out a random word from our list we could use the built-in random filter:

{% set wordlist = ['apple', 'grapple', 'wroom'] -%}
Word of the day is: {{ wordlist|random }}

This would print out a string with one of the strings randomly chosen, for example:

Word of the day is: apple

I recommend going through the list of built-in filters in Jinja and get to know most of the filters.

Another feature which is nice is tests for variables, mostly because of syntactic sugar that makes the templates easier to read. Tests are done by adding an is behind the variable followed by a function. For example we can combine a lot of what we've discussed into our very own Jinja fizzbuzz generator:

{% for number in range(1, 22) -%}
    {% set modulo3 =  number is divisibleby(3) -%}
    {%- if modulo3 %}fizz{% endif %}

    {%- if number is divisibleby(5) %}buzz
    {%- elif not modulo3 %}{{ number }}{% endif %}
{% endfor -%}

This will output the fizzbuzz counter for a few numbers. In this we use a for loops, if and set statements. We print out variables and we use a built-in test called divisibleby that checks if the variable is divisible by a provided number. There are a few built-in tests in Jinja that can make the code easier to read.

But let's get back to the series. We're here to learn about Salt!

Using Jinja in Salt

In our last post we had defined what packages to install based on roles in our pillar. However, we were still hard coding these packages in our states. We should aim to keep configurations in our pillar and the state logic, not the configuration, in our states.

So to remind ourselves, the pillar has a variable called packages which contains a dictionary of package names as keys, and a description as the value. For example, this was the pillar for our cow host:

cow-e9e01577bbd8:
    ----------
    packages:
        ----------
        cowsay:
            A program with a cow
        fortune:
            A program for a fortune teller

Let's create a new state file and call it roles.sls. This file is going to have the logic we need to install the packages based on the pillar with the help of Jinja.

We still have the same problem as before, where Ubuntu installs in a different location than Alpine so we must create symbolic links. Now you however understand the logic a little bit better. This was the trick we did:

{% if grains['os'] == 'Ubuntu' %}
make cowsay available on ubuntu:
  file.symlink:
    - name: /usr/bin/cowsay
    - target: /usr/games/cowsay
{% endif %}

We used Jinja's if statement to check if the operating system defined in the grains is Ubuntu. If it is, then we create this state. If not, then we don't do anything.

Salt adds grains and the pillar to the global context of Jinja, and makes them available as variables. That's why we're able to write grains['os']. Similarly we can use the Jinja global variable pillar to access the pillar.

Back to the roles.sls. We'll put this Jinja/YAML logic into the file:

{% if 'packages' in pillar %}
install role tools:
  pkg.installed:
    - pkgs: {% for package in pillar['packages'] %}
      - {{ package }}
      {% endfor %}

{% if grains['os'] == 'Ubuntu' %}
{% for package in pillar['packages'] %}
make {{ package }} available on ubuntu:
  file.symlink:
    - name: /usr/bin/{{ package }}
    - target: /usr/games/{{ package }}
{% endfor %}
{% endif %}

{% endif %}

Let's go through this step by step. First we check if the pillar has some packages for us. If it does we define a state to install all role tools. This state calls the installed function in the pkg module and then supplies the list of all packages names by looping through the packages we get from the pillar.

Thereafter we create a separate state with a symbolic link for every package (again accessing the pillar packages, but only if we are on an Ubuntu machine). Now this assumes that all packages we install will be installed in /usr/games on Ubuntu. That may not be the case, but I'll leave it as an exercise for you to use the pillar value and some logic in the state to only do it if we explicitly tell salt to create this symbolic link.

In the next post I'll show you one way to achieve it.

Deploying our package logic

We can now safely drop the rest of the sls files we created for each of the roles. Go ahead and remove cow.sls, fortune.sls, and locomotive.sls from the state directory.

Then we need to update our top.sls state file to use the new roles.sls instead of the hard-coded state files we just deleted. Thankfully this is easy because we define packages in the pillar and only worry about state logic.

We could just apply this to all salt minions, because we do check if there are packages in the pillar before we apply them. However, let's just limit ourselves to those minions that have a role defined in the grains (so we can use this opportunity to remind ourselves about how to target with grains). Our top.sls file will then look like this now:

base:
  'roles:*':
    - match: grain
    - roles

Let's try it out. Again, we should kill all the machines and start with a clean slate. Type in these commands and answer yes when prompted.

docker-compose stop
docker-compose rm
docker-compose up -d
docker-compose exec salt bash

Once we're attached to the salt master, we accept the salt keys for the new minions (press enter to accept them when prompted):

salt-key -A

Then we simply apply the state to all machines:

salt '*' state.apply

You should see all states succeed. That's super.

So now we're at a place where, if we want to define new packages for a new role, we just define it in the pillar with some simple configurations and the rest is automatic.

Now we're starting to see the true potential of using Salt to manage IT infrastructure.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Cover image by Motokoka: Awashima jinja shrine