A Makefile for Rasa

As a programmer, I can be lazy and have a list of aliases I use when working with everything on the command line, Rasa included.

At Nearly Human we use make for a lot of different things, and I decided to utilize the muscle memory for typing make all the time for my Rasa development.

If you're curious, currently I work on Windows 10, and I do the bulk of my Rasa development using VSCode and use the integrated WSL Ubuntu terminal. I'm working on another tutorial about setting up VSCode for Rasa development, but haven't finished that one yet.

Installing make

From an WSL Ubuntu terminal install make

sudo apt-get update -y
sudo apt-get install -y make

When that's completed make sure it's all happy with

make --version

You should get something like this - possibly with a different version. The important thing here is that you don't get an error.

GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

We also use commit hooks with pre-commit to keep some formatting standards. Lets get that installed too.

This is really easy as well. It's just

pip install pre-commit

We'll need a configuration for this, so create a .pre-commit-config.yml file in the root of your project.

You can copy / paste in this generic configuration

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files

To test this lets check all your files manually first, then we'll create a make recipe for it.

Run this

pre-commit run --files *.yml data/*

IF all goes well you'll see something like this

Trim Trailing Whitespace.......................................Passed
Fix End of Files...............................................Passed
Check Yaml.....................................................Passed
Check for added large files....................................Passed

If you get a Failed, read why, it should tell you what happened. What's great about pre-commit is that it will fix most common formatting issue on it's own. So run that again if you saw a Failed line and chances are this time they'll all pass... unless of course you really botched a file badly.

I'll presume you have that all working now.

The Makefile

Now to create the fun part. In the root of your Rasa project, create a new file named Makefile <-- there's no extension, it's all one word with a lowercase 'f'

You can read online all about make and how to construct recipes, but these are the ones I use. Essentially you have a recipe name, and a set commands you want to execute, one per line

Clean

This removes past model files. If you're like me you train often and this directory can get pretty large depending on the size of your model file.

In your Makefile add this super easy recipe

clean:
    rm models/*.gz -f

Save that, and from the terminal run make clean and it'll execute and remove your model files. The -f forces the recipe to run and not fail if the command fails. So for example if there are no models to delete, that command technically fails and without the -f flag, the recipe would fail as well.

Validate

This is a crazy valuable recipe that runs the validate command. Mostly because I have a very complex data directory (I do NOT have a domain.yml file in the root of my project, so I have to specify a parameter)

Add a new line and add this recipe to your Makefile (** see the note below)

validate:
    rasa data validate --domain data

** You may need to remove the --domain data to get this to work depending on your setup.

To execute this one you can run make validate

Training

The next most important recipe is the training.
This recipe is fantastic. It'll run the clean recipe removing the model files, then check your formatting, then validate, then train.

Add this recipe to your Makefile

train: clean
    pre-commit run --files *.yml data/*
    rasa data validate --domain data
    rasa train --domain data --out models

You'll note that clean is on the same line as the recipe name, this is on purpose, it'll allow you to run another recipe from this one.
As you would expect, run make train to execute this time-saver.

Testing

Now that we have a model trained you can test it. This command typically has many parameters so it's very handy to use a recipe for.

Add this to your Makefile

test: check
    rasa test --stories tests/conversation_tests.yml --fail-on-prediction-errors

run make test for this one.

This runs a check using my custom conversational_tests.yml file. You can change these parameters to whatever you use to run your tests.

Almost done

I have two more for you. I like to run the rasa server api locally (not rasa shell) and that has many parameters. Also in conjunction with that, I like to fire up the action server in another terminal window so here's two more recipes that are useful

Action Server

runactions:
    rasa run actions --action actions --debug --auto-reload

Fire up the action server in debug mode with auto-reload enabled with make runactions There are circumstances where the auto-reload flag doesn't actually reload your changes, so just be aware of that, but it's not related to the Makefile

Rasa Server (not shell)

run:
    rasa run -m models --enable-api --log-file out.log --cors "*" --debug

To launch this recipe type make run the rasa server will fire up with the api active, create a log file for you, enable cors for external communications and do it all in debug mode (very useful). It will sit there waiting for connections from your chat interface, whether that is a web ui, slack, messenger, whatever.

That's It

Hopefully that'll save you some keystrokes and you can become a lazy developer like me :)

If you have any recipes or commands you'd like to share, do that in the comments and I just may steal ... er use them in my workflow.