What is PyGeneses? What is this blog post about? Well if you have no idea what PyGeneses is then I would suggest you to first go through the introductory post. If you know the answer to these two questions, then you are in the right place. Let’s get started.

In this post we will go through installation process and PyGeneses’ training and hyperparameter tuning packages.

Installation

PyGeneses can be installed using pip in either your local system or a cloud based platform. The steps for installation will be the same for both cloud and own system. Let’s install PyGeneses:-

user@programmer:~$ pip install git+https://github.com/Project-DC/pygeneses

It will soon be available in pypi, till then installation will be done from the official GitHub repo for PyGeneses.

Let's start training

pygeneses.envs

Now that we have installed PyGeneses, let’s write some code to train the PrimaVita agents. First, we will look at the most basic code example to train agents with default hyperparameter setting and REINFORCE algorithm (we will be adding more RL algorithms in near future).

Yeah it is just these 3 lines. The three steps to train agents are:-

1) Import PrimaVita (First life) class from envs.prima_vita.

2) Instanstiate the PrimaVita class.

3) Call the run() method on the PrimaVita object.

That was super simple wasn’t it. This default setting will allow you to train the agents in ‘bot’ mode which means you will not be able to see the environment while training, in case you want to see the environment throughout training you just need to change the mode to ‘human’:-

Let us now look at an example where the default hyperparameters are changed:-

You just have to pass a dictionary with keys as the name of hyperparameter (here the initial energy of agents and the time in ticks after which model will be updated are changed) and the values as the value you want to specify. For a more detailed list of available hyperparameters refer to the official docs. Just one more line and you can customize your training.

pygeneses.hypertune

Now let’s move on to hyperparameter tuning package (hypertune) of PyGeneses. PyGeneses has two different algorithms for trying out different hyperparameter values:-

1) Grid Search: This takes in a pool of hyperparameter values as input and trains the model with all the possible combinations of these values. This is useful when you want to try out different values of a hyperparameter or group of hyperparameters.

Following is a snippet of code for applying a grid search over pool of hyperparameters using hypertune:-

This too in 3 lines of code

Again the three steps are as follows:-

1) Import HyperTune class from pygeneses.hypertune.

2) Instantiate HyperTune class with the name of environment class, hyperparameter list, hyperparameter values (2D list) and number of logs to be generated for each run.

3) Finally call hypertuner( ) method of HyperTune class.

The values are to be listed in order in which the hyperparameters are listed.

Moving on to the next algorithm:-

2) Randomized Search: This is similar to grid search in that it too takes a pool of hyperparameter values but instead of training model with all possible combinations it selects a subset out of those combinations randomly based on a probability.

Here is the code example:-

The only change in this code and the grid search code is the extra parameter randomize_percent which tells the randomized search algorithm the percentage of combinations to pick up randomly from the pool of available hyperparameters.

This was the end to this part of PyGeneses blog post series. In the next part we will talk about VitaBoard — an interactive dashboard for visualization of training results in PyGeneses which helps makes sense of the actions taken by agents during training and studying their behaviour. See you there :)