If I try to summarize PyGeneses in a single line then it would be something like — “PyGeneses is a PyTorch based Deep Reinforcement Learning (Deep RL) framework that helps users to simulate artificial agents in bio-inspired environments”.

Let’s break down the above line:-

• PyTorch: PyTorch is a machine learning framework developed by Facebook AI Research (FAIR). It has an intuitive python like syntax which allows faster prototyping and serving of ML/DL models. We use PyTorch for the deep reinforcement learning algorithms that we provide as part of our framework.

• Deep Reinforcement Learning: Deep Reinforcement Learning is a technique in Artificial Intelligence that brings together the best of Deep Learning and Reinforcement Learning. Deep learning deals with the concept of neural networks which are capable of learning complex functions from data, and Reinforcement Learning is based on the idea of learning from experience.

• Artificial Agents: Artificial agents are simulated and simplified versions of species found in nature. The rules these agents follow closely mimic that of nature which allows them to display behaviour close to what animals around us (or even human beings) display. Note: These agents are nowhere close to the complexity of living beings in our planet and are just a simplified version of them.

• Bio-inspired environments: Bio-inspired environments mimic conditions of survival and rules that we see in our natural environment in simulation.

About the framework

PyGeneses is a collection of four different packages:-

pygeneses.envs

This module allows you to create, configure and tweak the in-built bio-inspired environments. As of now, this only provides a single environment called Prima Vita (First Life), but there’s more coming soon! This lets you set up the entire environment and the species in just a few lines of code and provides both high level API and low level control over the environment. Training using the API includes logging of every action of an agent so that it can be studied using VitaBoard.

pygeneses.models
The ‘models’ module is what allows us to import the neural networks which the species uses to learn what to do. As of now, only the default model’s (REINFORCE) implementation is provided, but we will be adding support for custom pluggable networks from v0.2 onwards.

pygeneses.hypertune
The ‘HyperTune’ package allows us to configure and test out various hyperparameters we can provide for an environment and species (a list of hyperparameters is provided in the Classes section of this documentation). This contains single hyperparameter testing, grid search and randomized search. This allows us to find the best set of hyperparameters to display a type of behavior. This also produces logs which we can study using Vitaboard.

pygeneses.vitaboard
Vitaboard provides an advanced, interactive dashboard to study agents after the training phase. After each agent dies, his/her actions are written into a log file. And vitaboard allows us to visualize the agent’s life. It provides us with a life visualizer, group statistics and a genetic history visualizer. It allows us to identify and understand behaviours exhibited by an agent while interacting with the environment or with other agents in the environment.

Use cases

Pygeneses can be used for a variety of purposes, the limit only being your creativity and imagination. That being said, some of the use-cases of pygeneses can be:

1) To understand basic psychology — Since every agent has a limited actions to choose from in each time step, the simulation can be studied to infer the basic psychological patterns displayed, this can help to understand the similarities/dissimilarities between the agents and real life organisms.

2) To create applications/games based on the framework — You can create basic applications/games based on our framework. An example can be a productivity app that adds food to the simulation only when the user completes a task, hence staking the lives of the agents of the user’s own simulation in exchange for productivity.

3) To learn more about Deep Reinforcement Learning — Users can just tinker around with different hyperparameters and models and observe the results visually, this makes for a interesting way to get an insight into the world of Deep Reinforcement Learning.

Target Audience

Due to the simple high level API of PyGeneses I believe that this tool can be helpful for both programmer and non-programmers who want to understand the nature of behaviours in animals or understand concepts of Deep Reinforcement Learning.

Most of our API requires only 3–4 lines of code to modify the environment according to your needs, thus allowing users with little to no knowledge of ML or Deep RL to use our tool easily.

We (the developers of PyGeneses) believe that understanding the reason of display of certain behaviours is not only an active area of study but also a fun experiment and through our framework we want to provide a platform for carrying out both of these tasks.

Final words

If you are someone who is keen to learn about ML and AI, or a scientist who studies behaviour or even someone who just wants to have some fun experimenting with these artificial agents then get your local installation of PyGeneses today.

For a detailed introduction (installation and code examples) follow the subsequent articles on this:-

1) Getting started with PyGeneses

2) Studying logs with VitaBoard

If you want to contribute to this project or simply want to know more, then follow the official docs.