Inter-language communication between Erlang (rebar3) and Python using Erlport

When it comes to fault-tolerant systems, there are very few languages that can beat Erlang. While Erlang is a great language for building fault-tolerant systems, it is not the best language for building AI/ML applications. There are instances where you would want to use Python for building AI/ML applications and Erlang for building fault-tolerant systems. In such cases, you would want to use Erlang and Python together. This is where Erlport library comes in.

But before understanding Erlport, you need an understanding of ports in Erlang.

Ports

In simple terms, ports are used to communicate with external programs which are not written in Erlang. So if you have a C program or a Python program and you want to communicate with it from Erlang, you can use ports. It's not a restriction that you can only use ports to communicate with external programs, you can also use ports to communicate with other Erlang nodes.Port provides you with a byte-oriented interface. Which means, you send a list of bytes and receive a list of bytes during the communication. This also means you need to handle the encoding and decoding of the data at both ends by yourself.

Erlport

Erlport is a library that internally uses the port mechanism for communication and provides a wrapper around it. This makes the integration of Erlang and Python very easy. Right now, Erlport supports Python and Ruby. In this article, we will be focusing on Erlang and Python implementation.

Installation

If you are using rebar3 for building your Erlang application, you can add the following dependency to your rebar.config file.

...
{deps, [
    erlport
]}.
...

Or you can use the GitHub link for the dependency.

{erl_opts, [debug_info]}.
{deps, [
  {erlport, {git, "https://github.com/erlport/erlport.git", {tag, "v0.10.1"}}}
]}.

You'll also need to tell Erlang to wait for the dependencies to be available before starting the app. This can be done by adding the entry in *.app.src file.

 {applications,
   [kernel,
    stdlib,
    erlport
   ]},

Python program

Let's create an add program in Python and keep the file in the priv directory.

def add(a, b):
    return a + b

Erlang program

The Erlang program will initialize the Python port using python:start/1 function. And will call the add function of the Python program using python:call/4 function. The python:call/4 function takes the Pid of the Python port, the module name, the function name, and the list of arguments to be passed to the function.

Here, my Python module is named, math and the function is add with 2 and 4 as parameters.

-module(main).
-export([call_python/0]).

call_python() ->
  PythonCodePath = code:priv_dir(aiml_model_wrapper),
  {ok, P} = python:start([{python_path, PythonCodePath}, {python, "python3"}]),
  python:call(P, math, add, [2, 4]).

Running the program

Running the program is as simple as compiling and running the Erlang program.

reabr3 compile
reabr3 shell
1> main:call_python().

Conclusion

In this article, we saw how we can use Erlang and Python together using Erlport. This is a very simple example of how you can use Erlang and Python together. You can use this to build complex systems where you can use Erlang for building fault-tolerant systems and Python for building AI/ML applications. At GreyOrange I created an Erlang wrapper for our AI/ML models using the same approach.