DEV Community

Cover image for Inter-process communication: pipes
Leandro Proença
Leandro Proença

Posted on

Inter-process communication: pipes

After learning how OS processes can use file descriptors for IPC, it's time to analyse another IPC approach: pipes.


Let's recap the following example:

$ echo 'my precious' > rawcontent.txt
$ base64 < rawcontent.txt
Enter fullscreen mode Exit fullscreen mode
  • the program echo sends data to the redirected STDOUT
  • the echo output is used as the redirected STDIN for the program base64

Note the pattern here: it looks like a pipeline of data transformation, where the output of the first program is "enqueued" to the input of the second program.

UNIX pipelines

UNIX-like systems provide a mechanism for IPC called pipeline.

Instead of writing such a sentence in multiple lines, the OS allows us to write the entire sentence in a single line using the operator | between programs.

$ echo 'my precious' | base64

bXkgcHJlY2lvdXMK
Enter fullscreen mode Exit fullscreen mode

You've seen this pipe stuff elsewhere, am I right?

It's called anonymous pipe.

Anonymous Pipe

Anonymous pipes employ a FIFO (first-in, first-out) communication channel for one-way IPC.

By one-way, it means the data flows in one-direction only. It's the opposite of bi-direction communication, where data flows in both directions in a full-duplex way.

$ ps ax | grep docker | tail -n 3

62374 s039  S+     0:05.31 /usr/local/bin/com.docker.cli run -it ubuntu bash
65442 s040  S+     0:02.93 docker run -it ubuntu bash
65445 s040  S+     0:02.86 /usr/local/bin/com.docker.cli run -it ubuntu bash
Enter fullscreen mode Exit fullscreen mode

When a pipe (|) is created, it's opened a pair of file descriptors, one for writing and other for reading.

Because pipes are anonymous, both file descriptors last only as long as the processes, so they are automatically closed when both processes terminate.

Named pipes

Similar to anonymous pipes, named pipes also employ FIFO (first-in, first-out) communication channel for one-way IPC.

The main difference is that named pipes are created explicitly using the program mkfifo <filename>. A single file in the filesystem is then created, which will be opened for reading by one process and for writing by another process.

$ mkfifo myqueue
Enter fullscreen mode Exit fullscreen mode

A file called myqueue is created. The reader process can then open the pipe:

$ cat myqueue
Enter fullscreen mode Exit fullscreen mode

The process keeps blocked until a message arrives in the pipe.

Meanwhile, another process can write to the pipe:

# The echo message STDOUT is being redirected to the pipe, because it's a file!
$ echo 'some message' > myqueue
Enter fullscreen mode Exit fullscreen mode

Now look at the message arriving in the reader process:

$ cat myqueue

some message
Enter fullscreen mode Exit fullscreen mode

Be aware that named pipes live beyond the processes they are bound to, hence they should be removed manually when become unused:

rm myqueue
Enter fullscreen mode Exit fullscreen mode

Yay!


Conclusion

In this article we learned how UNIX-like systems use pipes for IPC.

Primarily, pipes are a one-way communication channel for IPC.

Anonymous pipes use a pair of file descriptors and are closed automatically when the process is finished.

Named pipes use a named file in the filesystem and should be closed manually when they are no longer used.

I hope you could understand a bit more about IPC. In the next post I'll cover UNIX sockets for inter-process communication.

Top comments (2)

Collapse
 
svenskanda profile image
Alexander Bombis

Hej - that’s very helpful - thanks for your type to give us the knowledge about Files and Pipes❣️

Do you know, if you write the article about sockets?

Collapse
 
leandronsp profile image
Leandro Proença

Hello, I'm planning to write about sockets soon, I'll let you know, thanks!