This post gives a brief introduction to developing peer-to-peer applications using libp2p using the Go programming language.
Table of Contents
- Introduction
- Coding the Node
- Sending and Receiving Data
- Finding Additional Peers
- Complete Code
- Conclusion
Introduction
This section explains the concepts we are going to see in this post.
What is libp2p?
From the libp2p docs:
libp2p is a modular system of protocols, specifications and libraries that enable the development of peer-to-peer network applications.
What are peer-to-peer network applications?
A pure peer-to-peer network application is one in which
the machines connected to it act like both as clients and servers, thus sharing their own hardware resources to make the network function.
Instead of clients and servers, machines connected peer-to-peer networks are usually called "nodes".
Coding the Node
Creating libp2p hosts
The code below simply creates a new libp2p host with default options.
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"github.com/libp2p/go-libp2p"
)
func main() {
ctx := context.Background()
host, err := libp2p.New(ctx)
if err != nil {
panic(err)
}
defer host.Close()
fmt.Println(host.Addrs())
sigCh := make(chan os.Signal)
signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
<-sigCh
}
Running the code I got the following output:
[/ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/43937 /ip6/::1/tcp/43937 /ip4/192.168.1.68/tcp/45559 /ip4/127.0.0.1/tcp/45559]
We are able to see that libp2p automatically chose IPv4 and IPv6 addresses on all interfaces for the host to listen for connections. By doing this, our node can now act as a server for others to connect to.
If those address strings look odd, don't worry. We'll dive deeper into node addressing in the next section as we'll need it to connect nodes.
Connecting to the node (from another node)
Before we can connect to the node from the previous section, let's see how node addressing works in libp2p. We'll explore 2 concepts required to connect to a libp2p node: multiaddr
and node IDs.
Multiaddress
libp2p does a lot to work on top of different network transports (i.e. the technology used to send and receive bits on the wire). That requires a flexible addressing scheme.
The address we saw in the output of the node execution are encoded using multiaddr
(see the spec). multiaddr
allows the encoding of many protocols on top of each other along with their addressing information.
Let's dissect the output of the previous section's node execution:
/ip4/127.0.0.1/tcp/45559
There are two protocols encoded in this multiaddr
string: /ip4/127.0.0.1
which tells us to use the 127.0.0.1
address of the IPv4 protocol and /tcp/45559
which is telling us to layer (on top of IP) the TCP protocol on port 45559.
Node ID
libp2p defines the /p2p
protocol and the addressing part of its multiaddr
string is the ID of the node we want to connect to. That means the address of a node would be something like:
/ip4/127.0.0.1/tcp/3000/p2p/NODE_ID
Where NODE_ID
is the node's ID.
Nodes need to generate a cryptographic key pair in order to secure the connections with other nodes (or peers).
The node's ID is simply a multihash of its public key.
That way (besides identifying different nodes) IDs are unique, can be made permanent and provide a way for other nodes to verify the public key sent by another node.
Connecting the nodes
With all that said, we can get back to writing the code to connect two nodes.
First we'll print the host's addresses and ID:
fmt.Println("Addresses:", host.Addrs())
fmt.Println("ID:", host.ID())
Starting the node again we get:
Addresses: [/ip4/192.168.1.68/tcp/44511 /ip4/127.0.0.1/tcp/44511 /ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/46471 /ip6/::1/tcp/46471]
ID: Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
So the p2p address string for this node would be (I'll be using the IPv4 address):
/ip4/127.0.0.1/tcp/44511/p2p/Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
In order to connect to other nodes we can extend our code to accept a peer address as argument and some connection logic:
package main
import (
"context"
"flag"
"fmt"
"os"
"os/signal"
"syscall"
"github.com/libp2p/go-libp2p"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/multiformats/go-multiaddr"
)
func main() {
// Add -peer-address flag
peerAddr := flag.String("peer-address", "", "peer address")
flag.Parse()
// Create the libp2p host.
//
// Note that we are explicitly passing the listen address and restricting it to IPv4 over the
// loopback interface (127.0.0.1).
//
// Setting the TCP port as 0 makes libp2p choose an available port for us.
// You could, of course, specify one if you like.
host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
if err != nil {
panic(err)
}
defer host.Close()
// Print this node's addresses and ID
fmt.Println("Addresses:", host.Addrs())
fmt.Println("ID:", host.ID())
// If we received a peer address, we should connect to it.
if *peerAddr != "" {
// Parse the multiaddr string.
peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
if err != nil {
panic(err)
}
peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
if err != nil {
panic(err)
}
// Connect to the node at the given address.
if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
panic(err)
}
fmt.Println("Connected to", peerAddrInfo.String())
}
sigCh := make(chan os.Signal)
signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
<-sigCh
}
Sending and Receiving Data
When we want to send and receive data directly from other peers, we can use a libp2p stream.
Let's make the nodes start a counter for each new connection (inbound and outbound) and send it through a stream every second. At the same time, nodes will keep reading the counters sent on that same stream.
First, we create a function to write data to the stream:
func writeCounter(s network.Stream) {
var counter uint64
for {
<-time.After(time.Second)
counter++
err := binary.Write(s, binary.BigEndian, counter)
if err != nil {
panic(err)
}
}
}
After that, we create a function to read data from the stream:
func readCounter(s network.Stream) {
for {
var counter uint64
err := binary.Read(s, binary.BigEndian, &counter)
if err != nil {
panic(err)
}
fmt.Printf("Received %d from %s\n", counter, s.ID())
}
}
Then we modify the code to do 2 additional things:
- Setup a stream handler using the
SetStreamHandler
function (the handler function is called every time a peer opens a stream) - Create a new stream using the
NewStream
function after connecting to a peer
After we create the host instance we can set up the stream handler function with the following code:
// This gets called every time a peer connects
// and opens a stream to this node.
host.SetStreamHandler(protocolID, func(s network.Stream) {
go writeCounter(s)
go readCounter(s)
})
After we connect to a peer we can open a new stream by doing:
s, err := host.NewStream(
context.Background(),
peerAddrInfo.ID,
protocolID,
)
if err != nil {
panic(err)
}
go writeCounter(s)
go readCounter(s)
Finding Additional Peers
Peer-to-peer networks do not require a central server for machines to make a connection. All that is required is the address of one of the nodes in the network.
But what happens if that node goes offline? We lose our connection.
To prevent that from happening, we want to find and remember the address of additional peers in the network.
Each node in the network will maintain a list of peers they know. Each node will also announce to the peers they know their own addresses in order to be found by others.
As the last step in this post, let's implement peer discovery.
First, we need a new type to define a method to be called when the discovery service finds peers.
type discoveryNotifee struct{}
func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
fmt.Println("found peer", peerInfo.String())
}
HandlePeerFound
will be called by the discovery service every time a peer is found (even if we already knew it).
Next, we create an instance of the discovery service. In this example we are using the mDNS protocol which tries to find peers in the local network.
discoveryService, err := discovery.NewMdnsService(
context.Background(),
host,
time.Second,
discoveryNamespace,
)
if err != nil {
panic(err)
}
defer discoveryService.Close()
discoveryService.RegisterNotifee(&discoveryNotifee{})
After we add this piece of code we should be able to start nodes that can connect to other nodes directly and start sending them the counter values. The node will also be periodically searching for peers in the local network and printing their IDs and address.
Complete code
This is the complete final code we developed in this post:
package main
import (
"context"
"encoding/binary"
"flag"
"fmt"
"os"
"os/signal"
"syscall"
"time"
"github.com/libp2p/go-libp2p"
"github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/libp2p/go-libp2p/p2p/discovery"
"github.com/multiformats/go-multiaddr"
)
const protocolID = "/example/1.0.0"
const discoveryNamespace = "example"
func main() {
// Add -peer-address flag
peerAddr := flag.String("peer-address", "", "peer address")
flag.Parse()
// Create the libp2p host.
//
// Note that we are explicitly passing the listen address and restricting it to IPv4 over the
// loopback interface (127.0.0.1).
//
// Setting the TCP port as 0 makes libp2p choose an available port for us.
// You could, of course, specify one if you like.
host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
if err != nil {
panic(err)
}
defer host.Close()
// Print this node's addresses and ID
fmt.Println("Addresses:", host.Addrs())
fmt.Println("ID:", host.ID())
// Setup a stream handler.
//
// This gets called every time a peer connects and opens a stream to this node.
host.SetStreamHandler(protocolID, func(s network.Stream) {
go writeCounter(s)
go readCounter(s)
})
// Setup peer discovery.
discoveryService, err := discovery.NewMdnsService(
context.Background(),
host,
time.Second,
discoveryNamespace,
)
if err != nil {
panic(err)
}
defer discoveryService.Close()
discoveryService.RegisterNotifee(&discoveryNotifee{h: host})
// If we received a peer address, we should connect to it.
if *peerAddr != "" {
// Parse the multiaddr string.
peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
if err != nil {
panic(err)
}
peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
if err != nil {
panic(err)
}
// Connect to the node at the given address.
if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
panic(err)
}
fmt.Println("Connected to", peerAddrInfo.String())
// Open a stream with the given peer.
s, err := host.NewStream(context.Background(), peerAddrInfo.ID, protocolID)
if err != nil {
panic(err)
}
// Start the write and read threads.
go writeCounter(s)
go readCounter(s)
}
sigCh := make(chan os.Signal)
signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
<-sigCh
}
func writeCounter(s network.Stream) {
var counter uint64
for {
<-time.After(time.Second)
counter++
err := binary.Write(s, binary.BigEndian, counter)
if err != nil {
panic(err)
}
}
}
func readCounter(s network.Stream) {
for {
var counter uint64
err := binary.Read(s, binary.BigEndian, &counter)
if err != nil {
panic(err)
}
fmt.Printf("Received %d from %s\n", counter, s.ID())
}
}
type discoveryNotifee struct {
h host.Host
}
func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
fmt.Println("found peer", peerInfo.String())
}
Conclusion
We developed a basic example to show various features of peer-to-peer networking (direct connections, data streams and peer discovery). I believe these can give you a taste of the types of applications that can be built using this kind of architecture (well-known examples include BitTorrent and the InterPlanetary File System).
And finally, I hope you liked the content and that this gets you a bit more interested in getting started in the peer-to-peer networks world.
Top comments (7)
Hey Felipe,
Thanks for writing this article. This is really helpful.
When I am executing this code, I found few issues.
1) Firstly when I imported go-libp2p I got some issues likes packages not found but I solved it by upgrading go version. I recommend others to use go latest version to use the above libraries.
2)
github.com/libp2p/go-libp2p-core/peer
this package is deprecated. Please use the following package.github.com/libp2p/go-libp2p/core/peer
Hope this helps someone.
Hey Vineeth,
Thanks for this, since the post was written a while ago.
I'll try to find some time to update it with your suggestions.
I enjoed that one, good job Felipe Rosa!
Hey thanks for the writeup. I'm planning to delve into Libp2p. I'm new to Golang as well as Rust. My plan is to learn Go / Rust as I start learning Libp2p. Which do you think I should go with Go or Rust?
Hey!
I would suggest the one you are the most familiar with (in this case it seems like it is Go).
Very interesting. Thanks
Thanks! Glad you liked it :)