DEV Community

Cover image for Getting started with libp2p in Go
Felipe Rosa
Felipe Rosa

Posted on

Getting started with libp2p in Go

#go #distributedsystems #p2p #libp2p

This post gives a brief introduction to developing peer-to-peer applications using libp2p using the Go programming language.

Table of Contents

Introduction

This section explains the concepts we are going to see in this post.

What is libp2p?

From the libp2p docs:

libp2p is a modular system of protocols, specifications and libraries that enable the development of peer-to-peer network applications.

What are peer-to-peer network applications?

A pure peer-to-peer network application is one in which

the machines connected to it act like both as clients and servers, thus sharing their own hardware resources to make the network function.

Instead of clients and servers, machines connected peer-to-peer networks are usually called "nodes".

Coding the Node

Creating libp2p hosts

The code below simply creates a new libp2p host with default options.

package main

import (
    "context"
    "fmt"
    "os"
    "os/signal"
    "syscall"

    "github.com/libp2p/go-libp2p"
)

func main() {
    ctx := context.Background()

    host, err := libp2p.New(ctx)
    if err != nil {
        panic(err)
    }
    defer host.Close()

    fmt.Println(host.Addrs())

    sigCh := make(chan os.Signal)
    signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
    <-sigCh
}
Enter fullscreen mode Exit fullscreen mode

Running the code I got the following output:

[/ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/43937 /ip6/::1/tcp/43937 /ip4/192.168.1.68/tcp/45559 /ip4/127.0.0.1/tcp/45559]
Enter fullscreen mode Exit fullscreen mode

We are able to see that libp2p automatically chose IPv4 and IPv6 addresses on all interfaces for the host to listen for connections. By doing this, our node can now act as a server for others to connect to.

If those address strings look odd, don't worry. We'll dive deeper into node addressing in the next section as we'll need it to connect nodes.

Connecting to the node (from another node)

Before we can connect to the node from the previous section, let's see how node addressing works in libp2p. We'll explore 2 concepts required to connect to a libp2p node: multiaddr and node IDs.

Multiaddress

libp2p does a lot to work on top of different network transports (i.e. the technology used to send and receive bits on the wire). That requires a flexible addressing scheme.

The address we saw in the output of the node execution are encoded using multiaddr (see the spec). multiaddr allows the encoding of many protocols on top of each other along with their addressing information.

Let's dissect the output of the previous section's node execution:

/ip4/127.0.0.1/tcp/45559
Enter fullscreen mode Exit fullscreen mode

There are two protocols encoded in this multiaddr string: /ip4/127.0.0.1 which tells us to use the 127.0.0.1 address of the IPv4 protocol and /tcp/45559 which is telling us to layer (on top of IP) the TCP protocol on port 45559.

Node ID

libp2p defines the /p2p protocol and the addressing part of its multiaddr string is the ID of the node we want to connect to. That means the address of a node would be something like:

/ip4/127.0.0.1/tcp/3000/p2p/NODE_ID
Enter fullscreen mode Exit fullscreen mode

Where NODE_ID is the node's ID.

Nodes need to generate a cryptographic key pair in order to secure the connections with other nodes (or peers).

The node's ID is simply a multihash of its public key.

That way (besides identifying different nodes) IDs are unique, can be made permanent and provide a way for other nodes to verify the public key sent by another node.

Connecting the nodes

With all that said, we can get back to writing the code to connect two nodes.

First we'll print the host's addresses and ID:

fmt.Println("Addresses:", host.Addrs())
fmt.Println("ID:", host.ID())
Enter fullscreen mode Exit fullscreen mode

Starting the node again we get:

Addresses: [/ip4/192.168.1.68/tcp/44511 /ip4/127.0.0.1/tcp/44511 /ip6/2804:d45:3613:5400:4b34:ed8f:df00:5055/tcp/46471 /ip6/::1/tcp/46471]
ID: Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
Enter fullscreen mode Exit fullscreen mode

So the p2p address string for this node would be (I'll be using the IPv4 address):

/ip4/127.0.0.1/tcp/44511/p2p/Qmdfuscj69bwzza5nyC1RCMRkV1aoYjQq2nvDYqUYG8Zoq
Enter fullscreen mode Exit fullscreen mode

In order to connect to other nodes we can extend our code to accept a peer address as argument and some connection logic:

package main

import (
    "context"
    "flag"
    "fmt"
    "os"
    "os/signal"
    "syscall"

    "github.com/libp2p/go-libp2p"
    "github.com/libp2p/go-libp2p-core/peer"
    "github.com/multiformats/go-multiaddr"
)

func main() {
    // Add -peer-address flag
    peerAddr := flag.String("peer-address", "", "peer address")
    flag.Parse()

    // Create the libp2p host.
    //
    // Note that we are explicitly passing the listen address and restricting it to IPv4 over the
    // loopback interface (127.0.0.1).
    //
    // Setting the TCP port as 0 makes libp2p choose an available port for us.
    // You could, of course, specify one if you like.
    host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
    if err != nil {
        panic(err)
    }
    defer host.Close()

    // Print this node's addresses and ID
    fmt.Println("Addresses:", host.Addrs())
    fmt.Println("ID:", host.ID())

    // If we received a peer address, we should connect to it.
    if *peerAddr != "" {
        // Parse the multiaddr string.
        peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
        if err != nil {
            panic(err)
        }
        peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
        if err != nil {
            panic(err)
        }

        // Connect to the node at the given address.
        if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
            panic(err)
        }
        fmt.Println("Connected to", peerAddrInfo.String())
    }

    sigCh := make(chan os.Signal)
    signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
    <-sigCh
}
Enter fullscreen mode Exit fullscreen mode

Sending and Receiving Data

When we want to send and receive data directly from other peers, we can use a libp2p stream.

Let's make the nodes start a counter for each new connection (inbound and outbound) and send it through a stream every second. At the same time, nodes will keep reading the counters sent on that same stream.

First, we create a function to write data to the stream:

func writeCounter(s network.Stream) {
    var counter uint64

    for {
        <-time.After(time.Second)
        counter++

        err := binary.Write(s, binary.BigEndian, counter)
        if err != nil {
            panic(err)
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

After that, we create a function to read data from the stream:

func readCounter(s network.Stream) {
    for {
        var counter uint64

        err := binary.Read(s, binary.BigEndian, &counter)
        if err != nil {
            panic(err)
        }

        fmt.Printf("Received %d from %s\n", counter, s.ID())
    }
}
Enter fullscreen mode Exit fullscreen mode

Then we modify the code to do 2 additional things:

  • Setup a stream handler using the SetStreamHandler function (the handler function is called every time a peer opens a stream)
  • Create a new stream using the NewStream function after connecting to a peer

After we create the host instance we can set up the stream handler function with the following code:

// This gets called every time a peer connects 
// and opens a stream to this node.
host.SetStreamHandler(protocolID, func(s network.Stream) {
    go writeCounter(s)
    go readCounter(s)
})
Enter fullscreen mode Exit fullscreen mode

After we connect to a peer we can open a new stream by doing:

s, err := host.NewStream(
    context.Background(), 
    peerAddrInfo.ID, 
    protocolID,
)
if err != nil {
    panic(err)
}

go writeCounter(s)
go readCounter(s)
Enter fullscreen mode Exit fullscreen mode

Finding Additional Peers

Peer-to-peer networks do not require a central server for machines to make a connection. All that is required is the address of one of the nodes in the network.

But what happens if that node goes offline? We lose our connection.

To prevent that from happening, we want to find and remember the address of additional peers in the network.

Each node in the network will maintain a list of peers they know. Each node will also announce to the peers they know their own addresses in order to be found by others.

As the last step in this post, let's implement peer discovery.

First, we need a new type to define a method to be called when the discovery service finds peers.

type discoveryNotifee struct{}

func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
    fmt.Println("found peer", peerInfo.String())
}
Enter fullscreen mode Exit fullscreen mode

HandlePeerFound will be called by the discovery service every time a peer is found (even if we already knew it).

Next, we create an instance of the discovery service. In this example we are using the mDNS protocol which tries to find peers in the local network.

discoveryService, err := discovery.NewMdnsService(
    context.Background(),
    host,
    time.Second,
    discoveryNamespace,
    )
if err != nil {
    panic(err)
}
defer discoveryService.Close()

discoveryService.RegisterNotifee(&discoveryNotifee{})
Enter fullscreen mode Exit fullscreen mode

After we add this piece of code we should be able to start nodes that can connect to other nodes directly and start sending them the counter values. The node will also be periodically searching for peers in the local network and printing their IDs and address.

Complete code

This is the complete final code we developed in this post:

package main

import (
    "context"
    "encoding/binary"
    "flag"
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "time"

    "github.com/libp2p/go-libp2p"
    "github.com/libp2p/go-libp2p-core/host"
    "github.com/libp2p/go-libp2p-core/network"
    "github.com/libp2p/go-libp2p-core/peer"
    "github.com/libp2p/go-libp2p/p2p/discovery"
    "github.com/multiformats/go-multiaddr"
)

const protocolID = "/example/1.0.0"
const discoveryNamespace = "example"

func main() {
    // Add -peer-address flag
    peerAddr := flag.String("peer-address", "", "peer address")
    flag.Parse()

    // Create the libp2p host.
    //
    // Note that we are explicitly passing the listen address and restricting it to IPv4 over the
    // loopback interface (127.0.0.1).
    //
    // Setting the TCP port as 0 makes libp2p choose an available port for us.
    // You could, of course, specify one if you like.
    host, err := libp2p.New(context.Background(), libp2p.ListenAddrStrings("/ip4/127.0.0.1/tcp/0"))
    if err != nil {
        panic(err)
    }
    defer host.Close()

    // Print this node's addresses and ID
    fmt.Println("Addresses:", host.Addrs())
    fmt.Println("ID:", host.ID())

    // Setup a stream handler.
    //
    // This gets called every time a peer connects and opens a stream to this node.
    host.SetStreamHandler(protocolID, func(s network.Stream) {
        go writeCounter(s)
        go readCounter(s)
    })

    // Setup peer discovery.
    discoveryService, err := discovery.NewMdnsService(
        context.Background(),
        host,
        time.Second,
        discoveryNamespace,
    )
    if err != nil {
        panic(err)
    }
    defer discoveryService.Close()

    discoveryService.RegisterNotifee(&discoveryNotifee{h: host})

    // If we received a peer address, we should connect to it.
    if *peerAddr != "" {
        // Parse the multiaddr string.
        peerMA, err := multiaddr.NewMultiaddr(*peerAddr)
        if err != nil {
            panic(err)
        }
        peerAddrInfo, err := peer.AddrInfoFromP2pAddr(peerMA)
        if err != nil {
            panic(err)
        }

        // Connect to the node at the given address.
        if err := host.Connect(context.Background(), *peerAddrInfo); err != nil {
            panic(err)
        }
        fmt.Println("Connected to", peerAddrInfo.String())

        // Open a stream with the given peer.
        s, err := host.NewStream(context.Background(), peerAddrInfo.ID, protocolID)
        if err != nil {
            panic(err)
        }

        // Start the write and read threads.
        go writeCounter(s)
        go readCounter(s)
    }

    sigCh := make(chan os.Signal)
    signal.Notify(sigCh, syscall.SIGKILL, syscall.SIGINT)
    <-sigCh
}

func writeCounter(s network.Stream) {
    var counter uint64

    for {
        <-time.After(time.Second)
        counter++

        err := binary.Write(s, binary.BigEndian, counter)
        if err != nil {
            panic(err)
        }
    }
}

func readCounter(s network.Stream) {
    for {
        var counter uint64

        err := binary.Read(s, binary.BigEndian, &counter)
        if err != nil {
            panic(err)
        }

        fmt.Printf("Received %d from %s\n", counter, s.ID())
    }
}

type discoveryNotifee struct {
    h host.Host
}

func (n *discoveryNotifee) HandlePeerFound(peerInfo peer.AddrInfo) {
    fmt.Println("found peer", peerInfo.String())
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

We developed a basic example to show various features of peer-to-peer networking (direct connections, data streams and peer discovery). I believe these can give you a taste of the types of applications that can be built using this kind of architecture (well-known examples include BitTorrent and the InterPlanetary File System).

And finally, I hope you liked the content and that this gets you a bit more interested in getting started in the peer-to-peer networks world.

Top comments (7)

Collapse
 
vineethpeddi profile image
Vineeth Peddi •

Hey Felipe,
Thanks for writing this article. This is really helpful.

When I am executing this code, I found few issues.
1) Firstly when I imported go-libp2p I got some issues likes packages not found but I solved it by upgrading go version. I recommend others to use go latest version to use the above libraries.
2) github.com/libp2p/go-libp2p-core/peer this package is deprecated. Please use the following package.
github.com/libp2p/go-libp2p/core/peer

Hope this helps someone.

Collapse
 
feliperosa profile image
Felipe Rosa •

Hey Vineeth,
Thanks for this, since the post was written a while ago.

I'll try to find some time to update it with your suggestions.

Collapse
 
kataras profile image
Gerasimos (Makis) Maropoulos •

I enjoed that one, good job Felipe Rosa!

Collapse
 
jithinks profile image
Jithin KS •

Hey thanks for the writeup. I'm planning to delve into Libp2p. I'm new to Golang as well as Rust. My plan is to learn Go / Rust as I start learning Libp2p. Which do you think I should go with Go or Rust?

Collapse
 
feliperosa profile image
Felipe Rosa •

Hey!

I would suggest the one you are the most familiar with (in this case it seems like it is Go).

Collapse
 
serajam profile image
Fedir Petryk •

Very interesting. Thanks

Collapse
 
feliperosa profile image
Felipe Rosa •

Thanks! Glad you liked it :)

Read next

sarvabharan profile image

System Design 10 - Distributed Logging and Monitoring: Keeping an Eye on Your System’s Every Move

Sarva Bharan -

architagr profile image

πŸŽ„ Advent of Code 2024: Day 2 Completed! πŸš€

Archit Agarwal -

sty6x profile image

I built my own search engine

FDiaz -

skysingh04 profile image

Why we Built a Mini-Language for a Golang Hackathon

Akash Singh -