Introduction to ic-event-hub library

This tutorial is dedicated to Rust smart-contract (canister) development on Internet Computer (Dfinity) platform. Completing it, you’ll know how to use ic-event-hub library APIs in order to perform efficient cross-canister integrations.

Before digging into this tutorial it is recommended to learn the basics of smart-contract development for the Internet Computer. Here are some good starting points: official website, dedicated to canister development on the IC; IC developer forum, where one could find an answer to almost any technical question.

This tutorial is intended for new developers who want to understand the basics of ic-event-hub. If you already finished it, it is recommended to check out my other tutorials about this library:

• Tutorial: Connecting A Token With Multiple Ledgers Using ic-event-hub
• Tutorial: Efficient Canister Networking With ic-event-hub

Motivation

Currently, canisters on the IC use request/response model to communicate with each other. While this model covers almost 100% of any needs, sometimes it limits integration simplicity. It's simpler to go through an example here.

Imagine the following scenario:

• you've built a token canister that accepts transfer transactions from users and re-transmits the info about these transactions to any other canister that will ask for it;
• there are many ledger canisters (developed by third parties who want to integrate them with your token), each of which is specialized on specific transaction type:
  • one of them only keeps track of transactions made by US citizens;
  • another one only keeps track of transactions made to/from charity organizations;
  • a bunch of personal ledgers, which keep track of transactions of a particular person or organization;
  • and each month there is at least one more ledger appears with some new specific requirements;
• besides all of that you want to make your token as efficient as possible:
  • if the token experiences high load (e.g. 100+ tx/block), you don't want to send 100+ messages to each ledger;
  • since messages you send to ledgers are small (only contain some generic transfer data: from, to, amount, timestamp), you want to pack these messages into a single batch and send them all at once - this way you could save a lot of cycles.

It looks like in order to implement such a scenario, one would need a system that fits into the following requirements:

1. We want to "fire-and-forget" data without bothering about who exactly is going to receive it or if there is an error at the receiver's side.
2. We want receivers to be able to specify conditions of desired data, e.g. "if this peace of data equals to - send it to me, otherwise ignore me".
3. There should be some kind of data-batching functionality, that will help us stay on the budget.

These declarations imply that an event-based protocol is what we actually want. Such a protocol, where there are emitters which emit events and listeners which receive these events based on some conditions.

ic-event-hub library implements that event-based protocol for us.

ic-event-hub

This library defines two types of actors: emitters and listeners. They communicate via events - special data structures optimized for conditional subscription. Emitters "emit" events, listeners (subscribers) "subscribe" to events they are interested in.

Here is an example of an event:

#[derive(Event)]
pub struct TransferEvent {
    #[topic]
    pub from: Principal,
    #[topic]
    pub to: Principal,
    pub amount: Nat,
    pub timestamp: u64,
}

Notice two things:

1. The whole struct is annotated with #[derive(Event)] macro. This macro will implement the IEvent trait for us - the trait that transforms our custom structure into the Event structure.
2. Some fields are also annotated with #[topic] macro. By annotating a field with this macro we enable listeners to filter all emitted events and only receive events which have these two fields set to some specific value.

By annotating a struct with #[derive(Event)] macro you'll also be automatically provided with special EventFilter struct. This struct is used by listeners in order to specify events they are interested in.

For example, annotating the TransferEvent struct defined above with #[derive(Event) would also generate the following structure:

pub struct TransferEventFilter {
    pub from: Option<Principal>,
    pub to: Option<Principal>,
}

If the listener wants to receive all TransferEvents they would initialize this structure this way:

TransferEventFilter { from: None, to: None }

If they want to receive only those TransferEvents related to some particular sender, they would use this structure the following way:

TransferEventFilter { from: Some(some_principal), to: None }

Emitter canister

Emitter canisters emit events by calling emit() function:

emit(TransferEvent {
    from,
    to,
    amount,
    timestamp,
});

As you might notice, there is no .await after the invocation. This is because of "fire-and-forget" policy - the emitter sends data in form of events to "nowhere". It doesn't even bother if there is any listener out there - the library handles it automatically.

It is important to say, that the "fire-and-forget" policy implementation is virtual. Under the hood the emitter canister still waits for responses for sent event batches in the background, but it just ignores them once they appear.

The event won't be transmitted immediately. Instead, the library will push it to an event queue (unique for each listener). Once this queue grows big enough or once enough time has passed, only then this queue is transformed into a batch and sent to the listener canister.

This implies that even if one writes something like this:

emit(event_1);
emit(event_2);
emit(event_3);

These events will be merged into a single event batch and sent all at once.

Both parameters: the queue's size in bytes and the desired delay before the transmission are configurable. You can pass them into the library-initializing macro like this: implement_event_emitter!(max_time_interval_nano, max_batch_size_bytes). What about this macro - it initializes the state of the library and make functions like emit() available for usage, so don't forget to call it somewhere in your canister's code.

In order to activate the transmission of events we have to use the #[heartbeat] function:

#[heartbeat]
pub fn tick() {
    send_events();
}

In order to receive events, a listener canister has to subscribe to them. An emitter canister's developer could allow any listener to subscribe to events emitted by this canister. To do this, the developer would have to call implement_subscribe!() macro somewhere inside the canister's code.

This macro supports guard function declaration inside: implement_subscribe!(guard = "guard_function").

Listener canister

After that a listener canister is able to call subscribe() function of the emitter canister, like this:

let event_filter = TransferEventFilter { from: Some(some_principal), to: None };

emitter_canister_principal
    .subscribe(SubscribeRequest {
        callbacks: vec![CallbackInfo {
            filter: event_filter.to_event_filter(),
            callback_method_name: String::from("events_callback"),
        }],
    })
    .await
    .expect("Subscription failed");

As you might notice, a listener is able to specify the callback method name for each event filter. ic-event-hub on the emitter's side will use this callback function name to send all the events which fit into the provided EventFilter. This callback should be implemented like this:

#[update]
fn events_callback(events: Vec<Event>) {
    for event in events {
        if event.get_name().as_str() == "TransferEvent" {
            let ev: TransferEvent = TransferEvent::from_event(event);
            print(format!("Got event: {:?}", ev).as_str());
        }
    }
}

It should also accept a single argument of type Vec<Event> and return nothing. If you need extra security (e.g. forbid some canisters from sending events to you), you can also express it inside this function.

Combining everything together

Okay, let's define a basic example of how one could use ic-event-hub in order to implement a solution for the scenario we described earlier.

On the token canister side we would need to put a code like this:

// define an event data type
#[derive(Event)]
pub struct TransferEvent {
    #[topic]
    pub from: Principal,
    #[topic]
    pub to: Principal,
    pub amount: Nat,
    pub timestamp: u64,
}

// initialize ic-event-hub state and enable the `emit()` function
implement_event_emitter!(1_000_000_000 * 60, 100 * 1024 * 1024);

// emit events somewhere in your code (for example - inside the `transfer()` function of the token
emit(TransferEvent {
    from,
    to,
    amount,
    timestamp,
});

// enable listeners to subscribe
implement_subscribe!()

// activate event sending
#[heartbeat]
pub fn tick() {
    send_events();
}

On a ledger canister side:

// define the same event data type
#[derive(Event)]
pub struct TransferEvent {
    #[topic]
    pub from: Principal,
    #[topic]
    pub to: Principal,
    pub amount: Nat,
    pub timestamp: u64,
}

// define event filter(s) describing the data you want to receive
let event_filter = TransferEventFilter { from: Some(some_principal), to: None };

// subscribe for events using the filter
emitter_canister_principal
    .subscribe(SubscribeRequest {
        callbacks: vec![CallbackInfo {
            filter: event_filter.to_event_filter(),
            callback_method_name: String::from("events_callback"),
        }],
    })
    .await
    .expect("Subscription failed");

// define the callback function to process incoming events
#[update]
fn events_callback(events: Vec<Event>) {
    for event in events {
        if event.get_name().as_str() == "TransferEvent" {
            let ev: TransferEvent = TransferEvent::from_event(event);
            print(format!("Got event: {:?}", ev).as_str());
        }
    }
}

That's it. ic-event-hub will handle everything else: the transmission, listener subscription, encoding/decoding process, event filter matching and event batching process.

Afterword

The ic-event-hub library is not only an alternative way of transmitting data between canisters. It is also an implementation of Inversion of Control pattern.

In the standard request/response model, the receiver defines the data type it want to receive and the sender decides whether it will send the data or not.

In the ic-event-hub's event-based model, the sender defines the data type it will send and the receiver decides whether it wants to receive that data or not.

In the standard request/response model, a developer has to keep track of many different things: an address (canister id) of the remote program, a signature of the remote function and the right time to transfer the data. In exchange for doing this the model rewards a developer with the ability to process the response of the remote call.

In the ic-event-hub's event-based model, a developer gives up on response processing and in exchange is rewarded with simplicity and high throughput.

It is recommended to use this library as an alternative protocol alongside the standard request/response protocol, not as a replacement for it. This way you, as a developer, would enjoy the best experience possible.

Thanks for reading and don't forget to take a look at ic-event-hub's source code: