SObjectizer Tales – 28. If I had a magic wand…

During our way back home, just before reaching the corporate parking lot, we bumped into Dan, an experienced developer who recently explored SObjectizer to help the team enhance some aspects of calico. Dan used the library for a while and now he is eager to share his feedback on what he finds most challenging or frustrating.

In this article, we’ll delve into Dan’s complaints to offer a perspective on certain features and aspects that can be awkward. As the author of this series, I believe it’s essential to candidly discuss areas where I feel less comfortable with the library and, sometimes, suggest potential improvements from a user’s standpoint. The opinions in this article are my own and might be different from what other people think about SObjectizer.

Dropping work at shutdown

The most notable missing feature for me is the automatic interruption of agent’s work at shutdown. This issue strikes a nerve with me, given that I often work on backend services requiring rapid shutdown, particularly in the context of modern deployment options like Docker. As discussed in a previous article, pending events in an agent’s queue will be processed at some point, even though the shutdown procedure is triggered. Therefore, if we need to drop pending messages, we must design agents accordingly to do so. However, all the solutions we discussed are essentially workarounds that require mixing agent logic with message management, often leading to a violation of the Single Responsibility Principle (SRP). While we may introduce more generic solutions to address this issue, they would inevitably involve some degree of SRP violation and may not be a standard solution in every case.

While I don’t have a specific proposal for implementing this feature into SObjectizer, as it would necessitate significant changes to its internals, I do have a suggestion for enabling it. We could introduce an optional “drop events at shutdown” toggle on the context object, similar to how message limits and agent priorities are configured:

class my_agent final : public so_5::agent_t
{
public:
    my_agent(so_5::agent_context_t ctx)
        : agent_t(ctx + drop_events_at_shutdown), ...

Akin to other toggles, the drop_events_at_shutdown is a “hint” for dispatchers to understand that the particular agent needs to drop any pending demands at shutdown. This means, the dispatcher should just discard any pending messages awaiting processing during the shutdown process. Also, this approach would allow for an incremental implementation. It could start with a spike for a specific dispatcher and then be extended to others if deemed necessary.

If I had a magic wand…I would introduce a feature that allows configuring agents to drop pending work at shutdown. This feature would be available on any dispatcher, similar to message limits, providing more control over shutdown behavior.

SObjectizer Lite

SObjectizer makes extensive use of features like RTTI, exceptions, dynamic dispatching, and heap allocations, which are essential for its expressiveness and power. However, these features also limit its usability in scenarios where they are not permitted, such as deterministic systems like embedded and real-time environments.

This wish is highly utopic, as it would necessitate a complete redesign of the entire framework. Moreover, achieving the same level of functionality without the aforementioned C++ features is practically unfeasible. However, maybe it would be possible at some point to create a subset of SObjectizer – a sort of “SObjectizer Lite” – that is implemented only in terms of some allowed C++ features.

If I had a magic wand…I would provide a “SObjectizer Lite” version tailored for real-time systems.

Telemetry is not strongly typed

When I bumped into SObjectizer’s runtime monitoring feature for the first time, I appreciated the consistency of telemetry data being sent as messages to a predefined message box. However, I found it somewhat cumbersome that filtering different quantities required working with strings. I understand that using a single type, like quantity<T>, is an effective and future-proof way to represent telemetry data. For instance, if we need to store all telemetry information blindly to a non-relational database, we can simply subscribe to quantity<size_t> and receive everything.

However, I believe that quantity should also contain a stronger type identifier for the kind of data transmitted, such as “demand count” or “active_obj event count”. While I don’t have strong opinions on this matter (it might be a scoped enumeration or something else), I would prefer to avoid using strings for filtering such information, especially standard information not provided by the user.

If I had a magic wand…I would strengthen telemetry data type by adding more information in order to simplify and make filtering more efficient.

Default handlers

A nice thing to have for me is the possibility to subscribe for “any” message type. My idea for the syntax would be either:

mhood_t<void>

That would not require introducing any special type, or:

mhood_t<unhandled>

where unhandled is provided by SObjectizer. Another alternative could involve using std::any, enabling functions like any_cast() and others. However, I’m not particularly fond of these techniques as they can often be seen as design shortcuts that may lead to complications later on.

One possible use case for this feature is discussed in the previous article, primarily to address agents like image_tracer or fps_estimator, which only need to observe traffic over a channel. Conversely, another useful scenario where this type might come in handy is the opposite case: ensuring that only certain types of messages are sent to a channel:

void so_define_agent() override
{
    so_subscribe(m_channel).event([](const std::string& name) {
        // ... ok
    }).event([](const my_message& another) {
        // ... ok
    }).event([](mhood_t<void>) {
        throw std::runtime_error("not allowed...");
    });
}

Here above, if types other than string and my_message are sent to m_channel, an exception will be thrown. This approach could prove useful for maintaining control over the design and, upon introducing a new message type into the system, breaking the program, possibly first by running unit tests.

If I had a magic wand…I would introduce a way to subscribe for a default message type.

Different channel abstractions

mbox_t serves as a powerful and unified abstraction, representing a versatile carrier for both messages and signals. However, it’s important to note that behind the scenes, message boxes come in three distinct flavors:

1. Multi-Producer Multi-Consumer: these message boxes can be subscribed to by any “sink” (e.g. agent), and any entity can send them a message;
2. Multi-Producer Single-Consumer: these message boxes can only be subscribed to by their “owner”, but any entity can still send them a message;
3. Converted from message chains: these message boxes cannot be subscribed to by any entity, but any entity can send them a message.

While I understand the convenience of having only a single type, mbox_t, to pass around, I must raise a concern about the resulting design. It could potentially be confusing, as it requires knowing exactly which “kind” of mbox_t we are dealing with, especially when inheriting code. Typically, this necessitates examining the source code or referring to documentation (if available). The primary concern is that mbox_t effectively acts as a weaker type, causing so_subscribe() to behave differently based on internal information that cannot be retrieved at compile-time.

Thus, another design – clearly more intrusive – would be to introduce a hierarchy, such as:

• mbox_sink: corresponding to the third flavor mentioned above, only allows data to be sent to it; attempting to subscribe to it would result in a compilation error;
• mbox_single_target, corresponding to the second flavor mentioned above, inherits from mbox_sink and adds the capability to subscribe to it from a single agent. Although it’s not possible to enforce checking the “owner” at compile-time, static analysis can help identify issues;
• mbox_multi_target, corresponding to the first flavor mentioned above, inherits from mbox_single_target and adds the capability to subscribe to it from any entity.

In practice:

• mbox_sink is created when a message chain is converted to a message box using as_mbox().
• mbox_single_target is the result of calling so_direct_mbox() on any agent.
• mbox_multi_target is the result of creating a Multi-Producer Multi-Consumer (MPMC) channel, such as calling environment.create_mbox().

The initial proposal for this could involve using type aliases. All such types could simply be aliases of mbox_t, with the mentioned functions returning them as appropriate. While errors wouldn’t be prevented at compile-time, users could opt to use these aliases to make their intentions a bit more expressive.

If I had a magic wand…I would introduce stronger channel abstractions to empower developers to maintain code and create more polished interfaces.

Takeaway

In this episode we have learned:

• If I had a magic wand…I would introduce a feature that allows configuring agents to drop pending work at shutdown;
• If I had a magic wand…I would provide a “SObjectizer Lite” version tailored for real-time systems;
• If I had a magic wand…I would strengthen telemetry data type by adding more information in order to simplify and make filtering more efficient;
• If I had a magic wand…I would introduce a way to subscribe for a default message type;
• If I had a magic wand…I would introduce stronger channel abstractions to empower developers to maintain code and create more polished interfaces;

As usual, calico is updated and tagged (even though this installment does not introduce any commit).

What’s next?

We’re nearing the end of the series, and it’s been quite a journey. I hope you found it interesting, even though some episodes may have been longer than intended due to my enthusiasm for the topic.

In the upcoming and final article – the Epilogue – we’ll conclude the series and offer suggestions for delving into more topics regarding SObjectizer.

Thanks to Yauheni Akhotnikau for having reviewed this post.