Message History Pattern Revisited

We recently ran a book club on Enterprise Integration Patterns here at work.

One of the patterns detailed in the book is the Message History pattern (go read the pattern if you're unfamiliar with it). To give some context, let's start with a possible implementation of a message history.

Given the following pipes and filters setup:

[appA] -> [comp1] --msg1--> [comp2] --msg2--> [comp3] --msg3--> [appB]
                                         \--> [comp4] --msg4--> [appC]

----
app = application
comp = component
msg = message

A messageHistory header could have the following values in each step (using / as a separator):

For the sake of representation I've assumed that messageHistory is a simple additive string but it could be some other data structure such as a linked list or similar.

In a very long chain of components (or with long component names à la ERPProductUpdateAdapterService) this header may become prohibitively long for a message header. What then? Do we codify our component names as abbreviations or some other short but unique ID?

Another aspect is what happens with our tracing capabilities in a pub/sub scenario where we will have parallel messages fanning out with their own message histories in subsequent steps? One solution could be allowing a wild card search for comp1/* in some kind of tracking system. At least correlation between component logs and messages is fairly straight forward, but clearly we have to think carefully about our implementation so that we cover our various use cases. One of the major advantages is that you can get a complete overview of the message history by just looking at the message.

In the wild I have seen two other patterns that solve the same problem and are well worth describing, Message ID Chaining and Tracking ID.

Message ID Chaining

A unique reference ID is set in the header of each new message that points back to the previous message. This allows for collecting all reference points and building up an event chain. Optionally the field could also contain the component that generated the message.

Advantages:

• Smaller header.
• Possibly simpler to implement from a messaging viewpoint.

Disadvantages:

• As mentioned in the book, what should an Aggregator that is combining several messages into one set?
• A single message is no longer enough to get the big picture.
• Requires some kind of tracking system to piece together and re-build the chain.

Tracking ID

A tracking ID (or tracing ID) is a unique identifier attached to the first message (or even before the first message) in a sequence of messages and then propagated along with all subsequent messages.

Advantages:

• Smaller header.
• Possibly simpler to implement from a messaging viewpoint.
• Not inherently tied to a message and therefore more natural/easier to reference in other places such as logs, RPC calls and similar.

Disadvantages:

• Does not contain component information, which must therefore be maintained in some other way.
• Does not maintain sequence. Sequence must be maintained through some other means e.g. timestamp (which is susceptible to clock skew) or a sequence number of some sort.
• Requires some kind of tracking system that can piece together events.

End Discussion

Are these two patterns actually patterns or simply variations of Message History? Hard to say. They definitely cover a common concern: tracing messages as they pass through message based systems. Regardless of your domain this is a powerful capability that can reduce maintenance and monitoring costs over time.