Encounters with the Event Bus
Preface
Now you’ll probably be wondering that this is just another pretentious take on the well-documented concept on how an Event bus works, and for the most part, you’ll be correct but I feel that collating one’s thoughts and putting it down on paper lets you retrospect on how well you understand a concept. This allows for constructive criticism outside my echo chamber. I have been working with the Event bus over the years with distributed services to form an opinion. So this is me giving it a shot.
Introduction
Let us begin with an illustration.
If this diagram is anything to go on you can immediately derive that the event bus enables the pub-sub pattern and decouples the process of sending the event from receiving it.
In the simplest form, it allows publishers to publish an event and subscribers to subscribe to those events and then do something with those events. Yet this simplicity is necessary and allows for some of the most complex architectures to flourish.
Use Case
An event bus like the name suggests is used in an event-based system. From the application layer to the platform layer it could be applied anywhere. In this post, we will closely look at how it works with microservices with an event-based architecture and how it allows for this architecture to flourish.
decoupling , decoupling ,decoupling
In this illustration, you can see a simplistic breakdown of microservices. In this case, if Service A wants to communicate with Service B then Service A will need to know about Service B. It will need to know its address and its specification. In terms of designing a microservice, this is a big red flag since we want to build atomic services that do not depend on the context of other services, and knowing other services violates that principle. This happens because we rely on synchronous communication between services. So how do we solve this with an Event Bus?
choreography over orchestration.
In this example, the sending/receiving of the events is decoupled from the services. In a sense, this is a reactive approach to doing something rather than ordering your services what they need to do. In our previous example, all our service components would have required references to each other whereas in this illustration Service A just broadcasts an event into the void, and anybody interested can listen to it. For the sake of illustration, we have added a consumer and a receiver but that can vary in your architecture.
just subscribe
This mechanism of choreography can be very powerful when you have a ton of services. Imagine the nightmare of having to manage 20+ services that have references to other services. The cognitive load on the development team will be always high in such a scenario.
To alleviate that each team can build its service knowing they have to subscribe to a particular event and just implement the business logic around that service instead of worrying about inter-service communication and gluing them together. This probably doesn’t solve all our problems and we will run into issues when presented with a considerable event catalog and workflows that are driven by events that we might want to orchestrate using an event bus + workflow orchestrators. But I am probably getting ahead of myself and is a topic for another post.
If you want to understand how these orchestrators work then go ahead and take look at a few of them.
- https://github.com/uber/cadence
- https://netflix.github.io/conductor
- https://zeebe.io
- https://camunda.com
Events and Commands
I think it is important to understand communication patterns that the event bus will eventually be plugged into.
The important distinction to make here is between events and commands.
For example : userPolicyupdated vs UpdateUserPolicy
The latter which is a “command” asks another system to do something as if it knows about what needs to be done whereas the event “userPolicyUpdated” is just something that is put out in the world for others to react to. There might be no particular response that happens based on that event. The events that end up in the event bus must be “events” and not “commands” to ensure loose coupling.
Here is Martin fowler talking about it at GOTO 2017
Considerations
While using/building an event bus you have to realize that this is one of the core pieces of the architecture and interactions with the bus will be significantly high. Especially in a distributed system architecture.
Must be scalable
Needless to say, it must scale with increasing traffic/interactions. It will be resource-intensive since it orchestrates most of your services. This will depend on the number of services in your system and how chatty these services are. A scaling policy must be put into place by considering these factors.
Must be resilient
In case the event bus goes down there must be measures in place to prevent loss of items in the event bus queue. Temporarily persisting queue items specified by some policy is the way to go.
Delivery semantics
The delivery mechanism might depend on the internal implementation details of the bus.
These could vary between
- at most once
- at least once
- exactly once
Depending on the build-in mechanism you might want to include or leave out building event idempotency in your event store or any place where events are persisted or processed.
e.g In case of the “at least once delivery” mode you might run into cases where the same event is delivered twice. To avoid the receivers from processing the same event multiple times it is important to distinguish duplicate events
Message Structure and Validations
The key is to keep the event payload lean. Bombarding the event bus with large bytes of data is not desirable. Use references within the payload whenever possible.
Event ordering
Since the event bus will internally maintain a queue it is essential to define if we will require events to be processed in a particular order.
- FIFO queues and Random queues
Conclusion
Implementing an EventBus pattern can be beneficial as it helps loose coupling and promotes a publish-subscribe pattern. It also helps components interact without being aware of each other.