SOA vs. EDA: Is Not Life Simply a Series of Events?

When should you use an API? When should you use an event? Most contemporary software architectures are some mix of these two approaches. I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events.

Some background first—I wrote a post last year about how event streaming platforms are “not your dad’s message bus anymore.” This picture with the Volkswagen bus and bullet train pretty much sums up the sentiment:

We typically think about message brokers or queuing systems in the context of enabling asynchronous processing, and a queue/topic being a conduit to get a message from many sources to many destinations. However, we have progressed much further (and hence, the mildly snarky position—“not your dad’s message bus anymore”).

Message brokers have evolved into intelligent event streaming platforms. The message brokers nowadays can be analytical engines by their own right. They, in some cases, are more resilient than databases. Also, a lot of data that needs to be analyzed is in most cases transient in nature. It does not have to be stored, and its meaning can be “derived” while in flight. Streaming analytics has become mainstream. Pun intended!

At this point, I should have explained what an EDA is before talking about something a bit advanced like streaming analytics…

And, therefore, wearing my philosophical skeptic’s hat I firmly espouse that events are what this is all about. And, if I were talking to Yoda, he would have without question ascribed this to life. Is not life simply a series of events?

Philosophy aside and back to technology, this is ultimately a discussion about SOA vs. EDA, or in other words, API vs. events. To start with, perhaps we should play with some questions first. We often ask these when seeking a technical solution to solve a business problem:

• Should we build something that is request and response?
• Should we build it to be a batch process instead?
• Should the response be synchronous or asynchronous?
• Should the batch be a fire and forget?
• Can a fire and forget become synchronous? If it can, should it?
• Should blocking calls always be synchronous?
• How is an API different from event processing?
• Why do we have voicemail when we have mobile phones?

If you read the last question, then I have your attention! All these questions and more are asked constantly by architects and engineers, and though they may look disjointed, they all have something in common: They are all about processing some state changes—state changes caused by events that took place in the realm of some business function.

Starting with SOA (be it good old SOA or more contemporary like microservices), we almost always associate it with SOAP or REST APIs. In the past, there were many other RPC-style interfaces with a fairly lengthy evolutionary process—so much so, that I event wrote a poem about this called In the Beginning was the Socket…

Anyway, SOAP or REST APIs are geared towards client-driven, request-response interactions. The limitations with such a synchronous request-response pattern is that the client dictates which service will process the request, and the client must wait for the response even if it could be doing other things.

In many cases, the client-driven nature of SOA restricts the flexibility and scalability of the system. On a humorous note, perhaps SOA should really be called CDA for client-driven architecture. Augmenting SOA with EDA can overcome these restrictions. In an EDA, when an event is sent it is fire and forget. An event is simply an observation about the world—a fact that describes something that physically happened. It is immutable. What happened in the past cannot change. It is not a request. It is an observation about the world which we use as a method of communication.

EDAs create environments where components react to events as they arrive, enabling a subscribe and parallel processing model that spans multiple services. More importantly, it reduces the physical and temporal coupling between request-response clients and services. Compared to SOA, EDA is a relatively connectionless approach.

Yoda would have said, “Physical and temporal decoupling it is!”

Applications built using an event-driven architecture tend to have a simple relationship between the components, which is unidirectional in nature. You have event sources and event handlers, also referred to as sources and sinks. Both of these components can be producers and consumers.

Typically a producer will send events to multiple consumers. In order to facilitate independent and asynchronous computing, we need a middleware component to manage the delivery of events. This is commonly known as an event broker, an intermediary that manages the delivery of events from producers to consumers. One can standardize these to be event adapters (producers), an event hub (broker) and event processors (consumers). Note that consumers can become producers also. This is illustrated in the following diagram:

Compared to SOA, the essence of an EDA is that the services involved communicate through the capture, propagation, processing and persistence of events. This resulting pattern of communicating through a dataflow is quite different from the SOA approach of requests and responses. Here are some reasons why the EDA patterns can alleviate some of the challenges traditional SOA patterns bring:

	SOA	EDA
Pull vs. Reactive	Client makes a request of a service and expects a response. It’s great for persisted, static data, but gets a little hard when data keeps changing. You have to poll to detect changes.	Subscription model pushes events to consumers.
Coupling	Client has to know details of the API and its location at runtime.	Producers have no knowledge of consumer which will ultimately receive the event. There is still some minimal coupling in terms names of queues/topics and event formats.
Service Availability	A service must be available at the time a request is made by a client even if you are doing an asynchronous response handling.	Events do not require a reply and are inherently asynchronous. Events can be persisted for future consumption. With a highly fault-tolerant broker, the event producer does not need to know whether the consumers are available. Thus, we achieve higher resilience to network and compute failure, and this allows event producers to avoid blocking.
Process Modification and Extension	Processing logic is a request-response API that is hardwired into a service endpoint (with or without service discovery). If the logic needs to change or be extended, or if new logic needs to be introduced, the definition (not contract) of the service must be updated. This introduces change management and regression risk.	Additional event producers and consumers can be added to a system without any explicit process definition.
Consistency Between Process Interaction and Internal State Management	State changes are managed based on requests. For example, a request to “withdraw money” mutates the state of an account. The distinct processes of a request, a change in state and its persistence in case of failure must be tied together transactionally. This often leads users to deploy expensive distributed transaction protocols like eXtended architecture (XA).	EDA provides better support for consistency between process interaction and persisted internal state transitions. This is done through the event sourcing pattern, where the communication protocol (the event) is also the persistence mechanism (the event log). The current state of a system can be built or rebuilt from the log of events.
Retaining the Exact State Transitions That Customers or Services Perform	In SOAs, data is typically “mutated in place” in a database. This is a lossy process where each state change loses the information about the state changes that happened previously.	EDAs are event sourced, meaning every state change is captured, providing a truthful journal of the exact state changes that every customer or every service made over time. This journal lets operators rewind time to view or replay previous events exactly as they happened. It is also important for analytics that review customer (or system) behavior to derive insight.
*Streaming Analytics	SOA is incapable of deriving analytics of data in flight. This requires the ability to detect a pattern from multiple state changes both temporally and spatially.	EDA is fully capable of detecting patterns across multiple event sources over many different types of time windows. Also, deriving analytics of data in flight is a means of continuous intelligence.
The Timing of Consistency and of Intelligence	Synchronous communication makes it a bit easier to create consistent state across services from a client’s perspective. Intelligence from the consistent state are derived eventually—that is, eventual intelligence and continuous consistency	Events, being asynchronous, mean that different services become consistent with one another only in eventuality: There is no control over the timeliness of the process of event propagation.

*There is a significant difference!

Ultimately, the choice between SOA or EDA depends on what you are trying to achieve and the problems you are trying to solve. What is your priority? Is it eventual intelligence–to maintain state first and then later derive insight, or is it continuous intelligence–responding to change and deriving insight as soon as it happens? These are very different things. The first systems are simple information custodians. In the second, the business functions as a deeply connected central nervous system. In fact, the most fascinating concept I have come across in recent times is one that is impossible to achieve without an EDA. It is the concept of a digital twin, but I will save that for another post.

All that said, EDA does not supplant SOA, meaning that SOA, or traditional state persistence and data management, does not go away. EDA is not a cure for all computing woes. There will always be the need for synchronous request-response, API-based computing where a service consumer needs to make a request and wait for the response to come back. There will also be cases where the complexity and cost of managing events deliver no justifiable benefit. In order to deliver strong lasting solutions, one must combine event-driven patterns with more traditional patterns to achieve the optimal integration complexity and appropriate for developer productivity.

If you’d like to know more, you can download the Confluent Platform to get started with the leading distribution of Apache Kafka.

P.S. If you have read this far, you may have been looking for the answer to the question, “Why do we have voicemail when we have mobile phones?” Well, at this point, you already know the answer…Hint: See “Service Availability” in the table above.