Loose coupling: why you want to avoid REST communication between microservices
A developer asked me yesterday why we’re using an event bus to integrate all our microservices, and since I’m very convinced of why this is a good thing, I decided to write a blog post about it.
Microservices are everywhere, and yet very few companies seem to get their microservice architectures right. One of the mistakes I have seen almost everywhere is the distributed big ball of mud: a collection of microservices that communicate through REST interfaces, usually synchronously. This is an excellent way to combine the worst of both worlds of monolithic and distributed systems.
I remember the first ‘official’ microservice architecture I encountered (I say ‘official’ because we have been doing similar things long before the term existed. Gregor Hohpe and Bobby Woolf already taught us how to design the communication between systems and the SOA principles have existed for decades). The boundaries of the services were actually quite well done, designed roughly around aggregates. But the only method of communication was synchronous REST calls, and even worse, every microservice contained the client libraries of almost every other microservice. It’s no surprise there were a lot of problems: independent deployability was non-existent, as was resilience, because Netflix hadn’t released Hystrix yet.
The strange thing is that although everyone knows the Enterprise Integration Patterns book, very few people actually make the connection with microservice architectures. Countless times have I heard arguments like ‘but this is how it’s done in Spring’ and ‘Netflix did it successfully, so it must be good’.
The situation has improved over the last year or so, and I think technologies like Kafka have helped a lot in that area. Let’s face it, we’re all developers, and playing with new toys is always more fun than following patterns that were invented many years ago. Still, let’s look at one of the reasons you should consider centering your microservices architecture around a message bus or event stream: coupling.
High Cohesion, Loose Coupling is a principle that is often mentioned in the context of microservices. Everything that logically belongs together should be in 1 microservice, and the different microservices should be loosely coupled. Loose coupling is harder to achieve than you might think, because there are many kinds of coupling:
When one system needs to wait for the response of another system before it can continue processing, those systems have temporal coupling. This is the case when using synchronous requests, so these should obviously be avoided. An event stream is one way to avoid it, but it can be done with asynchronous REST calls as well.
One service should not need to know the location (i.e. IP-address) of another service in order to communicate with it. If you rely on physical addresses, horizontal scaling and resilience become problematic. Discovery tools like Netflix Eureka help to solve this problem. A message bus or event stream does this as well, as long you make sure you don’t make it your single point of failure.
A service should not know anything about the behavior of the services it communicates with. If possible, this means communication between microservices should be based on immutable events rather than commands, because commands imply that the sending system tells the receiving system what to do, whereas events just state that something has happened in the sending system. What the receiving system does with that information is completely up to that system. The event stream pattern is a very good way to avoid behavioral coupling.
One of the often stated advantages of microservice architectures is the ability to have polyglot systems, allowing you to use the right tool or language for the job. What you often see is one service using the client library of another service to communicate with it. This introduces implementation coupling. The only thing services should share is a contract, for example the schema of the events that can be emitted. Ideally the contract is designed in such a way that it can evolve without breaking the systems that use it.
Another kind of temporal coupling that you might want to consider is the evolution of business needs over time. Using a persistent event stream like Kafka allows your system to evolve into areas you did not yet think of today, without impacting any of the source systems. Just connect the new service to the event stream and let it consume the events from the beginning of time and you’re good to go.
Admittedly, most of these problems of coupling can be solved when using pure REST. The Netflix stack is a nice example of tools to do this, with circuit breakers, discovery servers, libraries for easily creating REST clients and so on. But when you add up all those tools and libraries, I feel they make the whole architecture much more complex than it needs to be. Using an event stream like Kafka as the basis for your microservices architecture is much more elegant and much more powerful, especially when you consider future business needs. One thing you need to take into account is the resilience of your event stream, so a solid streaming platform and a good operations team are invaluable.
Do note that REST interfaces have their use even in an architecture like this, but this is usually limited to the edge services, for example when exposing APIs to the public or to partner companies.