To me asynchronous messaging means asynchronous, persistent, transactional messages; think MSMQ, MQ series, Rabbit MQ that sort of thing. Asynchronous messaging is still a popular communications pattern that’s been in use since the mainframe (read overpriced space heater) era. It’s still a very popular solution for providing scalable, reliable messaging between disparate systems. Modern enterprise application design principles advocate avoiding single large monolithic solutions, hence the focus on patterns like micro-services. Instead, they focus on loosely coupled subsystems, preferably with separate ecosystems, all operating in concert, providing a single unified system.
Loose coupling is a great concept designed to reduce the resulting side-effects of any change to core features. Loose coupling acts like a developmental circuit breaker, minimizing any downstream ripples that can affect related features when making application changes. Asynchronous messaging is one of the more powerful methods for providing an application with a systematic solution for loose coupling.
Transactional message handling, much like traditional database operations, must be atomic. This means, all of the tasks undertaken in handling a message must succeed if any are to succeed at all. Any faults encountered have the effect of cancelling out the results of any tasks completed prior to encountering a fault. As you might imagine, this is often a very attractive feature that guarantees application state consistency. But like all powerful features, it comes at a price. The cost of transaction management across multiple subsystems like database connections and message queue services can be really appreciable. Thus, it’s wise to consider the extra cost of transaction control when handling asynchronous messaging, particularly when messages are being persisted. This begs the question, when should I employ asynchronous transactional persistent messaging?
1. Crossing bounded Contexts
A bounded context is a concept associated with Domain Driven Design and often discussed in SOA related conversations. Bounded contexts are a divide-and-conquer solution to domain complexity, dividing a large application’s domain into clearly defined contexts (often around functional business areas) and declaratively managing their interrelationships. One of the prime goals of these contexts is loose coupling. Employing asynchronous persistent transactional messaging is often a very successful strategy.
2. Extensibility Points
I’ve already written a prior article about the benefits of application extensibility. Message queues are tailor-made extension points within any application. The pub/sub pattern can be readily employed to add functionality when handling a message.
3. Parallel Processing (many threads)
The pub/sub pattern makes naive parallelism for message handling relatively trivial to implement. I say naïve parallelism because the asynchronous nature of the pattern implies that concurrency is facilitated, not required. Some of the features inherent with a more traditional approach to parallelism can be awkward to implement, such as waiting for a group of parallel tasks to complete before progressing further.
4. Sequential Task Processing (single thread)
Similar to the scenario above. A single message handler instance makes processing tasks sequentially relatively easy. You might be wondering why this might be useful. Some tasks must occur in sequential order in order to complete at all. For example I recently had to hash the contents of file, block by block. If the progressive hash function had not been applied sequentially to the file blocks, the message digest would never have been calculated correctly, or even been consistently wrong.
5. Out-of-band tasks (new transactions, fire and forget and non-transactional tasks)
Please note that, transaction scope support can vary wildly from platform to platform. Many tasks don’t require a transaction or must complete regardless of whether the main task is completed successfully or not. Message handling provides an easy way to change the transaction scope and hence process tasks outside the scope of the original transaction.
6. Facilitating Retry Semantics
Transactional messaging implies utilizing a retry strategy. This can vary from simple to very sophisticated with timed retries etc. When a robust, yet powerful retry mechanism is required for complex tasks, it’s often convenient to leverage the features supported by the message platform and avoid having to write extensively robust components from scratch without any appreciable business value.
7. Transaction Auditing
Messages are a convenient way to track transactions for audit trails. While they don’t tend to indicate transaction progress, they can be a very succinct way of determining what sort of transaction is undertaken.
