Message Brokers
“Message matters. Message matters almost as much as actions.” — Ron Suskind
“Message brokers are an inter-application communication technology to help build a common integration mechanism to support cloud-native, microservices-based, serverless and hybrid cloud architectures.”
That’s how the definition of Message Broker, it’s written in a formal way of using cool words in the vocabulary by looking at that no one want’s to argue. 😆
Well, actually we need an explanation in a simple modular manner. Let’s get into that.
Modern software and application is not just one unit running on a single node/single workstation. Now the multiple systems are communicating with each other. the rise of microservices is now happening and components within the same applications are communicating.
Now, these application follows a polyglot approach, their development technologies are different from one another, their communication methods are from one another. This is where the Message Brokers come into, it helps to communicate with other systems or components. If the communication protocol and technologies are different then it will do a translation process for common messaging protocols. this happens at both ends when receiving data and sending data.
But the architecture and use of the Message brokers are not limited only to that. and message broker has many capabilities.
When the message broker received a message it validates it, store, route, and delivers to the destination where it should go, without letting them know the client about the sender's details.
Message broker has to ensure the safety of these messages, and deliver them without corruption or deliver without loss, for this message brokers are using “Message Queue”, which is a First-in-First-Out type data structure it holds the messages in their received order till the receiver is able to get them and send confirm the receipt.
Another issue is to deal with that due to the low latency and network issues it ensures that the messages are not corrupted and received only once, making sure there are no duplicates.
Components of a Message Broker
- Producer - The application responsible for creating messages. the producer is connected to the message broker. this is also known as the publisher in the publish-subscribe pattern.
- Consumer - This is the endpoint that consumes the messages in the message broker. his is also known as the subscriber in the publish-subscribe pattern.
- Queue / Topic - This is where the message broker stores the messages.
Message Broker Patterns
Point to Point Messaging
Here the relationship between Producer and Consumer is a one-to-one relationship, each message is sent once and consumed by the only one Consumer.
Publish / Subscribe Messaging
This distribution pattern is also known as the pub/sub. A producer is also known as a publisher who publishes a topic. and there are multiple consumers are there how are subscribe to that topic that they want to receive updates from, in this pattern consumers also called subscribers. This is a broadcast type pattern which the Producer / Publisher does not know about the receivers.
Message Brokers and APIs
REST APIs are very common these days to communicate with web applications and microservice component communications. REST API uses HTTP to communicate within the different services. The issue here is that the REST Calls making through the HTTP are synchronous, which means the request made through the HTTP has to wait till the current process finishes its execution and comes back, this is a blocked approach. But the REST architecture is about being asynchronous and processing multiple requests and responses. Due to the limitation of the current HTTP/1.0, it is not possible.
As a solution for this scenario Message Brokers enables asynchronous between the services and the handling of the requests and responses done by the message broker, sever does not have to wait for a request to finish its execution. and message broker keeps the state of all the consumers and making the transmission process more resilient.
Message brokers are now becoming a de-facto of most modern systems most of the popular areas that message brokers used are the financial sector on managing the payments and transactions, ECommerce application and specially maintaining their supply chain management cycle and when transiting highly sensitive data, for these type of scenarios message brokers are important.
As for disadvantages, for the first time, there is a steep learning curve and configurations to do. and since this follows an asynchronous approach the debug can be hard to capture the exact point and find the error. Introducing a message broker system increases the system complexity so expert help is a must.
The most popular Message brokers are now Apache Kafka, RabbitMQ and for cloud-based solutions AWS SQS, and GCP Pub/Sub, You can find more here.
References
