fbpx
Learn to build large language model applications: vector databases, langchain, fine tuning and prompt engineering. Learn more

Message queuing

Data Science Dojo
Syed Saad Peerzada
| July 9

Data Science Dojo has launched a RabbitMQ offering to the Azure Marketplace with pre-configured settings which helps the user to take deep dive into RabbitMQ with just a single click.

What is message queuing?

A message queue is a system that allows a sender and recipient process to communicate with one another asynchronously.

Key terminologies for RabbitMQ

  1. Publisher: A Publisher is a sender who sends a message first.
  2. Consumer: A consumer is attached at the end of the queue and consumes a published message.
  3. Exchange: When a publisher sends a message to the broker, the exchange accepts it and passes it to the proper queue.
  4. Queue: A queue is a channel for transferring and storing messages or buffers.
  5. Binding Key: A binding key is a key through which an Exchange is bound with a queue.
  6. Routing Key: Whenever a publisher publishes a message, it also specifies a routing key along with a message.

Challenges for individuals

If a publisher publishes a message directly to the consumer, there is a high chance of messages getting lost to a consumer as there might be a scenario in which the receiver is not available or running up.

What we provide

RabbitMQ is the most widely deployed, cross-platform, lightweight open source message broker that implements different messaging protocols including Advanced Message Queuing Protocol(AMQP). The biggest advantage of RabbitMQ is that published messages are never lost as they always reside in a safe place until they are received by the consumer. This safe place acts as an intermediary agent and is referred to as “Queues” in RabbitMQ. Through this offer, a learner can fire and forget about who will process the published message and when.

Working of RabbitMQ

When a message is published by a publisher, it first gets received by an exchange. The exchange then forwards the message to the queue based on the exchange type. RabbitMQ includes four important exchange types that cover the majority of messaging scenarios:

  1. Fan-out Exchange: This exchange broadcasts the message to all of the queues attached to the exchange.
  2. Direct Exchange: The message is forwarded only to the queue by an exchange whose Binding key exactly matches the Routing key specified by the publisher.
  3. Topic Exchange: The topic exchange allows the forwarding of messages based on partial matching of Binding and Routing keys.
  4. Header Exchange: This exchange takes advantage of the structure of AMQP messages and may do complex routing based on the AMQP message’s headers (including custom ones). Each message sent via AMQP has metadata attached to it called headers.
RabbitMQ Workflow
RabbitMQ Workflow

Use case of RabbitMQ

RabbitMQ Use case
Use case of RabbitMQ

Let us look at a use case of RabbitMQ in an online banking application. In the modern era, large-scale applications are built using microservice architecture in which each microservice is responsible for its task. Let us suppose in the given use case we have two microservices ‘Email service’ and ‘SMS service’. As soon as a user makes a transaction, he or she must be notified via email and SMS service. In technical jargon, a publisher publishes a message to the exchange along with the routing key ‘email’. In this scenario, we assume the exchange type as a direct exchange. The exchange then compares this routing key against the binding key of each queue. Based on the matching of keys, the message will be forwarded to the Email notification queue. The Email microservice then consumes the message from the Email notification queue and performs its task i.e., it will send a notification to the customer’s email address about the successful transaction. If for any reason, the server is down on which the email service is deployed or the service is not working for any reason, the published message is retained in the email notification queue and is not lost. When the email service starts working again, the message is consumed and the customer is notified via the email service.

Key features of RabbitMQ

Our solution provides RabbitMQ combined with the power of Azure services to enable web servers to quickly respond to requests rather than resource-heavy procedures on spot. Features included in this offer:

  • Multiple message protocols are supported through Asynchronous messaging. 
  • Provides messaging across different programming languages.
  • To achieve increased throughput and high availability, they can be deployed as clusters. Federate across multiple availability zones and regions.
  • RabbitMQ also provides TLS and LDAP authentication.
  • Continuous integration, operational metrics, and integration to other enterprise systems can be achieved through different tools & plugins in RabbitMQ.
  • Monitor every aspect of messaging through an interactive UI.

Conclusion

RabbitMQ messaging enables software applications to connect and communicate asynchronously which ensures that the message is received by the intended recipient and never lost. The offering leverages the power of Microsoft Azure services to run effortlessly with outstanding responsiveness. RabbitMQ offered by Data Science Dojo is ideal to start working with RabbitMQ without the need to worry about configurations and computing resources.

Install the RabbitMQ offer now from the Azure Marketplace by Data Science Dojo, your ideal companion in your journey to learn data science!

Try RabbitMQ!

Related Topics

Statistics
Resources
Programming
Machine Learning
LLM
Generative AI
Data Visualization
Data Security
Data Science
Data Engineering
Data Analytics
Computer Vision
Career
Artificial Intelligence