Pub/Sub

Pub/Sub, also known as Publisher-Subscriber, is a communication model where services exchange messages through an intermediary service called a broker. The publisher publishes a message to a topic managed by the broker. Subscribers receive and consume messages from that topic without knowing anything about the producer of the message.

The core components of the Pub/Sub communication model are as follows:

Producers:

Producers create and publish messages to a topic. For example, in an e-commerce website, the producer may be the order service, which publishes order-related messages to the appropriate topic.

Subscribers:

Subscribers are services that receive and consume messages from a topic managed by the broker. They process messages without knowing anything about the producers of those messages. For example, in an e-commerce website, the payment service might be the consumer or subscriber that receives order messages and processes payments accordingly.

Broker:

The broker manages topics and handles the receiving and delivery of messages within those topics. The message itself contains the data sent by the producer, which is then used by the subscriber.

The advantages of the Pub/Sub mechanism are:

• Reduced coupling: It reduces coupling between the producer and subscriber. The publisher does not need to have direct knowledge of the subscriber, and vice versa. This makes it easier to add new subscribers without changing the publisher itself.
• Improved scalability: We can increase the number of subscribers when more messages are produced. If needed, we can also independently scale the producer.
• Low-latency communication: With proper broker configuration, payload size, network conditions, and processing time, the system can support very low-latency message delivery.

The disadvantages are:

• Increased complexity: With the introduction of a broker, there is a whole new system to manage alongside the producers and consumers. This adds another level of complexity to the system.
• More difficult debugging: Since messages are produced and consumed independently, it becomes more difficult to trace the flow of messages.
• Message ordering challenge: Depending on the broker and its configuration, messages may not always be processed in a strictly sequential order.

The common use cases of Pub/Sub are:

• Real-time messaging and chat applications
• Distributed systems and microservices
• Event-driven architectures
• Stream processing and pipelines
• Notification and background processing

The common Pub/Sub brokers are:

• Apache Kafka
• Redis Pub/Sub
• Google Cloud Pub/Sub

Example

Code snippets given below demonstrate the publisher which publishes count message to the topic messages managed by Redis every 200 ms and the subscribers which consumes message and prints count to the console.

// publisher.js
const Redis = require("ioredis");

const redis = new Redis();

async function run() {
  let i = 0;
  while (true) {
    redis.publish("messages", JSON.stringify({ count: i++ }));
    await new Promise((r) => setTimeout(r, 200));
  }
}

run().catch(console.error);

// subscriber.js
const Redis = require("ioredis");

const redis = new Redis();

async function run() {
  redis.subscribe("messages", (err) => err && console.log(err));
  redis.on("message", (channel, message) => {
    if (channel === "messages") {
      const parsed = JSON.parse(message);
      console.log(parsed.count);
    }
  });
}

run().catch(console.error);

References
https://docs.cloud.google.com/pubsub/docs/overview
https://redis.io/glossary/pub-sub/
https://www.youtube.com/watch?v=FMhbR_kQeHw