Module 3 - Async Processing

Pub/Sub Pattern

Decouple producers from consumers with publish-subscribe messaging.

1The Newspaper Analogy

Simple Analogy
Think of a newspaper subscription. The publisher doesn't know who reads it. You subscribe to topics you care about (sports, business). When new content is published, all subscribers automatically receive it.

Pub/Sub works the same way. Publishers send to topics, subscribers listen to topics they care about.

Publish-Subscribe (Pub/Sub): A messaging pattern where senders (publishers) send messages to topics, and all subscribers to that topic receive the message.

2Pub/Sub vs Point-to-Point

Point-to-Point (Queue)

Producer → Queue → ONE Consumer
Message consumed by one receiver
Load distribution across consumers

Pub/Sub (Topic)

Publisher → Topic → ALL Subscribers
Message received by all subscribers
Event broadcasting to many services

3Use Cases

Event Broadcasting

Order placed → notify inventory, shipping, email, analytics

Real-time Updates

Stock price changes → all subscribers see updates

Microservice Communication

Decouple services through events

Log Aggregation

Multiple services publish logs to central topic

Cache Invalidation

Data changed → notify all cache instances

Webhooks

Event occurs → notify all registered endpoints

4Popular Implementations

Apache Kafka

High-throughput event streaming

Distributed event streaming. Persists messages, replay supported.

AWS SNS

AWS-native event fanout

Managed pub/sub service. Integrates with SQS, Lambda.

Google Pub/Sub

GCP event-driven systems

Global, scalable messaging. At-least-once delivery.

Redis Pub/Sub

Real-time notifications

Simple, in-memory. No persistence, fire-and-forget.

5Trade-offs

Advantages

  • ✓ Loose coupling between services
  • ✓ Easy to add new subscribers
  • ✓ Scales independently
  • ✓ Async, non-blocking

Challenges

  • ✗ Message ordering complexity
  • ✗ Duplicate handling needed
  • ✗ Debugging distributed flow
  • ✗ Eventual consistency

6Key Takeaways

1Pub/Sub decouples publishers from subscribers via topics
2Publishers don't know subscribers-loose coupling
3All subscribers receive the message-fan-out pattern
4Use for event broadcasting, microservice communication
5Kafka, SNS, Google Pub/Sub are popular implementations

?Quiz

1. In pub/sub, how many subscribers receive each message?

2. When should you use pub/sub over a queue?