Module 6 - Distributed Systems

CAP Theorem

The fundamental trade-off in distributed systems: you can only pick two of three.

1The ATM Analogy

Simple Analogy
Imagine two ATMs for the same bank account with $100. You withdraw $60 at ATM-A while your partner withdraws $60 at ATM-B simultaneously. The network between ATMs is down.

Option 1 (Consistency): Both ATMs refuse the transaction until network recovers. Safe but unavailable.

Option 2 (Availability): Both allow withdrawal. You now have -$20 balance. Available but inconsistent.

CAP Theorem: In a distributed system, during a network partition, you must choose between Consistency and Availability. You cannot have both.

2The Three Properties

C

Consistency

Every read receives the most recent write or an error. All nodes see the same data at the same time.

Example: After updating your profile, everyone sees the new version immediately.

A

Availability

Every request receives a response (not an error). The system is always operational.

Example: The website always responds, even if showing slightly stale data.

P

Partition Tolerance

System continues operating despite network partitions between nodes.

Example: If US and EU data centers can't communicate, both keep working.

3The Trade-off Reality

Key insight: Network partitions WILL happen in distributed systems. So you're really choosing between CP (consistency + partition tolerance) or AP (availability + partition tolerance).

CP Systems

Sacrifice availability during partition. Returns error rather than stale data.

MongoDB (with majority writes)HBaseRedis ClusterZookeeper

AP Systems

Sacrifice consistency during partition. Returns stale data rather than error.

CassandraDynamoDBCouchDBDNS

4Real-World Dry Run

Scenario: E-commerce Inventory During Network Partition

1
Initial state
10 items in stock. Data replicated to Node A (US) and Node B (EU).
2
Network partition occurs
US and EU nodes can't communicate.
3
US customer orders 3 items
Node A: stock = 7
4
EU customer orders 5 items
Node B: stock = 5
If CP (Consistent)

One or both regions return "Service Unavailable" until partition heals. No overselling, but lost sales.

If AP (Available)

Both orders succeed. After partition heals: 7 + 5 = 12 sold, but only 10 in stock. Oversold by 2.

5How to Choose

Banking / FinancialCP

Wrong balance is worse than temporary unavailability. Money must be accurate.

Social Media FeedsAP

Showing slightly stale posts is fine. Users expect availability over perfect freshness.

Shopping CartAP

Cart can merge later. Losing items in cart is worse than brief inconsistency.

Inventory (Airline Seats)CP

Overselling is very costly. Better to show unavailable than overbook.

6Key Takeaways

1CAP Theorem: During partition, choose Consistency OR Availability
2Network partitions will happen-plan for them
3CP systems (MongoDB, HBase): consistency over availability
4AP systems (Cassandra, DynamoDB): availability over consistency
5Choose based on business requirements, not technical preference

?Quiz

1. Network partition occurs. Your bank app shows 'Service Unavailable'. This is:

2. Twitter shows you a tweet from 2 seconds ago instead of the latest. This is: