SYSTEM DESIGN 101
DO THE WORDS
LR CACHE REPLICAS SHARDS STORAGE CDN QUEUE REDUNDANCY ORDERABILITY
BLUEPRINT - SOCIAL APP
LOAD BALANCER
USERS
SYSTEM DESIGN
101
System design 101.
? PEOPLE HEAR...
LOAD BALANCER
load balancer, cache,
QUEUE
DATABASE REPLICA
CDN
SHARDING
and it sounds like
KAFKA
CACHE
MICROSERVICES
DOOKER
PROMETHEUS
RATE LIMITER
S3
NGINX
DATABASE REPLICA
CONSISTENT HASHING
GRPC
CDN
DYNAMODB
SHARDING
CQRS
MONGODB
RABBITMQ
LAMBDA
ROUTE 53
CLOUDFLARE
a giant collection of
PROBLEM
Sounds like
random jargon
random infrastructure
But system design is
ROLE
It's really just
three things
START WITH
A PRODUCT REQUIREMENT
"Users post photos, like, and message."
MAKE IT WORK
RELIABLY
every request, every time
WHEN USERS GET
HUGE
1
USERS
really just this.
ROLE
It's really just
three things
START WITH
A PRODUCT REQUIREMENT
"Users post photos, like, and message."
MAKE IT WORK
RELIABLY
every request, every time
WHEN USERS GET
HUGE
16,402,673
USERS
really just this.
ROLE
It's really just
three things
START WITH
A PRODUCT REQUIREMENT
"Users post photos, like, and message."
MAKE IT WORK
RELIABLY
every request, every time
WHEN USERS GET
HUGE
99,961,950
USERS
really just this.
THE MENTAL MODEL
09 THE MENTAL MODEL
8 USERS
ROLE
Build it from
scratch.
Let's build the mental
model from scratch.
SEAL
A simple social
app
01 ONE SERVER
1 USER
Imagine a simple social
app
A user opens it,
loads their feed, likes
a post, uploads a
photo, sends a message.
ROLE
Every tap is a
request
REQUESTS
GET /home
GET /feed
POST /likes
POST /photos
POST /messages
At first you run
everything on one server.
One backend, one database,
one machine.
For a thousand users,
that's completely fine.
Then traffic grows. One
server can't handle
PROBLEM
One server can't
keep up
02 SCALE BUT
250K USERS
So you scale horizontally.
Instead of one back-end
server, you run several.
But now users need
system.
That's the load balancer.
ROLE
A load balancer
It receives traffic and
distributes requests
across the healthy servers.
ROUND ROBIN
FAILED HEALTH CHECK
across the healthy
servers.
But all those servers
PROBLEM
Every server hits
one database
hit the same database
bottleneck.
03 THE DATABASE
1M USERS
So you reduce unnecessary
reads, add a cache
like Redis.
PROBLEM
One profile, read
10,000s
CACHE
Redis - in memory
DATABASE
primary - writes
READS
READS
gets requested 10,000
serve it from memory
instead of querying the
database every time.
PROBLEM
Reads keep
Growing
READS ↑
2.5M USERS
Reads keep growing, so
you add read replicas.
ROLE
Writes: primary.
Reads: replicas.
Writes, replicas handle
reads.
Eventually, even one
database cluster isn't
enough.
ROLE
Split the data by
user range
USERS
NAME ID JOINER
Ana
v_3847
2019
Ben
v_3848
2019
Chloe
v_3854
2021
Dev
v_3861
2022
Maya
v_3868
2022
Noah
v_3875
2022
Omar
v_3882
2021
Sam
v_3889
2021
Tara
v_3893
2022
Zoe
v_3903
2023
SHARD 1
Users A-M
Ara
Chloe
Maya
Ben
Dev
SHARD 2
Users N-Z
Tore
Omar
Sam
So you partition the
data, users in one
database, others go
elsewhere.
That's sharding.
84 FILES
20M USERS
Now files. You don't
want profile pictures and
videos inside your
relational database.
Put them in object
storage like S3 or
R2.
PROBLEM
Files clog the
databases
RELATIONAL DATABASE
3.8 TB
VIDEOS
BLOBS DON'T BELONG HERE
PHOTOS
AMAZON S3
STORAGE
SD ID ID
CLOUDFLARE R2
PROBLEM
For users wait on
the ocean
WHERE THE PHOTOS ARE SERVED FROM
VIRGINIA
TOKYO
ONE PHOTO, FROM TOKYO
TO THE ORIGIN IN VIRGINIA
180 ms
TO THE EDGE IN TOKYO
12 ms
CloudFront - Cloudflare: hundreds of edge cities
Then use a CDN
to cache those files
closer to users around
the world.
FIX
A CDN: copies
files near users
DS SLOW WORK
50M USERS
Now someone uploads a
SHOULD the user wait
while the back-
end processes thumbnails,
notifications, analytics,
and transcoding?
PROBLEM
The user sits and
waits
THUMBNAILS 1.2 s
NOTIFICATIONS 0.8 s
ANALYTICS 8.6 s
TRANSCODING 11.6 s
USER WAITS 14.2 S
No. So you introduce
a queue,
ROLE
A queue: do it
later
QUEUE
jobs exit here
the request finishes
quickly, and background
workers process the slow
jobs asynchronously.
PROBLEM
What is
something crashes?
06 FAILURE
50M USERS
FOUR REGIONS, ONE APP
US-EAST-1
RE-CENTRAL-1
AP-NORTHEAST-1
STATUS US-EAST-1
DATABASE
primary
DOWN
SERVERS
UNREACHABLE
REGION
Virginia
OFFLINE
One database goes down,
one region disappears.
This is where redundancy
matters.
ROLE
Redundancy:
spaces for
everything
MULTIPLE SERVERS
one dies, three keep serving
REPLICATED DATA
every write lives in 2+ places
HEALTH CHECKS
ping every 5s - pull the sick
RETRIES
failed call? try again, elsewhere
FAILOVER
primary dies, a replica takes over
Multiple servers,
replicated data, health
checks, retries, failover.
ROLE
Assume parts will
break
EXPECT
PARTS TO
BREAK.
SERVER 1
SERVER 2
SERVER 3
The system should expect
parts to break.
THE GOAL ISN'T
NOTHING EVER FAILS
IT'S
FAILURE STAYS CONTAINED
THE APP, AS SCALED COMPARTMENTS
FEED
LIKES
PHOTOS
MESSAGES
SEARCH
PROFILE
UPLOAD
ALERTS
LOGIN
COMMENTS
FELLOWS
STORIES
The goal isn't nothing
ever fails. It's failure
in one part doesn't
destroy everything.
07 OBSERVABILITY
50M USERS
Now add observability.
ROLE
Observability:
see inside
LOGS
15:57:02.114 INFO GET /feed 208 ms - user=mype
15:57:02.158 INFO POST /likes 201 ms - user=bb12
15:57:02.171 WARN cache miss - profile=maya
15:57:02.206 ERROR POST /photos 500 s - s3 timeout 3690 ms - user=ian
15:57:02.344 INFO GET /feed 208 ms - user=ian
METRICS
TRACES
load balancer 5 ms
api server 6 ms
cache - Redis 30 ms
database 30 ms
storage - S3 80 ms
queue 3 ms
Logs tell you what
happened.
Metrics tell you how
the system is behaving.
Tracing tells you how
one request moved across
services.
ROLE
Trace one
request's path
LOGS
?
METRICS
?
TRACES
?
Without those, debugging a
distributed system is a
nightmare.
08 THE MISTAKE
50M USERS
And here's where beginners
PROBLEM
Blind debugging
COST
More isn't better
COMPONENTS
6
They think system design
means adding as much
41
Microservices, Kafka,
Kubernetes, 10 databases.
That's not good system
NOT GOOD
SYSTEM DESIGN
...for an app with a thousand users?
design.
COST
Every part adds
complexity
COMPLEXITY
SHARDS +1
REPLICAS +1
CACHE +1
LOAD BALANCER +1
OBSERVABILITY +1
REDUNDANCY +1
CDN +1
STORAGE +1
SHARDS +1
REPLICAS +1
CACHE +1
LOAD BALANCER +1
Every component adds
complexity.
CACHE
CAN GO STALE
QUEUE
CAN DELIVER TWICE
REPLICAS
CAN LAG
A cache can go
stale, a queue can
deliver twice, replicas
can lag.
SHARDING
QUERIES GET HARDER
MICROSERVICES
NETWORK FAILURES
feed -x likes 504 TIMEOUT
Sharding makes queries
harder. Microservices
introduce network
failures.
09 THE METHOD
50M USERS
ONLY ADD
COMPLEXITY WHEN
IT SOLVES A
REAL
BOTTLENECK.
because it's popular
because something broke
complexity when it solves
That's the core of
system design.
THE CORE OF SYSTEM DESIGN
START SIMPLE
and servers?
FIND THE
BOTTLENECK
what broke first?
SMALLEST FIX
one component
REPEAT
as it grows
Start simple, find the
bottleneck. Introduce the
smallest component that
solves it. Repeat as
the system grows.
WHEN YOU HEAR...
LOAD BALANCER
CACHE
QUEUE
CDN
REPLICA
SHARD
DON'T MEMORIZE THE NAMES
Names are not the point
WHY WAS IT ADDED?
WHY WAS IT ADDED?
WHY WAS IT ADDED?
WHY WAS IT ADDED?
WHY WAS IT ADDED?
WHY WAS IT ADDED?
Ask one question: what
Ask: what problem forced
this?
WHAT PROBLEM FORCED US TO ADD
THIS?
One server can't keep
up
Same reads hammer the
DB
Slow work makes users
wait
Far-away users wait
Reads keep growing
One DB can't hold it all
LOAD BALANCER
CACHE
QUEUE
CDN
REPLICA
SHARD
forced us to add
this component?
SYSTEM DESIGN
101
That's system design 101.