Netflix System Design Masterclass
Discover the engineering blueprints behind the world's leading video streaming service. Learn how Netflix orchestrates AWS computing resources and physical **Open Connect Edge Appliances** inside ISPs globally to deliver high-fidelity video streams to **260+ million active subscribers** concurrently under absolute zero network friction.
The Operational Scale & Requirements
Netflix is not a standard video catalog; it is a global data distribution engine that at peak hours accounts for **over 15% of the entire world's internet traffic**. Scaling to this requirement requires solving extremely hard challenges: seamless video startup speeds (sub-second playing latency), zero buffering under active network degradation, massive horizontal concurrency, and resilience against physical internet fiber cuts.
Engineering Objectives
- • Sub-Second Playback: Buffer and decode the initial movie segment inside 800ms of clicking "Play".
- • Adaptive Bitrate Control: Dynamically change video resolution (H.264/HEVC encoding profiles) millisecond-by-millisecond depending on internet speed.
- • Active Edge Delivery: Route bandwidth-heavy video assets directly from inside the client's Internet Service Provider (ISP), bypassing backbone web lines.
- • Infinite Fault Tolerance: Guarantee continuous playback even if an entire AWS geographical server datacenter collapses.
Expected Scale Parameters
The Split Plane Topology (Control vs. Data)
Netflix scales by strictly segregating their architectures into two completely isolated operational levels: the **Control Plane** (residing entirely in AWS to handle search, user accounts, recommendation engines, and transcoding pipelines) and the **Data Plane** (residing inside local ISPs on custom Open Connect Edge Appliances to deliver raw video binaries directly).
The AWS Control Plane
The Control Plane resides in Amazon Web Services (AWS) using stateless microservices orchestrated over custom platform APIs. This handles low-bandwidth, transactional operations: processing search inputs, streaming recommended movie queues, processing monthly billing transactions, and orchestrating video media transcoding tasks.
The Open Connect Data Plane
When a user clicks "Play", the high-bandwidth demand (streaming massive 4K video files) completely bypasses AWS servers. The client app requests video segments directly from localized **Open Connect Appliances (OCAs)** installed physically inside the user’s local Internet Service Provider (ISP) network, cutting latency and global traffic costs.
Open Connect Global CDN
Instead of building standard datacenters or leasing third-party cloud CDN bands (which would cost billions in egress traffic), Netflix created **Open Connect**—a custom hardware global network. Netflix builds specialized server machines (Open Connect Appliances - OCAs) and installs them directly inside ISP local routing centers for free, delivering movie data locally.
How caching on Open Connect operates:
When Netflix releases a popular title like *Stranger Things*, they do not stream it from a single central database server to millions of active screens simultaneously.
- During off-peak night hours (when internet usage is low), Netflix pushes the encoded video media files from **AWS S3** to all **Open Connect Appliances (OCAs)** around the world.
- The OCA (configured with high-capacity flash SSD storage arrays and optimized FreeBSD cores) saves the files locally.
- The next evening, when millions of users in London tap "Watch Stranger Things", their client requests are dynamically redirected by the Zuul proxy to the **closest local ISP OCA** physically in London.
- Video streams originate directly from their local ISP exchange—**consuming zero global bandwidth** and avoiding buffering lags.
Cosmos Media Transcoding Pipeline
A movie filmed by Netflix is delivered in a high-fidelity master format that is several terabytes in size. This massive asset cannot be streamed directly. Netflix must convert it into hundreds of different profiles (codecs, resolutions, and bitrates) to suit diverse devices. This is managed by the **Cosmos Media Pipeline**—a massive parallel encoding framework in AWS.
1. Dynamic Video Chunking
Transcoding a whole 2-hour 4K movie as a single file on one machine would take hours and fail easily. Instead, the Cosmos workflow engine splits the master file into thousands of small, **5-second video chunks**. These chunks are encoded in parallel across a massive cluster of AWS worker instances, reducing encoding times from hours to minutes.
2. Manifest Compilation
After all chunks are successfully encoded into hundreds of codec and bitrate variations, the system merges them, compiling a master **Manifest File (.m3u8/.mpd)**. When the client taps watch, the device reads the manifest to dynamically request the best matching bitrate chunks (switching between SD, HD, and 4K) as network conditions fluctuate.
Active-Active Cassandra Replication
When you pause a video on Netflix on your phone and immediately open it on your TV, your playback position is updated instantly. This state must be persisted globally with high durability. Netflix relies on **Apache Cassandra** set up in a highly scalable, **Active-Active Multi-Region Replication** topology across several AWS datacenters.
Decentralized Design
Cassandra uses a masterless peer-to-peer ring topology. There is no single master node to bottleneck write limits. User viewing histories can be written to *any* local replica node. The cluster handles consistency in the background, replicating updates to other geographical regions asynchronously.
Dynamic Conflict Resolution
Bi-directional write paths create the risk of data conflict (e.g. updating play position from two devices simultaneously). Cassandra uses the **Last-Write-Wins (LWW)** conflict resolution algorithm: the write with the latest client physical timestamp overwrites older states, maintaining perfect coherence.
EVCache Caching Layer
Reading user catalogs, movie descriptors, and dynamic recommendation matrix algorithms from primary databases on every app load would crash Cassandra under high loads. Netflix shields databases with **EVCache** (Episodic Volatile Cache)—a highly customized, distributed in-memory cache system built on top of Memcached.
Linear Scalability
EVCache scales horizontally by adding Memcached instances to the cluster. High-bandwidth cache keys are automatically distributed across the nodes using **consistent hashing rings**, maintaining perfect stability under massive request surges.
Asynchronous Updates
When a primary Cassandra database write occurs (e.g. updating viewing logs), the application service triggers asynchronous EVCache invalidation requests, keeping cached objects in sync without blocking transaction writes.
Region Localized Caching
Every AWS geographical datacenter hosts an independent physical EVCache ring. Cache hits execute locally under 1ms, completely eliminating regional backbone data traffic overhead.
Chaos Engineering & Chaos Kong Failovers
Netflix is the pioneer of **Chaos Engineering**. Instead of waiting for hardware components to break in production unexpectedly, Netflix intentionally breaks its own production systems to verify self-healing resilience. Their ultimate defense drill is **Chaos Kong**—an automation that shuts down entire AWS geographic regions to test warm-failovers.
How regional warm failover functions under 5 mins:
When **Chaos Kong** triggers, it fully blocks an AWS geographical cloud zone (e.g. US-East). In most systems, this causes global downtime. On Netflix:
- Route 53 DNS servers detect the outage and start shifting global network traffic immediately.
- DNS routes 100% of the active connection requests from US-East to **US-West (Oregon)**.
- The stateless API servers in Oregon launch new instances dynamically to handle the massive traffic surge.
- Because Cassandra database records are mirrored bi-directionally, the US-West Cassandra cluster already has the user viewing history.
- Users continue watching their streams with **zero interruption or session logouts**.
System Design Interview Cheat Sheet
Use this cheat sheet to confidently tackle high-scale video streaming system design interview questions.
Q1. How does Netflix scale to deliver high-bandwidth video files without crushing internal networks?
Answer: Netflix utilizes a **Split-Plane architecture**. While transactional controls (search, user recommendations, profiles) are handled by stateless microservices in AWS, all high-bandwidth video assets are served via a custom global CDN called **Open Connect**. Physical **Open Connect Appliances (OCAs)** are installed directly inside local ISPs for free. These OCAs store cached video segments locally and stream them directly to subscribers over local ISP networks, bypassing backbone internet channels.
Q2. How is adaptive bitrate streaming orchestrated on the backend?
Answer: Master video files are ingested by the **Cosmos media framework** in AWS. Cosmos splits the file into thousands of **5-second chunks** and encodes them in parallel across a cluster of worker nodes into multiple formats (HEVC, AVC) and bitrates. The system compiles a **Manifest File (.m3u8/.mpd)**. During playback, the client device reads the manifest file and dynamically requests the best bitrate chunk to match real-time network speeds, ensuring zero buffering.
Q3. Why is Apache Cassandra used instead of transactional relational databases for playback state tracking?
Answer: Tracking real-time user playback states requires massive write throughput. A centralized relational database would throttle under lock queues. Apache Cassandra uses a **decentralized, masterless peer-to-peer ring topology** with no single point of failure, enabling linear write-scaling. Viewing history is written to local nodes and replicated across regions asynchronously in the background.
Q4. How is session persistence guaranteed if an entire cloud database region goes down?
Answer: By deploying a highly resilient **Active-Active Cassandra Multi-Region Replication** structure. When a user updates their state (e.g. pause playhead), the write goes to their closest active region and replicates asynchronously to a secondary region. If Region 1 crashes, Route 53 DNS redirects client traffic to Region 2 instantly. The stateless API servers in Region 2 read the user's data from the replicated Cassandra ring, resuming playback seamlessly.
Unified End-to-End System Flow
Below is the ultimate, unified architecture topology representing the complete Netflix ecosystem. It traces how metadata requests route to AWS control panels while raw video playback streams directly from Open Connect Edge CDNs.
Video Streaming Journey: Step-by-Step Lifecycle
Off-Peak Asset Pre-Caching (Data Plane Hydration)
Before any title is streamed by subscribers, Netflix's AWS **Cosmos parallel encoding clusters** fetch multi-terabyte master files from **S3**. Cosmos chunks them into 5-second increments and compresses them into hundreds of adaptive codec/bitrate variations. During off-peak night hours, these assets are pre-cached over ISP leased connections directly onto the solid-state storage arrays inside the local **Open Connect Edge Appliances (OCAs)**.
User Discovery, Searching, & API Navigation
When you open the Netflix app, your device sends HTTP catalog browsing, search inputs, and authorization requests to our **Zuul Ingress Proxy**. Zuul authenticates your session tokens, validates permissions, and forwards routing demands straight to AWS stateless microservice clusters (Control Plane) under **< 5ms**.
Shielded Caching & Dynamic Recommendation Lookup (EVCache Layer)
To prevent choking database endpoints under million-user peak spikes, microservice queries are intercepted by the high-throughput **EVCache** distributed cache layer. EVCache searches consistent hashing rings, returning compiled homepages, recommendations, and active movie lists under **< 2ms** with a **99%+ cache hit rate**.
Global State Playback Persistence (Cassandra Multi-Region Replication)
When you watch or pause a title, real-time playhead bookmarks are written continuously to local nodes in the closest geographical **Apache Cassandra Multi-Region Database**. Cassandra’s masterless ring design scales write workloads linearly and uses peer-to-peer pipelines to replicate your session states across regions asynchronously in the background.
Manifest Verification & Close-Proximity Redirects
The moment you click the watch play button, the AWS control plane returns a specialized **Manifest File** mapping out the locations of nearby cached video chunks. Your device’s connection requests are intercepted by Zuul and dynamically routed straight to the **closest physical Open Connect Appliance (OCA)** situated directly inside your local ISP exchange.
Zero-Lag Video Playout (OCA Delivery Edge)
The localized **Open Connect Appliance (OCA)** receives the request and immediately streams the 5-second video chunks directly over your local ISP’s network using optimized high-performance FreeBSD socket engines. Because the traffic does not travel through global backend internet backbones, video playback loads in **< 800ms** with zero buffering.
Real-time Adaptive Bitrate Adjustment
During active video playout, your client device continually monitors current download speeds and packet loss frequencies. As network capacity changes, the player seamlessly switches between different video profiles outlined in the manifest, requesting higher or lower quality 5-second chunks from the OCA on-the-fly to guarantee zero playout interruption.