Twitch System Design Interview: Live Video Ingest, Transcode, and Chat at Scale
Twitch runs tens of thousands of concurrent live channels at once, with peaks reported in the range of 6 million concurrent viewers watching more than 70,000 live streams (industry estimates, not an official real-time figure). Every one of those broadcasters pushes a single high quality RTMP stream that has to be transcoded into a full bitrate ladder and fanned out to viewers within a couple of seconds. The chat side is its own beast: Twitch has said its chat delivers hundreds of billions of messages per day.
Twitch is a live video platform, which makes it a very different design problem from Netflix or YouTube VOD. A broadcaster pushes one live RTMP stream, and Twitch has to accept it at a nearby point of presence, route it to an origin, transcode it into multiple renditions in real time, package it as HLS segments, and push those segments through a CDN to potentially millions of viewers, all while keeping glass-to-glass latency in the low single digit seconds. On top of that sits a real-time chat system that has to deliver a firehose of messages per channel with strict ordering and moderation. The hardest parts are the transcode fleet economics, the thundering herd when a huge streamer goes live and every viewer requests the first segment at once, keeping live latency low without breaking CDN cacheability, and building a chat fanout tier that can hold tens of millions of persistent connections. Twitch built custom systems for most of this, including Intelligest for ingest routing and a purpose built transcoder, and a Go based chat edge and pubsub layer. A good interview answer separates the video plane from the chat plane and treats them as two independent scaling problems that happen to share a channel identity.
Asked at: Asked at Amazon (which owns Twitch), YouTube, Meta, and most companies that run live video or real-time messaging. It also shows up at streaming startups and any interview loop that wants to probe live media, CDN behavior, and websocket fanout.
Why this question is asked
Interviewers like this problem because live video breaks a lot of the assumptions candidates lean on. You cannot pre-encode and cache everything like VOD, so the transcode fleet is on the critical path and its cost model matters. Latency is a hard constraint that fights directly with CDN cacheability, so the candidate has to reason about that tension instead of hand waving. It also has two very different real-time subsystems, video and chat, which tests whether someone can decompose a product into independent planes rather than one monolith. Finally, the thundering herd on a popular stream is a clean, concrete way to see if a candidate understands caching, request coalescing, and origin protection.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Let a broadcaster push a single live stream over RTMP (or newer protocols) from encoding software like OBS.
- Transcode the ingest stream into a bitrate ladder (for example 1080p60, 720p, 480p, 360p, 160p) plus a source passthrough rendition.
- Package renditions as HLS (and DASH) segments with a manifest so any player can adapt bitrate to its bandwidth.
- Deliver segments to millions of concurrent viewers through a CDN with the lowest practical latency.
- Provide per-channel chat where any viewer can post messages that all other viewers in the channel see in near real time.
- Support moderation: bans, timeouts, slow mode, subscriber-only mode, and automated filtering before messages are delivered.
- Record the live stream to VOD so it can be replayed after the broadcast ends.
- Show near real-time viewer counts, follows, subscriptions, and channel metadata.
- Support clips, where a viewer captures a short segment of a live or recorded stream.
Non-functional requirements
- Glass-to-glass latency in the low single digit seconds for the low latency profile, ideally 2 to 4 seconds.
- High availability for ingest and playback; a viewer should never see the platform hard-fail even if one origin dies.
- Elastic transcode capacity that scales with the number of concurrent broadcasts, which is spiky and event-driven.
- Chat message delivery within a few hundred milliseconds and correct per-channel ordering.
- Global reach with points of presence close to both broadcasters and viewers to cut network latency.
- Cost efficiency on transcode, since each stream consumes several CPU cores continuously while live.
- Graceful degradation: if the highest rendition cannot be produced, viewers still get source or a lower rendition.
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Concurrent broadcasts
~70,000 to 100,000 live channels at peak
Public trackers report Twitch regularly running tens of thousands of live channels. Each live channel is an independent ingest plus transcode pipeline, so this number, not viewer count, drives transcode fleet size. Estimate.
Transcode compute per stream
~4 to 6 CPU cores for a 1080p60 ladder
Real-time transcode of a 1080p60 source into a five rung ladder is roughly 4 to 6 cores per stream on general purpose CPUs. At 70,000 concurrent streams that is on the order of 300,000 to 420,000 cores of transcode capacity. Derived estimate; Twitch offloads much of this to custom hardware and FPGAs.
Peak concurrent viewers
~6 million+
Widely cited peak figure across popular events. Viewers are served almost entirely from CDN edge, so this scales with edge fleet and cache hit ratio rather than origin. Estimate, not an official real-time metric.
Chat messages per day
Hundreds of billions
Twitch has publicly described chat delivering hundreds of billions of messages per day. Delivery, not ingestion, is the load: one message in a channel with 1 million viewers is 1 million deliveries, so fanout dominates.
Video egress bandwidth
Multiple Tbps at peak
6 million viewers at an average sustained bitrate near 3 Mbps is about 18 Tbps of egress (6e6 * 3e6 bits/s). Real average is lower because many watch sub-1080p renditions, but it lands in the multi-Tbps range served by CDN edge. Derived estimate.
Segment size and manifest churn
2 second segments, manifest refreshed every ~2s
HLS chops each rendition into short segments (2 seconds for low latency). A one hour VOD at 2 second segments is 1,800 segments per rendition, times roughly six renditions equals about 10,800 objects per hour per channel to store. Derived.
High-level architecture
A broadcaster's encoder opens an RTMP connection to the nearest Twitch point of presence, not directly to a data center. At the PoP, an ingest gateway (Twitch calls its system Intelligest) terminates the RTMP stream, inspects its properties, and asks a routing service where to send it. The routing service picks an origin data center based on capacity, health, and locality, and the media proxy forwards the stream there. At the origin, a transcode fleet decodes the source and re-encodes it into a ladder of renditions at different resolutions and bitrates, then a packager cuts each rendition into short HLS segments and writes a manifest. Those segments are pushed out to the CDN. Viewers never hit the origin directly: their player fetches the manifest and segments from a nearby edge server, which caches each segment for a short window so that thousands of viewers of the same channel are served from one cached copy. Adaptive bitrate logic in the player watches its download speed and switches renditions up or down. In parallel, and completely independent of the video path, the chat system runs its own edge tier that holds each viewer's persistent connection (IRC over TCP or WebSocket); a message posted in a channel goes to an edge node, through a pubsub layer that fans it out to every other edge node holding a connection for that channel, after passing moderation checks. When the broadcast ends, the recorded segments are stitched into a VOD asset in object storage for later replay.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Ingest gateway (Intelligest media proxy)
Runs at each PoP and is the first Twitch component the broadcaster's RTMP stream touches. It terminates the live stream, extracts stream properties, and forwards media to the chosen origin. Terminating at the edge keeps the fragile broadcaster connection short and lets Twitch make smart routing decisions before committing to a data center.
Ingest routing service
A stateful control plane service (Twitch runs its IRS in AWS) that decides which origin each incoming stream should go to. It uses capacity, health, and rule-based policies so that compute is packed efficiently and a failing origin can be steered around. This is what lets Twitch drive its transcode fleet near full utilization at peak.
Transcode fleet
At the origin, this fleet decodes the single source stream and re-encodes it into multiple renditions in real time. Twitch built a purpose designed transcoder rather than relying only on stock FFmpeg because per-stream cost is the dominant economic factor at their scale. Each active stream consumes several cores continuously, so this fleet is the most expensive part of the system.
Packager and origin
Takes encoded renditions and cuts them into short HLS and DASH segments, writes and continuously updates the manifest, and serves as the authoritative source the CDN pulls from. It also writes segments to storage for VOD. The origin is protected from viewer load by the CDN in front of it.
CDN and edge cache
A large fleet of edge servers close to viewers that cache each segment for a short window (tens of seconds) and serve it to every local viewer of that channel. This is what makes one million viewers of a stream affordable: the origin produces each segment once and the edge multiplies it. Cache hit ratio at the edge is the single biggest lever on cost and origin protection.
Chat edge tier
A fleet of Go services that hold each viewer's persistent connection over IRC-style TCP or WebSocket. It handles connect, join, and message send, and delivers messages down to connected clients. It is the tier that has to survive holding tens of millions of open sockets, so it is designed to be horizontally sharded and stateless beyond the connections it owns.
Chat pubsub and moderation pipeline
An internal pubsub layer fans a message out from the edge node that received it to every edge node that has a subscriber for that channel, forming a hierarchical distribution tree that keeps per-message work low. Before delivery, a business-logic pipeline (Twitch calls its component Clue) applies checks like ban status, subscriber-only mode, and slow mode, and automated moderation filters the message.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
channelschannel_idowner_user_idstream_keytitlecategory_idis_liveOne row per broadcaster channel. The stream_key authenticates the RTMP push and must be revocable. is_live plus current origin location is updated when a broadcast starts and ends.
live_streamsstream_idchannel_idstarted_atorigin_dcingest_poprendition_ladderOne row per broadcast session. Tracks which PoP ingested it and which origin transcodes it, so the control plane and support tooling can find a live stream's physical location.
vod_assetsvod_idchannel_idstream_idstorage_prefixduration_screated_atCreated when a stream is recorded. storage_prefix points at the object-storage location holding the segment files and manifests; the actual bytes live in blob storage, not the database.
chat_messagesmessage_idchannel_iduser_idbodysent_atflagsChat is delivery-first, so messages are streamed through pubsub rather than read from a database on the hot path. A durable copy is written asynchronously for moderation history, replay, and analytics. flags carries badges and moderation state.
moderation_statechannel_iduser_idstateexpires_atmoderator_idPer-channel per-user moderation records: bans and timeouts with an expiry, and channel-level modes like subscriber-only. Read on the message path by the business-logic pipeline, so it is cached aggressively at the edge.
subscriptions_followsviewer_idchannel_idtypetiercreated_atRecords whether a viewer follows or subscribes to a channel, and at what tier. Used to grant chat privileges (subscriber-only mode, badges) and to drive notifications when a followed channel goes live.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
Ingest routing and why it is not just a load balancer
The naive design points broadcasters at a data center behind a generic load balancer like HAProxy. Twitch moved away from that because a live video stream is long-lived, stateful, and expensive to move once it lands, so the routing decision needs to happen up front and be smart about it. Their Intelligest system terminates the RTMP stream at a nearby PoP with a media proxy, then a separate stateful routing service decides which origin should receive it based on real-time capacity, health, and configurable rules. The payoff is that Twitch can pack streams onto transcode capacity tightly and drive utilization close to 100 percent at peak, and it can steer new streams away from a degraded origin without disturbing existing ones. In an interview, the key insight to surface is that ingest routing is a scheduling and bin-packing problem over an expensive fleet, not a stateless request-spraying problem.
Real-time transcoding and the bitrate ladder
A broadcaster sends one high quality stream, but viewers are on everything from fiber to a phone on cellular, so the origin has to produce a ladder of renditions such as 1080p60, 720p, 480p, 360p, and 160p, plus a source passthrough. This has to happen live, in real time, with no ability to fall behind, which is fundamentally different from VOD where you can transcode offline at your leisure. Cost dominates the design: a single 1080p60 ladder can take roughly four to six CPU cores continuously, and Twitch runs tens of thousands of concurrent streams, so a few percent efficiency is enormous money. That is why Twitch built its own transcoder and invested in hardware acceleration rather than running stock FFmpeg everywhere. A subtlety worth mentioning is transmuxing versus transcoding: the source rendition can often be repackaged (transmuxed) without re-encoding, which is nearly free, while the lower rungs require full re-encoding. Not every small channel gets the full ladder; transcoding is prioritized so popular streams always get all renditions and tiny streams may get source only.
HLS delivery, segment caching, and the latency versus cacheability tension
Playback uses HLS, which chops each rendition into short segments listed in a manifest that the player refreshes. The reason HLS scales is that it is plain HTTP, so ordinary CDN edge servers can cache segments and serve millions of viewers without the origin seeing that load. The tension is latency. Longer segments cache better and survive network hiccups, but they add delay because the player buffers whole segments. Shorter segments cut latency but increase manifest churn and request rate. Twitch tunes this toward short segments (around 2 seconds) and short edge cache windows (tens of seconds), long enough that many viewers of the same channel share a cached copy but short enough that the edge never serves stale data after a stream ends. The mental model to give an interviewer is that live HLS is a sliding window of a handful of recent segments, and the whole system is optimized so the common case is an edge cache hit on the newest segment.
Low latency live without breaking the CDN
Standard HLS historically ran 10 to 30 seconds behind real time, which is fine for a movie but bad for interactive streaming where chat reacts to what just happened. Twitch documented bringing latency down from around 15 seconds to a low latency profile targeting roughly 2 to 4 seconds, with the best regions getting close to 1.5 seconds. They did it with a custom low latency HLS scheme: the manifest advertises upcoming segment URLs before those segments fully exist, and when the player requests a future segment the server holds the request open and streams bytes as they are produced (chunked transfer), so the player starts playing the moment content is available instead of waiting for a manifest refresh cycle. The hard part is doing this while keeping the delivery cacheable at the edge, because a blocking request that streams partial content is not a normal cache object. The interview point is that low latency live is a codesign of the packager, the player, and the CDN, not a single knob.
The thundering herd when a big streamer goes live
When a huge channel starts or a raid sends a flood of viewers at once, hundreds of thousands of players ask for the same first segment within a second or two. If those requests all miss the edge cache and hit the origin, they can overwhelm it. The defenses are layered. First, the edge caches segments so the vast majority of that herd is served from one cached copy. Second, request coalescing (single flight) at the edge means that if a segment is not yet cached, only one request goes to the origin while the rest wait for that single fetch to populate the cache. Third, because live playback is a sliding window, all viewers converge on the same small set of newest segments, which is the best possible case for caching. The failure mode to call out is a cache stampede on segment expiry: if a hot segment expires and a thousand requests arrive before the refill completes, you get a spike, which is exactly what single-flight coalescing and short staggered TTLs are designed to prevent.
Chat fanout: holding tens of millions of connections
Chat is a separate system from video and arguably harder to reason about because the load is delivery, not ingestion. One message in a channel with a million viewers is a million deliveries, so a popular channel is a massive fanout. Twitch built this in Go as two tiers: an edge tier that holds each viewer's persistent connection (IRC over TCP or WebSocket) and a pubsub tier that distributes each message from the edge node that received it to every edge node that has a subscriber for that channel. Together they form a hierarchical distribution tree so no single node has to know about every connection. The design constraints are the sheer number of open sockets, which pushes you toward many horizontally sharded stateless edge nodes, and per-channel ordering, which you preserve by routing a channel's traffic through a consistent path. Before a message is delivered it runs through a business-logic pipeline that checks ban and timeout state, subscriber-only and slow mode, and automated moderation, all of which must be fast because they sit on the hot path of every message.
VOD recording, clips, and storage economics
Because segments are already being produced for live delivery, recording a VOD is largely a matter of persisting those segments plus their manifest to object storage rather than transcoding again. That keeps recording cheap, but it creates a storage volume problem: a single one hour broadcast at 2 second segments across six renditions is on the order of ten thousand objects or more, and Twitch has huge daily broadcast hours. So storage tiering matters, with hot recent VODs on fast storage and older content moved to cheaper tiers or expired per retention policy. Clips are a related feature: a viewer marks a short window, and the system extracts those existing segments into a small standalone asset rather than re-encoding from scratch, which is what makes clip creation feel instant. The takeaway for an interview is that live-first architecture gives you VOD and clips nearly for free if you persist the segments you already generated.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
Custom ingest routing (Intelligest) versus a generic load balancer
A generic layer 4 or 7 balancer treats connections as interchangeable, but live streams are long-lived and expensive to place, so Twitch built a stateful router that makes a smart origin choice up front. The cost is significant engineering and operational complexity, justified only because packing an expensive transcode fleet efficiently saves far more than it costs.
Custom transcoder versus stock FFmpeg everywhere
FFmpeg is flexible and free, but at tens of thousands of concurrent streams the per-stream CPU cost dominates the entire budget, so a purpose-built transcoder and hardware acceleration pay for themselves. The tradeoff is losing FFmpeg's generality and taking on the burden of maintaining a codec-heavy system in house.
HLS over HTTP versus a stateful streaming protocol like WebRTC
WebRTC can hit sub-second latency but is peer-oriented and does not ride ordinary CDN caching, so scaling it to millions of viewers is expensive and complex. HLS gives up some latency in exchange for riding the entire existing HTTP CDN ecosystem, which is the only affordable way to reach millions. Twitch closes most of the latency gap with custom low latency HLS instead of switching protocols.
Short segments and short cache TTL versus long segments
Short segments cut latency and make the sliding window tight, but they raise request rate and manifest churn. Long segments cache well and are resilient but add buffering delay. Twitch leans short because interactivity with chat is core to the product, and it absorbs the higher request rate with strong edge caching and coalescing.
Separate chat plane versus one unified real-time system
Video and chat have completely different traffic shapes: video is high bandwidth from few producers to many consumers via cacheable HTTP, while chat is low bandwidth many-to-many over persistent sockets. Keeping them as independent planes lets each scale on its own axis. The cost is coordinating shared concepts like channel identity and viewer presence across two systems.
Delivery-first chat with async durability versus write-then-read from a database
Reading every chat message from a database before delivery would add latency and a huge read load, so Twitch streams messages through pubsub and persists a durable copy asynchronously for moderation and history. The tradeoff is that the durable log lags slightly behind what viewers saw, which is acceptable for chat but would not be for something like payments.
How Twitch actually does it
Twitch has been unusually open about its architecture. The company documented Intelligest, its ingest routing system that terminates RTMP at PoPs and routes to origins via a stateful routing service, in its 2022 engineering post on ingesting live video at global scale, noting it replaced HAProxy and lets them approach full transcode utilization at peak. Its 2021 post on low latency, high reach describes bringing latency from around 15 seconds down to a 2 to 4 second low latency profile using a custom low latency HLS approach with future-segment URLs and blocking requests. Its 2017 transcoding series explains why they built a custom transcoder instead of relying on stock FFmpeg, driven by per-stream cost. On the chat side, Twitch has described a Go-based system with an Edge tier speaking IRC over TCP and WebSocket, a Pubsub tier that performs hierarchical fanout, and a business-logic component (Clue) for checks like bans and subscriptions, delivering hundreds of billions of messages per day. These are real systems described by Twitch itself, and a strong candidate cites the split between the cacheable HTTP video plane and the persistent-connection chat plane as the central architectural decision.
Sources
- Twitch Blog: Ingesting Live Video Streams at Global Scale (Intelligest, 2022)
- Twitch Blog: Low Latency, High Reach (2021)
- Twitch Blog: Live Video Transmuxing/Transcoding, FFmpeg vs TwitchTranscoder (2017)
- Twitch Blog: Twitch Engineering, An Introduction and Overview (chat, 2015)
- Twitch Blog: State of Engineering 2023
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Related system design interview questions
Practice these next. They lean on the same core building blocks as Twitch.