Slack System Design Interview: Real-Time Messaging for Millions of Persistent Connections
Slack holds millions of persistent WebSocket connections open at once so that a message typed in a channel shows up on every other member's screen in well under a second. Slack has published that its Flannel edge cache alone serves around 4 million simultaneous connections and 600,000 client queries per second at peak, and that its Vitess datastore handles roughly 2.3 million queries per second. The hard part is not storing the messages. It is keeping the fan-out fast, keeping presence and typing indicators cheap, and surviving the moment a data center blips and a few hundred thousand clients all reconnect at the same instant.
Slack is a persistent-connection problem wearing a chat app's clothes. Every logged-in client holds a WebSocket to a gateway server, and the server pushes a stream of events for the channels that client cares about. On top of that live socket you layer a normal request/response web tier for sending messages, editing, and search, plus a durable store for message history. The design splits cleanly into three planes: a real-time delivery plane built on WebSockets and a gateway fleet, an application plane of stateless web servers that write to sharded MySQL through Vitess, and an edge caching plane called Flannel that keeps hot workspace metadata (users, channels, bots) close to clients so boot and reconnect stay cheap. The interview rewards candidates who separate ephemeral state (presence, typing) from durable state (messages, membership), who shard message storage by channel rather than by whole workspace to avoid hot shards, who compute per-user per-channel unread counts without scanning history, and who have a concrete answer for the reconnect storm when a gateway dies. Slack's real system uses Solr for message search, Envoy for terminating the socket fleet, and Enterprise Grid to stitch many workspaces into one organization.
Asked at: Asked at Slack, Salesforce, Microsoft, Amazon, and most companies that run a chat, collaboration, or notification product. It also shows up as generic 'design a team chat' or 'design WhatsApp for work' prompts.
Why this question is asked
It forces you to reason about long-lived connections instead of the usual stateless request cycle, which trips up candidates who only know REST behind a load balancer. A good answer has to separate what must be durable from what can be lost, pick a sharding key that does not create a hot shard for the biggest customer, design message fan-out and unread tracking that scale with channel size rather than user count, and handle the failure mode where a whole gateway's clients reconnect at once. Interviewers like that the problem naturally branches into real-time delivery, storage, presence, search, and enterprise multi-tenancy, so they can steer toward whatever they want to probe.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Users send messages to channels (public, private) and direct messages, and see them appear in real time for all members.
- Support threaded replies hanging off a parent message, with their own reply feed and participant set.
- Show presence (who is active, away, or offline) and typing indicators within a channel.
- Track unread state per user per channel, including mention and thread badges, and mark-as-read on view.
- Support workspaces and channels, channel membership, joins and leaves, and message edits and deletes.
- Full-text search across message history that a user has access to, with recency and relevance ranking.
- Deliver notifications (push, email) for mentions and DMs when a user is not actively connected.
- Support shared channels across organizations (Slack Connect) and Enterprise Grid for large customers with many workspaces.
- Reflect edits, deletes, reactions, and pins to all connected clients without a full reload.
Non-functional requirements
- Real-time delivery latency in the low hundreds of milliseconds from send to render for connected members.
- Hold millions of concurrent persistent connections across regions with graceful failover.
- Durability for messages: an acknowledged message is never lost, even across gateway or datastore failure.
- Survive reconnect storms when a gateway or region drops without cascading into the datastore.
- Read-heavy scaling: message reads and history loads dominate writes by roughly an order of magnitude.
- Strong per-channel ordering so members see messages in a consistent sequence.
- Tenant isolation so one large customer's traffic cannot starve others.
- Search results available quickly after a message is posted, with history retained per retention policy.
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Concurrent WebSocket connections
Millions at peak (estimate)
Slack has publicly described maintaining millions of simultaneous WebSocket sessions during weekday peaks. Treat exact figures as moving targets; the design point is that connection count, not message count, is the primary scaling axis.
Flannel edge cache load
~4M connections, ~600K queries/sec (published)
Slack's Flannel post states the edge cache handles roughly 4 million simultaneous connections and 600,000 client queries per second at peak. Clients ask Flannel for user and channel objects on demand instead of loading everything at boot.
Datastore query rate
~2.3M QPS peak, 2M read / 300K write (published)
Slack's Vitess post reports about 2.3 million queries per second at peak, split roughly 2M reads and 300K writes. That 6:1 read/write skew justifies read replicas and aggressive caching of hot metadata.
Message storage growth
Billions of messages/day, growing indefinitely (estimate)
Assume tens of thousands of active channels per large tenant times sustained posting. Derived: history is append-only and never shrinks, so storage planning must assume unbounded growth and tiering of cold history, not a fixed working set.
Fan-out per message
1 write, N live pushes where N = channel members online (estimate)
A message is one durable write plus a push to each connected member of that channel. Derived: a 10,000-member channel with 6,000 online produces one insert and up to 6,000 socket pushes, which is why large shared channels are the expensive case.
Datastore latency target
~2 ms median, ~11 ms p99 (published)
Slack's Vitess numbers cite about 2 ms median and 11 ms p99 query latency. That budget is what lets the web tier assemble a channel view or an unread count inside a single user-facing request.
High-level architecture
A client boots by calling the web API over HTTPS, authenticates, and gets back a slim bootstrap payload plus a URL to open a WebSocket. It opens that socket to a gateway server, which upgrades the HTTPS connection to a WebSocket and becomes the client's live event feed. Rather than loading every user and channel object up front, the client talks to Flannel, an application-level edge cache deployed at points of presence, and pulls user, channel, and bot objects lazily as it needs them. When a user sends a message, the client hits a stateless web app server, which validates permissions, writes the message durably to the sharded datastore (MySQL fronted by Vitess), and publishes the event to the message-routing tier. That tier fans the event out to every gateway holding a connection for a member of the target channel, and each gateway pushes it down the relevant sockets. Presence and typing indicators ride the same socket layer but are treated as ephemeral, so they never touch durable storage. History reads and search go back through the web tier: message history comes from the datastore and read replicas, while search queries hit a separate Solr-based index. Enterprise Grid customers get many workspaces stitched under one organization, and Flannel's model is extended to resolve users and channels that span workspace boundaries.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Gateway / real-time servers
The fleet that terminates client WebSockets and holds them open for the life of the session. Each server tracks which channels its connected clients subscribe to and pushes matching events down the socket. Slack migrated this layer from HAProxy to Envoy so it could change backend endpoint lists without reloading and dropping long-lived connections.
Flannel edge cache
An in-region application-level cache that holds hot workspace metadata: users, channels, bots, and membership. Clients query it on demand instead of downloading the whole workspace at boot, which cut startup payloads by about 7x on a 1.5K-user team and 44x on a 32K-user team. It also absorbs reconnect traffic so returning clients hit cache rather than the origin databases.
Web application tier
Stateless PHP/Hack servers that handle sends, edits, membership changes, and history reads. They own authorization and all durable writes, then hand the resulting event to the routing tier for delivery. Being stateless, they scale horizontally behind the load balancer.
Vitess-fronted MySQL datastore
The durable store for messages, channels, and membership. Slack started with whole-workspace sharding and moved to Vitess so it could shard on finer-grained keys like channel and user id. VtGate routes each query to the right shard based on the sharding column, hiding shard topology from the application.
Message routing / fan-out tier
The internal pub/sub layer that takes a committed message event and delivers it to every gateway that has a subscriber for that channel. It decouples the write path from the delivery path so a slow or failing gateway does not block the durable write.
Search indexing and Solr
A pipeline ingests posted messages and builds an inverted index in Solr. Search runs a two-stage rank: Solr returns a candidate set ordered by selected features, and the application layer re-ranks with the full feature set, offering Recent (chronological) and Relevant (Lucene-scored) modes.
Notification service
Watches for mentions and DMs to users who are not actively connected and dispatches push and email through mobile push providers and mail infrastructure. It reconciles against presence so an active desktop user is not spammed with mobile pushes.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
messageschannel_idtsuser_idthread_tstextedited_atSharded by channel_id under Vitess. The ts timestamp doubles as the ordering key and message id within a channel. thread_ts is null for top-level messages and points at the parent for replies.
channelschannel_idworkspace_idtypenameis_sharedcreated_atOne row per channel. is_shared marks Slack Connect and multi-workspace channels, which are stored as C-encoded channels so every server and client sees one consistent channel id.
channel_memberschannel_iduser_idjoined_atlast_read_tsThe membership edge and the read cursor in one place. last_read_ts is how unread counts are computed without scanning history: count messages in the channel with ts greater than last_read_ts.
usersuser_idworkspace_iddisplay_namepresence_hinttzdeletedProfile data, cached hot in Flannel. presence_hint is a soft field; authoritative live presence lives in the real-time layer, not here.
workspacesworkspace_idorg_idnameplanretention_policyorg_id links workspaces into an Enterprise Grid organization. retention_policy drives how long history and its search index are kept.
reactionschannel_idmessage_tsuser_idemojicreated_atSharded alongside messages by channel_id so a message and its reactions live together. Reaction add and remove are pushed to connected members as their own real-time events.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
The persistent WebSocket gateway and how a client connects
Every active client keeps one WebSocket open for its whole session. The connection starts as ordinary HTTPS and then issues a protocol-switch upgrade to a WebSocket, after which the gateway streams events for the channels that client subscribes to. Slack runs separate real-time endpoints for messages, presence, and app interactions so a failure in one does not take down the others, and it exposes primary and backup socket hostnames for failover. The operational headache with this layer is that connections are long-lived, so you cannot casually restart the proxy in front of them. Slack originally used HAProxy and had to keep several old config versions running so it would not drop live sockets on a reload. Moving to Envoy fixed that: Envoy takes dynamic endpoint updates without a reload and supports hot restart that preserves existing connections, and the migration was rolled out with weighted DNS in 10, 25, 50, 75, 100 percent steps.
Flannel: the edge cache that makes boot and reconnect cheap
For a small team, a client can just download every user and channel object at startup. For a 32,000-person org that payload is huge and slow, and doing it again on every reconnect is worse. Flannel is Slack's answer: an application-level cache sitting at edge points of presence that holds users, channels, and bots in memory and answers client queries on demand. The client boots with a slimmed-down payload and asks Flannel for objects lazily, and Flannel even pushes objects proactively, for example sending a user's profile to clients in a channel the moment that user is mentioned. Flannel itself opens a WebSocket to the origin to stay current with events, and it uses consistent hashing to place a given user on a specific Flannel host so the cache stays warm for that user. Slack reported startup payload reductions of about 7x for a 1.5K team and 44x for a 32K team, and at peak Flannel handles on the order of 4 million connections and 600K queries per second.
Sharding message storage: workspace shard to Vitess
Slack's first datastore design put an entire workspace on one MySQL shard. That was simple and gave good locality, but it created a hard ceiling: when a single customer grew huge, all of their traffic funneled onto one shard, and that shard eventually hit the biggest hardware available with nowhere left to grow. The fix was Vitess, a clustering layer on top of MySQL that lets you shard on a chosen column without baking shard logic into the application. Slack moved from whole-workspace sharding toward finer keys such as channel id and user id, so one giant customer's load spreads across many shards instead of hammering one. Queries flow through VtGate, a routing tier that knows the table set, which backends host them, and the sharding column, so it sends each query to the right shard. Slack has cited roughly 2.3 million queries per second at peak (about 2M reads, 300K writes) with median latency near 2 ms and p99 near 11 ms.
Presence and typing indicators are surprisingly expensive
Presence looks trivial and is not. A naive design pushes a status change to everyone who might see that user, so in a large workspace one person going from away to active can trigger a broadcast to thousands of clients, and multiply that by everyone toggling all day. The trick is to treat presence as ephemeral, keep it out of the durable store entirely, and only compute it for people a given client is actually looking at. Clients subscribe to presence for the users currently visible in their UI rather than the whole roster, and the server can batch and rate-limit updates. Typing indicators are the same shape: high-frequency, low-value, and safe to drop. They are fire-and-forget events on the socket with short expiry, never written to disk, so losing one costs nothing. Getting this wrong is a classic way to melt the fan-out tier with traffic that no user would even notice missing.
Fan-out and unread counts per user per channel
Sending a message is one durable write followed by a push to each online member of the channel, so cost scales with channel size, not user count. Small channels are trivial. The expensive case is a very large channel or a broadly shared channel where thousands of members are online at once, which is why huge channels get special attention. Unread state is the other subtle piece. You do not want to store a read flag per message per user, and you do not want to scan history to count unreads. Slack keeps a last-read cursor per user per channel (a last_read_ts), and the unread count is simply the number of messages in that channel newer than the cursor. Mentions and thread activity are tracked as their own lightweight badges. This keeps read state to one small row per membership edge and makes mark-as-read a single cursor update rather than a bulk write.
Search over message history with Solr
History search is a separate system from the message store because the access pattern is different: full-text ranking over petabytes with per-user access control. Slack builds an inverted index in Solr through an indexing pipeline that ingests posted messages. Search offers two modes. Recent finds messages matching all terms and returns them newest first. Relevant relaxes recency and weighs the Lucene score of how well a document matches the query. To rank well without pulling everything into the application, Slack uses a two-stage approach: Solr returns a candidate set ordered by a few selected features, then the application layer re-ranks that set using the full feature set. Access control has to be enforced at query time so a user only ever sees results from channels they belong to, which means the index and the permission model have to stay in sync as membership changes.
The reconnect storm and how to avoid a thundering herd
The scariest failure mode is correlated reconnection. If a gateway or a whole region drops, every client on it reconnects at nearly the same moment. If each of those clients tries to re-fetch its full workspace state and re-establish everything at once, the surge lands directly on the databases and can turn a small blip into an outage. Slack attacks this from several sides. Flannel absorbs the reconnect traffic by serving boot and metadata from its warm regional cache instead of the origin, so returning clients hit memory, not MySQL. Reconnection is spread over time with jittered client backoff rather than everyone retrying on the same tick, and the server side rate-limits new connections. What gets sent on reconnect is deliberately minimized to essential channel metadata rather than full history. And because DNS spreads new connections across several nearby regions, the load of a failed region does not all pile onto one replacement, which itself dampens the herd.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
WebSocket push versus HTTP long-polling
Persistent WebSockets give true low-latency bidirectional delivery and are the natural fit for chat, but they turn connection count into your dominant scaling problem and make proxy restarts and failover hard. Long-polling is simpler to load-balance but wastes round trips and adds latency. Slack chose sockets and then invested heavily (Envoy, Flannel) to make the connection layer operable.
Whole-workspace sharding versus channel/user sharding
Workspace sharding gives great locality and simple queries, but a single massive customer becomes a hot shard with no room to grow. Channel and user sharding via Vitess spreads that customer across many shards and kills the hotspot, at the cost of cross-shard queries and more routing complexity. Slack moved to the finer grain once big customers hit the ceiling.
Ephemeral presence versus durable presence
Storing presence durably would make it queryable and consistent, but the write and fan-out volume is enormous and the data is stale within seconds. Treating presence and typing as ephemeral, in-memory, and best-effort trades perfect accuracy for a massive drop in load, which is the right call for data no user relies on being exact.
Lazy load via Flannel versus full state at boot
Loading the whole workspace at startup is simple and means the client has everything locally, but it is unusable for large orgs and brutal on reconnect. Lazy loading through an edge cache cuts payloads by up to 44x and tames reconnect storms, at the cost of running and keeping consistent a whole extra caching tier at the edge.
Separate search store (Solr) versus querying the message database
Running full-text ranking against the primary datastore would contend with the live send/read path and scale poorly. A dedicated Solr index is optimized for retrieval and ranking but adds an indexing pipeline, eventual consistency between post and searchability, and the burden of enforcing access control at query time.
Two-stage search ranking versus single-pass ranking
Ranking everything with the full feature set inside the search engine is expensive and inflexible. Letting Solr do a cheap first-pass ordering and then re-ranking a bounded candidate set in the application keeps the engine fast and puts richer ranking logic where it is easy to iterate, at the cost of a second hop and a capped candidate window.
How Slack actually does it
The specifics here come from Slack's own engineering writing. Flannel is described in Slack's 'Flannel: An Application-Level Edge Cache to Make Slack Scale', including the users/channels/bots cache, consistent-hashing host placement, and the 7x and 44x payload reductions. The move off whole-workspace sharding onto Vitess, the VtGate routing tier, and the ~2.3M QPS with 2ms median and 11ms p99 figures are from 'Scaling Datastores at Slack with Vitess'. The socket-layer migration from HAProxy to Envoy, the protocol upgrade path, and the separate real-time endpoints for messages, presence, and app interactions are from 'Migrating Millions of Concurrent Websockets to Envoy'. Search modes (Recent versus Relevant) and the two-stage Solr ranking come from 'Search at Slack'. Shared channels as C-encoded channels and Flannel's extension to external users come from 'How Slack Built Shared Channels'. Enterprise Grid supporting up to 500,000 users per organization is from Slack's product material.
Sources
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Related system design interview questions
Practice these next. They lean on the same core building blocks as Slack.