The hard part of live video review
Dropbox Replay's Live Review feature aims to recreate the feel of an in-person screening session for distributed teams. In that room, anyone can pause, scrub, or jump to a different frame at any moment. But a roomful of people issuing conflicting playback commands at once raises a deceptively tricky problem: how do you guarantee that every participant lands on the same frame at the same time?
Live Review sessions have to distinguish between two kinds of state. Single client state—like a cursor position or a drawing on a frame—can't conflict with other participants, so it just gets echoed to the server and out to the rest of the room. Shared client state is what keeps playback synchronized across every participant, and it's where the real coordination challenge lies.
The concurrency problem
Each client in a Live Review session opens a WebSocket connection (Dropbox uses the open-source Gorilla Go library) to the Replay server. The playback state itself is encoded with Protocol Buffers, which pair well with WebSockets: Protocol Buffers don't handle message framing, while WebSockets are content-agnostic and provide framed messages delivered reliably and in order, not to mention built-in ping/pong heartbeats that keep sessions alive through proxies.
Naively broadcasting every state change to everyone breaks down quickly when two people interact at once. Consider Patty and Steven: Patty skips to frame 120 and sends that update, Steven skips to frame 240 and sends his update before Patty's arrives, then each applies the other's change on arrival. Patty ends up at frame 240, Steven ends up at frame 120, and both believe the other's command came after their own.
Timestamps could help distinguish old messages, but only if all client clocks are synchronized—rarely a safe assumption. A simpler route is to let the server impose order rather than the clients.
The server as logical clock
The sync service already receives every message from every client. Using it, Replay can establish a happened-before relationship between playback messages without any client-side clock coordination: the order in which the server receives a message determines the order in which it broadcasts that message to everyone. Message ordering is therefore canonical across the whole session.
The challenge is then the flip side of the same coin: the server must also process incoming messages in the exact order they arrive. As long as each incoming playback message gets broadcast to all clients before the server starts processing the next, a strict total order exists.
type PlaybackState struct {
frame int64
rate float32
}
...
// Each connection is assigned a unique sessionId
// Handled by a seperate Goroutine
// Protobuf messages are decoded from binary websocket messages
func handleMessage(msg *SyncMessage, sessionId string) {
switch messageType := msg.GetMessageType().(type) {
...
case *Message_Playback: {
// mutex is a sync.RWMutex
mutex.Lock() // Lock the state or wait if another session is holding the lock
// Update the state
playbackState = PlaybackState{
frame: frame,
rate: rate,
}
// Send the new state to all users, including the sender
SendMessageToAllIncludingSender(sessionId, msg)
mutex.Unlock() // Unlock only once we've sent the message to all clients
}
...
}
}That alone solves the inconsistency problem, but it degrades the experience on one side. A crucial design choice makes better behavior possible: the server echoes every message back to the client that originally sent it. This lets each client know precisely when its own command has achieved canonical ordering.
Consider the earlier example with echoing in place. Steven starts paused at frame 120, presses play, then sees Patty's pause at frame 0 arrives; both converge at frame 0. But Patty starts paused at frame 120, skips to frame 0, and then—because Steven's play command was processed before her skip—briefly sees the video start playing from frame 120 before it jumps back to frame 0. Both clients end at the same frame, but Patty's screen flickers through a state she never asked for.
Skipping the stale state
Once a client knows the server echoes its own message, dropping stale updates becomes possible. Anything Patty receives between sending her skip and receiving her own echo must have been processed by the server before her skip. Those commands happened first, so her client can safely discard them.
The playback experience for Steven stays the same: paused at frame 120, press play, play, then Patty's pause at frame 0. Patty's is now live too: paused at 120, skip to frame 0, and stays paused there. No blinking, no phantom playback from an earlier frame.
Other designs could have reached the same convergence as the server-mediated then-echo approach—for instance, event replay across clients or operational transformation—but the solution that matters most is the one that produces the smoothest flow for the person watching a review session. Thinking through the participant experience first is what pointed the team toward a simpler algorithm rather than a more elaborate one.
