From API to Animation: How Hotstar Tracks Audience Mood in Real Time

During live sports events, home viewers want the same visceral outlet that stadium crowds get from cheering and waving placards. Emojis provide that outlet, but when millions of users send reactions simultaneously, the infrastructure behind a simple tap becomes a serious engineering challenge.

Hotstar’s Social Feed features a "Sports Bar" where emoji swarms reflect shifting audience sentiment in real time—a boundary demand when Dhoni bats, a wicket craving when he keeps. Building this required moving away from a third-party provider to an in-house system capable of ingesting billions of submissions. The same architecture later powered a Voting feature for reality shows.

Design Constraints

Three principles guided the system’s construction:

  • Scalability: The service had to scale horizontally under unpredictable traffic spikes. Load balancers and auto-scaling handle resource provisioning.
  • Decomposition: Smaller, independent components allow each piece of the pipeline to be scaled and maintained separately.
  • Asynchronous processing: Non-blocking execution maximizes concurrency, particularly at the ingestion layer.

Handling Client Requests

Clients submit emojis via an HTTP API. To avoid keeping connections open during heavy processing, the API writes data to a message queue for offline consumption. Hotstar uses Kafka via its internal data platform, Knol, which provides high throughput, low latency and consumer group support without the operational overhead of managing Kafka directly.

Queue writes can happen one of two ways:

  • Synchronous: The client waits for acknowledgment before receiving a success response. Retries at both server and client protect against failure, making this suitable for transactional data.
  • Asynchronous: Messages are buffered locally and flushed to the queue in the background. This introduces rare data-loss risk but delivers very low latency.

Emojis tolerate minimal loss (and have seen none in practice), so Hotstar chose the asynchronous approach. Golang handles the concurrency using Goroutines and Channels: submitted messages flow into a Channel, and a Producer Goroutine periodically flushes them to Kafka. Using client libraries like Confluent or Sarama, the flush configuration was tuned to a 500ms interval with a maximum of 20000 messages per request.

Computing Aggregates

The processing goal is straightforward: consume the stream from Kafka and compute data aggregates over a small interval, small enough to feel real time to viewers. After evaluating Flink, Spark, Storm and Kafka Streams, the team selected Spark for its micro-batching and aggregation support, along with stronger community backing.

A Spark streaming job computes aggregates over 2-second batches and writes the results to another Kafka queue.

Delivering Results

Delivery uses PubSub, Hotstar’s in-house real-time messaging infrastructure for the Social Feed. A Python Kafka consumer reads from the queue at a rate tied to the Spark batch duration. It normalizes the data and forwards the most popular emojis to PubSub, where clients pick them up and render the animation.

Impact and Beyond

The system captured around 5 Billion emojis from 55.83 Million users during the ICC Cricket World Cup 2019 and has processed more than 6.5 Billion to date.

Emojis and Voting share a common problem: processing quantifiable user responses in near real time. Hotstar extended this pipeline to power the Voting feature, making the platform the sole voting destination for reality shows like Dance Plus and Bigg Boss (Telugu, Tamil, Malayalam). That infrastructure has handled around 3 Billion votes so far. The same system now also supports Polls and Trivia contests out of the box.