Live Streaming workflow

Live streaming has become a ubiquitous part of our digital lives, allowing us to broadcast events, share experiences, and connect in real time. In this article, we'll delve into the essentials of live streaming, covering protocols and technologies, latency considerations, and how to set up a live-streaming server.

In this article, we'll dive into the traditional RTMP and HLS-based streaming workflows, how FFmpeg handles video encoding, and the system design behind it. We'll also explore the evolution towards low latency streaming using protocols like SRT, WHIP, and WHEP, which are increasingly used for real-time requirements such as sports broadcasts.

RTMP and HLS-Based Flow

1. Ingest: The video stream is captured by a camera or encoding software and sent to the RTMP server. This is the first step where the live video is ingested into the streaming infrastructure.
2. Processing: The RTMP server processes the incoming stream and makes it available for further encoding and distribution.
3. Queue: The stream is then sent to a message queue (e.g., Kafka, RabbitMQ) for scalable processing.
4. Encoding: Workers consume the stream from the queue and use FFmpeg to transcode it into different formats and bitrates.
5. Segmenting and Packaging: The encoded streams are segmented into smaller chunks suitable for HTTP Live Streaming (HLS).
6. Delivery: These segments are then distributed via a CDN to the end-users.

Latency: The typical glass-to-glass latency (time from camera capture to playback on the user’s device) in this workflow is around 10 seconds.

Low Latency Streaming: SRT, LL-HLS WHIP, and WHEP

1. SRT/WHIP Source: The video stream starts from an SRT or WHIP source (camera/encoder).
2. Media Server: The stream is ingested by the media server optimized for low latency.
3. Message Queue: The stream is placed onto a message queue for low-latency processing.
4. Worker Nodes: Worker nodes pull the stream from the queue, and use FFmpeg or similar tools to transcode with low latency settings.
5. Delivery (WHEP or LL-HLS): The stream is delivered via WebRTC or low-latency HLS (LL-HLS) to the end users.