Low-Latency Live Streaming Upgraded Based on WebRTC (CSS)

#cloud #cloudstreaming #livebroadcasting #webrtc

1.Live Event Broadcasting (LEB) Overview
The fast advancement of the live streaming industry has ignited the development of a wealth of low-latency live streaming scenarios, typically live shopping and online education. For these use cases, the key requirement is real-time audio/video interaction, which cannot be well supported by traditional HLS- and FLV/RTMP-based live streaming technologies that feature a relatively high latency of several seconds. Therefore, Live Event Broadcasting (LEB) adopts WebRTC to implement a live streaming product solution with a latency of milliseconds. Besides live shopping and online education, the product can well meet the requirement of real-time interaction at a low latency in other scenarios such as sports and game live streaming.

2.WebRTC-Based LEB Scheme
After years of development, live streaming has had a standardized linkage. streamers use PCs or mobile phones to implement audio/video capturing and encoding on the client and push streams over RTMP to the cloud platform for live streaming. Then, the audio/video data is transcoded and transferred to users' devices via FLV and HLS protocols over the CDN network. On the entire linkage, the highest latency comes from RTMP stream push, CDN transfer, caching on the device, and playback after decoding. The traditional RTMP/FLV/HLS methods are based on the TCP protocol, which means that data tends to build up under poor network connections. What's more, to defend against TCP network fluctuations, device players usually need to cache one to two GOPs to ensure smooth playback.
WebRTC is based on the RTP/RTCP protocol and leverages an excellent congestion control algorithm to ensure low latency and high performance under poor network connections in the real-time audio/video field. Exactly based on WebRTC, LEB reconstructs stream pull in LVB to implement highly compatible, cost-effective, and large-capacity low-latency live streaming. The system reuses the cloud data processing capabilities in the original live streaming architecture to make the live streaming access side and CDN edge both WebRTC-based, so that the former can receive WebRTC streams, and the latter can support WebRTC negotiation and remuxing distribution in addition to the original FLV/HLS distribution capabilities. In this way, the low-latency LEB is not only compatible with LVB's cloud media processing features such as stream push, transcoding, recording, screencapturingscreen-capturing, and porn detection, but also has the strong edge distribution capabilities of the traditional CDN network, which is able to support millions of concurrent online users. You can smoothly migrate your business from the existing LVB platform to LEB to implement low-latency live streaming applications.
LEB also uses WebRTC to achieve low latency across platforms. Most mainstream browsers, including Chrome and Safari, have supported WebRTC, so we can offer standard WebRTC capabilities via browsers. In addition, the well-established, open-source WebRTC SDK makes optimization and customization easy, allowing for the customization of an SDK with improved low-latency streaming features.

3.WebRTC Upgrade and Extension
The audio/video encoding format supported by the standard WebRTC no longer meets the requirements of the live streaming industry. Specifically, the video encoding formats supported by the standard WebRTC are VP8/VP9 and H.264, and the supported audio encoding format is Opus; however, H.264/H.265+AAC is used for audio/video stream push. In addition, to implement superior low-latency communications, the standard WebRTC doesn't support B-frame encoding, although B-frame encoding has been widely used in the live streaming industry as it can improve the compression ratio and save bandwidth costs. Therefore, transcoding is required to connect the standard WebRTC to existing live streaming systems, introducing extra latency and costs. It's necessary to upgrade the standard WebRTC to make it compatible with AAC (audio), H.265 (video), and B-frame encoding. The following details the WebRTC upgrade and extension in LEB.

4.LEB SDK and Demo
The LEB SDK uses the native WebRTC for customization and extension. In addition to the standard WebRTC, it also supports: a) decoding and playback in AAC, including AAC-LC, AAC-HE, and AAC-HEv2; b) decoding and playback in H.265, including software and hardware; c) B-frame decoding in H.264 and H.265; d) SEI callback; e) encryption disablement; f) image screencapturingscreen-capturing, rotation, and zooming. The LEB SDK optimizes the performance of the native WebRTC, such as first image frame latency, frame sync, sync, jitter buffer, and NACK policies. It removes modules irrelevant to stream pull and playback and is about 5 MB in size after packaging. It includes ARM64 and ARM32 architectures. To facilitate connection, it provides a complete SDK and demo. The demo for web shows how to pull streams via the standard WebRTC on the web, and the demos for Android and iOS come with the stream pull and playback SDK, demo, and connection documentation.

4.1 Demo for web
http://webrtc-demo.tcdnlive.com/httpDemo.html