SRS Cloud: For multiple path router. Multi-path streaming, aggregation router solution, OpenWRT+MIPS support.

[winlinvip]

First, the most important thing: Currently, SRS does not support aggregation routers, and is still in development, so please stay tuned...

Now, there will be a live broadcast every week, mainly for SRS Q&A, pushing RTMP streams to video numbers, and sometimes network jitter can cause stuttering.

If the uplink push stream has poor network quality, it will cause everyone to freeze. Therefore, live broadcasts basically need to guarantee uplink push streams, unless it is a scenario where a network camera streams 24 hours a day and no one watches.

Solution

The current market solution is aggregation routers, which generally involve plugging multiple network cards into a router and deploying a dedicated server to aggregate the streams from these network cards and then push them to the live source station. There are several issues with this:

1. This is a solution from router manufacturers, and it is a completely physical layer multi-path transmission. In fact, if you do live streaming-specific multi-path, application layer multi-path push streaming would be simpler and allow for more business processing. Refer to MPTCP.
2. Multi-path does not have redundancy. If a link has poor quality or jitter, a more complex algorithm is needed to handle it. For example, if the RTT of a link changes from 10ms to 100ms, it is quite complicated to calculate the link quality, as some packets do not show the final result. If redundant push streaming is used, you can simply look at whether the FPS of these streams is normal and choose a stream with a more stable FPS for forwarding.
3. The server is generally deployed by the router manufacturer, and it is more troublesome to deploy it yourself, and it is a link layer server. It is more intuitive to deploy a dedicated video server, which will have many video indicators.

So, I think the application layer aggregation router solution would be better. For a specific solution, refer to: Architecture Diagram

Goal

This is an open-source aggregation router solution, with the main goals being:

• Provide an open-source aggregation router solution to achieve highly reliable live streaming.
• An example of using SRS on OpenWRT, which can be extended to other scenarios, such as enterprise SRS gateways.
• SRS supports OpenWRT (MIPS) platform, and ST supports MIPS.

This is not a commercial product, and the goal is not to produce a directly commercializable product. Because to truly make it a commercial product, many things need to be considered, and this is not something an open-source project should do.

What open source needs to do is to run the link and show various possibilities. Main tasks:

• SRS supports OpenWRT, modifies ST to adapt to MIPS system, refer to #2637
• SRS supports MPTCP multi-path transmission.
• SRS supports MP-QUIC multi-path transmission.

HiWifi Y1 + EC20

HiWifi has a low configuration but is widely available, with a USB port that can plug in an EC20 4G network card, enabling WiFi + 4G dual-channel transmission.

First, flash the HiWifi Y1 to the OpenWRT system, refer to the following OpenWRT Install.

Then, you need to install the USB 4G network card driver. Windows uses RNDIS, and Linux uses CDC-ECM. The difference is explained here. In OpenWRT, you can install it like this:

opkg update
opkg install kmod-usb-net-rndis

Next, restart the router. Plug the EC20 into the USB port, and you can see that it is recognized, and a new network card called usb0 is available:

root@OpenWrt:~# dmesg
[   99.149068] usb 1-1: new high-speed USB device number 2 using ehci-platform
[   99.229163] rndis_host 1-1:1.0 usb0: register 'rndis_host' at usb-101c0000.ehci-1, RNDIS device, fa:50:57:e0:1e:d6

root@OpenWrt:~# ifconfig usb0
usb0      Link encap:Ethernet  HWaddr 86:41:4A:15:66:2A  
          BROADCAST MULTICAST  MTU:1500  Metric:1

Finally, add a new network interface.

OpenWRT Install

If the official download is slow, you can download the flashing image and kmod from the domestic Tencent or Tsinghua mirror sites.

Note: kmod cannot be self-compiled, and an error Cannot satisfy dependencies / kernel is not compatible will be reported. You must use the corresponding version and hash number with the kernel, which means you can only install and not compile kmod separately. Note that using --force-depends --force-checksum is also not allowed.

For the flashing image, select the version and your router model from the download page. You can find it from Targets, for example, HiWifi Y1 is ramips/mt7620, and then find the corresponding brand image, such as hiwifi_hc5661 on the download page:

• https://downloads.openwrt.org/releases/21.02.0/targets/ramips/mt7620
• http://mirrors.cloud.tencent.com/openwrt/releases/21.02.0/targets/ramips/mt7620

After flashing the router, you can first connect to another WiFi to access the network, and then you can use opkg to install.

First, back up the official opkg source and set it to the Tencent Cloud mirror:

cp /etc/opkg/distfeeds.conf /etc/opkg/distfeeds.conf.bk-`date +%s`
sed -i 's|downloads.openwrt.org|mirrors.cloud.tencent.com/openwrt|g' /etc/opkg/distfeeds.conf

Finally, update and install:

opkg update
opkg install kmod-usb-net-rndis

Explain other packages that may be used:

• kmod-usb-net-cdc-ether: This is the USB network card driver, CDC ECM mode. Installing kmod-usb-net-rndis will install this package by default. For details, please check CDC-ECM and RNDIS.
• kmod-usb-serial-option: If you need to operate the EC20 module, such as AT serial command operation /dev/ttyUSB2, set the EC20 mode, you need to install this module. Generally, this module is not needed.
• libstdcpp: This is the C++ dynamic library. If the program uses C++, you need to install this library.
• libopenssl: This is the openssl dynamic library, which is used for encryption and decryption.
• gdb: You can debug programs on OpenWRT.

OpenWRT Build

The router is definitely based on OpenWRT, and basically, all current routers are developed based on this open-source project.

Since OpenWRT is quite large, downloading it in China is either slow or fails. I found some mirror sources to speed up the download. Please refer to openwrt-docker.

This Docker is an Ubuntu 20 environment. The Dockerfile will download the OpenWRT code and the dependent feeds code. After the image is built, you can directly configure and compile it. Subsequently, you can directly git pull to update the code, which is much faster.

Downloading to docker is because compiling OpenWRT requires a Linux file system, and the file systems of Mac and Windows are not case-sensitive.

OpenWRT: srs-server

SRS can run on OpenWRT, refer to Build package srs-server.

ST(state-threads) supports MIPS, refer to #21.

Start the SRS server:

/usr/bin/srs-server -c /etc/srs/conf/openwrt.conf

Push the stream to SRS:

ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://192.168.1.1/live/livestream

Watch HTTP-FLV: http://192.168.1.1:8080/live/livestream.flv

MPTCP

Currently, live aggregation routers use a transport layer multi-path transmission solution. Related projects are as follows:

• Open MPTCP Router, an integrated live aggregation router that includes the MPTCP protocol stack, OpenWRT, and other projects. Official website: openmptcprouter.com
• RFC8684: MPTCP, the RFC standard document for MPTCP. Official website: multipath-tcp.org
• Kernel: Multi-Path TCP, kernel MPTCP module.
• Multipath UDP tunnel, multi-path UDP tunnel.

MPTCP is a protocol based on TCP, as shown in the figure below:

The following is the architecture diagram:

The main purpose of multi-path transmission is to increase the transmission bandwidth, and incidentally, improve the jitter resistance.

Here is a test plan Wireless 4G Video Transmission Solution, using 3 4G network cards with a speed limit of 800Kbps to transmit a 2Mbps live stream. Note that when plugging and unplugging the network card, there will be a second-level freeze and mosaic (maybe commercial products can solve this).

MP-QUIC

Zhihong mentioned that QUIC also has a multipath version, referring to multipath-quic.org. The official says that the quic-go QUIC library already supports it.

I think there are several key advantages of MP-QUIC over MPTCP:

1. Application layer protocol, no need to go through a very long upgrade chain, App can use it by writing its own code, which greatly speeds up the popularization. No need to modify the kernel, routers, or underlying libraries, it can be done at the application layer.

2. UDP protocol, which is naturally easy to support. QUIC itself has also considered connection migration, but not simultaneous transmission. From the RFC, we can see that MP-QUIC is only 28 pages, while MPTCP has 58 pages.

3. QUIC is the foundation of the future Internet, and multipath can be applied far beyond just TCP, and can be used by all applications.

The impact this can bring is very different. It's fast, simple, and widespread.

Commercial Products

There are quite a few aggregation router products on the market, mainly aimed at increasing transmission bandwidth, and of course, supporting transmission fault tolerance.

• LiveU Solo, Simultaneously bond up to 4 network links including support for 2x 4G LTE cellular or WiMAX modems, Wifi, and Ethernet, supporting 4 parallel network transmissions (2x 4G, Wi-Fi, wired). Compatible with domestic mainstream online media such as Mudu Live, Sohu Video, Tencent Video, Weibo, WeChat, Bilibili, Youku Tudou, and other overseas mainstream content distribution networks and online video platforms.
• TVU One, Reliably transmits high picture quality video even in congested or challenging environments, Uses patented IS+ transmission algorithm, Support for HDR and 1080P60 encoding at as little as 800kbps, Long internal battery life, HEVC/H.265 video compression, CNCed from high-density aluminum. Can dynamically aggregate bandwidth from different network connections, including cellular, fiber, satellite, WiFi, and Ethernet.
• Teradek BondFamily, The Bond Backpack is an all-in-one professional HEVC/H.264 streaming solution for broadcasters seeking the best in signal reception and video quality. High Gain Antennas, Node Modems, Gold or V Mount Battery Plate, Integrated Locking Cables.

[online2311]

I can't understand some of the multi-line configurations for the SRS live broadcast aggregation router.

[MattUan]

Looking at your architecture diagram, the approximate effect achieved is that the SRS server shunts the push and the receiving end selects the best link for forwarding? In this case, it serves as a redundancy, while MPTCP serves to stack the speed of multiple links, such as stacking three 10M broadband lines to achieve 30M to increase available bandwidth. I've looked at the latest documentation for openmptcprouter, and it seems to have added hot standby redundancy features. I haven't studied the specific implementation yet, but in any case, it should be a different direction from what you're saying. However, integrating SRS aggregation should effectively reduce fault points and lower latency?

[winlinvip]

@MattUan Yes, it's different from MPTCP. If it were the same, there would be no need to do it, and MPTCP would be good enough.

What I want to do mainly addresses the redundancy issue when bandwidth requirements are met, and of course, redundancy can also be achieved with multiple camera positions.

In addition, MPQUIC may be more suitable than MPTCP, but it is not yet mature.