can we configure different camSimulcastEncodings on two different users of the same call in SFU?

Hi!

while creating callObject, can we configure camSimulcastEncodings with 3 layers(maxBitrate: 80000, 200000, 680000) for desktop users and camSimulcastEncodings with 2 different layers (maxBitrate: 80000, 520000) for mobile users?

for desktop:

camSimulcastEncodings: [

{

maxBitrate: 80 * 1000,

scaleResolutionDownBy: 4,

maxFramerate: 10

},

{

maxBitrate: 200 * 1000,

scaleResolutionDownBy: 2,

maxFramerate: 15

},

{

maxBitrate: 680 * 1000,

scaleResolutionDownBy: 1,

maxFramerate: 24

}

]

for mobile:

camSimulcastEncodings: [

{

maxBitrate: 80 * 1000,

scaleResolutionDownBy: 4,

maxFramerate: 10

},

{

maxBitrate: 520 * 1000,

scaleResolutionDownBy: 1,

maxFramerate: 24

}

]

Best Answer

  • vr000m
    vr000m Dailynista
    Answer ✓

    Related to 1.

    Network Link Conditioner should work, what are you observing when the link conditioner is turned on?

    I usually only set latency or packet loss impediments and not bandwidth limitations. if you set bandwidth throttling, this can be a hit or a miss. Do make sure that you are applying the impediments before starting the tests, that way, you can compare with baseline, when you did not have the link conditioner turned on. One other thing to check is -- if the impediment applies to all traffic on the system or local area or external traffic, i.e., in the peer-to-peer if you are running the test between two tabs or browsers in the same computer, making sure the impediments are applied to it. I apologies if what I am saying is too generic, my suggestion would be to open another thread with screenshots or a loom video of what is happening. So that experts from our team: @Dom@vanessa and @Dom could weigh in.


    Related to 2.

    In Peer-to-peer, there is no simulcast, as Web browser endpoints (incl. React Native) cannot receive simulcast, just send simulcast. For simulcast, the connection has to be through the SFU which will receive the simulcast streams and depending on track subscriptions and available bandwidth to a particular participant, send the appropriate simulcast stream (low, medium, high quality) to that user.

    Assuming the sender is sending 3 simulcast streams (example the settings in your original post/screenshot: camSimulcastEncodings of 80kbps, 200kbps, 680kbps, so a total of 960 kbps, and resolutions of 180p, 360p, 720p, respectively).

    The sending endpoint simultaneously sends the single video camera capture encoded at those particular resolutions to the SFU. The SFU then decides based on the available network of the receiving user, what resolution to send. So if the receiving user has low capacity, lets say 100kbps, it would only receive the lower quality video from this sender. If another user had 1Mbps, it would receive the highest quality stream. So with an SFU. endusers with differing network performance would get the appropriate quality stream from the sender and the sender does not need to worry about the receiver's network.

    In a similar vein, if the sender does not have 1Mbps to carry the aggregate of 960kbps video, the sender will automatically detect and send fewer simulcast streams, so the steps would be as follows:

    • if the sender has higher than, it will send all three simulcast streams
    • if the sender has above 280kbps but lower than 900kbps, it will send the two lower streams of 200kbps and 80kbps, the aggregate is 280kbps and turn off the highest stream of 680kbps
    • if the sender has about 100kbps send only the lowest stream and turn off all other simulcast streams, namely, the 200kbps and 680kbps
    • if the sender has lesser than 80kbps, sender will turn off video sending.

    I hope this sheds light on how simulcast works at the sender and helps receivers with varying network performance.

Answers

  • vr000m
    vr000m Dailynista
    edited November 2022

    Yes. The intuition and the API that you’re using is correct. We do recommend that low powered devices such as mobile or IoT use two simulcast streams, it’s less cpu and battery intensive for these limited-power devices. One can manipulate the settings by initializingcamSimulcastEncodings with the appropriate settings.

    See our large call guide in the docs and I’ve added a screenshot of the slightly different recommended settings for desktop and mobile, respectively.


  • Hi @vr000m

    Thanks 🙂

    I have two questions.

    1. I have used chrome throttle settings and Mac's Network Link conditioner, both were not helpful. Can you also suggest how to throttle Network?
    2. I have read this doc  large call guide, I have a question here. if the call is in Peer to Peer and resolution settings are made as shown below by using setBandwidth() method. if the user is having bad network, what feed he would be receiving from other user? will the resolution and bandwidth of the receiving video scales down to suits the network quality of the user? Can you please explain how it affects the call if the user is having low network but kbs and trackConstraints are configured for high resolution as below?

    callObject.setBandwidth({

    kbs: 1660,

    trackConstraints: {

    width: 1280,

    height: 720,

    frameRate: 24,

    },

    });

  • Hi @vr000m

    Thanks. now I got understand how exactly it works.