Multi Stream in SDK 2.8 Using Aux Streams

What is Multi-stream?

With multi-stream, you'll see the video of the most active participants in your meetings. Each stream is referred to as an auxiliary stream. You can open a certain number of auxiliary videos to display active participants. The most active speaker who is talking now will display on the main remote video view, and the other active speakers will display on the auxiliary video views.

Typical Scenario

In a meeting with more than two participants, if you want to see the active speaker along with other joined participants, you can use multi-stream to achieve it.

Limitations

You can only open 4 auxiliary video streams at most. You cannot specify a stream showing a participant, and which is only determined by the activity of the participants.

How to Use

In order to implement multi-stream, the client should implement MultiStreamObserver protocol. The following is the main steps you should follow.

class MyVideoCallViewController: UIViewController,MultiStreamObserver {
    var onAuxStreamAvailable: (()-> MediaRenderView?)?
    var onAuxStreamChanged: ((AuxStreamChangeEvent) -> Void)?
    var onAuxStreamUnavailable: (() -> MediaRenderView?)?

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
        self.call.multiStreamObserver = self
    }
}

1. Provide a view to open an auxiliary stream

When there is a new available auxiliary stream, such as a new participant join in the meeting, SDK will callback onAuxStreamAvailable and the client should give SDK a view for rendering, and the auxStreamOpenedEvent would be triggered indicating whether the stream is successfully opened.

var onAuxStreamAvailable: (() -> MediaRenderView?)? = {
    // Callback when a new multi stream media is available. Return a MediaRenderView let the SDK open it automatically.
    return self.auxiliaryStreamUI.filter({$0.notInUse}).first?.mediaRenderView
}

2. Provide a view to close the auxiliary stream

When there is an auxiliary stream unavailable, such as a participant left the meeting and the number of joined participants is smaller than the number of opened streams, SDK will callback onAuxStreamUnavailable and the client should give SDK a view handle which will be closed or if the given view is null, SDK will automatically close the last opened stream if needed.

var onAuxStreamUnavailable: (() -> MediaRenderView?)? = {
   return self.auxiliaryStreamUI.filter({$0.inUse}).last?.mediaRenderView
}

3. Display the view of the auxiliary stream

When an auxiliary stream is opened successfully or not, auxStreamOpenedEvent will be triggered. On this event, the client can display the view of the auxiliary stream if the result is successful.

4. Hide the view of the auxiliary stream

When an auxiliary stream is closed successfully or not, auxStreamClosedEvent will be triggered. On this event, the client can hide the view of the auxiliary stream if the result is successful.

var onAuxStreamChanged: ((AuxStreamChangeEvent) -> Void)? = { event in
    switch event {
        // Callback for open an auxiliary stream results
        case .auxStreamOpenedEvent(let view, let result):
            switch result {
                case .success(let auxStream):
                    self.updateAuxiliaryUIBy(auxStream: auxStream)
                case .failure(let error):
                    print("========\(error)=====")
            }
         // This might be triggered when the auxiliary stream's speaker has changed.
         case .auxStreamPersonChangedEvent(let auxStream,_,_):
             self.updateAuxiliaryUIBy(auxStream: auxStream)
         // This might be triggered when the speaker muted or unmuted the video.
         case .auxStreamSendingVideoEvent(let auxStream):
             self.updateAuxiliaryUIBy(auxStream: auxStream)
         // This might be triggered when the speaker's video rendering view size has changed.
         case .auxStreamSizeChangedEvent(let auxStream):
             self.updateAuxiliaryUIBy(auxStream: auxStream)
         // Callback for close an auxiliary stream results
         case .auxStreamClosedEvent(let view, let error):
             if error == nil {
                 self.closedAuxiliaryUI(view: view)
             } else {
                 print("=====auxStreamClosedEvent error:\(String(describing: error))")
             }
     }
}