Multi Stream in SDK 3.5 Using Categories

What is Multi-stream?

With multi-stream, you'll see the video of the most active participants in your meetings.

NOTE: The below new APIs are available from SDK version 3.5.0 onwards.

• Stream order will be allocated in the order of joining the meeting.

• 2 Categories are present - Category-A, Category-B.

• Category-A: It will contain 1 stream and it will be the active speaker participant in the meeting.

• Category-B: It will contain all other streams in the meeting. It can also contain the duplicate active speaker stream.

• Category-B can be restricted to contain the number of streams. If more participant joins the meeting above the limit they will go into the queue.

• If someone from the queue will speak he will get into the allocated stream and will replace the stream who is less active in the meeting.

• If someone from the allocated stream leaves the meeting, the random participant from the queue will replace the stream.

• The Active Speaker API can be used to check if the participant is an active speaker.

• The ActiveSpeakerChangedEvent is triggered if active speaker changes.

Typical Scenario

In a meeting with more than two participants, if you want to see the active speaker along with other joined participants, you can use multi-stream to achieve it.

How to Use

To implement multi-stream, the client register callback onMediaStreamAvailabilityListener . The following are the main steps you should follow.

 oall.onMediaStreamAvailabilityListener = { available, stream in 
     //if available {
     //     new stream available
     //} else {
     //     stream unavailable
     //}
 }

1. Provide a view for rendering the stream

When there is a newly available stream, such as a new participant joining the meeting, the above onMediaStreamAvailabilityListener calback will be fired and the client should provide the view to SDK for rendering the video of the stream.

// stream is MediaStream object returned by callback
 let view = MediaRenderView()
 stream.renderView = view

2. OnMediaStreamInfoChanged Listener

We support different event types - MediaStreamChangeEventType. If any of the types changes concerning the stream, the event will get triggered.

enum MediaStreamChangeEventType {
    case Size
    case Membership
    case Video
    case Audio
}

The client can register the info change listener to receive the event change.

// stream is MediaStream object returned by onMediaStreamAvailabilityListener callback
stream.setOnMediaStreamInfoChanged { type, info in
                    //type: MediaStreamChangeEventType
                    //info: MediaStreamChangeEventInfo
                }

3. setMediaStreamCategoryA

We can add the active speaker stream with the specified parameters if it does not already exist otherwise, update the active speaker stream with the specified parameters. Category-A will contain only 1 stream and it will always be an active speaker stream.

// duplicate if true, the same active speaker will also be included in Category-B streams.
    fun setMediaStreamCategoryA(duplicate: Boolean, quality: MediaStreamQuality)

enum MediaStreamQuality {
    case Undefined
    case LD  // 180p
    case SD  // 360p
    case HD  // 720p
    case FHD // 1080p
}

4. setMediaStreamsCategoryB

Set all Category-B streams to the specified params.

• If the number of existing B streams is less than numStreams, it will add B streams as necessary.
• If the number of existing B streams is more than numStreams, it will remove the extra B streams.

    fun setMediaStreamsCategoryB(numStreams: Int, quality: MediaStreamQuality)

5. removeMediaStreamCategoryA

Remove the Active Speaker stream. To add the active stream back, use setMediaStreamCategoryA().

• After removal of Category-A, if Category-B exists, all streams will be of the same priority and streams will not change based on the active speaker.

    fun removeMediaStreamCategoryA()

5. removeMediaStreamsCategoryB

Remove all Category-B streams. To change the number of B streams, use setMediaStreamsCategoryB().

• After removal of Category-B, if Category-A exists then only the active speaker stream will be visible and the participant who speaks will be visible in the Category-A stream.

    fun removeMediaStreamsCategoryB()