Questions regarding quality and fuzzy streams

[pablo67340]

Hello! I am currently using this library in combination with my Java Discord bot that supports SiriusXM Satellite Radio using ffmpeg, so this was a perfect drop-in solution for me(launching the node script per server instance blah blah). It works great!! I have managed to find some "WEB-x264" mp4 files (usually the smaller ones, like 150-250MB per episode), which play really well and has no audio lag. I think it's .mkv. Essentially I have an automated process that drops my .mkv file into a special folder inside my htdocs, so the bot is able to connect to the file and stream it. That said, the ONLY thing I am dealing with now, is quality fuzziness. My question is:

The discord server I use is fully upgraded with Nitro (just a group of friends and I pay for), and it doesn't appear to increase our stream quality, but it got me thinking, should my user account also be Nitro? Would this help fuzziness? Or is this a programmatic issue? It feels like the bitrate is way lower, than the 384 we have our server set to.

Are there any tricks I can use with ffmpeg to re-encode media to ensure the B-Frames don't mess up the audio? Is bitrate considered while streaming? What is the most optimal video/format I can find for a video? (e.g mp4, NO AAC, blah blah, libopus) So many questions gahhh!

Thank you so much - to everyone who's helped contribute to this library to make it what it is now! Absolutely outstanding work, I've always wanted to stream a video, or my 3D printer streams into my 3D Printing Discord servers. This is the way!

If I can figure out the quality issues, I'll probably re-implement this library onto VLCJ for Java as well, so the Java folks out there can use this too.

[dank074]

The discord server I use is fully upgraded with Nitro (just a group of friends and I pay for), and it doesn't appear to increase our stream quality, but it got me thinking, should my user account also be Nitro?

If your server is already maxed out on boosts then all the users within your server should have HD streaming. Regardless, Discord does not actually enforce the rule at the back-end, and your bot can send HD video even in a non-boosted server and without any Nitro Subscription (at your own risk, since this could raise suspicions from Discord and lead to an account ban on your self bot)

Would this help fuzziness? Or is this a programmatic issue? It feels like the bitrate is way lower, than the 384 we have our server set to.

Could also be the following flags that are affecting your video quality:
-tune zerolatency
-preset ultrafast

You can pass your own preferred ffmpeg flags by doing something similar to this example: https://github.com/dank074/Discord-video-stream/tree/master/examples/custom-stream-copy-codec

Are there any tricks I can use with ffmpeg to re-encode media to ensure the B-Frames don't mess up the audio?

You can use the flag -bf 0 to remove b frames, but this has nothing to do with the audio. The audio and video are sent separately

Is bitrate considered while streaming?

Bitrate is considered while streaming, the higher the bitrate, the better the quality usually, but other ffmpeg flags can also affect it (like the ones I listed above).

What is the most optimal video/format I can find for a video? (e.g mp4, NO AAC, blah blah, libopus)

The most optimal way to stream the video is probably to have it pre-encoded in the format you wish to stream it (if you are planning to stream h264 video for example, have it be an mp4 video file with h264 video) and then just copy the stream without doing any re-encoding. See https://github.com/dank074/Discord-video-stream/tree/master/examples/custom-stream-copy-codec for an example of how to do this

[pablo67340]

The discord server I use is fully upgraded with Nitro (just a group of friends and I pay for), and it doesn't appear to increase our stream quality, but it got me thinking, should my user account also be Nitro?

If your server is already maxed out on boosts then all the users within your server should have HD streaming. Regardless, Discord does not actually enforce the rule at the back-end, and your bot can send HD video even in a non-boosted server and without any Nitro Subscription (at your own risk, since this could raise suspicions from Discord and lead to an account ban on your self bot)

Would this help fuzziness? Or is this a programmatic issue? It feels like the bitrate is way lower, than the 384 we have our server set to.

Could also be the following flags that are affecting your video quality: -tune zerolatency -preset ultrafast

You can pass your own preferred ffmpeg flags by doing something similar to this example: https://github.com/dank074/Discord-video-stream/tree/master/examples/custom-stream-copy-codec

Are there any tricks I can use with ffmpeg to re-encode media to ensure the B-Frames don't mess up the audio?

You can use the flag -bf 0 to remove b frames, but this has nothing to do with the audio. The audio and video are sent separately

Is bitrate considered while streaming?

Bitrate is considered while streaming, the higher the bitrate, the better the quality usually, but other ffmpeg flags can also affect it (like the ones I listed above).

What is the most optimal video/format I can find for a video? (e.g mp4, NO AAC, blah blah, libopus)

The most optimal way to stream the video is probably to have it pre-encoded in the format you wish to stream it (if you are planning to stream h264 video for example, have it be an mp4 video file with h264 video) and then just copy the stream without doing any re-encoding. See https://github.com/dank074/Discord-video-stream/tree/master/examples/custom-stream-copy-codec for an example of how to do this.

As for the b-frames, I was referring to a response I saw: #72 (comment)

And after more research, it looked liked WebRTC is unable to send B-Frames regardless. When I checked my mp4 file that had severe audio lag from the immediate beginning, i ran ffprobe on the file and sure enough it's packed full of b-frames pretty much from right after the second frame. Not sure if it was a coincidence, or what was causing the mp4 to playback weirdly, but the audio was super delayed compared to the video. It was h264 AAC Audio. I tried another h264 without AAC and this worked fine. So this is why, with the research from WebRTC, and that one response, and my test results, it pointed towards B frames, or something with encoding still not running smoothly. I also saw a PR response about a delay in frames: #72 (comment)

I am probably completely wrong, just trying to learn the boundaries, and optimal settings for files to throw at this. So far it seems like it heavily depends on the type of h264 file I throw at it. I'd like to see how smooth we can get it!

Thank you so much for the quick reply! I will mess around with my ffmpeg, but also check into the alternative example showing the stream copy method, I should be able to create an adaption exactly to what I need. Much appreciate the information, definitely learned a ton!

[dank074]

And after more research, it looked liked WebRTC is unable to send B-Frames regardless. When I checked my mp4 file that had severe audio lag from the immediate beginning, i ran ffprobe on the file and sure enough it's packed full of b-frames pretty much from right after the second frame. Not sure if it was a coincidence, or what was causing the mp4 to playback weirdly, but the audio was super delayed compared to the video. It was h264 AAC Audio. I tried another h264 without AAC and this worked fine. So this is why, with the research from WebRTC, and that one response, and my test results, it pointed towards B frames, or something with encoding still not running smoothly. I also saw a PR response about a delay in frames: #72 (comment)

That's interesting and I wonder if it's related to #52 (comment)

[dank074]

You mind sharing some of the mp4 files that were having delayed audio issue? @pablo67340

[pablo67340]

Absolutely! If possible, please add me on Discord so I can shoot over some links.

Discord: @Vajoner

[longnguyen2004]

WebRTC doesn't support B-frames, I tried stream copying a Twitch stream and it was very glitchy due to the B-frames.

Regarding quality, the library defaults to using x264 ultrafast preset, which is fast but is pretty low quality. You can copy the streamLivestreamVideo function and either change the preset, or switch to using a hardware encoder (be sure to add -bf 0 to disable B-frames).