Using Video Effects

[NGame1]

Hi there,
Could anyone help me in using video effects?
I want to know where can I exactly learn how to use them. for example, I'm trying to a blur effect on an area on the video but I wasn't able to do it.
in some cases, I could randomly blur the whole video but not an area.
and also about mixing multiple filters with each other.
Maybe if someone can provide an extension library for them it should be nice.

public static class FFMpegVideoHelpers
{
    /// <summary>
    /// Crop the input video to given dimensions.
    /// </summary>
    /// <param name="source"></param>
    /// <param name="width">The width of the output video. It defaults to iw. This expression is evaluated only once during the filter configuration, or when the ‘w’ or ‘out_w’ command is sent.</param>
    /// <param name="height">The height of the output video. It defaults to ih. This expression is evaluated only once during the filter configuration, or when the ‘h’ or ‘out_h’ command is sent.</param>
    /// <param name="x">The horizontal position, in the input video, of the left edge of the output video. It defaults to (in_w-out_w)/2. This expression is evaluated per-frame.</param>
    /// <param name="y">The vertical position, in the input video, of the top edge of the output video. It defaults to (in_h-out_h)/2. This expression is evaluated per-frame.</param>
    public static void CropVideo(this FFmpegMediaSource source, int width, int height, int x, int y)
    {
        source.SetFFmpegVideoFilters($"crop=w={width}:h={height}:x={x}:y={y}");
    }

    /// <summary>
    /// 
    /// </summary>
    /// <param name="source"></param>
    /// <param name="sizeX"></param>
    /// <param name="sizeY"></param>
    /// <param name="planes"></param>
    public static void Blur(this FFmpegMediaSource source, int sizeX = 0, int sizeY = 0, int planes = 15)
    {
        source.SetFFmpegVideoFilters($"avgblur=sizeX={sizeX}:sizeY={sizeY}:planes={planes}");
    }
}

[brabebhin]

Hi

For video filters things are a bit tricky.

you can find the documentation here
https://ffmpeg.org/ffmpeg-filters.html

However, not all filters are supported. For examples, filters which change resolution (lile crop), bitmap properties or otherwise modify the metadata of the video stream won't work. This is due to limitations of MF. We could theoretically make them work if we know that those filters are added in advance.

Blur, grayscale, color matrix filters will work.

[NGame1]

honestly, Crop worked for me! I don't know how you say it doesn't 😂
But if you give a me sample for blur it should be nice
I don't know, Box blur or AvgBlur, or what works?
But it should be nice if I have a sample that I can give it x,y, and size for blurring a box or blurring all the video except somewhere.
Thank you.

[brabebhin]

I think this would be the blur

https://ffmpeg.org/ffmpeg-filters.html#avgblur

Personally i use frame server mode and win2D to do video filters.

[lukasf]

The downside of using FFmpeg video filters is the fact that they are applied on software bitmaps. So while decoding usually happens on the GPU, for ffmpeg video filters, the frames must be copied from GPU to CPU memory, then the filters are applied. And at the end for rendering, they are copied back to the GPU again. For high-res video (like 4K) this can cause performance issues.

Resizing and cropping does work because after the filters are applied, the swscaler will be used to scale the result back to our original desired resolution. This might have a minor quality impact.

It is possible to do high performance video effects on the GPU using Win2D. But it is a lot more complicated, because you acually need to program them. There is no easy language such as ffmpeg video filters.

[brabebhin]

For crop specifically, it's more effective to use this property of MediaPlaybackSession

https://learn.microsoft.com/en-us/uwp/api/windows.media.playback.mediaplaybacksession.normalizedsourcerect?view=winrt-22621#windows-media-playback-mediaplaybacksession-normalizedsourcerect