feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

gtxaspec · 2023-07-12T00:01:33Z

initial testing seems ok, 1080p/360p 30fps OK via webrtc / hls web streaming, go2rtc binary running on device.

rtsp OK

audio works, aac+rtsp, libopus+webrtc

test with frigate: frigate recordings are good.

if testing is good, we will deprecate v4l2rtspserver in favor of go2rtc.

gtxaspec · 2023-07-12T02:04:54Z

added in latest master. feel free to leave feedback here.

gtxaspec · 2023-07-12T02:31:25Z

example to view stream in browser:
http://192.168.x.x:1984/stream.html?src=1080p&mode=webrtc
still:
http://192.168.x.x:1984/api/frame.mp4?src=1080p
stream links:
http://192.168.x.x:1984/links.html?src=1080p

endertable · 2023-07-14T18:56:02Z

So the v4l2rtspserver can be completely disabled? Will the cmd commands run without it running?

gtxaspec · 2023-07-14T21:07:29Z

Yes, and yes.

They are independent of each other.

rickgitdone · 2023-07-24T15:19:15Z

is there a new build we can download to switch over to webrtc? or it is already ready and we just pull new git?

gtxaspec · 2023-07-24T18:22:12Z

Download master, it's ready

KyleStilkey · 2023-07-27T13:59:03Z

I've updated my cameras and I can't seem to figure out how to use this new release, is there a setting that must be enabled?

I've tried your example links and nothing comes up.

http://(Cams IP):1984/stream.html?src=1080p&mode=webrtc and nothing shows up.

gtxaspec · 2023-07-27T14:00:43Z

What model camera? Does the web UI show up at http://(Cams IP):1984/ ?

Can you share your wz_mini.conf?

KyleStilkey · 2023-07-27T14:02:59Z

I have V3's I'm using and no, I do not get a web UI at all going to that URL for the cameras.

This is the wz_mini.cof from the cameras.

#wz_mini configuration

W E B CAMERA###

##THIS MODE DISABLES EVERYTHING AND IT WILL

WORK AS A WEB CAMERA FOR YOUR PC ONLY

WEB_CAM_ENABLE="false"
WEB_CAM_BIT_RATE="8000"
WEB_CAM_FPS_RATE="25"

#####NETWORKING#####
CUSTOM_HOSTNAME="WCV3"

ENABLE_USB_ETH="false"
ENABLE_USB_ETH_MODULE_AUTODETECT="true"
ENABLE_USB_ETH_MODULE_MANUAL=""

ENABLE_USB_DIRECT="false"
USB_DIRECT_MAC_ADDR="02:01:02:03:04:08"

ENABLE_USB_RNDIS="false"

ENABLE_IPV6="false"

ENABLE_WIREGUARD="false"
WIREGUARD_IPV4=""
WIREGUARD_PEER_ENDPOINT=""
WIREGUARD_PEER_PUBLIC_KEY=""
WIREGUARD_PEER_ALLOWED_IPS=""
WIREGUARD_PEER_KEEP_ALIVE=""

ENABLE_IPTABLES="false"

ENABLE_NFSv4="false"

ENABLE_RTL8189FS_DRIVER="true"
ENABLE_ATBM603X_DRIVER="true"

NETWORK INTERFACE BONDING

BONDING_ENABLED="false"
BONDING_PRIMARY_INTERFACE="eth0"
BONDING_SECONDARY_INTERFACE="wlan0"
BONDING_LINK_MONITORING_FREQ_MS="100"
BONDING_DOWN_DELAY_MS="5000"
BONDING_UP_DELAY_MS="5000"
BONDING_FAIL_OVER_MAC="0"

#####ACCESSORIES#####
REMOTE_SPOTLIGHT="false"
REMOTE_SPOTLIGHT_HOST="0.0.0.0"

#####VIDEO STREAM#####
RTSP_LOGIN="admin"
RTSP_PASSWORD="wyze"
RTSP_PORT="8554"

RTSP_HI_RES_ENABLED="true"
RTSP_HI_RES_ENABLE_AUDIO="true"
RTSP_HI_RES_FPS=""
RTSP_HI_RES_MAX_BITRATE=""
RTSP_HI_RES_TARGET_BITRATE=""
RTSP_HI_RES_ENC_PARAMETER=""

RTSP_LOW_RES_ENABLED="false"
RTSP_LOW_RES_ENABLE_AUDIO="false"
RTSP_LOW_RES_FPS=""
RTSP_LOW_RES_MAX_BITRATE=""
RTSP_LOW_RES_TARGET_BITRATE=""
RTSP_LOW_RES_ENC_PARAMETER=""

RTSP_AUTH_DISABLE="true"

#####RTMP STREAMING#####

REQUIRES RTSP

AND RTSP AUDIO

RTMP_STREAM_ENABLED="false"
RTMP_STREAM_FEED="video1_unicast"
RTMP_STREAM_SERVICE="youtube"
RTMP_STREAM_DISABLE_AUDIO="false"
RTMP_STREAM_YOUTUBE_KEY=""
RTMP_STREAM_TWITCH_KEY=""
RTMP_STREAM_FACEBOOK_KEY=""

#####GENERAL#####
LIBCALLBACK_ENABLE="true"
ENABLE_SWAP="true"
ENABLE_USB_STORAGE="false"
ENABLE_EXT4="false"
ENABLE_CIFS="false"
DISABLE_FW_UPGRADE="true"
AUDIO_PROMPT_VOLUME="50"
ENABLE_MP4_WRITE="false"
NIGHT_DROP_DISABLE="false"
DISABLE_MOTOR="false"
ENABLE_FSCK_ON_BOOT="false"
ENABLE_CAR_DRIVER="false"
ENABLE_LOCAL_DNS="false"
ENABLE_CRONTAB="false"

#####DEBUG#####
#drops you to a shell via serial, doesn't load app_init.sh
DEBUG_ENABLED="false"
DEBUG_INITRAMFS_ENABLED="false"
DEBUG_PASSWORD="false"
ENABLE_SYSLOG_SAVE="false"

####WEB####
WEB_SERVER_ENABLED="false"

#####SCRIPTING#####
CUSTOM_SCRIPT_PATH=""

gtxaspec · 2023-07-27T15:03:05Z

you have an old version of wz_mini.conf.

you can either do a fresh install, or copy /opt/wz_mini/etc/wz_mini.conf.d to /opt/wz_mini/wz_mini.conf (this will overwrite your old config) to get the new go2rtc options.

mega3st · 2023-07-28T00:29:49Z

@gtxaspec first of all thanks for implementing go2rtc as a potential replacement for v4l2rtspserver!

long story short after lots of testing with tinycam ( #340):

it seems like tinycam doesn’t like the aac_he profile
but it is flawlessly working with low latency and no stuttering by using opus with the following parameters

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 4 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

for compression_level values between 0 and 10 are allowed
- 0 gives fastet encoding but lowest audio quality
- 10 is the default value and gives slowest encoding but best audio quality
  (this should also be the reason for stuttering with the current config)
- not sure if 4 is already the sweetspot but for now it is more than good enough
for -frame duration the values 2.5, 5, 10, 20, 40, 60 are allowed
- 20 ms is the default value with low packet overhead and high audio quality
- 10 ms is according to the docs the sweetspot for low latency with the best compromise between audio quality and packet overhaed
- initial test with compression_level 5 and 20 ms frame duration resulted in high latency
- second test with compression_level 4 and 10 ms frame duration works like a charm

@mrlt8 maybe this is also interesting for your wyze-bridge

Other findings while testing

if more than one audio output is defined in go2rtc.yml and "audio=all" is set in the rtsp link tinycam picks the first defined audio output (currenty pcm_s16le)
it therefore makes sense to use specific instead of generic rtsp link
e.g.
rtsp://[IP-ADDRESS]:8554/1080p?video=h264&audio=opus
instead of
rtsp://[IP-ADDRESS]:8554/1080p?video=all&audio=all
no matter what parameters I’ve tried the result for aac_he with tinycam was at best an established audio connection with only short random noises
the default aac profile (lc) with the following configuration worked immediately but had very high latency

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libfdk_aac -afterburner 1 -channels 1 -vbr 3 -muxdelay 0 -muxpreload 0 -ac:a 1 -f rtsp {output}

CONCLUSION

At least from my perspective go2rtc is working fantastic and after changing back to opus as default audio output or adding some notes for tinycam to the wiki I see no reason not to deprecate v4l2rtspserver!

gtxaspec · 2023-07-28T01:29:17Z

Have you tried with the regular aac codec? (Not libfdk)

mega3st · 2023-07-28T12:32:45Z

No, but since I’m going to figure out the lowest compression level for opus that still delivers audio quality on par with the original stream during the weekend, it shouldn’t be a problem to do some further testing wit aac as well

gtxaspec · 2023-07-28T13:23:46Z

I've got some other latency fixes coming, if they work out.

Hopefully we can eliminate using v4l2loopback.ko and ALSA altogether, and pipe them straight to ffmpeg.

mega3st · 2023-08-03T11:57:48Z

Latency fixes are always welcome and cutting out v4l2loopback.ko and ALSA sounds like big potential - really looking forward to test them!

Below my additional test results.

All tests were performed with wyzeCam V3, tinyCam app on Android 13 an the following settings:

GO2RTC_SERVER_ENABLED="true"
GO2RTC_HI_RES_ENABLED="true"
GO2RTC_HI_RES_ENABLE_AUDIO="true"
GO2RTC_LOW_RES_ENABLED="false"
GO2RTC_LOW_RES_ENABLE_AUDIO="false"
NTP_SERVER="[LOCAL_NTP_SERVER_IP]"
DISABLE_FW_UPGRADE="true"
ENABLE_SELFHOSTED_MODE="true"
WEB_SERVER_ENABLED="false"

the embedded aac encoder from ffmepg has very low latency by default but by far the worst audio quality
- did various tests with VBR and CBR as well as combinations with the available lc profiles
- audio was always unusable due to constant stuttering
- but latency seems to be the same as libopus with 10ms frame duration
libfdk_aac delivers perfekt audio quality but only with high latency (nearly 2 seconds)
- did various tests with the lc profile and different VBR and CBR combinations
- audio quality was very good even down to 16kbps
- even without afterburner the latency was always on the same high level
libopus delivers best overall experience with perfekt audio quality and low latency
- lowest compression_level value doesn’t deliver lowest latency - it is actually the other way around
- the highest value that worked fine with 10ms frame duration on V3 was 7
  - 7 has the lowest latency as well as perfect audio quality
  - latency increases minimally with each step down
  - if I remenber correctly 4 was the lowest value that delievered acceptable audio and latency
  - surprisingly 8 increased the latency again by a quite a bit
  - and from 9 or with 10 (default if nothing is set) go2rtc crashed in combination with 10ms frame duration
  - maybe 7 is close to the edge and gets unstable with parallel low quality stream or on really hot days!?
- I will do a few tests on V2 with T20 soon
- For now I’ll stay with the following setting on V3 which worked best for me without any issues so far

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

jemershaw · 2023-08-10T19:59:06Z

@gtxaspec when you added it to frigate did you use go2rtc in frigate? If so what was the config you used? I tried setting this:

streams:
  camera1:
    - ffmpeg:rtsp://camera1:8554/1080p?video&audio=pcml#video=copy#audio=copy
    - ffmpeg:camera1#video=copy#audio=aac
    - ffmpeg:camera1#video=copy#audio=opus
  camera1.sub:
    - ffmpeg:rtsp://camera1:8554/360p?video&audio=pcml#video=copy#audio=copy
    - ffmpeg:camera1.sub#video=copy#audio=aac
    - ffmpeg:camera1.sub#video=copy#audio=opus

When I try to view it from the frigate server the page just keeps restarting and I don't see a connection. Maybe I'm using the wrong configuration?

UPDATE:

turns out it was a go2rtc problem since ffmpeg didn't support pcml. I changed the cameras to look like this

  camera1:
    - ffmpeg:rtsp://camera1:8554/1080p?video&audio=aac#video=copy#audio=copy
    - ffmpeg:camera1#video=copy#audio=opus

gtxaspec · 2023-09-18T23:54:31Z

Positive feedback:
#617 (reply in thread)

gtxaspec · 2023-09-18T23:55:08Z

@mega3st have 6ou tried with straight pcm audio using copy?

unintended-consequences · 2023-10-14T09:11:49Z

@mega3st - exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

This is brilliant. Working great for me. Wyzecam v3, Tinycam Pro on Android. For the less technically minded like me:

Loaded the go2rtc webpage at http://[cam-ip]:1984
Click Config
Add # to all other lines starting - exec:ffmpeg -hide_banner...
Uncommented the last '-exec:ffmpeg' line with libopus and updated it to match your settings above
Click Save & Restart
Wait 2 mins to reboot.

KyleStilkey · 2023-10-15T18:34:18Z

Ok I've had some time to mess with this, I got one camera up and running. It's a v3 but I'm having issues seeing video. Clicking on the stream button for 1080p displays a black screen but I do hear Audio, not sure if I'm missing a something on my end.

pdobrien3 · 2023-10-22T02:41:38Z

@jemershaw curious if you followed #550 (comment) for your wz_mini setup? I can pull streams in VLC but as soon as I limit go2rtc to - exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output} I cant pull anything. Havent tried in frigate yet but if you have version 3s working?

gtxaspec · 2023-10-22T03:13:18Z

@pdobrien3 have you checked the go2rtc logs? what do they say when it fails

pdobrien3 · 2023-10-22T04:16:24Z

So I actually ended up getting a working config. It is late and I am tired so this isn't all inclusive. My wz_mini config is pretty basic. I ended up leaving the go2rtc config at the default and setup Frigate like this:

go2rtc:
  streams:
    1_camera:    
      - ffmpeg:rtsp://192.168.xx.xxx:8554/1080p?video&audio=aac#video=copy#audio=copy
      - ffmpeg:1_camera#video=copy#audio=opus

camera config:

  1_camera:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://192.168.xx.xxx:8554/360p
            - detect
        - path: rtsp://localhost:8554/1_camera
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record

The streams are a little blotchy and I am sure they can be improved upon but, I wanted to just get something up and running. Audio is also a little off sync but really not bad. Will be interesting to see if Frigate makes it through the night without any ffmpeg crashes.

I honestly have no clue what I am doing and feel pretty lucky to get working streams in Frigate. Any advice is greatly appreciated!!!

pdobrien3 · 2023-10-22T11:43:49Z

woke up to a bunch of this, any idea?

2023-10-22 05:23:06.709167904  [2023-10-22 05:23:06] frigate.video                  ERROR   : 1_camera: Unable to read frames from ffmpeg process.
2023-10-22 05:23:06.709628439  [2023-10-22 05:23:06] frigate.video                  ERROR   : 1_camera: ffmpeg process is not running. exiting capture thread...
2023-10-22 05:23:15.194541440  [2023-10-22 05:23:15] watchdog.1_camera    ERROR   : Ffmpeg process crashed unexpectedly for 1_camera.
2023-10-22 05:23:15.194745284  [2023-10-22 05:23:15] watchdog.1_camera    ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-10-22 05:23:15.195005108  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] decode_slice_header error
2023-10-22 05:23:15.195229589  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] no frame!
2023-10-22 05:23:15.195419995  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] non-existing PPS 0 referenced
2023-10-22 05:23:15.195624832  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   :     Last message repeated 1 times
2023-10-22 05:23:15.195832875  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] decode_slice_header error
2023-10-22 05:23:15.196025966  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] no frame!
2023-10-22 05:23:15.196236635  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] non-existing PPS 0

pdobrien3 · 2023-10-22T20:47:30Z

so, I changed:

output_args:
        record: preset-record-generic-audio-aac

to:

output_args:
        record: preset-record-generic-audio-copy

I will edit my post above for future visitors.

and frigate has been running rock solid all day. Thanks @gtxaspec for the dev work on go2rtc. I have been struggling with my V3s for a long time. There is some minor audio sync issues but nothing critical and haven't played with any settings. I have all the options in the go2rtc un-hashed, which doesn't jive with this string but things are working.

ajbucci · 2023-12-12T18:10:28Z

I'm currently using go2rtc with all defaults and am experiencing serious compression artifacts on a wired connection. Does anyone know how to reduce compression/increase bitrate in go2rtc? I presume it could be possible by modifying the go2rtc config?

I didn't realize that the VIDEO settings in the wz_mini.config file also apply to go2rtc... I just bumped up the bitrate there and set to capped quality and now all is good.

mcondren · 2024-04-04T14:48:04Z

Can anyone share the go2rtc config file line from the wyzecam that sends audio out with raw pcma or pcmu? I saw @gtxaspec mention PCM above but I am unsure what to add to the go2rtc file on the cam to make that happen. For codec issues downstream I need the cams to actually send raw pcm instead of opus or aac. Thanks!

gtxaspec added the enhancement New feature or request label Jul 12, 2023

gtxaspec self-assigned this Jul 12, 2023

gtxaspec pinned this issue Jul 12, 2023

gtxaspec mentioned this issue Jul 12, 2023

No audio in tinycam app due to audio codec used in RTSP stream #340

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

gtxaspec commented Jul 12, 2023 •

edited

Loading

gtxaspec commented Jul 12, 2023 •

edited

Loading

gtxaspec commented Jul 12, 2023 •

edited

Loading

endertable commented Jul 14, 2023

gtxaspec commented Jul 14, 2023

rickgitdone commented Jul 24, 2023

gtxaspec commented Jul 24, 2023

KyleStilkey commented Jul 27, 2023

gtxaspec commented Jul 27, 2023

KyleStilkey commented Jul 27, 2023

gtxaspec commented Jul 27, 2023

mega3st commented Jul 28, 2023 •

edited

Loading

gtxaspec commented Jul 28, 2023

mega3st commented Jul 28, 2023 •

edited

Loading

gtxaspec commented Jul 28, 2023 •

edited

Loading

mega3st commented Aug 3, 2023

jemershaw commented Aug 10, 2023 •

edited

Loading

gtxaspec commented Sep 18, 2023

gtxaspec commented Sep 18, 2023

unintended-consequences commented Oct 14, 2023

KyleStilkey commented Oct 15, 2023

pdobrien3 commented Oct 22, 2023

gtxaspec commented Oct 22, 2023

pdobrien3 commented Oct 22, 2023 •

edited

Loading

pdobrien3 commented Oct 22, 2023

pdobrien3 commented Oct 22, 2023

ajbucci commented Dec 12, 2023 •

edited

Loading

mcondren commented Apr 4, 2024

feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

Comments

gtxaspec commented Jul 12, 2023 • edited Loading

gtxaspec commented Jul 12, 2023 • edited Loading

gtxaspec commented Jul 12, 2023 • edited Loading

endertable commented Jul 14, 2023

gtxaspec commented Jul 14, 2023

rickgitdone commented Jul 24, 2023

gtxaspec commented Jul 24, 2023

KyleStilkey commented Jul 27, 2023

gtxaspec commented Jul 27, 2023

KyleStilkey commented Jul 27, 2023

W E B CAMERA###

WORK AS A WEB CAMERA FOR YOUR PC ONLY

NETWORK INTERFACE BONDING

REQUIRES RTSP

AND RTSP AUDIO

gtxaspec commented Jul 27, 2023

mega3st commented Jul 28, 2023 • edited Loading

gtxaspec commented Jul 28, 2023

mega3st commented Jul 28, 2023 • edited Loading

gtxaspec commented Jul 28, 2023 • edited Loading

mega3st commented Aug 3, 2023

jemershaw commented Aug 10, 2023 • edited Loading

gtxaspec commented Sep 18, 2023

gtxaspec commented Sep 18, 2023

unintended-consequences commented Oct 14, 2023

KyleStilkey commented Oct 15, 2023

pdobrien3 commented Oct 22, 2023

gtxaspec commented Oct 22, 2023

pdobrien3 commented Oct 22, 2023 • edited Loading

pdobrien3 commented Oct 22, 2023

pdobrien3 commented Oct 22, 2023

ajbucci commented Dec 12, 2023 • edited Loading

mcondren commented Apr 4, 2024

gtxaspec commented Jul 12, 2023 •

edited

Loading

gtxaspec commented Jul 12, 2023 •

edited

Loading

gtxaspec commented Jul 12, 2023 •

edited

Loading

mega3st commented Jul 28, 2023 •

edited

Loading

mega3st commented Jul 28, 2023 •

edited

Loading

gtxaspec commented Jul 28, 2023 •

edited

Loading

jemershaw commented Aug 10, 2023 •

edited

Loading

pdobrien3 commented Oct 22, 2023 •

edited

Loading

ajbucci commented Dec 12, 2023 •

edited

Loading