[FEAT] Make cuda available

[mricharz]

Is this a new feature request?

• I have searched the existing issues

Wanted change

I would like to have the ability to activate cuda as it is supported by piper itself.
The piper documentation says we need to install onnxruntime-gpu and add --cuda on the start script.

Reason for change

Piper should work much faster with cuda support on nvidia cards.

Proposed code change

1. Install onnxruntime-gpu in docker container.
2. Add a variable to docker image with will be passed to the start script like EXTRA_PARAMS
   With such a variable the user could add additional parameters to the piper start script as they like.

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[LinuxServer-CI]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

[4-FLOSS-Free-Libre-Open-Source-Software]

    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    runtime: nvidia # https://docs.linuxserver.io/images/docker-plex/#nvidia

Append --cuda cmd there https://github.com/linuxserver/docker-piper/blob/main/root/etc/s6-overlay/s6-rc.d/svc-piper/run#L7-L16 ?

[LinuxServer-CI]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

[mricharz]

    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    runtime: nvidia # https://docs.linuxserver.io/images/docker-plex/#nvidia

Append --cuda cmd there https://github.com/linuxserver/docker-piper/blob/main/root/etc/s6-overlay/s6-rc.d/svc-piper/run#L7-L16 ?

Yes. Should be the correct place.

[mricharz]

You can find the documentation here

If you'd like to use a GPU, install the onnxruntime-gpu package:

.venv/bin/pip3 install onnxruntime-gpu

and then run piper with the --cuda argument. You will need to have a functioning CUDA environment, such as what's available in NVIDIA's PyTorch containers.

[srvoets]

You can find the documentation here

If you'd like to use a GPU, install the onnxruntime-gpu package:

.venv/bin/pip3 install onnxruntime-gpu

and then run piper with the --cuda argument. You will need to have a functioning CUDA environment, such as what's available in NVIDIA's PyTorch containers.

yes you can do that in the console, but it needs to survive updates etc.

I found this due to the same issue.

[LinuxServer-CI]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

[4-FLOSS-Free-Libre-Open-Source-Software]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

The proposed solution involves installing the onnxruntime-gpu package and passing the --cuda argument to Piper’s start script.