fix(docker): update build script with respect to multi-container design

[amadeuszsz]

Description

Fix for #5558 & #5514

How was this PR tested?

Notes for reviewers

None.

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[esteve]

@amadeuszsz I can confirm that this worked for me, is there anything that needs to be done before this is ready for review?

[amadeuszsz]

@amadeuszsz I can confirm that this worked for me, is there anything that needs to be done before this is ready for review?

@esteve
I don't know how to specify targets, as it is now. Pointing target results with something which I don't understand:

#16 54.67 Installing 'autoware.dev_env:0.1.0' to '/root/.ansible/collections/ansible_collections/autoware/dev_env'
#16 54.69 Created collection for autoware.dev_env:0.1.0 at /root/.ansible/collections/ansible_collections/autoware/dev_env
#16 54.69 autoware.dev_env:0.1.0 was installed successfully
#16 54.71 �[36mansible-playbook autoware.dev_env.rosdep --extra-vars prompt_install_nvidia=y --extra-vars install_devel=y --extra-vars prompt_download_artifacts=y --extra-vars data_dir=/root/autoware_data --extra-vars rosdistro=humble --extra-vars rmw_implementation=rmw_cyclonedds_cpp --extra-vars base_image=ros:humble-ros-base-jammy --extra-vars autoware_base_image=ghcr.io/autowarefoundation/autoware-base:latest --extra-vars autoware_base_cuda_image=ghcr.io/autowarefoundation/autoware-base:cuda-latest --extra-vars cuda_version=12.4 --extra-vars cudnn_version=8.9.7.29-1+cuda12.2 --extra-vars tensorrt_version=10.7.0.23-1+cuda12.6 --extra-vars pre_commit_clang_format_version=17.0.5 �[m
.
.
.
#19 73.89 Installing 'autoware.dev_env:0.1.0' to '/root/.ansible/collections/ansible_collections/autoware/dev_env'
#19 73.92 Created collection for autoware.dev_env:0.1.0 at /root/.ansible/collections/ansible_collections/autoware/dev_env
#19 73.92 autoware.dev_env:0.1.0 was installed successfully
#19 73.94 �[36mansible-playbook autoware.dev_env.openadkit --extra-vars prompt_install_nvidia=n --extra-vars cuda_install_drivers=false --extra-vars install_devel=y --extra-vars prompt_download_artifacts=N --extra-vars data_dir=/root/autoware_data --extra-vars module=all --extra-vars rosdistro=humble --extra-vars rmw_implementation=rmw_cyclonedds_cpp --extra-vars base_image=ros:humble-ros-base-jammy --extra-vars autoware_base_image=ghcr.io/autowarefoundation/autoware-base:latest --extra-vars autoware_base_cuda_image=ghcr.io/autowarefoundation/autoware-base:cuda-latest --extra-vars cuda_version=12.3 --extra-vars cudnn_version=8.9.5.29-1+cuda12.2 --extra-vars tensorrt_version=8.6.1.6-1+cuda12.0 --extra-vars pre_commit_clang_format_version=17.0.5 �[m

Briefly speaking, I updated amd64.env and tried to rebuild images. In the second part of the log we can see cudnn_version=8.9.5.29-1+cuda12.2 --extra-vars tensorrt_version=8.6.1.6-1+cuda12.0 which doesn't match version specified in updated amd64.env file (see first part of the log) and they (versions in wrong part of the log) don't exist in any of my local file. Even if I cleared all docker builder caches, seems somehow docker layer is loaded from existing images. It's not what I expected (force build all stages till target) and users might not even notice it. At the end, in docker image I see old dependencies versions. If I don't specify target, script proceed with all stages from Dockerfile and everything works correctly.

For this PR I'm removing arguments which don't apply, but I would prefer to keep them if we could force build target with use of local changes. If anyone know how to do it, please revert this commit and apply necessary changes.

What I know is we can't build image using both hcl files at once, otherwise both hcl files will be processed at once resulting pulling repository from github registry, because triggered local build of base images didn't finish yet.

[youtalk]

Since not only the build.sh but also .hcl files have been modified, it will also affect container image builds in CI.
If you just want to build a specific stage locally, I think you can achieve that by setting environment variables and running docker build --target. Isn’t that sufficient?

[amadeuszsz]

Since not only the build.sh but also .hcl files have been modified, it will also affect container image builds in CI. If you just want to build a specific stage locally, I think you can achieve that by setting environment variables and running docker build --target. Isn’t that sufficient?

I found why I can't apply my changes into docker images - for docker-bake-base.hcl we need to build both (cuda and non-cuda) images. Otherwise for second docker buildx call in script we will just pull image from github registry.

I modified PR and now I believe we can merge it.

[youtalk]

Thank you for your update.