Skip to content

Commit

Permalink
Improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
jakki-amd committed Dec 20, 2024
1 parent 8d78419 commit d37ed5c
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 1 deletion.
3 changes: 2 additions & 1 deletion docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Creates a docker image with publicly available `torchserve` and `torch-model-arc

- To create a GPU based image with cuda 10.2. Options are `cu92`, `cu101`, `cu102`, `cu111`, `cu113`, `cu116`, `cu117`, `cu118` for CUDA and `rocm60`, `rocm61`, `rocm62` for ROCm.

- GPU images are built with either NVIDIA CUDA base image amd AMD ROCm base image. If you want to use ONNX, please specify the base image as shown in the next section.
- GPU images are built with either NVIDIA CUDA base image or AMD ROCm base image. If you want to use ONNX, please specify the base image as shown in the next section.

```bash
./build_image.sh -g -cv cu117
Expand Down Expand Up @@ -183,6 +183,7 @@ Creates a docker image with `torchserve` and `torch-model-archiver` installed fr
./build_image.sh -bt dev -g [-cv cu121|cu118] -cpp
```

- For more ROCm support (*experimental*), refer to [this documentation](../docs/hardware_support/amd_support.md).

## Start a container with a TorchServe image

Expand Down
27 changes: 27 additions & 0 deletions docs/hardware_support/amd_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,33 @@ If you have 8 accelerators but only want TorchServe to see the last four of them
> ⚠️ Setting both `CUDA_VISIBLE_DEVICES` and `HIP_VISIBLE_DEVICES` may cause unintended behaviour and should be avoided.
> Doing so may cause an exception in the future.

## Docker¨

**In Development**

`Dockerfile` and `build_image.sh` provides ROCm support for TorchServe.

Building and running `dev-image`:

```bash
./build_image.sh -bt dev -g -rv rocm62 -t torch-serve-dev-image-rocm
docker run -it --rm -device=/dev/kfd --device=/dev/dri torch-serve-dev-image-rocm bash
```

Building and running `ci-image`:

```bash
./build_image.sh -bt ci -g -rv rocm62 -t torch-serve-ci-image-rocm
docker run -it --rm --device=/dev/kfd --device=/dev/dri torch-serve-ci-image-rocm
```

Building and running `production-image`:

```bash
./build_image.sh -bt production -g -rv rocm62 -t torch-serve-production-image-rocm
docker run -it --rm --device=/dev/kfd --device=/dev/dri torch-serve-production-image-rocm
```

## Example Usage

After installing TorchServe with the required dependencies for ROCm you should be ready to serve your model.
Expand Down

0 comments on commit d37ed5c

Please sign in to comment.