Can hashgrid be distributed on multiple GPUs?

[cadop]

I want to use a hashgrid with queries across multiple GPUs, what are the possibilities for this?

Can I use the same hashgrid and query it along multiple GPUs assuming they are CUDA peer access supported?

For example, I want to split a particle simulation amongst multiple GPUs.

[shi-eric]

Hey @cadop, I'm not very experienced with wp.HashGrid but I do have a system that supports peer-access between two GPUs. It seems like the following works:

1. Create a wp.HashGrid on cuda:0 from a points array.
2. Call grid.build()
3. Create a copy of the points array on cuda:1.
4. Run queries against grid in a kernel launched on cuda:1.

The main trick is enabling peer access. Assuming you are on a system that supports the mempool memory allocators, you need to add

wp.set_mempool_access_enabled(wp.get_cuda_device(0), wp.get_cuda_device(1), True)
wp.set_mempool_access_enabled(wp.get_cuda_device(1), wp.get_cuda_device(0), True)

Without this, I get a lot of Warp CUDA error 700: an illegal memory access was encountered errors when I try to launch the kernel on cuda:1.