Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate & improve the cross-compatible compute backend #1055

Open
cgokmen opened this issue Dec 12, 2024 · 0 comments
Open

Investigate & improve the cross-compatible compute backend #1055

cgokmen opened this issue Dec 12, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@cgokmen
Copy link
Contributor

cgokmen commented Dec 12, 2024

This is meant as a follow up to #1051

  1. Writing our own cross-backend API is something that is really annoying for us to do and also very annoying for a user to learn. It probably also has a ton of consequences in terms of whether GPU pipeline can work (remember the need for out= arguments), whether higher-level functions can be jitted, and how IDEs interact with the code. We should instead vendor something like https://data-apis.org/array-api-compat/ which conforms to a known common interface
  2. These setattr calls that mutate the cb interface at file load time are a huge problem imo because the interface ends up with undocumented functions. We should accept the untidyness of either moving these rather specific functions into the cb, or wrap them in an if/else inside the user class itself
  3. We currently do this only for the control interface, but should really consider applying the cb across the board and just setting the Isaac Sim frontend to use numpy/torch based on user choice.
  4. This has huge implications on whether full GPU pipelining can be done at all, so we should be careful. Even just the current control-only conversion might mean that there is suddenly no acceptable way to put control on GPU in a reasonable way. It might effectively mean the whole torch conversion was entirely pointless.
@cgokmen cgokmen added the enhancement New feature or request label Dec 12, 2024
@cgokmen cgokmen self-assigned this Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant