Inquiry on IQL Performance with One-Leg Assembly Task

[pengzhi1998]

Dear Authors,

First and foremost, I would like to express my gratitude for the exceptional work on this repository, which serves as a valuable benchmark for long-horizon assembly tasks!

I am reaching out to seek your insight regarding an issue I encountered with the Implicit Q-Learning (IQL) algorithm's performance on the one-leg assembly task. Following the instructions provided in your tutorial, I executed the example evaluation code using the command:

python implicit_q_learning/test_offline.py --env_name=FurnitureSimImageFeature-v0/one_leg --config=implicit_q_learning/configs/furniture_config.py --ckpt_step=1000000 --run_name one_leg_full_iql_r3m_low_sim_1000 --randomness low

Subsequently, I downloaded the pre-trained model one_leg_full_iql_r3m_low_sim_1000.42. Apart from a single adjustment made to address a bug—modifying _, params = variables.pop('params') to params = variables.pop('params') at this line (modify it to solve a small bug) — I adhered strictly to the provided setup. However, I've observed that the model's performance is suboptimal. Specifically, it occasionally completes only the first two phases (Grasp tabletop and Place it to corner) or even fails to progress through the first phases.

Included below are two screenshots capturing the moments when the robot arm becomes stuck during the task execution:

Given these observations, I am eager to hear your suggestions and recommendations. Is the issue rooted in the model itself, or is there a possibility that I've overlooked a critical step?

Thank you once again for developing and sharing this invaluable resource. I look forward to your guidance and advice!

Best regards,
Pengzhi

[pengzhi1998]

Here is the output during execution:

Importing module 'gym_38' (/home/pengzhi/courses_delft/master_thesis/isaac_gym/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/pengzhi/courses_delft/master_thesis/isaac_gym/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
/home/pengzhi/anaconda3/envs/isaacgym/lib/python3.8/site-packages/jax/_src/api_util.py:229: SyntaxWarning: Jitted function has invalid argnames {'distribution'} in static_argnames. Function does not take these args.This warning will be replaced by an error after 2022-08-20 at the earliest.
  warnings.warn(f"Jitted function has invalid argnames {invalid_argnames} "
PyTorch version 2.2.2+cu121
Using /home/pengzhi/.cache/torch_extensions/py38_cu121 as PyTorch extensions root...
Device count 1
/home/pengzhi/courses_delft/master_thesis/isaac_gym/isaacgym/python/isaacgym/_bindings/src/gymtorch
Emitting ninja build file /home/pengzhi/.cache/torch_extensions/py38_cu121/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Using SDF cache directory '/home/pengzhi/.isaacgym/sdf_V100'
~!~!~! Loaded/Cooked SDF triangle mesh 0 @ 0x531675a0, resolution=512, spacing=0.000317
  ~!~! Bounds:  (-0.081250, 0.081250) (-0.015685, 0.015565) (-0.081250, 0.081251)
  ~!~! Extents: (0.162500, 0.031250, 0.162501)
  ~!~! Resolution: (512, 99, 512)
~!~!~! Loaded/Cooked SDF triangle mesh 1 @ 0x53165eb0, resolution=512, spacing=0.000171
  ~!~! Bounds:  (-0.015000, 0.015000) (-0.056250, 0.031250) (-0.014383, 0.015618)
  ~!~! Extents: (0.030000, 0.087500, 0.030001)
  ~!~! Resolution: (176, 512, 176)
~!~!~! Loaded/Cooked SDF triangle mesh 2 @ 0x522de620, resolution=512, spacing=0.000172
  ~!~! Bounds:  (-0.015000, 0.015000) (-0.056562, 0.031376) (-0.015438, 0.014563)
  ~!~! Extents: (0.030000, 0.087938, 0.030001)
  ~!~! Resolution: (175, 512, 175)
~!~!~! Loaded/Cooked SDF triangle mesh 3 @ 0x52ae1f70, resolution=512, spacing=0.000171
  ~!~! Bounds:  (-0.015000, 0.015000) (-0.056250, 0.031250) (-0.014375, 0.015625)
  ~!~! Extents: (0.030000, 0.087500, 0.030000)
  ~!~! Resolution: (176, 512, 176)
~!~!~! Loaded/Cooked SDF triangle mesh 4 @ 0x52e74e60, resolution=512, spacing=0.000171
  ~!~! Bounds:  (-0.015000, 0.015000) (-0.056250, 0.031250) (-0.015618, 0.014383)
  ~!~! Extents: (0.030000, 0.087500, 0.030001)
  ~!~! Resolution: (176, 512, 176)
/home/pengzhi/anaconda3/envs/isaacgym/lib/python3.8/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
/home/pengzhi/anaconda3/envs/isaacgym/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/pengzhi/anaconda3/envs/isaacgym/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)
Observation space Dict(image1:Box([-inf -inf -inf ... -inf -inf -inf], [inf inf inf ... inf inf inf], (2048,), float32), image2:Box([-inf -inf -inf ... -inf -inf -inf], [inf inf inf ... inf inf inf], (2048,), float32), robot_state:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf], (14,), float32))
Action space Box([[-1. -1. -1. -1. -1. -1. -1. -1.]], [[1. 1. 1. 1. 1. 1. 1. 1.]], (1, 8), float32)
I0409 01:53:10.627077 140285573837184 xla_bridge.py:603] Unable to initialize backend 'cuda': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I0409 01:53:10.627183 140285573837184 xla_bridge.py:603] Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
I0409 01:53:10.627608 140285573837184 xla_bridge.py:603] Unable to initialize backend 'tpu': INVALID_ARGUMENT: TpuPlatform is not available.
W0409 01:53:10.627674 140285573837184 xla_bridge.py:617] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
INFO: Found collision-free init pose

[letsgofeng]

Dear Authors,

First and foremost, I would like to express my gratitude for the exceptional work on this repository, which serves as a valuable benchmark for long-horizon assembly tasks!

I am reaching out to seek your insight regarding an issue I encountered with the Implicit Q-Learning (IQL) algorithm's performance on the one-leg assembly task. Following the instructions provided in your tutorial, I executed the example evaluation code using the command:

python implicit_q_learning/test_offline.py --env_name=FurnitureSimImageFeature-v0/one_leg --config=implicit_q_learning/configs/furniture_config.py --ckpt_step=1000000 --run_name one_leg_full_iql_r3m_low_sim_1000 --randomness low

Subsequently, I downloaded the pre-trained model one_leg_full_iql_r3m_low_sim_1000.42. Apart from a single adjustment made to address a bug—modifying _, params = variables.pop('params') to params = variables.pop('params') at this line (modify it to solve a small bug) — I adhered strictly to the provided setup. However, I've observed that the model's performance is suboptimal. Specifically, it occasionally completes only the first two phases (Grasp tabletop and Place it to corner) or even fails to progress through the first phases.

Included below are two screenshots capturing the moments when the robot arm becomes stuck during the task execution:

Given these observations, I am eager to hear your suggestions and recommendations. Is the issue rooted in the model itself, or is there a possibility that I've overlooked a critical step?

Thank you once again for developing and sharing this invaluable resource. I look forward to your guidance and advice!

Best regards, Pengzhi

Hi, I also have the same problem. I would like to know if this problem has been solved?