Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gpr dataset integration #8

Merged
merged 18 commits into from
Dec 5, 2024
Merged

Gpr dataset integration #8

merged 18 commits into from
Dec 5, 2024

Conversation

alik-git
Copy link

@alik-git alik-git commented Dec 4, 2024

Integrate GPR h5 data to be used via LeRobot Dataset

First clone the repo and do the install instructions in readme

then just run

python lerobot/common/datasets/push_dataset_to_hub/gpr_h5_format.py --raw_dir  /path/to/h5_files/

# for me its 
python lerobot/common/datasets/push_dataset_to_hub/gpr_h5_format.py --raw_dir  /home/kasm-user/ali_repos/sim/runs/h5_out/stompypro/2024-12-02_20-04-51/all_h5

To visualize:

python lerobot/scripts/visualize_dataset.py \
    --repo-id gpr_test \
    --root /home/kasm-user/.cache/huggingface/lerobot/gpr_test \
    --local-files-only 1 \
    --episode-index 0

@alik-git alik-git requested a review from budzianowski December 4, 2024 07:26

def check_format(raw_dir) -> bool:
"""Verify HDF5 files have expected structure"""
print(f"[DEBUG] Checking format for directory: {raw_dir}")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logging.debug everywhere please


if "next.success" in batch:
rr.log("next.success", rr.Scalar(batch["next.success"][i].item()))
# Choose visualization function based on dataset type

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove that please and add more ifs if necessary

file_type = get_krec_file_type(file_path)

if file_type == "krec":
return load_krec_direct(file_path)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return load_krec_direct(file_path)
return krec.KRec.load(krec_file_path)

simplicity always wins, no langchain patterns

@budzianowski budzianowski self-requested a review December 5, 2024 03:19
@budzianowski budzianowski merged commit e5e4a80 into main Dec 5, 2024
2 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants