Flashing rootfs to NVMe and all other partitions to eMMC

[jpeeee]

I am following the instructions here with branch scarthgap SHA f89eeb2.
My aim is to flash my dev kit such that the rootfs A/B partitions are located on the NVMe drive and everything else on the eMMC.
I built my image with TNSPEC_BOOTDEV = "nvme0n1p1" and flashed with:

sudo ./initrd-flash -u ~/horizon/horizon-yocto-secureboot/horizon_keys/pkc0_priv.pem -v ~/horizon/horizon-yocto-secureboot/horizon_keys/sbk.key

Looking at the output below, it seems all partitions are written to the /dev/sda (nvme0n1).
Rebooting, I can see that the rootfs is mounted on /dev/nvme0n1p1 and the EFI /dev/nvme0n1p11 on /boot/efi.

What is the expectation from the script? Should it only put the rootfs on the NVMe as I expect or is it only able to flash everything on either the eMMC or the NVMe disk?

Secondary point, the flashing is extremely slow. I do not have bmaptool and it takes 30min for each 15GB APP partition. Is that expected?

Finally, should I be using this script at all to achieve my goal or is there a better approach?

Starting at 2024-11-26T16:57:56+00:00
Machine: horizon-orin-devkit
Rootfs device: nvme0n1p1
Found Jetson device in recovery mode at USB 1-2.1
== Step 1: Signing binaries at 2024-11-26T16:57:56+00:00 ==
Partition not found: A_cpu-bootloader
== Step 2: Boot Jetson via RCM at 2024-11-26T16:58:33+00:00 ==
Found Jetson device in recovery mode at USB 1-2.1
== Step 3: Sending flash sequence commands at 2024-11-26T16:58:37+00:00 ==
Waiting for USB storage device flashpkg from 3ae1e9b3.......[/dev/sda]
Device size in blocks: 262144
Unmounted /dev/sda.
== Step 4: Writing partitions on external storage device at 2024-11-26T16:59:02+00:00 ==
Waiting for USB storage device nvme0n1 from 3ae1e9b3.......[/dev/sda]
Creating partitions
[03] name=A_kernel start=0 size=262144 sectors
[04] name=A_kernel-dtb start=0 size=1536 sectors
[05] name=A_reserved_on_user start=0 size=64768 sectors
[06] name=B_kernel start=0 size=262144 sectors
[07] name=B_kernel-dtb start=0 size=1536 sectors
[08] name=B_reserved_on_user start=0 size=64768 sectors
[09] name=recovery start=0 size=163840 sectors
[10] name=recovery-dtb start=0 size=1024 sectors
[11] name=esp start=0 size=131072 sectors
[12] name=recovery_alt start=0 size=163840 sectors
[13] name=recovery-dtb_alt start=0 size=1024 sectors
[14] name=esp_alt start=0 size=131072 sectors
[15] name=UDA start=0 size=819200 sectors
[16] name=reserved start=0 size=982016 sectors
[01] name=APP start=0 size=29360128 sectors
[02] name=APP_b start=0 size=29360128 sectors
Writing partitions
Writing boot.img (size=46899200) to /dev/sda3 (size=134217728)...
Writing kernel_tegra234-p3737-0000+p3701-0000-nv.dtb (size=249445) to /dev/sda4 (size=786432)...
Writing boot.img (size=46899200) to /dev/sda6 (size=134217728)...
Writing kernel_tegra234-p3737-0000+p3701-0000-nv.dtb (size=249445) to /dev/sda7 (size=786432)...
Writing esp.img (size=67108864) to /dev/sda11 (size=67108864)...
Writing horizon-image-full.ext4 (size=15032385536) to /dev/sda1 (size=15032385536)...
Writing horizon-image-full.ext4 (size=15032385536) to /dev/sda2 (size=15032385536)...
[OK: /dev/sda]
== Step 5: Waiting for final status from device at 2024-11-26T17:58:44+00:00 ==
Waiting for USB storage device flashpkg from 3ae1e9b3...[/dev/sda]
Unmounted /dev/sda.
Final status: SUCCESS
Successfully finished at 2024-11-26T17:58:46+00:00
Host-side log: log.initrd-flash.2024-11-26-16.57.56
Device-side logs stored in: device-logs-2024-11-26-16.57.56

[dwalkes]

What is the expectation from the script? Should it only put the rootfs on the NVMe as I expect or is it only able to flash everything on either the eMMC or the NVMe disk?

There are some limitations about what can go on NVMe vs eMMC which I don't know off the top of my head but ultimately the partition layout file will control where partitions are written. I'd start with NVIDIA's documentation about where partitions can be placed.

Secondary point, the flashing is extremely slow. I do not have bmaptool and it takes 30min for each 15GB APP partition. Is that expected?

I've never timed it without bmaptool but I do think it's quite slow and I wouldn't be surprised if this is expected. I would suggest trying bmaptool.

Finally, should I be using this script at all to achieve my goal or is there a better approach?

If you want to write rootfs partitions to NVMe I think short of doing something yourself at first boot initrd-flash is the correct (and only AFAIK) way to do it.