Set Rootfs to AB for UEFI based L4T (nvidia jetson xavier AGX)

[elbbit01]

Hi,
I updated in the end of the year my system from Dunfell to Kirkstone (and most importantly, from cboot to UEFI).
In previous versions, I could set the variable CONFIG_ENABLE_A_B_SLOT=1 and then AB rootfs would be configured.
For the current version, I was redirected to the documentation, where this is described to be made with the following command:
sudo ROOTFS_AB=1 ROOTFS_RETRY_COUNT_MAX=3 ./flash.sh [options] <target_board> <rootdev>

Actually, this worked for me with the predefined ubuntu image from nvidia using specifically (and after this, nvbootctl works too, which is what I need as an interface for Rauc):
sudo ROOTFS_AB=1 ./flash.sh jetson-agx-xavier-devkit mmcblk0p12

Why mmcblk0p12 I can not say, but it is the recommendation of the flashing script for using emmc. I dont really understand it but I can live with that :)

The question is, how do I bring this variable for the meta-tegra generated scripts? As far as I can look, there is just the tegraflash.py script used, and I dont find a variable that could help me to set this.

Am I missing something? Could you help me to set ROOTFS_AB for the generated scripts?

Thanks in advance,
Alvaro.

[kekiefer]

This has changed a few times in the last couple releases, but this is my current understanding. Someone will please correct me if this is wrong. In the stock flash script, the setting ROOFS_AB does two things:

1. it populates (and signs where appropriate) the _b partitions
2. It reconfigures two values in the L4TConfiguration.dtbo:

• RootfsRedundancyLevel -> {data = [ 01 00 00 00 ];runtime;locked;};
• RootfsRetryCountMax -> {data = [ ${retry_count_max} 00 00 00 ];runtime;locked;}; (where retry_count_max : 1~3)

In meta-tegra, we handle (1) by calling out allowing you to customize the partition map (e.g.) PARTITION_LAYOUT_TEMPLATE ?= "flash_t194_sdmmc_rootfs_ab.xml" and the flash script will automatically stuff the _b partitions if they're in there.

For (2) you can change the value specified for TEGRA_BOOTCONTROL_OVERLAYS to effectively set the overlay data at build time. The documentation for constructing your own uefi overlays isn't very apparent, but you can run dtc -I dtb -O dts L4TConfiguration.dtbo > L4TConfiguration.dtso (or look at the edk2-tegra sources) to see how it's formatted. Your overlay might look something like this:

/dts-v1/;

/ {
	overlay-name = "A/B Configuration Settings";

	fragment@0 {
		target-path = "/";

		board_config {
			sw-modules = "uefi";
		};

		__overlay__ {
			firmware {
				uefi {
					variables {
						gNVIDIAPublicVariableGuid {
							RootfsRedundancyLevel {
								data = [ 01 00 00 00 ]; runtime; locked;
							};
							RootfsRetryCountMax {
								data =  [ 03 00 00 00 ]; runtime; locked;
							};
						};
					};
				};
			};
		};
	};
};

You would compile this overlay, and append it to the TEGRA_BOOTCONTROL_OVERLAYS list.

[elbbit01]

Hi @kekiefer ,
thanks a lot for your answer, I understood now better how the uefi variables are set.

I adapted the device tree with the RootfsRedundancyLevel to 01 (I actually couldnt add it as an overlay, because something in between is ignoring my overlay at least as I use it for a standart raspi, but I managed to patch it, so now is set).

When I call nvbootctl now, I can read the current slot, I can see that there are 2 slots available, and so on.

Curiously, nvbootctl -t rootfs is-rootfs-ab-enable answers nothing (before the device tree changes, it answered that it is not implemented, so this is something).

get current slot and set current slot work, also dump slots info. For set-active-boot-slot is of course write protected, so I remount the efivars as rw, and chattr on the files (all of them as a test, since I dont know where exactly nvbootctl is trying to write) so the inmutable bit is removed, and then try again nvbootctl -t rootfs set-active-boot-slot and the device basically dies, and does not want to boot anymore unless I flash it again.

Could I be missing something? May be some other fixed variable on the uefi var that has to be set on the device tree?

It is actually complicated to test this without the sources of nvbootctl script or the description of all uefivars.

Thanks in advance and best regards,
Alvaro.

[elbbit01]

Ok, actually after some research I got something.

strace on nvbootctrl on the nvidia ubuntu distro (where it works) writes a file mounted on /opt/nvidia/esp/EFI/NVDA/Variables/BootChainFwNext.

Comparing with my distro built using meta-tegra, I found out that actually from 3 files that should be mounted there, I have the other 2, but not this one (BootChainFwNext). That might trigger the kernel panic.

Does somebody know something about it? Am I missing something? I am grepping over the layer and the workspace unsuccessfully trying to find out where are those Variables mounted and how am I skipping one of them.

Best regards,
Alvaro.

[elbbit01]

And third post today, excuse me.

I found out myself that this issue was more or less reproduced in this post #1113 and the middle solution proposed there patching the machine name from setup-nv-boot-control at least manually work. (Apparently the ESP file is mounted after you are able to set the next boot slot for the first time).

I am not sure it is very clean but I can work further.

Thanks for your work and sorry for the duplication.

Regards,
Alvaro.

[kekiefer]

/opt/nvidia/esp is the esp partition, a vfat filesystem that contains the l4tloader and some shared variable storage specific to nvidia's implementation (presumably due to problems with interacting with the "real" efi variables storage on emmc platforms). This is normally mounted at boot, by setup-nv-boot-control.service

As this isn't the "real" efi variables, the variables in here are just files that will not be there until they are created by something (be in the loader in uefi, or update tools in linux), at which point the loader will use them.

patching the machine name from setup-nv-boot-control at least manually work.

I think I understand -- are you using a custom machine name, so the TARGET value doesn't match anything? If so, one way to deal with this is to customize the TNSPEC_MACHINE variable, which will give things that need it a way to match some predefined values against the TNSPEC. I have an entry like this in my machine.conf:

TNSPEC_MACHINE:append = "-jetson-agx-xavier-devkit"

The tools just try to match a partial string against a few pre-defined values. So the TARGET value would then become "mymachine-jetson-agx-xavier-devkit" or "mymachine-jetson-xavier-maxn" if you need it shorter (there are limits in some places to the length of this string). Tools that check for jetson-agx-xavier-devkit or jetson-xavier-maxn will be happy with these values.

[elbbit01]

Thank you, it works and I understood it better now :)

I still have 2 related questions:

1 - Do you know how to mark-slot-successful / unsuccessful now for L4T 35.2.1 is not an option any more for nvbootctrl?

2 - Is it possible that in the adaption of L4T 35.2.1 a bug was introduced and units can not be softly rebooted without getting a kernel panic? I have built with the previous version and the error is not there, and the following commit where I just changed to 35.2.1 has that problem for me. It might be a tricky dependency just from my implementation, but asking is never bad :)

Regards and thank you again.

[kekiefer]

1. I short circuited such checks in my implementation because I kept running into corruption of uefi variables on emmc rendering the system unbootable, so I won't be much help with that. I see that nvbootctrl verify deals with BootChainFwStatus, and I have a link from the prior release bookmarked here to deal with similar issues.
2. I have a seen a number of things cause this, some of which result in RAS errors, others not. The most recent one was caused by a timeout removing the GPU driver, which I solved by making sure that /sys/devices/gpu.0/railgate_enable was set before trying to reboot or power down (it is disabled if you run jetson_clocks). If it's something other than this you should probably create new issue.

[dwalkes]

I've got a PR started at #1428 which proposes incorporating this into the BSP layer. Would appreciate any review/testing from folks who have already implemented in their custom MACHINEs