run_once: execute /boot/run_once when booting

[fasmide]

Based on Ned McClain's ideas found in https://github.com/nmcclain/raspberian-firstboot this pull request will enable newly flashed Raspberry Pi's to be easily provisioned by simply adding an executable to the /boot partition right after flashing sd cards.

Let me know what you think :)

[fasmide]

Documentation pull request: raspberrypi/documentation#1701

[lurch]

I wonder if there's a security risk in that any arbitrary binary placed onto the FAT32 partition (named firstboot) will automatically get executed as root ? 🤔

I think that Raspberry Pi OS still requires root access to modify files in /boot, but if e.g. you've got multiple OSes installed using NOOBS, I think some of those other OSes may auto-mount FAT32 partitions as being user-modifiable?

[fasmide]

I wonder if there's a security risk in that any arbitrary binary placed onto the FAT32 partition (named firstboot) will automatically get executed as root ?

I have been having these thoughts as well and I thought about adding ConditionFirstBoot=yes as well - that way it will not execute any newly added firstboot executables unless systemd figures out this really is the first boot...

That way I don't think its possible, even for root, to add new firstboot executables (unless they also remove stuff in /etc directory - to make systemd think this is the very first boot once again)

I decided against using it - in a totally headless environment (no monitor - no serial - no way of knowing the raspberry pi's IP address) I believe its a nice addition to be able to pull the sd card - mount /boot again on a desktop pc and edit/add a new firstboot file

Adding ConditionFirstBoot=yes is absolutely not a deal-breaker for my intended use cases - but I'm not sure it will increase security - Think about the other consequences of having /boot mounted in a way that allows user edits - they could change out the entire kernel if they wanted to

[fasmide]

@XECDesign Judging on the commit logs - it seems you got quite the history in this repository so I'm reaching out to you :)

Is there a formal process for getting things like this merged into the Raspberry Pi OS ?

[XECDesign]

Sorry about the delay. Isn't the name 'firstboot' a bit misleading then?

In the past, we have pushed against this idea in favour of a simple text file with user-friendly options. However, it has been over a year since I looked at that last and I don't think I'll get to it in the near future. Also, no matter what options we add, somebody will ask for their use case to be supported as well and everything quickly becomes complicated. The idea of just running a script is good because it's simple for advanced users and supports all the use cases anybody might have.

So yes, I think this is a good addition. It might take a little while before it's merged though because I'll need to consult a few more people and test it out when I get some time.

[lurch]

See also https://www.pibakery.org/
I think it's abandoned now, but IIRC that also worked by writing stuff it needed to do into a script file on the boot partition.

[XECDesign]

Aside from the name, another issue somebody raised is what happens if it's a network boot and there's no /boot write access. Some kind of meaningful error if /boot is read-only would be useful.

[fasmide]

Aside from the name, another issue somebody raised is what happens if it's a network boot and there's no /boot write access. Some kind of meaningful error if /boot is read-only would be useful.

Thanks for taking the time to look at this :)

I'm not sure I agree on the naming scheme being misleading - I think it clearly states the intended purpose, other names I can think of, initialize and provision have the same problem: users will be able to add another if they want.

We could of cause just call it exec, nextboot, one-shot or similar - these names kind of loses the intended purpose, and having to deal with a systemd unit called exec.service seems confusing - I don't know - let me know that you think :)

The network boot issue is a nice catch - I think addingConditionPathIsReadWrite=/boot should do the trick - that way it should not run at all if it has no way of ensuring it will only run once - and systemctl status firstboot should state that conditions are not met.

[fasmide]

I just added the ConditionPathIsReadWrite=/boot - It should be there netboot or not :)

[lurch]

run_once ? 🤷

[XECDesign]

I like run_once.

Should ConditionFileIsExecutable also be there?

[fasmide]

run_once ?

I like run_once.

I think we should go with run_once then - I still feel firstboot is more descriptive, but I get the reasoning...

Should ConditionFileIsExecutable also be there?

It certainly wouldn't hurt - it properly won't make a difference since everything is executable on the /boot filesystem. But may help clarify future debugging if this should change.

The branch has been updated :)

[lurch]

It certainly wouldn't hurt - it properly won't make a difference since everything is executable on the /boot filesystem.

Everything in /boot is executable if it's the FAT partition on an SD card, but things in /boot might not be executable if you're booting over the network 😉

[n0nk]

Hey, just a short question. I am new to github therefore i am not sure if this is the right place to ask this question...

Why was this not merged into the new release from 2nd december? I just wanted to tryout because its a pretty good feature!

Regards

[XECDesign]

I haven't had a chance to test it and this was a fairly big release with other changes that needed pre-release attention.

[n0nk]

Ah okay, thanks for the quick answer. Then i will wait for the next release to use this. If i can do something to help regarding this firstboot script testing, please contact me.

Regards

[lurch]

Perhaps @fasmide can provide some guidance for how @n0nk can help test this? 🙂

[fasmide]

@lurch @n0nk Here's how I'm testing it quick and dirty

First, clone my repository of the project

git clone [email protected]:fasmide/raspberrypi-sys-mods.git

Then, to build the deb, use the usual debuild tool, with a few extra parameters:

cd raspberrypi-sys-mods/
debuild -uc -us -aarmhf

-uc and -us disables deb signatures as we can't really sign these in any useful way without being the raspberry pi foundation - -aarmhf chooses architecture. There should be no issues "cross building" the package as it has no architecture-dependent stuff in it...

Now if everything went right, we should be presented with a new deb file in the parent directory, it's just a matter of moving this to an existing raspberry pi, installing it - and look for expected behavior :)

> cd ..
> ls -al
drwxrwxr-x  4 fas fas  4096 Dec 14 21:45 .
drwxrwxr-x 23 fas fas  4096 Nov  2 22:25 ..
drwxrwxr-x 12 fas fas  4096 Oct 20 21:09 pi-gen
drwxrwxr-x  8 fas fas  4096 Oct 20 20:40 raspberrypi-sys-mods
-rw-r--r--  1 fas fas  4575 Dec 14 21:45 raspberrypi-sys-mods_20200812_armhf.build
-rw-r--r--  1 fas fas  5748 Dec 14 21:45 raspberrypi-sys-mods_20200812_armhf.buildinfo
-rw-r--r--  1 fas fas  1667 Dec 14 21:45 raspberrypi-sys-mods_20200812_armhf.changes
-rw-r--r--  1 fas fas 10996 Dec 14 21:45 raspberrypi-sys-mods_20200812_armhf.deb
-rw-r--r--  1 fas fas   797 Dec 14 21:45 raspberrypi-sys-mods_20200812.dsc
-rw-r--r--  1 fas fas  8440 Dec 14 21:45 raspberrypi-sys-mods_20200812.tar.xz

scp raspberrypi-sys-mods_20200812_armhf.deb [email protected]:
ssh [email protected]
sudo dpkg -i raspberrypi-sys-mods_20200812_armhf.deb

So far so good - now it's just a matter of putting anything armhf executable in /boot/run_once and see if it's executed and moved out of the way on boot:

pi@raspberry:~ $ sudo -i
root@raspberry:~# cat > /boot/run_once
#!/bin/bash

echo "hello world"

<<CTRL+D>>

root@raspberry:~# reboot

When the pi has booted, check the output of systemctl status run_once for confirmation:

$ systemctl status run_once
● run_once.service - run_once initialization
   Loaded: loaded (/lib/systemd/system/run_once.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2020-12-14 21:40:24 CET; 12min ago
  Process: 453 ExecStart=/boot/run_once (code=exited, status=0/SUCCESS)
  Process: 459 ExecStartPost=/bin/mv /boot/run_once /boot/run_once.${INVOCATION_ID} (code=exited, status=0/SUCCESS)
 Main PID: 453 (code=exited, status=0/SUCCESS)

Dec 14 21:40:24 raspberry systemd[1]: Starting run_once initialization...
Dec 14 21:40:24 raspberry run_once[453]: hello world
Dec 14 21:40:24 raspberry systemd[1]: run_once.service: Succeeded.
Dec 14 21:40:24 raspberry systemd[1]: Started run_once initialization.

We can confirm that the exit code of the program run_once exited with status 0 and also the ExecStartPost succeeded in moving the executable away... Also, we have "hello world" in the logs :)

[fasmide]

@lurch @XECDesign I've just updated the documentation to once again match this pull request
raspberrypi/documentation#1701

[kccarbone]

I just want to +1 this idea and thank you all for working on it! This will be a huge help for doing some quick initial customizations.

[XECDesign]

Ah, it is quite flexible. In this case you would drop a YAML file onto the card that contains your customisations without requiring any remote servers.

Some examples:
https://cloudinit.readthedocs.io/en/latest/topics/examples.html

[nmcclain]

As the original author of firstboot.sh, I always felt like cloud-init was the "right" answer.

Note that you don't have to use YAML with cloud-init - you can simply paste in a shell script:
https://cloudinit.readthedocs.io/en/latest/topics/format.html#user-data-script

[XECDesign]

Thanks for the heads up, that's a very useful bit of information.

[lurch]

I'd not heard of cloud-init before, but just had a quick flick through its docs. Seems like it'd be something that plugs into https://github.com/raspberrypi/piserver quite nicely too?

[n0nk]

@XECDesign

Just a short comment from my side. I really like the cloud-init idea. But i also like it to keep things simple.

Having the possibility to add a simple executable or script to the boot partition and do whatever i want it to do is much more easier for everyone who initially starts developing/playing with the rasperry. Having cloud init is nice but its also a hurdle to take.

I want to say:
Everyone whos playing with raspberry and rasberrypi os, knows the boot partition, knows wpa_supplicant.conf, the "ssh enabling file" and (i think so) knows bash scripts. But not everyone knows cloud-init and from the first "google" it looks much more complicated than a simple bash script (i know it isn't, but most of the people will have to read the docs first to understand that)

And even if i know cloud-init and like to use it. Maybe its .. in german we say "Shoot sparrows with cannons" for most usecases, if you know what i mean?

Greetings
n0nk

[XECDesign]

@n0nk that's why nmcclain's comment is interesting. It can work with a shell script OR a YAML file in addition to other options, based on what's actually in the file.

[lurch]

I'm sure that if/when cloud-init does get added to RaspiOS, there'll also be easy-to-follow instructions (on how to do the basics) added to the Raspi documentation site 🙂

[n0nk]

@n0nk that's why nmcclain's comment is interesting. It can work with a shell script OR a YAML file in addition to other options, based on what's actually in the file.

oh sorry, i completely ignored this. Thanks for the hint

[geerlingguy]

I just wanted to mention, when reading the first few comments in this issue I kept thinking why not just use cloud-init?

Ubuntu for Pi has had it enabled for a while now, and it's a breath of fresh air to be able to have almost complete control over the first initialization of a Pi (has made me switch to running Ubuntu in many cases for headless installations).

I really do think something more than configuration of ssh via an ssh file, and wifi config via wpa_supplicant.conf, would be used quite a lot for Pi OS.

I'm sure that if/when cloud-init does get added to RaspiOS, there'll also be easy-to-follow instructions (on how to do the basics) added to the Raspi documentation site 🙂

I know I've been considering doing a video just on cloud-init on Ubuntu on the Pi for some time. Having it be supported in Pi OS would be a great excuse to dust off my cloud-init docs and make a pretty good guide for the YT audience at least.

I can imagine there will be good coverage of how to configure a few basics like ssh-keys, networking, and running executables, at least.

[danielo515]

This is a perfect example of how perfection is the worst enemy of good. For more than half a year there is a merge request with an available solution, but here we are, discussing about the perfect solution and users meanwhile still have to burn the image, find the IP, ssh to it and configure it manually.

This is a 2 file PR, I don't think it will be problematic to merge it now and undo it later when the "perfect solution" arrives (if ever does 😉).
Just my 2cents

[MichaIng]

Agreed that cloud-init is an overkill if you want to have a local script executed and vote for merging this PR until there is someone actually requesting needs that are addressed with cloud-init only.

Two minor things, not mandatory IMO:

• Instead of renaming the script, I personally would disable the service via ExecStartPost=/bin/systemctl disable run_once.
• And I'd try to have not only the loopback interface configure but have the network online, so that this script can download and execute remote scripts, e.g. from a personal Git repo:

  Wants=network-online.target
  After=network-online.target
  

[geerlingguy]

@danielo515 - Perfect is, indeed, the enemy of the good. However, for my own purposes, the thing I was most interested in has been solved by the latest feature addition to the official Raspberry Pi Imager—you can hold down Shift+Ctrl+X (Cmd on Mac) and have WiFi credentials configured, an SSH pubkey inserted, even set things like the hostname and locale!

So at this point I don't have much skin in the game, but I do hope that the ability to get a script to run once is added, either via this or cloud-init. The advantage of cloud-init is it's pretty much a standard across many distributions (Ubuntu already includes it), so I can see why there might be hesitation to merge in something that's more or less a one-off/unique for Pi OS.

[danielo515]

the thing I was most interested in has been solved by the latest feature addition to the official Raspberry Pi Imager—you can hold down Shift+Ctrl+X (Cmd on Mac) and have WiFi credentials configured, an SSH pubkey inserted, even set things like the hostname and locale!

It's fun that you mention that. The way I found this issue is because I saw your post on reddit about that hidden feature of Raspberrypi Pi imager, so I thought, ok, maybe it's time to take a look at this "etcher like" tool. Reading the issues of that tool I found one issue about executing a script on first boot and that lead me here. The circle is now complete.

I, in fact, used that feature, I inspected the resulting image and guess what? there is a script called firstboot. So if the tool is able to do such thing, I don't understand why they are pointing to this issue.

This is all very confusing even for advanced users. Curious coming from the people that thing that an "show advanced options" button is too hard for normal people to grasp.

[MichaIng]

I, in fact, used that feature, I inspected the resulting image and guess what? there is a script called firstboot.

But I guess the imager as well created another script/service to call that firstboot script on first boot, doesn't it?

Wouldn't then make sense to make this exact method a native method, so that either the imager or the user only needs to create the firstboot script? Otherwise an imager-prepared image would have two of such systems running 😄.

[lurch]

Otherwise an imager-prepared image would have two of such systems running 😄 .

I just did a bit of digging, and RPi Imager does it's firstrun magic by simply adding some extra text to cmdline.txt . Looks like there's no custom "firstboot service" running in Raspberry Pi OS, it just relies on systemd. I guess there's many ways to skin a cat 😸
ping @maxnet in case he wants to provide any extra details.

[MichaIng]

Ah a native systemd feature: https://www.freedesktop.org/software/systemd/man/kernel-command-line.html#systemd.run=

Okay, knowing this (I didn't know this feature before), everyone can easily manually setup first boot scripts (This one is called firstrun.sh btw.), an idea would now be to ship Raspberry Pi OS images with these cmdline.txt entries and a dummy script OOTB, which only removes the cmdline.txt entry and itself, and may contain commented lines to perform usual tasks, like setting hostname, enabling SSH and such. So that script only needs to be edited/extended for own needs and the imager only re-creates the script based on selectors but does not need to take care of cmdline.txt anymore, at least not on a Raspberry Pi OS image.

[lurch]

an idea would now be to ship Raspberry Pi OS images with these cmdline.txt entries and a dummy script OOTB .... the imager only re-creates the script based on selectors but does not need to take care of cmdline.txt anymore, at least not on a Raspberry Pi OS image.

...but that would mean that RPi Imager wouldn't be able to "customise" any RasPiOS images that didn't already have these cmdline.txt customisations; whereas the current approach used by RPi Imager will work with any version of RasPiOS that has systemd installed 🙂

[maxnet]

Looks like there's no custom "firstboot service" running in Raspberry Pi OS, it just relies on systemd.

Correct, does not require anything special.

And should work on any Linux distribution that uses systemd, as long as you provide the correct location of where the script would be available inside the file system after boot.
E.g. on RPI OS it is:

systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

But if you where using say Ubuntu instead where FAT partition is mounted at /boot/EFI instead of /boot it would need to be:

systemd.run=/boot/EFI/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

I personally think it is always best to use standard constructs that are available in multiple Linux distributions.
While it could be argued that the /boot/run_once proposed in this PR would be easier to use than adding those kernel cmdline parameters, it will be teaching people skills that are not very useful if they ever have to work with Linux computers that are not Raspberry Pi.

an idea would now be to ship Raspberry Pi OS images with these cmdline.txt entries and a dummy script OOTB

Why not just teach advanced users how to add the cmdline.txt entries (and how to remove the entries at end of script)?

[MichaIng]

...but that would mean that RPi Imager wouldn't be able to "customise" any RasPiOS images that didn't already have these cmdline.txt customisations

True, especially it won't work with other distros. Easiest would be to first remove possibly present cmdline.txt entries (same method used by the script to disable itself) and also overwrite possibly present firstrun.sh scripts. The first point would be good anyway, as currently in theory, when a 3rd party image had those entries already, imager would double them. The second point is already done like that, if I'm not mistaken.

it will be teaching people skills that are not very useful if they ever have to work with Linux computers that are not Raspberry Pi.

cmdline.txt itself is not present on non-RPi, generally there is no generic way to manipulate the kernel command line at all, but it works different depending on the used bootloader (depending on the hardware). More generic (still based on systemd) would be to create a systemd service (like the one this PR aims to ship or the one that systemd.run triggers), which requires root file system access, hence usually ext4, which still causes problems when only Windows or macOS systems are available.

Why not just teach advanced users how to add the cmdline.txt entries (and how to remove the entries at end of script)?

We'll, everyone with enough knowledge or a good documentation can modify and apply any change and tweak to any image, but I guess the idea here was to make it easy. While teaching users skills is nice, it's a different goal then offering easy to use features. This PR aims to add an easy to use feature, similar to how RPi imager aims to make pre-configuring an image easy, and the aim is not to teach users something. That could be a topic for the documentation, e.g. as companion to explain how the first boot script feature works 🙂.

[CCorrado]

So...are we going to add support for a firstboot script on the official Raspberry Pi images? This is the easiest and most straight forward way IMHO and I am using an outdated version of the Pi image instead of the latest official one due to it...
Anyway...

I did run into this issue that was brought up on newer Ubuntu distros. Maybe the fix to this issue should get pulled into this PR? It's a good call.
nmcclain/raspberian-firstboot#13

[CCorrado]

per nmcclain/raspberian-firstboot#13 ...

Update After=network.target to include apt-daily.service and apt-daily-upgrade.service

[fasmide]

Seems reasonable

How does this behave on a system without network connectivity?

[MichaIng]

EDIT: Lol, now I see the comment was more about the unattended upgrade services 😄. I wonder if they have any effect since those services are not "enabled" to be started by systemd at boot, but they are started by their respective systemd timers. As such, AFAIK they cannot be used for boot ordering at all, but I have not tested it.

---

network.target is a passive unit, hence it has no meaning unless a network stack/provider gives it one (sorts itself Before=network.target). In case of Raspbian/Debian (Raspberry Pi OS), this is most of all the networking.service from the ifupdown package, which brings up the loopback interface and other manually configured auto <iface> interfaces. Theoretically it's as well the ifup@<iface>.service's, which are created for allow-hotplug <iface> interfaces, but since this unit is of Type=simple, they don't need to finish but only need to start (hence do not delay the target to be reached, even if e.g. DHCP or wpa_supplicant hangs or fails).

dhcpcd does NOT have any effect on this target, which is strange and somehow a problem since Raspberry Pi OS uses it for network configuration by default 🤔.

So if no network is configured, the target is reached once the loopback interface is available. If a network is configured, it depends on the network stack, but the way Raspberry Pi OS is configured, no additional delay happens.

Btw, network.target is not intended to indicate that the device is online. For this, network-online.target is intended. Since it is an active target, it needs to be pulled it:

Wants=network-online.target
After=network-online.target

But currently (with the current network stacks) it has not the intended meaning but is reached simply immediately after network.target.

More info: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

• But it has to be taken care since there is quite some discrepancy between the intended meaning by systemd and the actual meaning on the system, which depends on network stack and consumer services.
• The effective meaning can be best seen with:

  systemctl show -p Before,After network.target network-online.target

[dermotbradley]

Hi folks

Regarding cloud-init I just though I'd point out that I have been using it to bootstrap RPIs running Alpine Linux (as well as PCs, x86/x86_64/arm64 VMs and Cloud Servers) for some time using my own script.

For physical machines such as RPIs I'm creating a small partition to hold the YAML files. More info here: https://github.com/dermotbradley/create-alpine-disk-image/blob/main/Physical/RaspberryPi.md

[XECDesign]

Closing this, since it won't be done this way.

For people who require this functionality right now, there is the systemd.run method outlined earlier.

We're experimenting with a simple customisation toml file where I'll add similar functionality.

At some point, possibly for the bookworm release, we'll add support for cloud-init. It's not quite mature enough to support our use case right now, hence the stop-gap customisation method.