Skip to content

Commit

Permalink
Minor edits
Browse files Browse the repository at this point in the history
  • Loading branch information
billz committed Nov 20, 2024
1 parent cf4b670 commit 446f895
Showing 1 changed file with 18 additions and 18 deletions.
36 changes: 18 additions & 18 deletions docs/minwrite.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ Linux, and indeed most substantial operating systems, is frequently writing logs

What is more, most microSD cards were not designed with 24/7 operation in mind. Continuous writing to the card's flash memory shortens its lifespan. They often accumulate bad sectors rather quickly after a period of extended use. This is particularly true of so-called "budget" microSD cards.

Using a Raspberry Pi as an access point requires reliable operation over a long period of time. While "read-only mode" operation for the SD card is one approach to prolong its use, this prevents user settings from being persisted to storage — meaning that any changes will be lost if the device is disconnected from power. This makes it less than ideal for RaspAP, or indeed any application such as a web server or database that depends on persistent storage.
Using a Raspberry Pi as wireless router requires reliable operation over a long period of time. While _read-only mode_ operation for the SD card is one approach to prolong its use, this prevents user settings from being persisted to storage — meaning that any changes will be lost if the device is disconnected from power. This makes it less than ideal for RaspAP, or indeed any application that depends on persistent storage.

## Solution
Rather than force the system into a read-only mode, RaspAP has an alternative "minimal write mode" that substantially reduces the risk of SD card corruption and also helps to extend the card's lifespan.
Rather than force the system into a read-only mode, RaspAP has an alternative _minimal write mode_ that substantially reduces the risk of SD card corruption and also helps to extend the card's lifespan.

This solution involves moving logging, cache and temporary data to a RAM-based file system. The default system log processor `rsyslog` is replaced with an in-memory logger and several log-related services are disabled. The `tmpfs` filesystem is used for most processes that require write access, such as sessions used by `php-cgi`, as well as paths for transient and cache data including `/var/cache` and `/var/tmp`.

Expand All @@ -20,11 +20,11 @@ In addition, the system's boot options are modified to disable swap and file sys
The minimal microSD card write utility, **minwrite**, may be invoked by using RaspAP's [Quick installer](quick.md). This does _not_ (re)install RaspAP — only the [minwrite](https://github.com/RaspAP/raspap-webgui/blob/master/installers/minwrite.sh) shell script is loaded and executed. Users of this method are informed of which operations are performed at each step. Alternatively, [manual configuration](#manual-steps) steps are also provided. Notes specific to Armbian are given where applicable.

!!! warning "Warning"
These methods have been used successfully with many Debian-based systems. However, you still use this at your own risk. We recommend either creating a backup image of your SD card before proceeding, or begin with a baseline setup that you can easily recreate if needed.
These methods have been used successfully with many Debian-based systems. However, you still use this at your own risk. Best advice is to either create a backup image of your SD card before proceeding, or begin with a baseline setup that you can easily recreate if needed.

Both methods are reasonably straightforward. Bear in mind that RAM usage on your device will necessarily increase, since we'll be migrating the disk I/O activity of several system processes to the `tmpfs` ramdisk. For this reason, it's recommended to review the [memory considerations](#memory-considerations) before proceeding.
Both methods are reasonably straightforward. Bear in mind that RAM usage on your device will necessarily increase, since you'll be migrating the disk I/O activity of several system processes to the `tmpfs` ramdisk. For this reason, it's recommended to review the [memory considerations](#memory-considerations) before proceeding.

After we've enabled **minwrite** we'll look at a technique to evaluate its effectiveness.
After you've enabled **minwrite** we'll look at a technique to evaluate its effectiveness.

### Quick install
The [minwrite utility](https://github.com/RaspAP/raspap-webgui/blob/master/installers/minwrite.sh) may be invoked remotely from the [Quick installer](quick.md) like so:
Expand Down Expand Up @@ -52,7 +52,7 @@ a88aaaa8P' .d8888b. .d8888b. 88d888b. 88aaaaa88a a88aaaa8P
88 88 88. .88 88 88. .88 88 88 88
dP dP 88888P8 88888P 88Y888P 88 88 dP
88
dP version 2.8.8
dP version 3.2.1
The Quick Installer will guide you through a few easy steps
Expand Down Expand Up @@ -83,7 +83,7 @@ Proceed? [Y/n]:
These steps perform the same actions as the Quick install method. Details are provided so that you may choose to customize or skip some steps, if desired.

#### Remove packages
The goal here is to only remove packages that actively write to the filesystem, and that we intend to replace or disable entirely. In a subsequent step, `logrotate` will be replaced with `busybox-syslogd`.
The goal here is to only remove packages that actively write to the filesystem, and that will be replaced or disabled entirely. In a subsequent step, `logrotate` will be replaced with `busybox-syslogd`.
Additionally, `dphys-swapfile`, which manages a swapfile in the root filesystem on the SD card, is removed as it won’t be able to work.

Remove these packages with the following:
Expand All @@ -94,7 +94,7 @@ sudo apt-get autoremove --purge
```

#### Disable services
Linux is able to update packages autonomously without an external command. This task is scheduled by the `apt-daily.service`, which triggers the system to start `apt` tasks and scan installed packages for available updates. If updates are found, the `apt-daily-upgrade.service` downloads and installs them without user intervention. While useful for keeping your system updated, these are intensive processes in terms of disk I/O that we can safely disable and handle manually.
Linux is able to update packages autonomously without an external command. This task is scheduled by the `apt-daily.service`, which triggers the system to start `apt` tasks and scan installed packages for available updates. If updates are found, the `apt-daily-upgrade.service` downloads and installs them without user intervention. While useful for keeping your system updated, these are intensive processes in terms of disk I/O that may be safely disabled and handled manually.

Disable the `bootlogd.service`, `apt-daily` and related services like so:

Expand All @@ -107,7 +107,7 @@ sudo systemctl disable apt-daily.service apt-daily.timer apt-daily-upgrade.timer
By disabling these services, you will need to manually check for package updates periodically with `sudo apt-get update && sudo apt-get upgrade`.

#### Replace logger
In this step we'll replace the default system logger `rsyslog` with an in-memory logger, `busybox-syslogd`. [BusyBox](https://manpages.debian.org/jessie/busybox-syslogd/syslogd.8.en.html) combines tiny versions of many common Linux utilities into a single small executable. It provides a fairly complete POSIX environment for any small or embedded system, including a minimal write Raspberry Pi.
In this step you'll replace the default system logger `rsyslog` with an in-memory logger, `busybox-syslogd`. [BusyBox](https://manpages.debian.org/jessie/busybox-syslogd/syslogd.8.en.html) combines tiny versions of many common Linux utilities into a single small executable. It provides a fairly complete POSIX environment for any small or embedded system, including a minimal write Raspberry Pi.

Install it like so and remove `rsyslog`:

Expand All @@ -119,7 +119,7 @@ sudo dpkg --purge rsyslog
Be aware that because `busybox-syslogd` writes system logs to RAM, these logs will be lost if your device is disconnected from power.

#### Disable swap
Next we'll modify system boot options to disable [swap](https://wiki.debian.org/Swap) and filesystem checks, as these are both intensive disk I/O processes. Edit this file with `sudo nano /boot/cmdline.txt` and append the following to the end:
Next you'll modify system boot options to disable [swap](https://wiki.debian.org/Swap) and filesystem checks, as these are both intensive disk I/O processes. Edit this file with `sudo nano /boot/cmdline.txt` and append the following to the end:

```
fsck.mode=skip noswap
Expand All @@ -137,9 +137,9 @@ Save your changes and quit out of the editor with ++ctrl+x++ followed by ++y++ a
By default Armbian does not use any SD card-based swap, so unless you’ve customized your installation there’s nothing to disable.

#### Move directories to RAM
As a final step, we'll move several directories to the `tmpfs` filesystem. By storing these directories on a ramdisk instead of the SD card, we can substantially reduce the volume of I/O operations on the card's flash memory. Writing to `tmpfs` also provides fast sequential read/write speeds. The tradeoff is that `tmpfs` is _volatile storage_ — meaning that you will lose all data stored on the filesystem if you lose power.
As a final step, several directories will be moved to the `tmpfs` filesystem. Storing these directories on a ramdisk instead of the SD card will substantially reduce the volume of I/O operations on the card's flash memory. Writing to `tmpfs` also provides fast sequential read/write speeds. The tradeoff is that `tmpfs` is _volatile storage_ — meaning that you will lose all data stored on the filesystem if your device loses power.

We'll select paths to migrate to `tmpfs` for transient and cache data, as well as those required for RaspAP's operation that are associated with disk I/O activity. Moving these directories to `tmpfs` is done by editing `fstab` with `sudo nano /etc/fstab`. Append the following lines to the end:
Paths are selected here to migrate to `tmpfs` for transient and cache data, as well as those required for RaspAP's operation that are associated with disk I/O activity. Moving these directories to `tmpfs` is done by editing `fstab` with `sudo nano /etc/fstab`. Append the following lines to the end:

```
tmpfs /tmp tmpfs nosuid,nodev 0 0
Expand All @@ -162,9 +162,9 @@ The `/var/tmp` directory is made available for programs that require temporary f
A reboot is required for the above steps to take effect: `sudo reboot`.

## Memory considerations
The **minwrite** configuration migrates as much as possible from SD card storage to the `tmpfs` ramdisk. As a result, a concomitant increase in memory utilization is expected. To benchmark this, we can compare the change in memory usage on a Pi 3 Model B+ with 1GB of RAM with a typical RaspAP installation.
The **minwrite** configuration migrates as much as possible from SD card storage to the `tmpfs` ramdisk. As a result, a concomitant increase in memory utilization is expected. To benchmark this, the change in memory usage on a Pi 3 Model B+ with 1GB of RAM and a typical RaspAP installation will be compared.

Here we use the following to return the amount of free system memory expressed as a percentage of total available:
Execute the following to return the amount of free system memory expressed as a percentage of the total available:

```
free -m | awk '/Mem:/ { total=$2 ; used=$3 } END { print used/total*100}'
Expand All @@ -177,13 +177,13 @@ free -m | awk '/Mem:/ { total=$2 ; used=$3 } END { print used/total*100}'
While this is a noticable increase in RAM usage, it's still well within the margin for reliable operation of the OS. If you have a higher rate of RAM utilization on your device, or have limited available system memory to begin with, bear this in mind before proceeding.

!!! note "Note"
Recall that we've disabled swap, so if the system runs out of physical memory (RAM) there is no partition available for the kernel to allocate virtual memory in its place. This will cause the kernel to throw an out of memory (OOM) error. Normally this causes the kernel to panic and stop functioning.
Recall that with swap disabled, if the system runs out of physical memory (RAM) there is no partition available for the kernel to allocate virtual memory in its place. This will cause the kernel to throw an out of memory (OOM) error. Normally this causes the kernel to panic and stop functioning.

## File system metrics
We can evaluate a **minwrite** configuration by using `iotop`, a utility that watches I/O usage information output by the Linux kernel. Install it like so:
A **minwrite** configuration may be futher evaluated by using `iotop`, a utility that watches I/O usage information output by the Linux kernel. Install the package like so:

```
sudo apt-get install iotop
sudo apt-get install iotop -y
```

Execute it with the following switches to monitor accumulated activity of processes doing actual I/O:
Expand All @@ -192,7 +192,7 @@ Execute it with the following switches to monitor accumulated activity of proces
sudo iotop -aoP
```

After a period of time, you will see disk I/O activity reported for a number of processes. Returning to our Pi 3 Model B+ test bench, we can compare the before and after results:
After a period of time, you will see disk I/O activity reported for a number of processes. Returning to the example Pi 3 Model B+ test bench, the before and after results may be compared:

**Pre-minwrite I/O**
```
Expand Down

0 comments on commit 446f895

Please sign in to comment.