Update boot order configuration for eMMC first, then network

Modify the first-boot script and documentation to set the EEPROM boot order to 0xf21, prioritizing eMMC boot followed by network boot. Adjust network boot settings for faster failure on DHCP timeouts and update related scripts and documentation to reflect these changes. Enhance the rescue script to directly modify EEPROM settings without requiring a chroot into eMMC, streamlining the recovery process for devices stuck in network-only boot. Update relevant documentation to ensure clarity on the new boot order and its implications.
2026-02-21 15:05:17 +02:00
parent ff6258c2af
commit 5238d457e8
13 changed files with 348 additions and 247 deletions
--- a/emmc-provisioning/network-boot-initramfs/README.md
+++ b/emmc-provisioning/network-boot-initramfs/README.md
@@ -1,33 +1,36 @@
 # Provisioning initramfs for network boot

-Minimal initramfs that runs **provisioning-client.sh** after bringing up the network. Used with Raspberry Pi 4 / CM4 (reTerminal) when booting via TFTP from the provisioning LXC.
+Alpine Linux-based initramfs that runs **provisioning-client.sh** after bringing up the network. Used with Raspberry Pi 4 / CM4 (reTerminal) when booting via TFTP from the provisioning LXC.
+
+Includes Python 3 and `rpi-eeprom-config` so EEPROM configuration can be modified directly from the initramfs without chrooting into eMMC.

 ## What it does

 1. Mounts `/proc`, `/sys`, `/dev`, `/dev/pts`.
-2. Ensures an IP (reuses kernel DHCP or runs `udhcpc` on eth0).
+2. Brings up eth0 and obtains a DHCP lease via `udhcpc`.
 3. Runs the provisioning client with `PROVISIONING_SERVER` (default `http://10.20.50.1:5000`, overridable via kernel cmdline).
-4. The client registers with the dashboard and polls for **Deploy** or **Backup**; on action it performs the dd + curl and exits.
+4. The client registers with the dashboard and polls for **Deploy** or **Backup**; on action it performs the dd + curl and reboots.

 ## Build

-**On x86_64 (e.g. your laptop):** the script uses **Podman** or **Docker** with `--platform linux/arm64` to run an arm64 container and copy busybox + curl into the initramfs. Your host must be able to *run* arm64 containers (via QEMU emulation).
+The build script uses Docker or Podman with `--platform linux/arm64` to create an Alpine aarch64 rootfs with Python 3, curl, and `rpi-eeprom-config`. Your host must support arm64 containers via QEMU emulation.

- **Fedora:** one-time setup to enable arm64 containers:
+### Prerequisites
+
+- **Docker** or **Podman** installed
+- **arm64 emulation** (QEMU user-static):
  ```bash
+  # Fedora
  sudo dnf install -y qemu-user-static
-  ```
-  Then run the build (Podman will use QEMU automatically):
-  ```bash
-  cd emmc-provisioning/network-boot-initramfs
-  ./build.sh
-  ```
- If you don’t install `qemu-user-static`, the script will fail with an error and print the same instructions and an alternative (build on a Pi).

-**On a Raspberry Pi 4 or other aarch64 host:** no Docker. Install deps and run:
+  # Debian/Ubuntu
+  sudo apt install -y qemu-user-static
+  ```
+
+### Build the initramfs

 ```bash
-sudo apt install -y busybox curl
+cd emmc-provisioning/network-boot-initramfs
 ./build.sh
 ```

@@ -37,6 +40,8 @@ Optional: pass an output path:
 ./build.sh /path/to/initrd.img
 ```

+The resulting `initrd.img` is approximately 25-35 MB compressed (Alpine base + Python + EEPROM firmware).
+
 ## Deploy to TFTP root

 1. Copy **initrd.img** to the LXC TFTP root (e.g. `/srv/tftpboot`):
@@ -51,7 +56,7 @@ Optional: pass an output path:
   initramfs initrd.img followkernel
   ```

-   So the firmware loads the kernel and then the initrd that “follows” it. The Pi will boot the kernel and run `/init` from the initrd.
+   So the firmware loads the kernel and then the initrd that "follows" it. The Pi will boot the kernel and run `/init` from the initrd.

 3. If your DHCP already points the Pi to this TFTP server and `start4cd.elf`, the Pi will load kernel + initrd from the same root. No NFS or extra server needed.

@@ -67,15 +72,43 @@ The init script reads `provisioning_server=` from `/proc/cmdline` and exports `P

 ### Rescue mode (stuck in network-only boot)

-If the device has **BOOT_ORDER=0x2** (network only), it never boots from eMMC. To get a shell and change boot order using the eMMC’s **rpi-eeprom-config**, add **provisioning_rescue=1** to the kernel cmdline (e.g. in the TFTP-served `cmdline.txt` for that device). The initramfs will then start an interactive shell instead of the provisioning client. Run **/rescue-eeprom.sh** to mount eMMC and chroot to run `rpi-eeprom-config --edit`; set `BOOT_ORDER=0x1` or `0x21`, save, then `reboot`. See **docs/NETWORK-BOOT-TROUBLESHOOTING.md** (“Stuck in network-only boot”) for full steps.
+If the device has **BOOT_ORDER=0x2** (network only), it never boots from eMMC. To fix the EEPROM config, add **provisioning_rescue=1** to the kernel cmdline (e.g. in the TFTP-served `cmdline.txt` for that device). The initramfs will start an interactive shell instead of the provisioning client.
+
+Run **/rescue-eeprom.sh** to set `BOOT_ORDER=0xf21` directly from the initramfs. The script:
+
+1. Reads the current EEPROM config using `rpi-eeprom-config` (included in the initramfs)
+2. Creates a modified config with `BOOT_ORDER=0xf21` and tuned network timeouts
+3. Embeds the config into a new EEPROM image using the bundled `pieeprom.bin` firmware
+4. Copies the update (`pieeprom.upd` + `pieeprom.sig`) to the eMMC boot partition
+
+No chroot, no EDITOR hack, no dependency on the eMMC OS.
+
+After running the script, disable network boot on the LXC and reboot so the bootloader boots from eMMC and applies the update.
+
+See **docs/NETWORK-BOOT-TROUBLESHOOTING.md** ("Stuck in network-only boot") for full steps.
+
+## What's included in the initramfs
+
+| Component | Purpose |
+|-----------|---------|
+| Alpine Linux base | Minimal rootfs with `apk` package manager |
+| BusyBox | Core Unix utilities (sh, mount, ip, udhcpc, dd, etc.) |
+| Python 3 | Required by `rpi-eeprom-config` |
+| curl | HTTP client for provisioning dashboard API |
+| rpi-eeprom-config | EEPROM configuration tool (from rpi-eeprom repo) |
+| pieeprom.bin | EEPROM firmware image (for creating update files) |
+| init | Boot script: mounts fs, DHCP, rescue or provision |
+| provisioning-client.sh | Registers with dashboard, executes deploy/backup |
+| rescue-eeprom.sh | Sets EEPROM boot order directly |
+| udhcpc.script | Applies DHCP lease (IP, route, DNS) |

 ## Flow summary

-1. Pi does DHCP → gets IP and TFTP server (e.g. 10.20.50.1).
+1. Pi does DHCP -> gets IP and TFTP server (e.g. 10.20.50.1).
 2. Pi loads via TFTP: start4cd.elf, fixup4cd.dat, config.txt, cmdline.txt, kernel8.img, **initrd.img**.
 3. Kernel boots with initrd as root; runs `/init`.
 4. Init mounts minimal fs, ensures network, runs `/provisioning-client.sh`.
-5. Client registers and polls; you choose Deploy or Backup in the dashboard; client runs dd + curl and exits.
-6. After deploy, power cycle the Pi so it boots from eMMC.
+5. Client registers and polls; you choose Deploy or Backup in the dashboard; client runs dd + curl and reboots.
+6. After deploy, device boots from eMMC.

 See **docs/NETWORK-BOOT-DEPLOYMENT-FLOW.md** for the full deployment flow.
--- a/emmc-provisioning/network-boot-initramfs/build.sh
+++ b/emmc-provisioning/network-boot-initramfs/build.sh
@@ -2,173 +2,132 @@
 # Build provisioning initramfs for Raspberry Pi 4 / CM4 (aarch64).
 # Produces initrd.img (gzip cpio) for TFTP boot (config.txt: initramfs initrd.img followkernel).
 #
-# On x86_64: tries Docker/Podman with --platform linux/arm64; if that fails (no
-# arm64 emulation), downloads prebuilt static aarch64 busybox and curl. No sudo needed.
-# On aarch64 (e.g. Raspberry Pi): uses local busybox and curl if installed.
+# Uses an Alpine Linux aarch64 base with Python 3 so rpi-eeprom-config can run
+# directly in the initramfs (no chroot into eMMC needed for rescue).
+#
+# Requires Docker or Podman with arm64 emulation:
+#   Fedora: sudo dnf install -y qemu-user-static
+#   Debian/Ubuntu: sudo apt install -y qemu-user-static

 set -e
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 OUTPUT="${1:-$SCRIPT_DIR/initrd.img}"
-BUILD_DIR=$(mktemp -d)
-trap "rm -rf $BUILD_DIR" EXIT
+ALPINE_VERSION="3.22"
+RPI_EEPROM_REPO="https://github.com/raspberrypi/rpi-eeprom"
+RPI_EEPROM_RAW="https://raw.githubusercontent.com/raspberrypi/rpi-eeprom"

-echo "Build dir: $BUILD_DIR"
-
-# Layout: /init, /provisioning-client.sh, /revision.txt, /bin/busybox, ...
-mkdir -p "$BUILD_DIR"/{bin,usr/bin,proc,sys,dev,dev/pts,lib,mnt,etc,usr/share/udhcpc}
-cp "$SCRIPT_DIR/init" "$BUILD_DIR/init"
-cp "$SCRIPT_DIR/provisioning-client.sh" "$BUILD_DIR/provisioning-client.sh"
-cp "$SCRIPT_DIR/rescue-eeprom.sh" "$BUILD_DIR/rescue-eeprom.sh"
-cp "$SCRIPT_DIR/udhcpc.script" "$BUILD_DIR/usr/share/udhcpc/default.script"
-chmod +x "$BUILD_DIR/init" "$BUILD_DIR/provisioning-client.sh" "$BUILD_DIR/rescue-eeprom.sh" "$BUILD_DIR/usr/share/udhcpc/default.script"
-# Revision shown on serial so you can confirm the device is running the latest initrd
-REV=$(date +%Y%m%d-%H%M 2>/dev/null || echo "unknown")
-[ -d "$SCRIPT_DIR/../.git" ] && REV="${REV}-$(git -C "$SCRIPT_DIR" rev-parse --short HEAD 2>/dev/null)" || true
-echo "$REV" > "$BUILD_DIR/revision.txt"
-
-ARCH=$(uname -m 2>/dev/null)
-if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ] || [ "$ARCH" = "armv8l" ]; then
-  if ! command -v busybox >/dev/null 2>&1 || ! command -v curl >/dev/null 2>&1; then
-    echo "On aarch64: install busybox and curl (apt install busybox curl) then re-run."
-    exit 1
+# Find container runtime
+CONTAINER_RUNTIME=""
+for cmd in podman docker; do
+  if command -v "$cmd" >/dev/null 2>&1; then
+    CONTAINER_RUNTIME="$cmd"
+    break
  fi
-  echo "Copying busybox and curl from host (aarch64)..."
-  cp "$(command -v busybox)" "$BUILD_DIR/bin/busybox"
-  cp "$(command -v curl)" "$BUILD_DIR/usr/bin/curl"
-  chmod +x "$BUILD_DIR/bin/busybox" "$BUILD_DIR/usr/bin/curl"
-  for f in $(ldd "$(command -v curl)" 2>/dev/null | awk '/=>/{print $3}'); do
-    [ -f "$f" ] && cp "$f" "$BUILD_DIR/lib/"
-  done
-  cp /lib/ld-linux-aarch64.so.1 "$BUILD_DIR/lib/" 2>/dev/null || true
-else
-  # x86_64: try container first; on "Exec format error" (no arm64 emulation) use static downloads
-  CONTAINER_RUNTIME=""
-  for cmd in docker podman; do
-    if command -v "$cmd" >/dev/null 2>&1; then
-      CONTAINER_RUNTIME="$cmd"
-      break
-    fi
-  done
-
-  CONTAINER_OK=0
-  if [ -n "$CONTAINER_RUNTIME" ]; then
-    echo "Trying $CONTAINER_RUNTIME (linux/arm64)..."
-    CNT_NAME="cm4-initramfs-build-$$"
-    # Use a container-internal dir (no bind mount); copy out with podman cp (works with rootless)
-    $CONTAINER_RUNTIME run --name "$CNT_NAME" --platform linux/arm64 debian:bookworm-slim bash -c '
-      apt-get update -qq && apt-get install -y -qq busybox curl
-      mkdir -p /out/bin /out/usr/bin /out/lib
-      cp /bin/busybox /out/bin/busybox
-      cp /usr/bin/curl /out/usr/bin/curl
-      chmod +x /out/bin/busybox /out/usr/bin/curl
-      ldd /usr/bin/curl 2>/dev/null | awk "/=>/{print \$3}" | while read f; do [ -n "$f" ] && [ -f "$f" ] && cp "$f" /out/lib/; done
-      cp /lib/ld-linux-aarch64.so.1 /out/lib/ 2>/dev/null || true
-    ' 2>/dev/null || true
-    # Always copy from container (avoids rootless bind-mount issues)
-    $CONTAINER_RUNTIME cp "$CNT_NAME:/out/bin/busybox" "$BUILD_DIR/bin/" 2>/dev/null && \
-    $CONTAINER_RUNTIME cp "$CNT_NAME:/out/usr/bin/curl" "$BUILD_DIR/usr/bin/" 2>/dev/null && \
-    $CONTAINER_RUNTIME cp "$CNT_NAME:/out/lib/." "$BUILD_DIR/lib/" 2>/dev/null || true
-    $CONTAINER_RUNTIME rm -f "$CNT_NAME" >/dev/null 2>&1
-    if [ -f "$BUILD_DIR/bin/busybox" ] && [ -f "$BUILD_DIR/usr/bin/curl" ]; then
-      CONTAINER_OK=1
-      echo "Container build succeeded."
-    fi
-  fi
-
-  if [ "$CONTAINER_OK" -ne 1 ]; then
-    echo "Using prebuilt static aarch64 binaries (no container/emulation needed)..."
-    DOWNLOAD_DIR=$(mktemp -d)
-    trap "rm -rf $BUILD_DIR $DOWNLOAD_DIR" EXIT
-    if command -v curl >/dev/null 2>&1; then
-      GET="curl -sL"
-      GET_O="curl -sL -o"
-    else
-      GET="wget -q -O -"
-      GET_O="wget -q -O"
-    fi
-    # Static busybox aarch64: try busybox.net, then Alpine busybox-static package
-    BB_OK=0
-    $GET_O "$DOWNLOAD_DIR/busybox" "https://busybox.net/downloads/binaries/1.35.0-defconfig-multiarch-musl/busybox-armv8l" 2>/dev/null || true
-    if [ -f "$DOWNLOAD_DIR/busybox" ] && [ -s "$DOWNLOAD_DIR/busybox" ]; then
-      BB_OK=1
-    fi
-    if [ "$BB_OK" -ne 1 ]; then
-      echo "Trying Alpine busybox-static aarch64..."
-      $GET_O "$DOWNLOAD_DIR/bb.apk" "https://dl-cdn.alpinelinux.org/alpine/v3.19/main/aarch64/busybox-static-1.36.1-r11.apk" 2>/dev/null || true
-      if [ -f "$DOWNLOAD_DIR/bb.apk" ] && [ -s "$DOWNLOAD_DIR/bb.apk" ]; then
-        case "$(head -c 4 "$DOWNLOAD_DIR/bb.apk" | od -An -tx1 2>/dev/null | tr -d ' ')" in
-          3c21444f|3c68746d) ;; # HTML response, skip extract
-          *)
-            (cd "$DOWNLOAD_DIR" && (tar xf bb.apk 2>/dev/null || tar xzf bb.apk 2>/dev/null) && [ -f data.tar.gz ] && tar xzf data.tar.gz 2>/dev/null)
-            ;;
-        esac
-      fi
-      if [ -f "$DOWNLOAD_DIR/bin/busybox" ] && [ -s "$DOWNLOAD_DIR/bin/busybox" ]; then
-        cp "$DOWNLOAD_DIR/bin/busybox" "$DOWNLOAD_DIR/busybox"
-        BB_OK=1
-      fi
-    fi
-    if [ "$BB_OK" -ne 1 ]; then
-      echo "Failed to download busybox (x86 host cannot run arm64 container without emulation)."
-      echo ""
-      echo "Option A - Enable arm64 containers (one-time, needs sudo):"
-      echo "  Fedora: sudo dnf install -y qemu-user-static"
-      echo "  Then re-run this script (Podman will use QEMU to run the arm64 build)."
-      echo ""
-      echo "Option B - Build on a Raspberry Pi 4 (aarch64):"
-      echo "  scp -r $(dirname "$SCRIPT_DIR") pi@<pi-ip>:~/  && ssh pi@<pi-ip> 'cd ~/emmc-provisioning/network-boot-initramfs && sudo apt install -y busybox curl && ./build.sh'"
-      echo "  Then scp pi@<pi-ip>:~/emmc-provisioning/network-boot-initramfs/initrd.img ."
-      exit 1
-    fi
-    chmod +x "$DOWNLOAD_DIR/busybox"
-    cp "$DOWNLOAD_DIR/busybox" "$BUILD_DIR/bin/busybox"
-    # Static curl aarch64 glibc (Raspberry Pi OS uses glibc)
-    $GET "https://github.com/stunnel/static-curl/releases/download/8.18.0/curl-linux-aarch64-glibc-8.18.0.tar.xz" -o "$DOWNLOAD_DIR/curl.tar.xz" || true
-    if [ ! -f "$DOWNLOAD_DIR/curl.tar.xz" ] || [ ! -s "$DOWNLOAD_DIR/curl.tar.xz" ]; then
-      echo "Failed to download static curl."
-      exit 1
-    fi
-    (cd "$DOWNLOAD_DIR" && tar xf curl.tar.xz)
-    CURL_BIN=$(find "$DOWNLOAD_DIR" -maxdepth 3 -name "curl" -type f 2>/dev/null | head -1)
-    if [ -n "$CURL_BIN" ] && [ -x "$CURL_BIN" ]; then
-      cp "$CURL_BIN" "$BUILD_DIR/usr/bin/curl"
-      chmod +x "$BUILD_DIR/usr/bin/curl"
-    else
-      echo "Could not find curl binary in tarball."
-      exit 1
-    fi
-    rm -rf "$DOWNLOAD_DIR"
-    trap "rm -rf $BUILD_DIR" EXIT
-  fi
-fi
-
-# Verify we have busybox (container or fallback must have left it)
-if [ ! -f "$BUILD_DIR/bin/busybox" ] || [ ! -s "$BUILD_DIR/bin/busybox" ]; then
-  echo "Error: busybox not found in $BUILD_DIR/bin. If the container ran, check Podman volume mount."
+done
+if [ -z "$CONTAINER_RUNTIME" ]; then
+  echo "Error: Docker or Podman is required to build the Alpine-based initramfs."
+  echo "Install one of them and ensure arm64 emulation is available:"
+  echo "  Fedora: sudo dnf install -y podman qemu-user-static"
+  echo "  Debian/Ubuntu: sudo apt install -y podman qemu-user-static"
  exit 1
 fi
-chmod +x "$BUILD_DIR/bin/busybox" 2>/dev/null || true

-# Busybox applet symlinks (mount, mkdir, etc.). When building arm64 on x86, busybox cannot be run so --list fails; create symlinks manually.
-APPLETS="sh ash mount umount mkdir cat ip udhcpc sleep echo grep cut awk hostname dd reboot chroot ls rm"
-cd "$BUILD_DIR/bin"
-if ./busybox --list >/dev/null 2>&1; then
-  ./busybox --list | while read applet; do
-    case " $APPLETS " in *" $applet "*) ln -sf busybox "$applet"; ;; esac
-  done
-else
-  for applet in $APPLETS; do
-    [ -e "$applet" ] || ln -sf busybox "$applet"
-  done
+echo "Using $CONTAINER_RUNTIME to build Alpine aarch64 initramfs..."
+
+# Revision shown on serial so you can confirm the device is running the latest initrd
+REV=$(date +%Y%m%d-%H%M 2>/dev/null || echo "unknown")
+if [ -d "$SCRIPT_DIR/../.git" ] || [ -d "$SCRIPT_DIR/../../.git" ]; then
+  REV="${REV}-$(git -C "$SCRIPT_DIR" rev-parse --short HEAD 2>/dev/null || echo 'nogit')"
 fi
-[ -e sh ] || ln -sf busybox sh
+
+CNT_NAME="cm4-initramfs-build-$$"
+trap "$CONTAINER_RUNTIME rm -f $CNT_NAME >/dev/null 2>&1; true" EXIT
+
+# Build the rootfs inside an arm64 Alpine container
+$CONTAINER_RUNTIME run --name "$CNT_NAME" --platform linux/arm64 \
+  "alpine:${ALPINE_VERSION}" /bin/sh -c "
+set -e
+
+# Install packages: Python for rpi-eeprom-config, curl for provisioning client, coreutils for dd
+apk add --no-cache python3 curl busybox coreutils
+
+# Download rpi-eeprom tools from GitHub (use curl -sL which follows redirects; busybox wget does not)
+curl -sL -o /usr/bin/rpi-eeprom-config \
+  '${RPI_EEPROM_RAW}/master/rpi-eeprom-config'
+curl -sL -o /usr/bin/rpi-eeprom-update \
+  '${RPI_EEPROM_RAW}/master/rpi-eeprom-update'
+curl -sL -o /usr/bin/rpi-eeprom-digest \
+  '${RPI_EEPROM_RAW}/master/rpi-eeprom-digest'
+chmod +x /usr/bin/rpi-eeprom-config /usr/bin/rpi-eeprom-update /usr/bin/rpi-eeprom-digest
+
+# Download latest stable CM4 (BCM2711) EEPROM firmware
+mkdir -p /lib/firmware/raspberrypi/bootloader/default
+# Use GitHub API to find the latest pieeprom-*.bin in firmware-2711/default/
+LATEST_FW=\$(curl -sL 'https://api.github.com/repos/raspberrypi/rpi-eeprom/contents/firmware-2711/default' \
+  | grep -o '\"name\" *: *\"pieeprom-[^\"]*\\.bin\"' | sed 's/\"name\" *: *\"//;s/\"//' | sort | tail -1)
+if [ -n \"\$LATEST_FW\" ]; then
+  echo \"Downloading EEPROM firmware: \$LATEST_FW\"
+  curl -sL -o /lib/firmware/raspberrypi/bootloader/default/pieeprom.bin \
+    \"${RPI_EEPROM_RAW}/master/firmware-2711/default/\$LATEST_FW\"
+else
+  echo 'WARNING: Could not determine latest firmware; rescue-eeprom.sh may not work'
+fi
+
+# Create required directories
+mkdir -p /proc /sys /dev/pts /mnt /tmp /run /usr/share/udhcpc /etc/default
+
+# Clean up unnecessary files to reduce size
+rm -rf /var/cache/apk/* /usr/share/man /usr/share/doc /usr/include \
+  /usr/lib/python*/test /usr/lib/python*/unittest /usr/lib/python*/idlelib \
+  /usr/lib/python*/tkinter /usr/lib/python*/ensurepip /usr/lib/python*/__pycache__/test* \
+  /usr/lib/python*/lib2to3 /usr/lib/python*/distutils \
+  /usr/share/terminfo/[a-s]* /usr/share/terminfo/[u-z]*
+
+echo 'Alpine rootfs ready'
+"
+
+echo "Container build done. Copying scripts into container..."
+
+# Copy our scripts into the container
+$CONTAINER_RUNTIME cp "$SCRIPT_DIR/init" "$CNT_NAME:/init"
+$CONTAINER_RUNTIME cp "$SCRIPT_DIR/provisioning-client.sh" "$CNT_NAME:/provisioning-client.sh"
+$CONTAINER_RUNTIME cp "$SCRIPT_DIR/rescue-eeprom.sh" "$CNT_NAME:/rescue-eeprom.sh"
+$CONTAINER_RUNTIME cp "$SCRIPT_DIR/udhcpc.script" "$CNT_NAME:/usr/share/udhcpc/default.script"
+
+# Write revision.txt
+echo "$REV" | $CONTAINER_RUNTIME cp /dev/stdin "$CNT_NAME:/revision.txt" 2>/dev/null || \
+  (echo "$REV" > /tmp/cm4-rev-$$.txt && $CONTAINER_RUNTIME cp /tmp/cm4-rev-$$.txt "$CNT_NAME:/revision.txt" && rm -f /tmp/cm4-rev-$$.txt)
+
+# Set permissions
+$CONTAINER_RUNTIME start "$CNT_NAME" >/dev/null 2>&1 || true
+$CONTAINER_RUNTIME exec "$CNT_NAME" chmod +x /init /provisioning-client.sh /rescue-eeprom.sh /usr/share/udhcpc/default.script 2>/dev/null || \
+  echo "Note: could not chmod in stopped container; permissions set by cp"
+
+# Export container filesystem and create cpio archive
+echo "Exporting filesystem and building initrd.img..."
+$CONTAINER_RUNTIME export "$CNT_NAME" | gzip -1 > /tmp/cm4-rootfs-$$.tar.gz
+
+BUILD_DIR=$(mktemp -d)
+trap "$CONTAINER_RUNTIME rm -f $CNT_NAME >/dev/null 2>&1; rm -rf $BUILD_DIR /tmp/cm4-rootfs-$$.tar.gz; true" EXIT
+
+cd "$BUILD_DIR"
+tar xzf /tmp/cm4-rootfs-$$.tar.gz
+rm -f /tmp/cm4-rootfs-$$.tar.gz
+
+# Remove container metadata that shouldn't be in initramfs
+rm -rf .dockerenv .containerenv
+
+# Ensure init is executable and at the root
+chmod +x init provisioning-client.sh rescue-eeprom.sh usr/share/udhcpc/default.script 2>/dev/null || true

 # Build cpio (gzip)
 echo "Building cpio..."
-( cd "$BUILD_DIR"; find . -print0 | cpio -o -H newc -0 2>/dev/null ) | gzip -9 > "$OUTPUT"
-echo "Written: $OUTPUT ($(stat -c%s "$OUTPUT" 2>/dev/null || stat -f%z "$OUTPUT" 2>/dev/null) bytes)"
+find . -print0 | cpio -o -H newc -0 2>/dev/null | gzip -9 > "$OUTPUT"
+
+SIZE=$(stat -c%s "$OUTPUT" 2>/dev/null || stat -f%z "$OUTPUT" 2>/dev/null || echo "?")
+echo ""
+echo "Written: $OUTPUT ($SIZE bytes, $(( ${SIZE:-0} / 1048576 )) MB)"
 echo ""
 echo "Next: copy initrd.img to TFTP root (e.g. /srv/tftpboot on LXC) and in config.txt add:"
 echo "  initramfs initrd.img followkernel"
-echo "Then ensure the kernel line loads the initrd (followkernel does that)."
-echo "Default provisioning server: http://10.20.50.1:5000 (override with kernel cmdline: provisioning_server=http://...)"
+echo "Default provisioning server: http://10.20.50.1:5000 (override with kernel cmdline: provisioning_server=http://...)"
--- a/emmc-provisioning/network-boot-initramfs/init
+++ b/emmc-provisioning/network-boot-initramfs/init
@@ -1,10 +1,11 @@
 #!/bin/sh
 # Init for provisioning initramfs: bring up minimal env and run provisioning-client.sh.
 # PROVISIONING_SERVER can be set via kernel cmdline: provisioning_server=http://10.20.50.1:5000
+# Based on Alpine Linux aarch64 with Python 3 and rpi-eeprom-config.

 set -e
-export PATH=/bin:/usr/bin
-export LD_LIBRARY_PATH=/lib
+export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+export LD_LIBRARY_PATH=/lib:/usr/lib

 echo "=== CM4 provisioning initramfs ==="
 # Revision is set at build time; cat /revision.txt to confirm you have the latest initrd on TFTP
@@ -52,7 +53,7 @@ export PROVISIONING_SERVER

 if [ "$RESCUE" -eq 1 ]; then
  echo "=== RESCUE MODE (provisioning_rescue=1) ==="
-  echo "Run /rescue-eeprom.sh to mount eMMC and change boot order (rpi-eeprom-config), then reboot."
+  echo "Run /rescue-eeprom.sh to set EEPROM boot order (runs rpi-eeprom-config directly), then reboot."
  # Ensure shell I/O goes to serial console (some setups drop output otherwise)
  [ -c /dev/console ] && exec </dev/console >/dev/console 2>&1
  exec /bin/sh -i
--- a/emmc-provisioning/network-boot-initramfs/initrd.img
+++ b/emmc-provisioning/network-boot-initramfs/initrd.img
--- a/emmc-provisioning/network-boot-initramfs/rescue-eeprom.sh
+++ b/emmc-provisioning/network-boot-initramfs/rescue-eeprom.sh
@@ -1,36 +1,138 @@
 #!/bin/sh
-# Rescue script: mount eMMC root and chroot to run rpi-eeprom-config.
-# Use this when stuck in network-only boot (BOOT_ORDER=0x2) to set BOOT_ORDER=0x1 or 0x21.
+# Rescue script: set EEPROM boot order directly from the initramfs (no chroot needed).
+# Uses rpi-eeprom-config + pieeprom.bin bundled in the Alpine-based initramfs.
+# Sets BOOT_ORDER=0xf21 (eMMC first, then network, restart) with fast network timeouts.
 # Run from the initramfs rescue shell (after booting with provisioning_rescue=1 in cmdline).
+# Pass --edit to open the editor manually instead of applying automatically.

 set -e
-ROOT="/mnt/emmc"
-BOOT="$ROOT/boot/firmware"
-[ -d "$ROOT/boot" ] && [ ! -d "$BOOT" ] && BOOT="$ROOT/boot"
+EEPROM_FW="/lib/firmware/raspberrypi/bootloader/default/pieeprom.bin"
+BOOT_MNT="/mnt/boot"
+MANUAL=0
+[ "$1" = "--edit" ] && MANUAL=1

-echo "=== Mounting eMMC for EEPROM config ==="
-# CM4 / reTerminal: eMMC is usually mmcblk0, p1=boot (FAT), p2=root (ext4)
-if [ ! -b /dev/mmcblk0p2 ]; then
-  echo "No /dev/mmcblk0p2 found. Try: ls /dev/mmcblk*"
+# Clean up any previous mounts
+umount "$BOOT_MNT" 2>/dev/null || true
+umount /mnt/emmc 2>/dev/null || true
+
+# --- Read current EEPROM config ---
+echo "=== EEPROM rescue (Alpine initramfs) ==="
+
+if ! command -v rpi-eeprom-config >/dev/null 2>&1; then
+  echo "ERROR: rpi-eeprom-config not found in initramfs."
+  echo "This initramfs was not built with the Alpine build script."
  exit 1
 fi

-mkdir -p "$ROOT"
-mount /dev/mmcblk0p2 "$ROOT" || { echo "Mount root failed"; exit 1; }
-if [ -b /dev/mmcblk0p1 ]; then
-  mkdir -p "$BOOT"
-  mount /dev/mmcblk0p1 "$BOOT" 2>/dev/null || true
+if [ ! -f "$EEPROM_FW" ]; then
+  echo "ERROR: EEPROM firmware not found at $EEPROM_FW"
+  echo "Rebuild the initramfs with build.sh to include it."
+  exit 1
 fi
-mount -t proc none "$ROOT/proc"
-mount -t sysfs none "$ROOT/sys"
-mount --bind /dev "$ROOT/dev"
-mount --bind /dev/pts "$ROOT/dev/pts" 2>/dev/null || true

-if [ -x "$ROOT/usr/bin/rpi-eeprom-config" ]; then
-  echo "Chroot to eMMC and run: rpi-eeprom-config --edit"
-  echo "Set BOOT_ORDER=0x1 (eMMC only) or 0x21 (network first, then eMMC), save, then exit and run: reboot"
-  chroot "$ROOT" /usr/bin/rpi-eeprom-config --edit
-else
-  echo "rpi-eeprom-config not found in eMMC. Chrooting anyway; run: apt install rpi-eeprom && rpi-eeprom-config --edit"
-  chroot "$ROOT" /bin/sh -i
+echo "Reading current EEPROM config from running bootloader..."
+CURRENT_CONF="/tmp/eeprom-current.conf"
+rpi-eeprom-config 2>/dev/null > "$CURRENT_CONF" || true
+
+if [ ! -s "$CURRENT_CONF" ]; then
+  echo "Could not read current EEPROM config via vcgencmd."
+  echo "Extracting config from firmware image instead..."
+  rpi-eeprom-config "$EEPROM_FW" > "$CURRENT_CONF" 2>/dev/null || true
 fi
+
+if [ ! -s "$CURRENT_CONF" ]; then
+  echo "ERROR: Could not read EEPROM config from either source."
+  exit 1
+fi
+
+echo "Current config:"
+cat "$CURRENT_CONF"
+echo ""
+
+# --- Manual mode: mount eMMC boot, chroot, edit ---
+if [ "$MANUAL" -eq 1 ]; then
+  echo "Manual mode: mounting eMMC for interactive editing..."
+  mkdir -p /mnt/emmc
+  mount /dev/mmcblk0p2 /mnt/emmc 2>/dev/null || true
+  BOOT="/mnt/emmc/boot/firmware"
+  [ ! -d "$BOOT" ] && BOOT="/mnt/emmc/boot"
+  if [ -b /dev/mmcblk0p1 ]; then
+    mkdir -p "$BOOT"
+    mount /dev/mmcblk0p1 "$BOOT" 2>/dev/null || true
+  fi
+  mount -t proc none /mnt/emmc/proc 2>/dev/null || true
+  mount -t sysfs none /mnt/emmc/sys 2>/dev/null || true
+  mount --bind /dev /mnt/emmc/dev 2>/dev/null || true
+  if [ -x /mnt/emmc/usr/bin/rpi-eeprom-config ]; then
+    chroot /mnt/emmc /usr/bin/rpi-eeprom-config --edit
+  else
+    echo "rpi-eeprom-config not found on eMMC. Dropping to shell."
+    chroot /mnt/emmc /bin/sh -i
+  fi
+  exit 0
+fi
+
+# --- Automatic mode: build new config and apply ---
+NEW_CONF="/tmp/eeprom-new.conf"
+
+# Keep settings we don't modify, strip the ones we replace
+grep -v '^BOOT_ORDER=' "$CURRENT_CONF" \
+  | grep -v '^NET_BOOT_MAX_RETRIES=' \
+  | grep -v '^DHCP_TIMEOUT=' \
+  | grep -v '^DHCP_REQ_TIMEOUT=' \
+  | grep -v '^TFTP_IP=' \
+  | grep -v '^NET_INSTALL_AT_POWER_ON=' \
+  > "$NEW_CONF" || true
+
+echo 'BOOT_ORDER=0xf21' >> "$NEW_CONF"
+echo 'NET_BOOT_MAX_RETRIES=3' >> "$NEW_CONF"
+echo 'DHCP_TIMEOUT=1500' >> "$NEW_CONF"
+echo 'DHCP_REQ_TIMEOUT=500' >> "$NEW_CONF"
+echo 'NET_INSTALL_AT_POWER_ON=0' >> "$NEW_CONF"
+
+echo "New config to apply:"
+cat "$NEW_CONF"
+echo ""
+
+# Create the modified EEPROM image with the new config embedded
+EEPROM_OUT="/tmp/pieeprom.upd"
+echo "Embedding config into EEPROM firmware image..."
+rpi-eeprom-config --config "$NEW_CONF" --out "$EEPROM_OUT" "$EEPROM_FW"
+
+if [ ! -f "$EEPROM_OUT" ] || [ ! -s "$EEPROM_OUT" ]; then
+  echo "ERROR: Failed to create modified EEPROM image."
+  exit 1
+fi
+
+# Generate the signature file (sha256 of the .upd, named .sig)
+EEPROM_SIG="/tmp/pieeprom.sig"
+sha256sum "$EEPROM_OUT" | awk '{print $1}' > "$EEPROM_SIG"
+
+# Mount eMMC boot partition and copy the update files
+echo "Mounting eMMC boot partition..."
+if [ ! -b /dev/mmcblk0p1 ]; then
+  echo "ERROR: /dev/mmcblk0p1 not found. Is eMMC present?"
+  exit 1
+fi
+
+mkdir -p "$BOOT_MNT"
+mount /dev/mmcblk0p1 "$BOOT_MNT" || { echo "ERROR: Could not mount boot partition"; exit 1; }
+
+cp "$EEPROM_OUT" "$BOOT_MNT/pieeprom.upd"
+cp "$EEPROM_SIG" "$BOOT_MNT/pieeprom.sig"
+sync
+
+echo ""
+echo "=== EEPROM update written to eMMC boot partition ==="
+echo "  BOOT_ORDER=0xf21 (eMMC first, then network, restart)"
+echo "  NET_BOOT_MAX_RETRIES=3, DHCP_TIMEOUT=1500ms"
+echo "  Files: pieeprom.upd + pieeprom.sig on /dev/mmcblk0p1"
+echo ""
+echo "The bootloader will apply this update on next boot from eMMC."
+echo ""
+echo "Next steps:"
+echo "  1. Disable network boot on the LXC (so next boot falls through to eMMC)"
+echo "  2. Reboot: reboot -f  (or: echo b > /proc/sysrq-trigger)"
+
+umount "$BOOT_MNT" 2>/dev/null || true
+rm -f "$CURRENT_CONF" "$NEW_CONF" "$EEPROM_OUT" "$EEPROM_SIG"