nearxos/reterminal-dm4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5238d457e849e9ab6e0aa60c054452aa78b89387
reterminal-dm4/emmc-provisioning/docs/EMMC-PROVISIONING-GUIDE.md

14 KiB
Raw Blame History

eMMC provisioning for reTerminal DM4 (CM4)

This guide covers:

  1. USB boot mode: When the reTerminal is in boot mode (eMMC disable jumper) and connected via USB, the host runs rpiboot to expose the eMMC, then the dashboard shows "Device connected (USB)". You choose Backup or Deploy in the portal — there is no auto-flash; the action runs only after your choice.
  2. Network boot: If the device boots over the network and runs the provisioning client (see network-client/), it registers with the dashboard and appears as "Device (Network)"; you then choose Backup or Deploy. Deploy streams the golden image to the device; Backup uploads the device eMMC to the server.
  3. Cloud-init: The golden image can include cloud-init so each device configures itself on first boot (hostname, network, packages, kiosk setup).

Part 1: USB boot mode — detect device, choose Backup or Deploy in portal

How it works

  • reTerminal has an eMMC disable jumper (see reTerminal docs; often “J2” or “nRPIBOOT”). When the jumper is fitted, the CM4 boots in USB device mode and waits for rpiboot from the host.
  • You connect the reTerminal’s USB slave port to a provisioning PC (Linux).
  • udev detects the Raspberry Pi Foundation USB device (vendor 2b8e) and runs a trigger script.
  • The trigger starts the provisioning script that:
    1. Runs rpiboot (from the usbboot project). The CM4 then exposes its eMMC as a USB mass-storage device.
    2. Finds the new block device (eMMC) and writes status so the dashboard shows "Device connected (USB boot mode). Choose Backup or Deploy in the dashboard."
    3. Waits for your choice in the portal — no automatic flash. When you click Backup or Deploy in the dashboard, the script runs that action (dd backup or dd deploy).
  • You remove the jumper and power cycle; the reTerminal boots from eMMC and can run cloud-init on first boot.

Provisioning host setup (Linux)

1. Build and install usbboot (rpiboot)

sudo apt-get install -y libusb-1.0-0-dev
git clone --depth=1 https://github.com/raspberrypi/usbboot
cd usbboot
make
sudo mkdir -p /opt/usbboot
sudo cp rpiboot /opt/usbboot/

2. Create golden image and config directory

  • Build your golden image (see Part 2) and place it where the script will find it, e.g.:

    sudo mkdir -p /var/lib/cm4-provisioning
sudo cp /path/to/your/golden-reterminal.img /var/lib/cm4-provisioning/golden.img

  • Or use a different path and set GOLDEN_IMAGE when installing the script (see below). On Proxmox with LXC, you can store backup images on a host directory by setting CM4_BACKUPS_HOST_PATH at deploy so that folder is bind-mounted into the LXC — see PROXMOX-LXC-DEPLOYMENT.md § Store backup images on a host directory.

3. Install the provisioning script and trigger

# From this repo (emmc-provisioning/host/)
cd emmc-provisioning/host

sudo mkdir -p /opt/cm4-provisioning
sudo cp flash-emmc-on-connect.sh /opt/cm4-provisioning/
sudo chmod +x /opt/cm4-provisioning/flash-emmc-on-connect.sh

# Optional: override paths via environment (create env file)
echo 'GOLDEN_IMAGE=/var/lib/cm4-provisioning/golden.img' | sudo tee /opt/cm4-provisioning/env
echo 'RPIBOOT_DIR=/opt/usbboot' | sudo tee -a /opt/cm4-provisioning/env
echo 'EMMC_SIZE_BYTES=8589934592' | sudo tee -a /opt/cm4-provisioning/env   # 8GB; use 17179869184 for 16GB
# Optional: shrink backups after dd (requires PiShrink; see "Shrinking backup and golden images" below)
# echo 'SHRINK_BACKUP=1' | sudo tee -a /opt/cm4-provisioning/env
# echo 'PISHRINK_COMPRESS=gz' | sudo tee -a /opt/cm4-provisioning/env   # or xz

sudo cp cm4-flash-trigger.sh /usr/local/bin/
sudo chmod +x /usr/local/bin/cm4-flash-trigger.sh

If your golden image path or rpiboot path is different, set GOLDEN_IMAGE, RPIBOOT_DIR, and optionally EMMC_SIZE_BYTES in /opt/cm4-provisioning/env and source it from the script, or pass them into the systemd-run call in the trigger (e.g. by making the trigger source the env file and export variables before systemd-run).

4. Install udev rule

# From emmc-provisioning/host/
sudo cp 90-cm4-boot-mode.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger

5. Enable provisioning (safety)

Provisioning runs only if the “enabled” file exists:

sudo mkdir -p /etc/cm4-provisioning
sudo touch /etc/cm4-provisioning/enabled

To disable provisioning (no device detection), remove that file: sudo rm /etc/cm4-provisioning/enabled.

6. Optional: pass environment into the provisioning job

If you use /opt/cm4-provisioning/env, update the trigger so the flash script sees those variables. For example change /usr/local/bin/cm4-flash-trigger.sh to:

#!/usr/bin/env bash
set -a
[[ -f /opt/cm4-provisioning/env ]] && source /opt/cm4-provisioning/env
set +a
export GOLDEN_IMAGE RPIBOOT_DIR EMMC_SIZE_BYTES
FLASH_SCRIPT="${CM4_FLASH_SCRIPT:-/opt/cm4-provisioning/flash-emmc-on-connect.sh}"
exec systemd-run --no-block --unit=cm4-flash-once --property=Environment="GOLDEN_IMAGE=$GOLDEN_IMAGE" ...

Or keep it simple and edit the defaults inside flash-emmc-on-connect.sh (e.g. GOLDEN_IMAGE, RPIBOOT_DIR, EMMC_SIZE_BYTES).

Usage

  1. Fit the eMMC disable jumper on the reTerminal.
  2. Connect the reTerminal USB slave port to the provisioning PC.
  3. Power the reTerminal (or apply power after USB).
  4. On the host, rpiboot will run automatically; when it exits, the script will dd the golden image to the eMMC. Watch logs: journalctl -u cm4-flash-once -f or journalctl -t cm4-flash -f.
  5. When done, remove the jumper and power cycle the reTerminal. It will boot from eMMC; cloud-init will run on first boot.

Part 2: Golden image with cloud-init

Raspberry Pi OS (recent versions) supports cloud-init using the NoCloud datasource: it reads user-data, meta-data, and optionally network-config from the boot (FAT32) partition.

Steps to prepare a cloud-init image for Raspberry Pi OS

  1. Obtain Raspberry Pi OS

    • Download from raspberrypi.com/software (desktop or Lite), or use Raspberry Pi Imager with "Edit settings" for locale/SSH. Recent Raspberry Pi OS has cloud-init built in.

  2. Flash the image

    • Flash to a spare SD card, or to a loop file for building without physical media: cp raspios.img golden-work.img

  3. Mount the boot partition

    • From an image file: sudo losetup -fP golden-work.img then e.g. sudo mount /dev/loop0p1 /mnt/boot
    • From SD: mount the first (FAT32) partition at /mnt/boot. On a running Pi, boot is often /boot/firmware.

  4. Add NoCloud files on the boot partition (root of FAT32, same level as config.txt):

    • user-data, meta-data, network-config
    • From this repo: cp emmc-provisioning/cloud-init/{user-data,meta-data,network-config} /mnt/boot/

  5. Customise user-data and network-config. Use the remote bootstrap script pattern (below) to avoid rebuilding the image when you change first-boot commands.

  6. Unmount and create golden image: sudo umount /mnt/boot, then copy the image to your GOLDEN_IMAGE path (e.g. /var/lib/cm4-provisioning/golden.img).

Remote bootstrap script (no image rebuild for script changes)

You can keep the golden image fixed and have cloud-init download a script from a file server and run it on first boot. When you change the script on the server, the next device gets the new commands without rebuilding the image.

  1. Host the script on an HTTP/HTTPS file server (e.g. nginx, or python3 -m http.server) at a URL the Pi can reach, e.g. http://192.168.1.10/provisioning/bootstrap.sh.

  2. In user-data, use runcmd to download and run it (runcmd runs after packages and network are up):

    #cloud-config
package_update: true
package_upgrade: false
packages:
  - curl   # or wget

runcmd:
  - curl -fsSL "http://YOUR_FILE_SERVER/provisioning/bootstrap.sh" -o /tmp/bootstrap.sh
  - chmod +x /tmp/bootstrap.sh
  - /tmp/bootstrap.sh

    With wget: wget -q -O /tmp/bootstrap.sh "http://.../bootstrap.sh" then chmod +x /tmp/bootstrap.sh and /tmp/bootstrap.sh.

  3. Bootstrap script: use a normal shell script (e.g. #!/bin/bash and set -e). It can install packages, configure kiosk, set hostname, register with a dashboard, etc.

  4. Notes: Ensure network-config (or DHCP) gives the Pi an IP before runcmd. For HTTPS, add ca-certificates to packages if needed. You can use one script for all devices or have the script call your server with serial/MAC for device-specific config.

Creating the golden image

  1. Flash Raspberry Pi OS (or your base image) to a spare SD card or a loop file.
  2. Mount the boot partition (first partition, FAT32). On the image file it might be at an offset; use losetup -P or mount the SD’s partition.
  3. Add cloud-init NoCloud files on the boot partition (same level as config.txt, not in a subfolder for default NoCloud):
    • user-data – main config (packages, runcmd, etc.)
    • meta-data – optional (instance-id, local-hostname)
    • network-config – optional (network config in netplan format)

You can use the examples in this repo:

# After mounting boot partition at e.g. /mnt/boot
# (On Raspberry Pi OS, boot is often /boot/firmware on the running system, or the first FAT partition of the image)
cp emmc-provisioning/cloud-init/user-data /mnt/boot/
cp emmc-provisioning/cloud-init/meta-data /mnt/boot/
cp emmc-provisioning/cloud-init/network-config /mnt/boot/
  1. Customise user-data and network-config (hostname, WiFi, packages, Chromium kiosk, or a single runcmd that downloads and runs a remote script—see "Remote bootstrap script" above).
  2. Copy your kiosk/Chromium scripts into the image rootfs only if you are not using a remote script; otherwise the remote script can pull what it needs.
  3. Unmount, then create a golden image from the SD or loop device (e.g. dd or copy of the whole block device). Use that as golden.img on the provisioning host. Optionally shrink it with PiShrink (see below) to save space and speed up deploy.

Shrinking backup and golden images (PiShrink)

Raw full-disk backups and golden images are the full size of the eMMC (e.g. 32 GB). PiShrink shrinks the last partition (must be ext2/3/4) to its minimum size and truncates the image file; on first boot the rootfs can expand back to fill the device. This reduces backup/golden image size (often to a few GB) and improves transfer times.

On the provisioning host:

  1. Install PiShrink and dependencies (run as root on the host, or via ssh root@HOST 'bash -s' < scripts/install-pishrink-on-host.sh):

    # From this repo (on the host or via ssh)
bash emmc-provisioning/scripts/install-pishrink-on-host.sh

    This installs parted, e2fsprogs, gzip, pigz, xz-utils and downloads pishrink.sh to /usr/local/bin/pishrink.sh.

  2. Shrink backups automatically after each Backup in the dashboard: add to /opt/cm4-provisioning/env and ensure the trigger sources it:

    echo 'SHRINK_BACKUP=1' | sudo tee -a /opt/cm4-provisioning/env
# Optional: compress after shrinking (smaller file; must decompress before using as golden image)
# echo 'PISHRINK_COMPRESS=gz' | sudo tee -a /opt/cm4-provisioning/env   # good balance
echo 'PISHRINK_COMPRESS=xz' | sudo tee -a /opt/cm4-provisioning/env   # minimum size (slower)

    With SHRINK_BACKUP=1, once a backup finishes, the script runs PiShrink on the .img file. Use PISHRINK_COMPRESS=xz for minimum size (smallest file, slower); or gz for a good balance. The file becomes .img.xz or .img.gz and must be decompressed before deploy (e.g. xz -dk backup.img.xz then copy to golden.img).

  3. Shrink a golden image manually (e.g. after building from Raspberry Pi OS):

    sudo pishrink.sh -n /path/to/large.img /var/lib/cm4-provisioning/golden.img

    -n disables update check. Omit the second argument to shrink in place. The shrunk image will expand the rootfs on first boot when deployed to eMMC.

Notes:

  • PiShrink only shrinks the last partition; it must be ext2/3/4 (standard Raspberry Pi OS root is ext4).
  • Compressed backups (.img.gz / .img.xz) are for archival; to use as golden image, decompress first (e.g. gunzip -k backup.img.gz then copy to golden.img).
  • Dashboard "Shrink" / "Compress" buttons run PiShrink on the host (not in the LXC). The dashboard writes a request file; the host runs run-shrink-on-host.sh when cm4-shrink.path sees it. Ensure PiShrink is installed on the host (see above) and that deploy has installed run-shrink-on-host.sh and enabled cm4-shrink.path.

Cloud-init file locations on the Pi

  • NoCloud: Boot partition root – user-data, meta-data, network-config.
  • Some images expect them in a subfolder cloud-init/ or on a separate vfat partition labeled cidata; check your OS docs. Standard Raspberry Pi OS NoCloud uses the boot partition root.

Per-device config (optional)

NoCloud can also use a seed partition or config drive. For per-device hostname/settings you can:

  • Use meta-data instance-id and local-hostname and generate different meta-data per device when imaging (e.g. script that writes meta-data before flashing), or
  • Use a first-boot script that calls a provisioning server (e.g. by serial number) and applies device-specific config; cloud-init can launch that script from runcmd.

Summary

Step Action
1 Build usbboot, install rpiboot on provisioning host.
2 Create golden image with cloud-init user-data, meta-data, network-config on boot partition.
3 Install flash-emmc-on-connect.sh, cm4-flash-trigger.sh, and udev rule; set GOLDEN_IMAGE and enable file.
4 Put reTerminal in boot mode (jumper), connect USB to host; image is written automatically.
5 Remove jumper, power cycle; device boots from eMMC and cloud-init runs on first boot.

This gives you automatic deployment of the golden image to eMMC when the reTerminal is in boot mode, plus first-boot configuration via cloud-init.