diff --git a/.gitignore b/.gitignore index 47f5c7b..6ec373f 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,9 @@ *.img !emmc-provisioning/network-boot-initramfs/*.img +# Deploy logs (generated by deploy-to-proxmox.sh when DEPLOY_LOG=1) +emmc-provisioning/scripts/deploy-*.log + # Backup/data from devices (large DBs and logs) backup-from-device/**/data/*.db backup-from-device/**/logs/ diff --git a/README.md b/README.md index 19bd49c..17b9f00 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ A single **revision number** is kept in `REVISION` and in a comment line in trac ## Quick start -1. Read **emmc-provisioning/docs/EMMC-PROVISIONING-GUIDE.md** for full setup. -2. Use **emmc-provisioning/scripts/sync-portal-files-to-lxc.sh** to sync first-boot assets (including kiosk) to the file server. -3. Provision devices via USB boot or network boot; first-boot configures kiosk, labwc, rotation, wallpaper, dark theme, and optional CM4 boot order. +1. **New deployment:** Follow **[emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md](emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md)** for step-by-step instructions (Proxmox host prep → LXC deploy → network boot → portal files). +2. **Sync first-boot assets** to the file server after deploy: + `./emmc-provisioning/scripts/sync-portal-files-to-lxc.sh root@` +3. Provision devices via USB boot or network boot; first-boot configures the Chromium kiosk, labwc Wayland desktop, screen rotation, wallpaper, dark theme, and CM4 boot order. diff --git a/archive/README.md b/archive/README.md index 401867f..f92aebb 100644 --- a/archive/README.md +++ b/archive/README.md @@ -1,10 +1,9 @@ # Archive -This folder holds files that are no longer part of the active reTerminal DM4 / eMMC provisioning workflow. Kept for reference only. +This folder holds files that are no longer part of the active reTerminal DM4 / eMMC provisioning workflow. Kept for historical reference only. | Subfolder | Contents | |-----------|----------| -| **chromium-setup-legacy/** | Old Chromium-setup guides and scripts: KDE installation, LED/buzzer control, audio config, touchscreen options, Flask apps, test scripts, revert-to-lxde. Kiosk assets (start-chromium.sh, chromium-kiosk.desktop) live in `emmc-provisioning/cloud-init/` and `emmc-provisioning/cloud-init/config-files/`. | -| **cloud-init-duplicates/** | Duplicate or superseded cloud-init files (e.g. plymouth-custom.script duplicate of `files-from-guard/plymouth-custom/custom.script`). | +| **chromium-setup-legacy/** | Old Chromium-setup guides and scripts: KDE installation, LED/buzzer control, audio config, touchscreen options, Flask apps, test scripts, revert-to-lxde. The active kiosk launcher lives at `emmc-provisioning/cloud-init/fileserver/start-chromium.sh` (Wayland/labwc). | -Do not rely on archived files for deployment; use the main tree under **emmc-provisioning/**. +Do not rely on archived files for deployment; use the active tree under **emmc-provisioning/**. diff --git a/emmc-provisioning/README.md b/emmc-provisioning/README.md index a4806f5..2939805 100644 --- a/emmc-provisioning/README.md +++ b/emmc-provisioning/README.md @@ -13,12 +13,19 @@ Revisions are tracked project-wide; see repo root **README.md** and `scripts/bum emmc-provisioning/ ├── README.md ← You are here ├── docs/ Documentation -│ ├── DEPLOY-NEW-PROXMOX.md Step-by-step: deploy to a new Proxmox instance -│ ├── EMMC-PROVISIONING-GUIDE.md Full setup and usage -│ ├── NETWORK-BOOT-LXC.md Network boot (PXE/dnsmasq) and LXC +│ ├── DEPLOY-NEW-PROXMOX.md ★ START HERE: full deploy guide (host prep, LXC, scripts, network boot) +│ ├── EMMC-PROVISIONING-GUIDE.md Golden image creation, cloud-init, PiShrink +│ ├── PROXMOX-LXC-DEPLOYMENT.md Reference: what is deployed, redeploy, troubleshooting +│ ├── NETWORK-BOOT-LXC.md Network boot architecture (PXE/dnsmasq, interfaces) +│ ├── NETWORK-BOOT-DEPLOYMENT-FLOW.md Full data flow for network boot provisioning +│ ├── NETWORK-BOOT-TROUBLESHOOTING.md Troubleshooting network boot issues │ ├── DEVICE-DNS-DHCP-RESOLVCONF.md Device DNS from DHCP, resolv.conf, cloud-init │ ├── DNSMASQ-DNS-FILESERVER.md dnsmasq DNS and file.server on LXC -│ ├── PROXMOX-LXC-DEPLOYMENT.md Proxmox LXC + host setup (reference) +│ ├── PROXMOX-HOST-COMPARISON.md Diff between Proxmox hosts (fixes checklist) +│ ├── PREPARE-IMAGE-FOR-CLOUDINIT.md How to shrink and prep a golden image +│ ├── BACKUP-DEVICE-CONFIG-AUDIT.md Audit of backup image contents +│ ├── DEVICE-REMOVABLE-PACKAGES.md Packages to purge from the device image +│ ├── EDIT-CLOUDINIT-ON-DEVICE.md Edit NoCloud files on-device or in .img.xz │ └── PORTAL_STYLING_GUIDE.md Dashboard UI styling reference ├── host/ Scripts that run on the provisioning host (Proxmox host) │ ├── flash-emmc-on-connect.sh rpiboot + wait for Backup/Deploy choice, then dd @@ -55,8 +62,8 @@ emmc-provisioning/ ## Quick start -1. **Read** [docs/EMMC-PROVISIONING-GUIDE.md](docs/EMMC-PROVISIONING-GUIDE.md) for setup and usage. -2. **Deploy to a new Proxmox:** Follow [docs/DEPLOY-NEW-PROXMOX.md](docs/DEPLOY-NEW-PROXMOX.md) for clear step-by-step instructions. -3. **Proxmox reference:** [scripts/deploy-to-proxmox.sh](scripts/deploy-to-proxmox.sh) and [docs/PROXMOX-LXC-DEPLOYMENT.md](docs/PROXMOX-LXC-DEPLOYMENT.md) for options, layout, and troubleshooting. -4. **Manual host:** Copy scripts from `host/` to the host and install the udev rule (see the guide). -5. Put **golden.img** in `/var/lib/cm4-provisioning/` (or your configured path). When a device is detected (USB or network), the **dashboard** asks **Backup** or **Deploy**. +1. **New deployment:** Follow **[docs/DEPLOY-NEW-PROXMOX.md](docs/DEPLOY-NEW-PROXMOX.md)** — covers Proxmox host prep, LXC creation, host scripts, network boot, and portal file sync. +2. **Full setup reference:** [docs/EMMC-PROVISIONING-GUIDE.md](docs/EMMC-PROVISIONING-GUIDE.md) for golden image creation, cloud-init, PiShrink. +3. **Redeploy / update:** Re-run `scripts/deploy-to-proxmox.sh root@HOST` — updates scripts, dashboard, udev, and systemd without touching your golden image or enabled flag. +4. **Sync portal files:** After deploy or when kiosk/first-boot assets change: `scripts/sync-portal-files-to-lxc.sh root@`. +5. **Troubleshooting:** [docs/PROXMOX-LXC-DEPLOYMENT.md](docs/PROXMOX-LXC-DEPLOYMENT.md) for USB errors, rpiboot failures, and monitoring. [docs/NETWORK-BOOT-TROUBLESHOOTING.md](docs/NETWORK-BOOT-TROUBLESHOOTING.md) for network boot issues. diff --git a/emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md b/emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md index 3ad860b..f19c776 100644 --- a/emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md +++ b/emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md @@ -1,33 +1,86 @@ -# Deploy CM4 eMMC Provisioning to a New Proxmox Instance +# Deploying the CM4 eMMC Provisioning Stack to Proxmox -Step-by-step guide to deploy the provisioning service (host + LXC) on a **new** Proxmox server. For redeploy/update and troubleshooting, see [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md). +Complete step-by-step guide for deploying the provisioning service (Proxmox host + LXC container) on a new or existing Proxmox server. Covers host preparation, network bridge configuration, LXC deployment, post-deploy setup, network boot, and first-boot asset sync. + +For reference details (troubleshooting, redeploy, architecture), see [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md). --- -## Prerequisites (before running the deploy script) +## Overview -| Requirement | Details | -|-------------|---------| -| **Proxmox host** | A Proxmox VE node (new or existing) where you want the service. | -| **SSH as root** | You must be able to run `ssh root@YOUR_PROXMOX_HOST` with **key-based auth** (no password prompt). | -| **Proxmox storage** | At least one active storage (e.g. `local` or `local-lvm`). Check on the host: `pvesm status`. | -| **Host internet** (recommended) | Needed so the deploy script can download the Debian 12 LXC template (if missing), and install **usbboot** and **PiShrink** on the host. Without internet, deploy still runs but you must install usbboot and PiShrink manually later. | +The provisioning stack consists of two parts that work together: -**Optional (set before deploy):** +| Component | Where it runs | What it does | +|-----------|--------------|--------------| +| **Host scripts + udev** | Proxmox host | Detects CM4 over USB, runs `rpiboot`, then `dd` to write/read the eMMC | +| **LXC container** (`cm4-provisioning`) | Proxmox LXC | Runs the Flask dashboard on port 5000; serves portal files and golden images | -- `DEPLOY_ROOTFS_STORAGE=local-lvm` — Skip interactive storage choice when creating the LXC. -- `DEPLOY_LXC_ROOT_PASSWORD=yourpassword` — Set LXC root password and enable SSH. -- `DEPLOY_LXC_SSH_KEY=/path/to/pub` — Copy this key into the LXC (default: `~/.ssh/id_ed25519.pub` or `id_rsa.pub`). -- `CM4_BACKUPS_HOST_PATH=/mnt/storage/cm4-backups` — Store backups on this host path (create the directory on the host if needed). -- **Network (WAN/LAN):** - `DEPLOY_LXC_WAN_BRIDGE=vmbr0` (default), `DEPLOY_LXC_WAN_IP=dhcp` (default), - `DEPLOY_LXC_LAN_BRIDGE=vmbr1`, `DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24` — To add eth1 as provisioning LAN. **Set these if you want the portal reachable from the LAN** (e.g. http://10.20.50.1:5000); the dashboard listens on all interfaces. +The host and LXC share `/var/lib/cm4-provisioning/` via a bind-mount, so images and status files are visible from both. + +``` +Workstation (this repo) + │ + │ deploy-to-proxmox.sh (SSH + rsync) + ▼ +Proxmox Host + ├── udev → cm4-flash-trigger.sh → flash-emmc-on-connect.sh + ├── /opt/cm4-provisioning/ (scripts, env) + ├── /var/lib/cm4-provisioning/ (golden.img, backups, status.json) ◄──────┐ + └── LXC: cm4-provisioning │ bind-mount + ├── Flask dashboard :5000 │ + ├── /opt/cm4-provisioning/dashboard/ │ + └── /var/lib/cm4-provisioning/ ◄────────────────────────────────────┘ +``` --- -## Step 1: Run the deploy script +## Part 1 — Proxmox Host Prerequisites -From your **workstation** (where the repo is cloned), run: +### 1.1 Hardware & OS + +- Proxmox VE 7 or 8 installed on a physical machine. +- At least one USB port accessible to the host (not passed through to a VM) for the reTerminal USB slave cable. +- At least one active storage (local or local-lvm). Check: `pvesm status`. +- Internet access on the host (needed for initial usbboot/PiShrink install and LXC template download). + +### 1.2 SSH key access from your workstation + +The deploy script connects as `root` using key-based auth. Set this up if not already done: + +```bash +# On your workstation — copy your public key to the Proxmox host +ssh-copy-id root@YOUR_PROXMOX_HOST +# Verify +ssh root@YOUR_PROXMOX_HOST "echo OK" +``` + +### 1.3 Proxmox network bridges + +The LXC needs at minimum one bridge for WAN access. For provisioning LAN (DHCP to devices), it needs a second bridge. + +#### WAN bridge (required) + +The default WAN bridge is `vmbr0`, which is created automatically by Proxmox during installation and connects to your primary network. No extra configuration needed. + +#### LAN bridge for provisioning (required for network boot / device DHCP) + +If you want the LXC to serve DHCP, TFTP, and DNS on a dedicated provisioning LAN (so devices can network-boot), create a second Linux bridge on the Proxmox host: + +1. Open **Proxmox Web UI → Node → Network → Create → Linux Bridge**. +2. Set: + - **Name:** `vmbr1` (or any unused bridge name) + - **Bridge ports:** the physical NIC connected to your provisioning LAN switch (e.g. `enp2s0`). Leave blank for an internal-only bridge (useful for testing with no physical switch). + - **IPv4/CIDR:** leave blank (the LXC handles the IP on this bridge, not the host). + - **Autostart:** checked. +3. Click **Create**, then **Apply Configuration**. + +> **Note:** If you connect the reTerminals via a switch that is also connected to `enp2s0`, traffic flows directly. If there is no physical NIC to dedicate, leave bridge ports blank and connect all provisioning devices as VMs/LXCs on the same internal bridge. + +--- + +## Part 2 — Running the Deploy Script + +From your **workstation** (where this repo is cloned), run a single command to deploy everything: ```bash cd /path/to/reTerminal\ DM4 @@ -35,9 +88,26 @@ cd /path/to/reTerminal\ DM4 ./emmc-provisioning/scripts/deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST ``` -Replace `YOUR_PROXMOX_HOST` with the Proxmox hostname or IP (e.g. `10.20.30.40`). +Replace `YOUR_PROXMOX_HOST` with the Proxmox hostname or IP address. -**Example with options:** +### 2.1 Deploy script environment variables + +Set these before running the script to customise the deployment: + +| Variable | Default | Description | +|----------|---------|-------------| +| `DEPLOY_ROOTFS_STORAGE` | *(interactive)* | LXC rootfs storage name (e.g. `local-lvm`). If not set, script lists storages and asks. | +| `DEPLOY_LXC_WAN_BRIDGE` | `vmbr0` | Proxmox bridge for WAN (eth0 in LXC). | +| `DEPLOY_LXC_WAN_IP` | `dhcp` | WAN address: `dhcp` or a static IP like `192.168.1.10/24`. | +| `DEPLOY_LXC_LAN_BRIDGE` | *(none)* | If set, adds eth1 as provisioning LAN on this bridge (e.g. `vmbr1`). | +| `DEPLOY_LXC_LAN_SUBNET` | `10.20.50.1/24` | LXC IP/prefix on the LAN bridge. Used only when `DEPLOY_LXC_LAN_BRIDGE` is set. | +| `DEPLOY_LXC_ROOT_PASSWORD` | *(default)* | Sets LXC root password and enables SSH inside the container. | +| `DEPLOY_LXC_SSH_KEY` | `~/.ssh/id_ed25519.pub` | Public key to add to LXC root's `authorized_keys`. Defaults to your workstation key. | +| `CM4_BACKUPS_HOST_PATH` | *(none)* | Host directory for backup images (e.g. `/mnt/storage/cm4-backups`). Bind-mounted into LXC. | +| `DEPLOY_EMMC_SIZE_GB` | `32` | eMMC size hint in GB (used only when multiple block devices appear after rpiboot). | +| `DEPLOY_LOG` | *(off)* | Set to `1` to write a timestamped log file in `scripts/`. | + +### 2.2 Full deploy example ```bash DEPLOY_ROOTFS_STORAGE=local-lvm \ @@ -47,117 +117,352 @@ DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24 \ ./emmc-provisioning/scripts/deploy-to-proxmox.sh root@10.20.30.40 ``` -- On **first run**, the script will ask you to choose LXC rootfs storage (unless `DEPLOY_ROOTFS_STORAGE` is set). It then creates the LXC, installs host scripts, udev, systemd units, and the dashboard in the LXC. -- The script prints **LXC IP (WAN)** and, if you set `DEPLOY_LXC_LAN_BRIDGE`, **LXC IP (LAN)**. The portal is reachable at `http://:5000` on both; use the LAN IP from devices on the provisioning LAN. +### 2.3 What the deploy script does + +The script runs five stages: + +1. **Check** — SSHes to the host, finds existing `cm4-provisioning` container by hostname (or lists storage for new container creation). +2. **Clean + Rsync** — Wipes `/tmp/emmc-provisioning-deploy` on the host and rsyncs the entire repo there (excluding `.git` and deploy logs). +3. **Remote install (host + LXC)** — Runs a remote heredoc that: + - Creates the LXC (Debian 12, 1 GB RAM, 8 GB rootfs) if it doesn't exist, or reuses it by hostname. + - Adds eth1 (LAN bridge) if `DEPLOY_LXC_LAN_BRIDGE` is set. + - Configures the bind-mount for `/var/lib/cm4-provisioning/`. + - Installs host scripts to `/opt/cm4-provisioning/` and udev rules to `/etc/udev/rules.d/`. + - Installs and enables systemd units: `cm4-flash.service`, `cm4-build-cloudinit.path/.service`, `cm4-shrink.path/.service`. + - Writes `/opt/cm4-provisioning/env` (golden image path, rpiboot dir, eMMC size). + - Installs `python3-flask` and `openssh-server` in the LXC (skipped if already present). + - Deploys the Flask dashboard and enables/restarts `cm4-dashboard.service` in the LXC. + - Installs usbboot (`rpiboot`) on the host if not already present. + - Installs PiShrink on the host if not already present. +4. **LXC start** — Starts the LXC if stopped. +5. **Summary** — Prints LXC WAN IP (and LAN IP if set), dashboard URL, and remaining manual steps. + +On **redeploy** (container already exists): host scripts, dashboard, env, systemd, and udev are always updated. LXC creation, bind-mounts, apt installs, usbboot, and PiShrink are skipped when already present. --- -## Step 2: Install usbboot on the host (if host had no internet during deploy) +## Part 3 — Post-Deploy: Required Manual Steps -USB flash/backup needs **rpiboot** on the Proxmox **host**. If the deploy log said usbboot install failed or was skipped: - -**From your workstation:** +### Step 1: Verify the dashboard is up ```bash -scp emmc-provisioning/scripts/install-usbboot-on-host.sh root@YOUR_PROXMOX_HOST:/tmp/ -ssh root@YOUR_PROXMOX_HOST "bash /tmp/install-usbboot-on-host.sh" +# Get the LXC IP from deploy output, or: +ssh root@YOUR_PROXMOX_HOST \ + "CID=\$(pct list | awk '\$3==\"cm4-provisioning\"{print \$1}'); pct exec \$CID -- hostname -I" + +# Open the dashboard +open http://:5000 ``` -**Or on the Proxmox host** (if `/tmp/emmc-provisioning-deploy` is still there): +The dashboard should show **"Waiting for device in USB boot mode"** on the home page. + +### Step 2: Verify host services + +SSH to the Proxmox host and confirm the host side is healthy: ```bash ssh root@YOUR_PROXMOX_HOST -bash /tmp/emmc-provisioning-deploy/scripts/install-usbboot-on-host.sh + +# Check udev rule is installed +ls /etc/udev/rules.d/90-cm4-boot-mode.rules + +# Check flash trigger script +ls /usr/local/bin/cm4-flash-trigger.sh + +# Check host scripts +ls /opt/cm4-provisioning/ +# Expected: flash-emmc-on-connect.sh, build-cloudinit-image.sh, +# run-shrink-on-host.sh, fix-gadget-bootcode-on-host.sh, env + +# Check systemd path units are active +systemctl status cm4-build-cloudinit.path cm4-shrink.path + +# Check auto-flash is enabled +ls /etc/cm4-provisioning/enabled ``` ---- +### Step 3: Add a golden image -## Step 3: Add a golden image (required for Deploy) +A **golden image** is required for **Deploy** (writing an image to the device's eMMC). Backup (reading from device) works without it. -To **write** an image to a device (Deploy), the host must have a **golden image** at `/var/lib/cm4-provisioning/golden.img`. Backup (read from device) works without it. - -**Option A — From the dashboard** - -1. Open **http://<LXC-IP>:5000** (use the LXC IP from the deploy output). -2. Build a cloud-init image or upload/set an existing backup as golden (see dashboard Admin). - -**Option B — Copy an image from your machine** +**Option A — Build via the dashboard:** +1. Open `http://:5000` → Admin tab. +2. Click **Build cloud-init image**: the host downloads the latest Raspberry Pi OS, injects your cloud-init `user-data`, and creates `golden.img`. +3. Click **Set as golden** once the build finishes. +**Option B — Copy an existing image:** ```bash scp /path/to/your-golden.img root@YOUR_PROXMOX_HOST:/var/lib/cm4-provisioning/golden.img ``` ---- +**Option C — Promote a backup:** +In the dashboard Admin → Images tab, select a backup and click **Set as golden**. -## Accessing the portal from the LAN +### Step 4: Enable SSH into the LXC (optional) -The dashboard listens on **all interfaces** (`0.0.0.0:5000`), so it is reachable on both WAN and LAN IPs when the LXC has two networks. +If you ran the deploy with `DEPLOY_LXC_ROOT_PASSWORD` or a default SSH key, the LXC already has SSH enabled. Otherwise: -- **Deploy with a LAN interface:** set `DEPLOY_LXC_LAN_BRIDGE=vmbr1` (and optionally `DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24`) when running the deploy script. The LXC will get eth1 with the LAN IP (e.g. 10.20.50.1). -- **From the provisioning LAN:** open **http://<LAN-IP>:5000** (e.g. http://10.20.50.1:5000). Devices on that subnet can use the portal without going through WAN. -- If you did not set a LAN bridge at deploy time, you only have one IP (WAN); use that for the portal. To add LAN later you would need to add eth1 to the container and reconfigure (see PROXMOX-LXC-DEPLOYMENT.md). +```bash +# From your workstation — adds your default SSH key and enables root SSH +./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST ---- - -## Step 4: (Optional) SSH into the LXC - -If you set `DEPLOY_LXC_ROOT_PASSWORD` or had a default SSH key, you can already run: +# Or with a specific key and password +ROOT_PASSWORD='YourPassword' \ +./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST ~/.ssh/id_ed25519.pub +``` +Then connect: ```bash ssh root@ ``` -Otherwise, enable root SSH and add your key: +--- + +## Part 4 — Sync Portal Files (First-Boot Assets) + +The LXC serves first-boot assets (kiosk scripts, desktop files, splash, theme, etc.) from `/var/lib/cm4-provisioning/portal-files/`. These must be synced from the repo. ```bash -./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST -# Or with password: ROOT_PASSWORD='YourPassword' ./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST ~/.ssh/id_ed25519.pub +# From your workstation +./emmc-provisioning/scripts/sync-portal-files-to-lxc.sh root@ ``` +This rsyncs everything under `emmc-provisioning/cloud-init/fileserver/` to the LXC portal-files directory. Run this every time you update kiosk assets or first-boot scripts. + +**What gets synced:** + +| File | Purpose | +|------|---------| +| `start-chromium.sh` | Wayland/labwc Chromium kiosk launcher | +| `five-tap-close-chromium.py` | 5-tap touch overlay to close Chromium | +| `chromium-kiosk.desktop` | Autostart: launches Chromium kiosk | +| `chromium-kiosk-no-select/` | Chromium extension: disables text selection | +| `set-rotation-at-login.sh/.desktop` | Per-login screen rotation | +| `01-set-rotation-once.sh/.desktop` | One-shot: rotation + dark theme + kanshi | +| `02-set-wallpaper-once.sh/.desktop` | One-shot: set wallpaper | +| `99-default-session.conf` | LightDM session = `rpd-labwc` | +| `custom.plymouth` + `custom.script` | Plymouth boot splash theme | +| `splash.png` | Boot splash / wallpaper image | +| `steps/01–13*.sh` | First-boot step scripts sourced by `first-boot.sh` | + --- -## Step 5: (Optional) Network boot (DHCP + TFTP on eth1) +## Part 5 — Network Boot Setup (Optional) -Only if you deployed with **`DEPLOY_LXC_LAN_BRIDGE`** (and optionally `DEPLOY_LXC_LAN_SUBNET`) and want to offer network boot to devices on that LAN: +Only needed if you want devices to **boot over the network** (PXE-style via TFTP) for provisioning, rather than via USB cable. + +### 5.1 Prerequisites + +- The LXC must have been deployed with a **LAN bridge** (`DEPLOY_LXC_LAN_BRIDGE` set). The LXC's eth1 will be the provisioning LAN gateway. +- Devices must be connected to the same LAN as the LXC's eth1. + +### 5.2 Run the network boot setup script + +From your workstation: ```bash ./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@ ``` -See [NETWORK-BOOT-LXC.md](NETWORK-BOOT-LXC.md) for details. +This SSH-connects to the LXC and runs the full setup inside the container. It performs: + +1. **Installs dnsmasq** (DHCP + DNS server) and the `vlan` package (for VLAN interfaces). +2. **Configures dnsmasq** on eth1: + - DHCP range: `.100` – `.200` (e.g. `10.20.50.100`–`10.20.50.200`). + - DNS: static record `file.server` → LAN gateway IP, so first-boot scripts can reach `http://file.server/...`. + - DHCP option 6: sends LXC as DNS server to all DHCP clients. +3. **Configures extra IPs on eth1**: `192.168.30.1/24`, `192.168.127.1/24` (for serving multiple subnets). +4. **Creates VLAN 40** interface `eth1.40` at `192.168.0.1/24` (for VLAN-tagged networks on the provisioning LAN). +5. **Enables IP forwarding** (`net.ipv4.ip_forward=1`) persisted in `/etc/sysctl.d/`. +6. **Configures NAT** (nftables or iptables fallback): masquerades all LAN traffic out eth0 so devices on the provisioning LAN get internet access. +7. **Enables and starts dnsmasq**. + +Config files written: +- `/etc/dnsmasq.d/network-boot.conf` — DHCP + DNS on eth1 +- `/etc/nftables.d/nat-lan.conf` — NAT rules +- `/etc/network/interfaces.d/70-cm4-extra-lan` — extra IPs and VLAN persisted + +### 5.3 Enable PXE / TFTP network boot + +TFTP is not enabled by default (dnsmasq is configured for DHCP + DNS only). To enable PXE/TFTP so devices can load a kernel and initramfs over the network: + +```bash +# SSH into the LXC +ssh root@ + +# Enable PXE/TFTP (adds the PXE options to dnsmasq and restarts it) +/opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable +``` + +This activates the PXE snippet at `/etc/dnsmasq.d/network-boot-pxe.conf` (DHCP options 66/67: next-server + boot file) and reloads dnsmasq. + +To disable PXE again (keep DHCP/DNS only): +```bash +/opt/cm4-provisioning/toggle-network-boot-dhcp.sh disable +``` + +### 5.4 Populate the TFTP boot files + +The TFTP root (`/srv/tftpboot`) needs Raspberry Pi 4 / CM4 boot files. From your workstation: + +```bash +./emmc-provisioning/scripts/populate-tftpboot-from-git.sh root@ +``` + +This downloads the official Raspberry Pi firmware `boot/` folder from GitHub into `/srv/tftpboot` on the LXC. + +To add the custom provisioning initramfs (Alpine-based, allows Backup/Deploy from network boot): + +```bash +# Ensure the initramfs image is built (or use the pre-built one in the repo) +ls emmc-provisioning/network-boot-initramfs/initrd.img + +# Copy it to the LXC TFTP root +scp emmc-provisioning/network-boot-initramfs/initrd.img root@:/srv/tftpboot/ + +# Then ensure config.txt references it +./emmc-provisioning/scripts/ensure-tftpboot-config-kernel-initrd.sh root@ +``` + +### 5.5 Configure device EEPROM for network boot + +For a reTerminal to boot from the network (when eMMC is empty or network boot order is set), its EEPROM `BOOT_ORDER` must include network boot. The recommended order is `0xf21` (eMMC first, then network): + +```bash +# Check current EEPROM boot order on a connected device +./emmc-provisioning/scripts/check-network-boot-priority.sh root@ +``` + +To set boot order via the provisioning dashboard (when device is in USB boot mode), use the **Update EEPROM** button in the dashboard. + +### 5.6 Verify network boot is working + +On the LXC, check the following: + +```bash +# Is dnsmasq running? +systemctl status dnsmasq + +# Is TFTP/PXE enabled? +/opt/cm4-provisioning/toggle-network-boot-dhcp.sh status + +# Are TFTP boot files present? +ls /srv/tftpboot/start4cd.elf + +# Any DHCP leases from devices? +cat /var/lib/misc/dnsmasq.leases + +# Monitor live DHCP/TFTP traffic when powering on a device +tcpdump -i eth1 -n port 67 or port 68 or port 69 +``` --- -## Step 6: (Optional) Install PiShrink on the host +## Part 6 — Installing usbboot Manually (if needed) -If the deploy log said PiShrink install failed (e.g. no internet), and you want **Shrink/Compress** in the dashboard to work: +If the deploy script could not install usbboot (e.g. no internet during deploy), install it manually: + +```bash +# From your workstation +scp emmc-provisioning/scripts/install-usbboot-on-host.sh root@YOUR_PROXMOX_HOST:/tmp/ +ssh root@YOUR_PROXMOX_HOST "bash /tmp/install-usbboot-on-host.sh" +``` + +Or, if `/tmp/emmc-provisioning-deploy` is still on the host: +```bash +ssh root@YOUR_PROXMOX_HOST "bash /tmp/emmc-provisioning-deploy/scripts/install-usbboot-on-host.sh" +``` + +After install, verify: +```bash +ssh root@YOUR_PROXMOX_HOST "ls /opt/usbboot/rpiboot && ls /opt/usbboot/mass-storage-gadget64/" +``` + +--- + +## Part 7 — Installing PiShrink Manually (if needed) + +PiShrink enables the dashboard **Shrink/Compress** function (shrinks backup images before compressing). Install if the deploy failed: ```bash ssh root@YOUR_PROXMOX_HOST "bash /tmp/emmc-provisioning-deploy/scripts/install-pishrink-on-host.sh" +# Or stream from workstation: +ssh root@YOUR_PROXMOX_HOST 'bash -s' < emmc-provisioning/scripts/install-pishrink-on-host.sh ``` -Or from your machine (stream the script): use the same pattern as in [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md) for `install-pishrink-on-host.sh`. - --- -## Summary checklist +## Part 8 — Updating / Redeploying -| Step | Action | Required? | -|------|--------|------------| -| 1 | Run `deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST` | **Yes** | -| 2 | Install usbboot on host (if deploy couldn’t) | For USB flash/backup | -| 3 | Add `golden.img` for Deploy | For Deploy only | -| 4 | SSH to LXC (or use setup-lxc-ssh.sh) | Optional | -| 5 | Run setup-network-boot-on-lxc.sh (if using eth1 LAN) | Optional | -| 6 | Install PiShrink on host (if deploy couldn’t) | For Shrink/Compress | +To push code changes (scripts, dashboard, udev, systemd) to an existing deployment: -**After deployment:** +```bash +./emmc-provisioning/scripts/deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST +``` -- **Dashboard:** http://<LXC-IP>:5000 (WAN). If you set `DEPLOY_LXC_LAN_BRIDGE`, also **http://<LAN-IP>:5000** (e.g. http://10.20.50.1:5000) from the LAN. -- **Golden image path (host and LXC):** `/var/lib/cm4-provisioning/golden.img` -- **Disable auto-flash:** `ssh root@YOUR_PROXMOX_HOST "rm /etc/cm4-provisioning/enabled"` -- **Enable again:** `ssh root@YOUR_PROXMOX_HOST "touch /etc/cm4-provisioning/enabled"` +The script finds the container by hostname (`cm4-provisioning`) and updates all files. It does **not** overwrite your `golden.img` or `/etc/cm4-provisioning/enabled`. -**If you see "rpiboot failed or no device connected":** The error is from the **Proxmox host** (where USB is connected). On the host run: `tail -50 /var/lib/cm4-provisioning/flash.log` to see the real rpiboot message. Ensure the reTerminal is in **boot mode** (eMMC disable jumper, USB slave port), then unplug/replug. See [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md) § "If rpiboot fails" for full steps. +To update **only the dashboard** (faster when only `app.py` or templates changed): -Full reference: [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md). +```bash +./emmc-provisioning/scripts/deploy-dashboard-to-lxc.sh root@ +``` + +To update **only the portal files** (kiosk assets, first-boot scripts): + +```bash +./emmc-provisioning/scripts/sync-portal-files-to-lxc.sh root@ +``` + +--- + +## Deployment Checklist + +| # | Action | Script / Command | Required? | +|---|--------|-----------------|-----------| +| **Host prep** | | | | +| 1 | SSH key access to Proxmox host | `ssh-copy-id root@HOST` | **Yes** | +| 2 | Create LAN bridge on Proxmox (`vmbr1`) | Proxmox Web UI | For network boot | +| **Deploy** | | | | +| 3 | Run deploy script | `deploy-to-proxmox.sh root@HOST` | **Yes** | +| 4 | Verify dashboard is up | `http://:5000` | **Yes** | +| 5 | Verify host services and udev rule | `ssh root@HOST "ls /etc/udev/rules.d/90-cm4-boot-mode.rules"` | **Yes** | +| **Post-deploy** | | | | +| 6 | Add golden image for Deploy | Dashboard Admin or `scp golden.img root@HOST:/var/lib/cm4-provisioning/` | For Deploy | +| 7 | Sync portal files (kiosk/first-boot assets) | `sync-portal-files-to-lxc.sh root@` | For first-boot provisioning | +| 8 | Enable SSH into LXC | `setup-lxc-ssh.sh root@HOST` | Optional | +| **Network boot** | | | | +| 9 | Run network boot setup on LXC | `setup-network-boot-on-lxc.sh root@` | For network boot only | +| 10 | Enable PXE/TFTP | `ssh root@ /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable` | For PXE boot | +| 11 | Populate TFTP boot files | `populate-tftpboot-from-git.sh root@` | For PXE boot | +| 12 | Copy provisioning initramfs to TFTP | `scp network-boot-initramfs/initrd.img root@:/srv/tftpboot/` | For provisioning via netboot | +| 13 | Configure device EEPROM boot order | Dashboard **Update EEPROM** or `rpi-eeprom-config` | For network boot | +| **If needed** | | | | +| 14 | Install usbboot manually | `install-usbboot-on-host.sh` | If deploy had no internet | +| 15 | Install PiShrink manually | `install-pishrink-on-host.sh` | If deploy had no internet | + +--- + +## After Deployment: Quick Reference + +| What | How | +|------|-----| +| **Dashboard (WAN)** | `http://:5000` | +| **Dashboard (LAN)** | `http://10.20.50.1:5000` (if LAN bridge was set) | +| **SSH to LXC** | `ssh root@` | +| **Get LXC IP** | `ssh root@HOST "pct list; pct exec -- hostname -I"` | +| **Golden image path** | `/var/lib/cm4-provisioning/golden.img` (same on host and in LXC) | +| **Disable auto-flash** | `ssh root@HOST "rm /etc/cm4-provisioning/enabled"` | +| **Re-enable auto-flash** | `ssh root@HOST "touch /etc/cm4-provisioning/enabled"` | +| **Flash log (on host)** | `ssh root@HOST "tail -f /var/lib/cm4-provisioning/flash.log"` | +| **Status JSON (on host)** | `ssh root@HOST "cat /var/lib/cm4-provisioning/status.json"` | +| **Full host snapshot** | `ssh root@HOST 'bash -s' < emmc-provisioning/scripts/monitor-from-host.sh` | +| **DHCP leases (LXC)** | `ssh root@ "cat /var/lib/misc/dnsmasq.leases"` | + +--- + +## Troubleshooting + +For USB flash errors (rpiboot failures, block device not found, USB transfer errors) and LXC/dashboard issues, see the full troubleshooting section in [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md). + +For network boot issues (DHCP not working, device not appearing in dashboard), see [NETWORK-BOOT-TROUBLESHOOTING.md](NETWORK-BOOT-TROUBLESHOOTING.md).