Update documentation and scripts for revision tracking and cloud-init enhancements</message>

<message>Introduce a revision tracking system across project files, allowing for easier identification of changes. Update the README files to include instructions for bumping revisions and auto-bumping on commits. Additionally, enhance cloud-init scripts with revision comments for better version control. Modify the dashboard API to improve build status management, including a forced clear option for stuck statuses, enhancing user experience and operational reliability.

README.md

@@ -1,7 +1,18 @@
+<!-- Revision: 2 -->
 # reTerminal DM4
 
 Project for **reTerminal DM4** (Seeed) with CM4: Chromium kiosk, eMMC provisioning (USB + network boot), and first-boot configuration via cloud-init.
 
+## Revisions
+
+A single **revision number** is kept in `REVISION` and in a comment line in tracked files (`# Revision: N` or `<!-- Revision: N -->`) so you can see what changed across hosts and deploys.
+
+- **Bump revision (update all files):** from repo root run  
+  `./emmc-provisioning/scripts/bump-revision.sh`
+- **Auto-bump on every commit:** install the pre-commit hook  
+  `cp emmc-provisioning/scripts/pre-commit-revision.sh .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit`  
+  Then every commit will bump the revision and update the revision line in all tracked files.
+
 ## Repository structure
 
 | Path | Purpose |

REVISION

@@ -0,0 +1 @@
+2

emmc-provisioning/README.md

@@ -1,7 +1,10 @@
+<!-- Revision: 2 -->
 # reTerminal DM4 eMMC provisioning
 
 Automatically **deploy** or **backup** the CM4 eMMC when the reTerminal is connected in **USB boot mode** or when it **boots over the network**. Uses **cloud-init** for first-boot configuration.
 
+Revisions are tracked project-wide; see repo root **README.md** and `scripts/bump-revision.sh`.
+
 ---
 
 ## Project layout

emmc-provisioning/cloud-init/01-set-rotation-once.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # One-shot: set screen rotation via kanshi (same as Control Center), then remove self.
 # Reads video=DSI-1:rotate=N from kernel cmdline and writes ~/.config/kanshi/config.
 # Runs once as user pi at first login; deletes its autostart and this script so it never runs again.

emmc-provisioning/cloud-init/02-set-wallpaper-once.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # One-shot: set desktop wallpaper for labwc (swaybg) and persist via labwc autostart, then remove self.
 # Runs as user pi at first login. Logs to /var/log/first-boot.log.
 FIRST_BOOT_LOG="/var/log/first-boot.log"

emmc-provisioning/cloud-init/first-boot.conf

@@ -1,4 +1,5 @@
-# first-boot.conf.example
+# Revision: 2
+# first-boot.conf (or .example)
 # Copy to first-boot.conf and edit. Loaded by first-boot.sh from:
 #   - same directory as first-boot.sh, or
 #   - /tmp/first-boot.conf (when run via cloud-init), or

emmc-provisioning/cloud-init/first-boot.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # First-boot: packages, Chromium kiosk, rpd-labwc + touch, reTerminal DM drivers.
 # Run by cloud-init (user-data-remote-gnss.example). Run as root.
 # Optional: copy first-boot.conf.example to first-boot.conf and edit; it is loaded

emmc-provisioning/cloud-init/fix-reterminal-display.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # One-time fix for reTerminal DM after first-boot: splash, Plymouth theme, rotation, wallpaper.
 # Run on the device as root (e.g. sudo bash fix-reterminal-display.sh).
 # Or run over SSH: ssh pi@DEVICE_IP 'sudo bash -s' < fix-reterminal-display.sh

emmc-provisioning/cloud-init/set-rotation-at-login.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # Set screen rotation via kanshi (same as Control Center). Reads video=DSI-1:rotate=N
 # from kernel cmdline and writes ~/.config/kanshi/config. Kanshi auto-reloads when the file changes.
 # Also sets GTK dark theme (PiXnoir / Adwaita-dark). Runs from autostart when user pi logs in; does not remove itself.

emmc-provisioning/cloud-init/start-chromium.sh

@@ -1,4 +1,5 @@
 #!/bin/bash
+# Revision: 2
 # Start Chromium in app mode. Optional env vars:
 #   CHROMIUM_APP_URL  - URL to open (default: http://127.0.0.1:8080)
 #   CHROMIUM_MODE     - "fullscreen" or "kiosk" (default: fullscreen)

emmc-provisioning/dashboard/app.py

@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# Revision: 2
 """
 Flask dashboard for CM4 eMMC provisioning.
 Public home: deploy only (status, logs, how to connect). No login.
@@ -1632,10 +1633,12 @@ def api_build_cloudinit_cancel():
 @app.route("/api/build-cloudinit-status-clear", methods=["POST"])
 @require_admin
 def api_build_cloudinit_status_clear():
-    """Clear build status to idle (so message is cleared after cancel/done/error)."""
+    """Clear build status to idle (so message is cleared after cancel/done/error).
+    Use ?force=1 to clear even when status is stuck (e.g. build died during finalizing)."""
     st = _build_status_read()
+    force = request.args.get("force") in ("1", "true", "yes")
     busy = st.get("phase") in ("downloading", "decompressing", "injecting", "finalizing", "resolving")
-    if busy:
+    if busy and not force:
         return jsonify({"ok": False, "error": "Cannot clear while build is in progress"}), 409
     _build_status_write("idle", "", None, None)
     return jsonify({"ok": True})

emmc-provisioning/dashboard/templates/admin.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">

emmc-provisioning/dashboard/templates/cloudinit_build.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">
@@ -155,7 +156,7 @@
           if (!window._buildClearScheduled) {
             window._buildClearScheduled = true;
             setTimeout(function() {
-              authFetch('/api/build-cloudinit-status-clear', { method: 'POST', headers: {'Content-Type':'application/json'} }).then(function() { fetchBuildStatus(); }).finally(function() { window._buildClearScheduled = false; });
+              authFetch('/api/build-cloudinit-status-clear?force=1', { method: 'POST', headers: {'Content-Type':'application/json'} }).then(function() { fetchBuildStatus(); }).finally(function() { window._buildClearScheduled = false; });
             }, 3000);
           }
         } else {
@@ -243,7 +244,7 @@
     var cancelBuildBtn = document.getElementById('buildCloudInitCancelBtn');
     if (cancelBuildBtn) cancelBuildBtn.onclick = cancelBuild;
     var dismissBuildBtn = document.getElementById('buildCloudInitDismiss');
-    if (dismissBuildBtn) dismissBuildBtn.onclick = function(e) { e.preventDefault(); authFetch('/api/build-cloudinit-status-clear', { method: 'POST', headers: {'Content-Type':'application/json'} }).then(function() { fetchBuildStatus(); }); };
+    if (dismissBuildBtn) dismissBuildBtn.onclick = function(e) { e.preventDefault(); authFetch('/api/build-cloudinit-status-clear?force=1', { method: 'POST', headers: {'Content-Type':'application/json'} }).then(function() { fetchBuildStatus(); }); };
     document.getElementById('buildVariant').onchange = function() {
       authFetch('/api/raspios-latest-url?variant=' + encodeURIComponent(document.getElementById('buildVariant').value)).then(function(r) { return r.json(); }).then(function(d) {
         document.getElementById('buildRaspiosUrl').textContent = (d.ok && d.filename) ? d.filename : (d.error || '');

emmc-provisioning/dashboard/templates/home.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">

emmc-provisioning/dashboard/templates/index.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">
@@ -916,7 +917,7 @@
           if (!window._buildClearScheduled) {
             window._buildClearScheduled = true;
             setTimeout(function() {
-              fetch('/api/build-cloudinit-status-clear', { method: 'POST', headers: { 'Content-Type': 'application/json' } }).then(function() { fetchBuildStatus(); }).finally(function() { window._buildClearScheduled = false; });
+              fetch('/api/build-cloudinit-status-clear?force=1', { method: 'POST', headers: { 'Content-Type': 'application/json' } }).then(function() { fetchBuildStatus(); }).finally(function() { window._buildClearScheduled = false; });
             }, 3000);
           }
         } else {
@@ -1024,7 +1025,7 @@
     var cancelBuildBtn = document.getElementById('buildCloudInitCancelBtn');
     if (cancelBuildBtn) cancelBuildBtn.onclick = cancelBuildCloudInit;
     var dismissBuildBtn = document.getElementById('buildCloudInitDismiss');
-    if (dismissBuildBtn) dismissBuildBtn.onclick = function(e) { e.preventDefault(); fetch('/api/build-cloudinit-status-clear', { method: 'POST', headers: { 'Content-Type': 'application/json' } }).then(function() { fetchBuildStatus(); }); };
+    if (dismissBuildBtn) dismissBuildBtn.onclick = function(e) { e.preventDefault(); fetch('/api/build-cloudinit-status-clear?force=1', { method: 'POST', headers: { 'Content-Type': 'application/json' } }).then(function() { fetchBuildStatus(); }); };
     var variantSel = document.getElementById('buildVariant');
     if (variantSel) variantSel.onchange = fetchRaspiosUrl;
     var templateLoadBtn = document.getElementById('buildTemplateLoad');

emmc-provisioning/dashboard/templates/login.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">

emmc-provisioning/dashboard/templates/portal_files.html

@@ -1,4 +1,5 @@
 <!DOCTYPE html>
+<!-- Revision: 2 -->
 <html lang="en">
 <head>
   <meta charset="UTF-8">

emmc-provisioning/host/README.md

@@ -1,3 +1,4 @@
+<!-- Revision: 2 -->
 # Host-side provisioning scripts
 
 These files run on the **provisioning host** (e.g. the Proxmox host where the reTerminal is connected via USB).
@@ -9,3 +10,23 @@ These files run on the **provisioning host** (e.g. the Proxmox host where the re
 | **90-cm4-boot-mode.rules** | udev rule: on USB add (vendor 2b8e), run the trigger. Install to `/etc/udev/rules.d/`. |
 
 See [../docs/EMMC-PROVISIONING-GUIDE.md](../docs/EMMC-PROVISIONING-GUIDE.md) for full setup. The [deploy script](../scripts/deploy-to-proxmox.sh) copies these into place on the Proxmox host and LXC.
+
+### Stuck “finalizing: Compressing image (xz)…” on the host
+
+If the dashboard still shows that message, run on the **host** (e.g. `ssh root@10.20.30.152`):
+
+```bash
+# 1) See current status and whether build/xz is still running
+PROV="${CM4_PROVISIONING_DIR:-/var/lib/cm4-provisioning}"
+cat "$PROV/build_cloudinit_status.json"
+ps aux | grep -E 'build-cloudinit|xz.*\.img' | grep -v grep
+
+# 2a) If a build is still running and you want to cancel it
+touch "$PROV/build_cloudinit_cancel"
+# Script checks every 2s; after it exits, status will show "cancelled"
+
+# 2b) If no build process is running but status is stuck, reset to idle
+echo '{"phase":"idle","message":"","output_name":null,"error":null,"updated":'$(date +%s)'}' > "$PROV/build_cloudinit_status.json"
+```
+
+Then refresh the dashboard or click **Dismiss** to clear the message.

emmc-provisioning/host/build-cloudinit-image.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Run on the Proxmox host when build_cloudinit_request.json appears in the provisioning dir.
 # Downloads Raspberry Pi OS image from URL, injects cloud-init NoCloud files, saves to BACKUPS_DIR.
 # Uses loop devices and mount (available on host, not in unprivileged LXC).
@@ -193,21 +194,31 @@ cp "$IMG_FILE" "$OUT_PATH"
 check_cancel
 write_status "finalizing" "Compressing image (xz)…" "" ""
 OUT_XZ="${OUT_PATH}.xz"
+XZ_PID=""
+cleanup_xz_and_exit() {
+  local phase="${1:-cancelled}"
+  if [[ -n "$XZ_PID" ]] && kill -0 "$XZ_PID" 2>/dev/null; then
+    kill "$XZ_PID" 2>/dev/null
+    wait "$XZ_PID" 2>/dev/null || true
+  fi
+  write_status "$phase" "Build cancelled." "" ""
+  rm -f "$REQUEST_FILE" "$CANCEL_FILE"
+  exit 0
+}
+trap 'cleanup_xz_and_exit "cancelled"' TERM INT
 xz -T 0 -z -k -f "$OUT_PATH" 2>/dev/null &
 XZ_PID=$!
 while kill -0 "$XZ_PID" 2>/dev/null; do
   if [[ -f "$CANCEL_FILE" ]]; then
-    kill "$XZ_PID" 2>/dev/null
+    cleanup_xz_and_exit "cancelled"
-    wait "$XZ_PID" 2>/dev/null
-    write_status "cancelled" "Build cancelled." "" ""
-    rm -f "$REQUEST_FILE" "$CANCEL_FILE"
-    exit 0
   fi
   sleep 2
 done
 wait "$XZ_PID" 2>/dev/null || true
+trap - TERM INT
 if [[ ! -f "$OUT_XZ" ]]; then
   xz -T 1 -z -k -f "$OUT_PATH" 2>/dev/null || true
+
 fi
 if [[ -f "$OUT_XZ" ]]; then
     rm -f "$OUT_PATH"

emmc-provisioning/host/cm4-flash-trigger.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Called by udev when CM4 in boot mode (0a5c:2711 or 2b8e) is connected.
 # Start the flash service via systemd so it runs under systemd, not udev —
 # otherwise udev kills the process when the device re-enumerates (first→second stage).

emmc-provisioning/host/flash-emmc-on-connect.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Provision CM4 eMMC when reTerminal is connected in boot mode (eMMC disable jumper).
 # On USB boot device (2b8e) detection: run rpiboot to expose eMMC, then wait for the user
 # to choose Backup or Deploy in the portal — no auto-flash; action runs only after portal choice.

emmc-provisioning/host/run-shrink-on-host.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Run on the Proxmox host when shrink_request.json appears in the provisioning dir.
 # Runs PiShrink on the requested backup image (shrink or shrink+compress).
 # Requires PiShrink on host (scripts/install-pishrink-on-host.sh). Triggered by cm4-shrink.path.

emmc-provisioning/scripts/bump-revision.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Revision: 2
+# Bumps REVISION at repo root and updates the "Revision: N" line in all tracked files.
+# Run from repo root, or use with: ./emmc-provisioning/scripts/bump-revision.sh
+# Optional: use in a pre-commit hook so every commit gets a new revision.
+
+set -e
+# Script lives in emmc-provisioning/scripts/; repo root is two levels up
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+REVISION_FILE="$ROOT/REVISION"
+
+# Files to update (relative to ROOT). Each must contain a line matching:
+#   # Revision: N   or   <!-- Revision: N -->
+TRACKED=(
+  emmc-provisioning/scripts/bump-revision.sh
+  emmc-provisioning/scripts/pre-commit-revision.sh
+  emmc-provisioning/host/build-cloudinit-image.sh
+  emmc-provisioning/host/flash-emmc-on-connect.sh
+  emmc-provisioning/host/run-shrink-on-host.sh
+  emmc-provisioning/host/README.md
+  emmc-provisioning/host/cm4-flash-trigger.sh
+  emmc-provisioning/dashboard/app.py
+  emmc-provisioning/dashboard/templates/index.html
+  emmc-provisioning/dashboard/templates/home.html
+  emmc-provisioning/dashboard/templates/cloudinit_build.html
+  emmc-provisioning/dashboard/templates/admin.html
+  emmc-provisioning/dashboard/templates/login.html
+  emmc-provisioning/dashboard/templates/portal_files.html
+  emmc-provisioning/cloud-init/first-boot.sh
+  emmc-provisioning/cloud-init/first-boot.conf
+  emmc-provisioning/cloud-init/start-chromium.sh
+  emmc-provisioning/cloud-init/01-set-rotation-once.sh
+  emmc-provisioning/cloud-init/02-set-wallpaper-once.sh
+  emmc-provisioning/cloud-init/set-rotation-at-login.sh
+  emmc-provisioning/cloud-init/fix-reterminal-display.sh
+  emmc-provisioning/scripts/deploy-to-proxmox.sh
+  emmc-provisioning/scripts/deploy-dashboard-to-lxc.sh
+  emmc-provisioning/scripts/sync-portal-files-to-lxc.sh
+  README.md
+  emmc-provisioning/README.md
+)
+
+current=1
+[[ -f "$REVISION_FILE" ]] && current=$(awk '/^[0-9]+$/ { print $1; exit }' "$REVISION_FILE" || true)
+[[ -z "$current" ]] && current=0
+next=$((current + 1))
+echo "$next" > "$REVISION_FILE"
+
+for rel in "${TRACKED[@]}"; do
+  f="$ROOT/$rel"
+  [[ "$rel" == "REVISION" ]] && continue
+  [[ ! -f "$f" ]] && continue
+  if grep -qE 'Revision: [0-9]+|<!-- Revision: [0-9]+ -->' "$f" 2>/dev/null; then
+    sed -i -E 's/(Revision: )[0-9]+/\1'"$next"'/; s/(<!-- Revision: )[0-9]+( -->)/\1'"$next"'\2/' "$f"
+  fi
+done
+
+echo "Revision bumped to $next"

emmc-provisioning/scripts/deploy-dashboard-to-lxc.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Deploy only the dashboard to the LXC by IP (no Proxmox host needed).
 # Uses rsync so all files (app, templates, service file, etc.) stay in sync.
 # Usage: ./deploy-dashboard-to-lxc.sh [user@lxc_ip]

emmc-provisioning/scripts/deploy-to-proxmox.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Deploy CM4 eMMC provisioning to a Proxmox host: host scripts, udev, systemd units,
 # LXC container (dashboard), usbboot (rpiboot), and PiShrink. Uses hostname "cm4-provisioning"
 # to find the container on redeploy; creates with next available ID if not found.

emmc-provisioning/scripts/pre-commit-revision.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# Revision: 2
+# Sample pre-commit hook: bump revision and update all tracked files, then stage them.
+# Install: cp emmc-provisioning/scripts/pre-commit-revision.sh .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit
+# Or: ln -sf ../../emmc-provisioning/scripts/pre-commit-revision.sh .git/hooks/pre-commit
+
+set -e
+ROOT="$(git rev-parse --show-toplevel)"
+cd "$ROOT"
+"$ROOT/emmc-provisioning/scripts/bump-revision.sh"
+git add REVISION
+git add emmc-provisioning/scripts/bump-revision.sh \
+  emmc-provisioning/scripts/pre-commit-revision.sh \
+  emmc-provisioning/host/build-cloudinit-image.sh \
+  emmc-provisioning/host/flash-emmc-on-connect.sh \
+  emmc-provisioning/host/run-shrink-on-host.sh \
+  emmc-provisioning/host/README.md \
+  emmc-provisioning/host/cm4-flash-trigger.sh \
+  emmc-provisioning/dashboard/app.py \
+  emmc-provisioning/dashboard/templates/index.html \
+  emmc-provisioning/dashboard/templates/home.html \
+  emmc-provisioning/dashboard/templates/cloudinit_build.html \
+  emmc-provisioning/dashboard/templates/admin.html \
+  emmc-provisioning/dashboard/templates/login.html \
+  emmc-provisioning/dashboard/templates/portal_files.html \
+  emmc-provisioning/cloud-init/first-boot.sh \
+  emmc-provisioning/cloud-init/first-boot.conf \
+  emmc-provisioning/cloud-init/start-chromium.sh \
+  emmc-provisioning/cloud-init/01-set-rotation-once.sh \
+  emmc-provisioning/cloud-init/02-set-wallpaper-once.sh \
+  emmc-provisioning/cloud-init/set-rotation-at-login.sh \
+  emmc-provisioning/cloud-init/fix-reterminal-display.sh \
+  emmc-provisioning/scripts/deploy-to-proxmox.sh \
+  emmc-provisioning/scripts/deploy-dashboard-to-lxc.sh \
+  emmc-provisioning/scripts/sync-portal-files-to-lxc.sh \
+  README.md \
+  emmc-provisioning/README.md
+exit 0

emmc-provisioning/scripts/sync-portal-files-to-lxc.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# Revision: 2
 # Sync portal (file server) content from the repo to the LXC.
 # Updates /var/lib/cm4-provisioning/portal-files/ so first-boot and the
 # dashboard /files/ serve the same scripts and assets as in the repo.