reterminal-dm4/emmc-provisioning/docs/NETWORK-BOOT-LXC.md

Network boot on the provisioning LXC (eth1 = LAN, eth0 = WAN)

The provisioning LXC can provide network boot (PXE-style) and internet access to devices connected on eth1, while eth0 is used as WAN for the LXC itself.

Roles

Interface	Role	Typical config
eth0	WAN	DHCP or static; default route; internet for the LXC
eth1	LAN (provisioning)	Static e.g. 10.20.50.1/24; DHCP server + TFTP server; NAT so clients get internet via eth0

Devices plugged into the same network as eth1 (e.g. reTerminals with network boot enabled) will:

1. Get an IP via DHCP (from the LXC on eth1).
2. Get TFTP boot files (Raspberry Pi firmware: start4.elf, fixup4.dat, kernel, etc.) for network boot.
3. Have internet via NAT through the LXC (eth0).

What you need on the LXC

1. DHCP server on eth1 only (e.g. dnsmasq), handing out addresses in e.g. 10.20.50.100–10.20.50.200 and advertising the TFTP server (next-server = LXC’s eth1 IP).
2. TFTP server (dnsmasq can provide this) with TFTP root containing Raspberry Pi 4 / CM4 boot files.
3. IP forwarding and NAT (nftables or iptables) so traffic from 10.20.50.0/24 is masqueraded out eth0.

One-time setup (inside the LXC)

From your machine, run the setup script on the LXC (replace with your LXC IP if different):

# From the repo (script runs inside the LXC)
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@10.130.60.141

Or SSH into the LXC and run the script there:

ssh root@10.130.60.141
# Copy or rsync the emmc-provisioning tree into the container, then:
bash /path/to/setup-network-boot-on-lxc.sh

The script will:

• Install dnsmasq (DHCP + TFTP).
• Configure dnsmasq to listen only on eth1, with a DHCP range and TFTP root.
• Create /srv/tftpboot and fetch Raspberry Pi 4 boot files from GitHub (raspberrypi/firmware, boot/ folder) if not already present.
• Enable IPv4 forwarding and NAT (nftables) so clients on eth1 use eth0 for internet.
• Enable and start the dnsmasq service.

Proxmox: adding eth1 to the LXC

If you create the container by hand or want a second interface:

1. On the Proxmox host, add a second network device to the container, e.g.:

   pct set <CTID> --net1 name=eth1,bridge=vmbr1,ip=10.20.50.1/24
   

   Use the bridge that corresponds to the physical LAN where reTerminals are connected (e.g. vmbr1 or a dedicated provisioning bridge).

2. Inside the LXC, ensure eth1 has a static address (e.g. in /etc/network/interfaces):

   auto eth1
   iface eth1 inet static
       address 10.20.50.1/24
   

Your current LXC already has eth0 (10.130.60.141) and eth1 (10.20.50.1); the setup script only adds DHCP, TFTP, and NAT.

After setup: reTerminal network boot

1. Set the reTerminal boot order to try network first (e.g. BOOT_ORDER=0x21; see cloud-init/first-boot).
2. Connect the reTerminal to the same network as the LXC’s eth1 (e.g. 10.20.50.0/24).
3. Power on; it will get an IP via DHCP and load boot files via TFTP from the LXC.
4. For provisioning (Backup/Deploy), the netboot environment must run network-client/provisioning-client.sh with PROVISIONING_SERVER=http://10.20.50.1:5000 so it talks to the dashboard on the LXC.

TFTP boot files (Raspberry Pi 4 / CM4)

The setup script automatically downloads the official Raspberry Pi firmware boot/ folder from GitHub (https://github.com/raspberrypi/firmware) into /srv/tftpboot when start4cd.elf is missing. No manual copy is needed.

To refresh or populate TFTP without re-running the full setup:

./emmc-provisioning/scripts/populate-tftpboot-from-git.sh root@<LXC-IP>

(Remove /srv/tftpboot/start4cd.elf on the LXC first if you want a full re-fetch.)

The TFTP root contains e.g. start4cd.elf, fixup4cd.dat, config.txt, cmdline.txt, kernel8.img, and other boot files. For a custom kernel or initramfs (e.g. for provisioning), add or replace files in /srv/tftpboot and adjust config.txt / cmdline.txt as needed.

DHCP leases

On the LXC, dnsmasq stores DHCP leases in /var/lib/misc/dnsmasq.leases (Debian/Ubuntu default). To see which devices got an IP on the provisioning LAN:

# On the LXC (or via SSH)
cat /var/lib/misc/dnsmasq.leases

Each line is: expiry_epoch MAC IP hostname client_id. Example: 1734567890 aa:bb:cc:dd:ee:ff 10.20.50.101 reterminal 01:aa:bb:cc:dd:ee:ff

Summary

Component	Where	Purpose
eth0	LXC	WAN; LXC’s internet
eth1	LXC	LAN; 10.20.50.1/24; DHCP + TFTP
dnsmasq	LXC	DHCP (on eth1) + TFTP
TFTP root	LXC	e.g. /srv/tftpboot with RPi boot files
NAT	LXC	10.20.50.0/24 → eth0 so LAN has internet