diff --git a/emmc-provisioning/docs/NETWORK-BOOT-LXC.md b/emmc-provisioning/docs/NETWORK-BOOT-LXC.md index fb2dc37..d1b6790 100644 --- a/emmc-provisioning/docs/NETWORK-BOOT-LXC.md +++ b/emmc-provisioning/docs/NETWORK-BOOT-LXC.md @@ -88,6 +88,21 @@ So changing `DEPLOY_LXC_LAN_SUBNET` and **re-running the deploy script** updates Then run **toggle enable** again if you use network boot: `ssh root@ /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable` +### Extra LAN IPs and VLAN (eth1.40) + +The setup script also configures **extra IPs on eth1** and a **VLAN interface** so the LXC can serve multiple subnets and provide internet (NAT) to all of them: + +| Address / interface | Purpose | +|--------------------|--------| +| **Primary** (e.g. `10.20.50.1/24`) | Set at deploy; used by dnsmasq for DHCP/TFTP/DNS | +| **192.168.30.1/24** | Extra LAN on eth1 | +| **192.168.127.1/24** | Extra LAN on eth1 | +| **eth1.40** **192.168.0.1/24** | VLAN 40 on eth1 | + +- Config is persisted in **`/etc/network/interfaces.d/70-cm4-extra-lan`** (installed when you run `setup-network-boot-on-lxc.sh`). +- **NAT** is applied to all four: primary LAN, 192.168.30.0/24, 192.168.127.0/24, and 192.168.0.0/24 (VLAN 40), so clients on any of these subnets get internet via eth0. +- For **VLAN 40** to receive tagged traffic, the Proxmox bridge connected to eth1 (e.g. vmbr1) must either be a trunk that passes VLAN 40, or you use a dedicated bridge (e.g. vmbr1.40) and attach the container to it as a second interface; the script creates eth1.40 inside the LXC for the in-container VLAN case. + ## After setup: reTerminal network boot 1. Set the reTerminal **boot order** to try eMMC first, then network (e.g. `BOOT_ORDER=0xf21`): use the dashboard **Update EEPROM** when the device is connected via USB boot, or set manually (usbboot recovery / `rpi-eeprom-config` on device). Not set by first-boot. diff --git a/emmc-provisioning/lxc/70-cm4-extra-lan b/emmc-provisioning/lxc/70-cm4-extra-lan new file mode 100644 index 0000000..16a36cd --- /dev/null +++ b/emmc-provisioning/lxc/70-cm4-extra-lan @@ -0,0 +1,17 @@ +# Extra LAN IPs on eth1 and VLAN 40 on eth1. +# Primary eth1 address is set by Proxmox/deploy (used by dnsmasq DHCP). +# Installed by setup-network-boot-on-lxc.sh; ensure /etc/network/interfaces +# includes: source-directory /etc/network/interfaces.d + +# Secondary addresses on eth1 (192.168.30.1, 192.168.127.1) +iface eth1 inet static + address 192.168.30.1/24 +iface eth1 inet static + address 192.168.127.1/24 + +# VLAN 40 on eth1 — 192.168.0.0/24 (gateway 192.168.0.1) +# Requires: apt install vlan; host bridge must pass VLAN 40 if using tagged uplink +auto eth1.40 +iface eth1.40 inet static + address 192.168.0.1/24 + vlan-raw-device eth1 diff --git a/emmc-provisioning/lxc/README.md b/emmc-provisioning/lxc/README.md index f6c9870..ed0f039 100644 --- a/emmc-provisioning/lxc/README.md +++ b/emmc-provisioning/lxc/README.md @@ -7,7 +7,9 @@ Config files for the **provisioning LXC** when using **eth1** as a provisioning | File | Purpose | |------|--------| | **dnsmasq-network-boot.conf** | Template: dnsmasq DHCP + TFTP on eth1. Setup script writes `/etc/dnsmasq.d/network-boot.conf` using values from `lan-subnet.conf`. | -| **nft-nat-lan.conf** | Template: nftables NAT for LAN→WAN. Setup script writes `/etc/nftables.d/nat-lan.conf` using `LAN_CIDR` from `lan-subnet.conf`. | +| **nft-nat-lan.conf** | Template: nftables NAT for LAN→WAN (primary + extra subnets + VLAN 40). Setup script writes `/etc/nftables.d/nat-lan.conf`. | +| **70-cm4-extra-lan** | Extra LAN IPs on eth1 (192.168.30.1, 192.168.127.1) and VLAN eth1.40 (192.168.0.1/24). Installed to `/etc/network/interfaces.d/` by setup script. | +| **toggle-network-boot-dhcp.sh** | Enable/disable PXE (TFTP) on the LXC; copied to `/opt/cm4-provisioning/` by setup script. | Setup is done by running (from your machine): diff --git a/emmc-provisioning/lxc/nft-nat-lan.conf b/emmc-provisioning/lxc/nft-nat-lan.conf index ae1a555..42b3b4c 100644 --- a/emmc-provisioning/lxc/nft-nat-lan.conf +++ b/emmc-provisioning/lxc/nft-nat-lan.conf @@ -1,10 +1,14 @@ -# nftables: NAT for LAN (eth1) so clients use WAN (eth0) for internet. +# nftables: NAT for LAN (eth1 + extra IPs + eth1.40) so clients use WAN (eth0) for internet. # Load with: nft -f /etc/nftables.d/nat-lan.conf -# When using setup-network-boot-on-lxc.sh, the subnet is taken from /opt/cm4-provisioning/lan-subnet.conf (LAN_CIDR). +# When using setup-network-boot-on-lxc.sh, the primary subnet is from lan-subnet.conf (LAN_CIDR). +# Extra subnets: 192.168.30.0/24, 192.168.127.0/24 (eth1), 192.168.0.0/24 (eth1.40 VLAN). table ip nat { chain postrouting { type nat hook postrouting priority srcnat; policy accept; ip saddr 10.20.50.0/24 oifname "eth0" masquerade + ip saddr 192.168.30.0/24 oifname "eth0" masquerade + ip saddr 192.168.127.0/24 oifname "eth0" masquerade + ip saddr 192.168.0.0/24 oifname "eth0" masquerade } } diff --git a/emmc-provisioning/scripts/setup-network-boot-on-lxc.sh b/emmc-provisioning/scripts/setup-network-boot-on-lxc.sh index dcd42ec..0f96f72 100755 --- a/emmc-provisioning/scripts/setup-network-boot-on-lxc.sh +++ b/emmc-provisioning/scripts/setup-network-boot-on-lxc.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# Setup network boot on the provisioning LXC: DHCP + TFTP on eth1, NAT so LAN uses eth0 for internet. +# Setup provisioning LXC: DHCP + DNS on eth1, extra LAN IPs, VLAN eth1.40, NAT so LAN uses eth0 for internet. +# Network boot (TFTP, PXE, initrd) sections are commented out; uncomment if you need PXE/netboot. # Run inside the LXC (as root), or from your machine: ./setup-network-boot-on-lxc.sh root@10.130.60.141 [SUBNET] # SUBNET optional: A.B.C.D/PREFIX (e.g. 10.100.1.1/24). When run with ssh target, writes lan-subnet.conf on LXC if SUBNET given. # When run with ssh target, rsyncs lxc/ and runs this script inside the container. Subnet is read from /opt/cm4-provisioning/lan-subnet.conf. @@ -14,12 +15,13 @@ if [[ -n "$TARGET" ]]; then REPO_DIR="$(dirname "$SCRIPT_DIR")" echo "Syncing lxc config and script to $TARGET ..." rsync -a "$REPO_DIR/lxc/" "$TARGET:/tmp/cm4-network-boot-lxc/" --exclude='.git' - if [[ -f "$REPO_DIR/network-boot-initramfs/initrd.img" ]]; then - echo "Copying initrd.img to $TARGET ..." - scp "$REPO_DIR/network-boot-initramfs/initrd.img" "$TARGET:/tmp/cm4-network-boot-lxc/initrd.img" - else - echo "Note: network-boot-initramfs/initrd.img not found (run build.sh first); skipping." - fi + # Network boot: copy initrd.img for TFTP + # if [[ -f "$REPO_DIR/network-boot-initramfs/initrd.img" ]]; then + # echo "Copying initrd.img to $TARGET ..." + # scp "$REPO_DIR/network-boot-initramfs/initrd.img" "$TARGET:/tmp/cm4-network-boot-lxc/initrd.img" + # else + # echo "Note: network-boot-initramfs/initrd.img not found (run build.sh first); skipping." + # fi scp "$SCRIPT_DIR/setup-network-boot-on-lxc.sh" "$TARGET:/tmp/cm4-network-boot-lxc/setup.sh" # If SUBNET_ARG given, write lan-subnet.conf on LXC so inner script uses the set subnet if [[ -n "$SUBNET_ARG" ]]; then @@ -67,18 +69,21 @@ else DHCP_RANGE_END="10.20.50.200" echo "No lan-subnet.conf and no SUBNET argument; using defaults: $LAN_CIDR." fi -echo "Configuring network boot (DHCP + TFTP on eth1, NAT via eth0) — LAN $LAN_CIDR (gateway $LAN_GW), DHCP ${DHCP_RANGE_START}-${DHCP_RANGE_END} ..." +echo "Configuring LAN (DHCP + DNS on eth1, NAT via eth0) — LAN $LAN_CIDR (gateway $LAN_GW), DHCP ${DHCP_RANGE_START}-${DHCP_RANGE_END} ..." -# 1) Install dnsmasq +# 1) Install dnsmasq and vlan (for eth1.40) if ! command -v dnsmasq >/dev/null 2>&1; then apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq dnsmasq fi +if ! command -v vconfig >/dev/null 2>&1; then + apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq vlan +fi -# 2) dnsmasq config for eth1 only (DHCP + TFTP + DNS); PXE options in network-boot-pxe.conf (toggle with toggle-network-boot-dhcp.sh) +# 2) dnsmasq config for eth1 only (DHCP + DNS). PXE/TFTP in network-boot-pxe.conf when needed (toggle-network-boot-dhcp.sh) mkdir -p /etc/dnsmasq.d cat > /etc/dnsmasq.d/network-boot.conf << DNSMASQ # DHCP + DNS on eth1 only (provisioning LAN) -# TFTP and PXE options in network-boot-pxe.conf, controlled by toggle-network-boot-dhcp.sh +# (TFTP/PXE options in network-boot-pxe.conf when network boot is enabled) interface=eth1 bind-interfaces dhcp-range=${DHCP_RANGE_START},${DHCP_RANGE_END},12h @@ -91,51 +96,72 @@ log-dhcp log-queries DNSMASQ mkdir -p /opt/cm4-provisioning -if [ -f /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh ]; then - cp /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh /opt/cm4-provisioning/ - chmod +x /opt/cm4-provisioning/toggle-network-boot-dhcp.sh - /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable +# Network boot: install toggle script and enable PXE/TFTP +# if [ -f /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh ]; then +# cp /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh /opt/cm4-provisioning/ +# chmod +x /opt/cm4-provisioning/toggle-network-boot-dhcp.sh +# /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable +# fi + +# 3) Network boot: TFTP root — fetch Raspberry Pi 4 boot files from GitHub if missing +# mkdir -p /srv/tftpboot +# if [[ ! -f /srv/tftpboot/start4cd.elf ]]; then +# echo "Fetching Raspberry Pi firmware boot files from GitHub ..." +# if ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then +# apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq curl +# fi +# tmpdir=$(mktemp -d) +# trap "rm -rf $tmpdir" EXIT +# if command -v curl >/dev/null 2>&1; then +# curl -sL "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" -o "$tmpdir/firmware.tar.gz" +# else +# wget -q -O "$tmpdir/firmware.tar.gz" "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" +# fi +# tar xzf "$tmpdir/firmware.tar.gz" -C "$tmpdir" +# cp -a "$tmpdir/firmware-master/boot/." /srv/tftpboot/ +# rm -rf "$tmpdir" +# echo "Copied RPi boot files to /srv/tftpboot" +# else +# echo "TFTP root already has boot files (start4cd.elf present), skipping fetch." +# fi + +# 3b) Network boot: copy provisioning initrd.img to TFTP root if provided +# if [[ -f /tmp/cm4-network-boot-lxc/initrd.img ]]; then +# cp /tmp/cm4-network-boot-lxc/initrd.img /srv/tftpboot/initrd.img +# echo "Copied initrd.img to /srv/tftpboot" +# if [[ -f /srv/tftpboot/config.txt ]] && ! grep -q 'initramfs initrd.img' /srv/tftpboot/config.txt 2>/dev/null; then +# echo "" >> /srv/tftpboot/config.txt +# echo "# Provisioning initramfs (network-boot-initramfs)" >> /srv/tftpboot/config.txt +# echo "initramfs initrd.img followkernel" >> /srv/tftpboot/config.txt +# echo "Added initramfs line to config.txt" +# fi +# fi + +# 4) Extra LAN IPs on eth1 (192.168.30.1, 192.168.127.1) and VLAN eth1.40 (192.168.0.1/24) +EXTRA_LAN_CONF="/etc/network/interfaces.d/70-cm4-extra-lan" +if [[ -f /tmp/cm4-network-boot-lxc/70-cm4-extra-lan ]]; then + mkdir -p /etc/network/interfaces.d + cp /tmp/cm4-network-boot-lxc/70-cm4-extra-lan "$EXTRA_LAN_CONF" + # Ensure main interfaces includes interfaces.d (Debian) + if [[ -f /etc/network/interfaces ]] && ! grep -q 'interfaces.d' /etc/network/interfaces 2>/dev/null; then + echo 'source /etc/network/interfaces.d/*' >> /etc/network/interfaces + fi + # Apply immediately: secondary IPs on eth1 + ip addr add 192.168.30.1/24 dev eth1 2>/dev/null || true + ip addr add 192.168.127.1/24 dev eth1 2>/dev/null || true + # Create and bring up eth1.40 (VLAN 40) + ip link add link eth1 name eth1.40 type vlan id 40 2>/dev/null || true + ip addr add 192.168.0.1/24 dev eth1.40 2>/dev/null || true + ip link set eth1.40 up 2>/dev/null || true + echo "Extra LAN: eth1 + 192.168.30.1/24, 192.168.127.1/24; eth1.40 192.168.0.1/24 (persisted in $EXTRA_LAN_CONF)" fi -# 3) TFTP root: fetch Raspberry Pi 4 boot files from GitHub if missing -mkdir -p /srv/tftpboot -if [[ ! -f /srv/tftpboot/start4cd.elf ]]; then - echo "Fetching Raspberry Pi firmware boot files from GitHub ..." - if ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then - apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq curl - fi - tmpdir=$(mktemp -d) - trap "rm -rf $tmpdir" EXIT - if command -v curl >/dev/null 2>&1; then - curl -sL "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" -o "$tmpdir/firmware.tar.gz" - else - wget -q -O "$tmpdir/firmware.tar.gz" "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" - fi - tar xzf "$tmpdir/firmware.tar.gz" -C "$tmpdir" - cp -a "$tmpdir/firmware-master/boot/." /srv/tftpboot/ - rm -rf "$tmpdir" - echo "Copied RPi boot files to /srv/tftpboot" -else - echo "TFTP root already has boot files (start4cd.elf present), skipping fetch." -fi - -# 3b) Copy provisioning initrd.img to TFTP root if provided -if [[ -f /tmp/cm4-network-boot-lxc/initrd.img ]]; then - cp /tmp/cm4-network-boot-lxc/initrd.img /srv/tftpboot/initrd.img - echo "Copied initrd.img to /srv/tftpboot" - if [[ -f /srv/tftpboot/config.txt ]] && ! grep -q 'initramfs initrd.img' /srv/tftpboot/config.txt 2>/dev/null; then - echo "" >> /srv/tftpboot/config.txt - echo "# Provisioning initramfs (network-boot-initramfs)" >> /srv/tftpboot/config.txt - echo "initramfs initrd.img followkernel" >> /srv/tftpboot/config.txt - echo "Added initramfs line to config.txt" - fi -fi - -# 4) IP forwarding (LAN clients use WAN) +# 5) IP forwarding (LAN clients use WAN) echo 'net.ipv4.ip_forward=1' > /etc/sysctl.d/99-cm4-network-boot.conf sysctl -p /etc/sysctl.d/99-cm4-network-boot.conf 2>/dev/null || sysctl -w net.ipv4.ip_forward=1 -# 5) NAT: LAN subnet -> eth0 (masquerade) +# 6) NAT: primary LAN + extra LAN subnets + VLAN 40 -> eth0 (masquerade) +# Subnets: primary (from lan-subnet.conf), 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 (eth1.40) if command -v nft >/dev/null 2>&1; then mkdir -p /etc/nftables.d cat > /etc/nftables.d/nat-lan.conf << NFT @@ -143,29 +169,37 @@ table ip nat { chain postrouting { type nat hook postrouting priority srcnat; policy accept; ip saddr ${LAN_CIDR} oifname "eth0" masquerade + ip saddr 192.168.30.0/24 oifname "eth0" masquerade + ip saddr 192.168.127.0/24 oifname "eth0" masquerade + ip saddr 192.168.0.0/24 oifname "eth0" masquerade } } NFT if ! nft list table ip nat 2>/dev/null | grep -q postrouting; then nft -f /etc/nftables.d/nat-lan.conf + else + nft -f /etc/nftables.d/nat-lan.conf fi # Ensure main config includes our drop-in (Debian) if [[ -f /etc/nftables.conf ]] && ! grep -q 'nftables.d/nat-lan' /etc/nftables.conf 2>/dev/null; then echo 'include "/etc/nftables.d/nat-lan.conf"' >> /etc/nftables.conf fi - echo "NAT rule added (nftables) and saved to /etc/nftables.d/nat-lan.conf" + echo "NAT rules added (nftables): ${LAN_CIDR}, 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 -> eth0" else # Fallback iptables - iptables -t nat -C POSTROUTING -s "${LAN_CIDR}" -o eth0 -j MASQUERADE 2>/dev/null || \ - iptables -t nat -A POSTROUTING -s "${LAN_CIDR}" -o eth0 -j MASQUERADE - echo "NAT rule added (iptables)." + for cidr in "${LAN_CIDR}" 192.168.30.0/24 192.168.127.0/24 192.168.0.0/24; do + iptables -t nat -C POSTROUTING -s "$cidr" -o eth0 -j MASQUERADE 2>/dev/null || \ + iptables -t nat -A POSTROUTING -s "$cidr" -o eth0 -j MASQUERADE + done + echo "NAT rules added (iptables) for primary LAN and extra subnets." fi -# 6) Enable and start dnsmasq +# 7) Enable and start dnsmasq systemctl enable dnsmasq systemctl restart dnsmasq -echo "Network boot setup done." -echo " - DHCP + TFTP on eth1 ($LAN_GW), range ${DHCP_RANGE_START}-${DHCP_RANGE_END}" -echo " - NAT: ${LAN_CIDR} -> eth0 (internet)" -echo " - TFTP root: /srv/tftpboot (RPi boot files; initrd.img if provided)" +echo "Setup done." +echo " - DHCP + DNS on eth1 ($LAN_GW), range ${DHCP_RANGE_START}-${DHCP_RANGE_END}" +echo " - NAT: ${LAN_CIDR}, 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 -> eth0 (internet)" +echo " - Extra LAN: eth1 also 192.168.30.1, 192.168.127.1; eth1.40 192.168.0.1/24 (VLAN 40)" +# echo " - TFTP root: /srv/tftpboot (RPi boot files; initrd.img if provided)" # when network boot enabled