Enhance provisioning documentation and scripts for improved network boot and DNS management</message>
<message>Add new documentation files for device DNS management via DHCP and dnsmasq configuration. Update cloud-init scripts to ensure proper handling of /etc/resolv.conf and DNS settings, allowing for seamless integration with file.server. Modify existing scripts to support dynamic LAN subnet configuration and improve overall network boot functionality. These changes enhance user experience and streamline the setup process for the CM4 eMMC provisioning service.
This commit is contained in:
nearxos
90
emmc-provisioning/docs/DEVICE-DNS-DHCP-RESOLVCONF.md
Normal file
90
emmc-provisioning/docs/DEVICE-DNS-DHCP-RESOLVCONF.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# Device DNS from DHCP and /etc/resolv.conf
|
||||
|
||||
This document describes how to configure provisioned devices (e.g. Raspberry Pi / reTerminal) so they **use DNS from DHCP** and do **not** have a fixed nameserver in `/etc/resolv.conf`. That way the LXC’s dnsmasq (option 6) is used, **file.server** resolves, and scripts can use `http://file.server/...` without hardcoding IPs.
|
||||
|
||||
## Summary of changes (what we did on the Pi)
|
||||
|
||||
1. **Do not overwrite `/etc/resolv.conf`**
|
||||
No script (e.g. bootstrap or first-boot) should write a fixed nameserver (e.g. `8.8.8.8`) into `/etc/resolv.conf`. DNS should come from DHCP.
|
||||
|
||||
2. **LXC sends DHCP option 6 (DNS server)**
|
||||
dnsmasq on the LXC must send the LXC’s eth1 IP as the DNS server so clients use it and get **file.server** resolution. See [DNSMASQ-DNS-FILESERVER.md](DNSMASQ-DNS-FILESERVER.md) and `scripts/setup-network-boot-on-lxc.sh` (`dhcp-option=6,${LAN_GW}`).
|
||||
|
||||
3. **Let NetworkManager manage `/etc/resolv.conf`**
|
||||
So that the nameserver in `/etc/resolv.conf` is the one from DHCP (option 6), either:
|
||||
- **Option A (recommended for full cloud-init):** Use **systemd-resolved** and make `/etc/resolv.conf` a symlink to the resolved stub; resolved gets DNS from NetworkManager.
|
||||
- **Option B (minimal cloud-init):** Use **NetworkManager** to manage `/etc/resolv.conf` via a symlink: add `rc-manager=symlink` in NetworkManager config so `/etc/resolv.conf` points to `/run/NetworkManager/resolv.conf`, which NM fills with the DHCP DNS.
|
||||
|
||||
4. **Ensure connection uses DHCP DNS**
|
||||
The NetworkManager connection should have `ipv4.ignore-auto-dns: no` (default) so it accepts option 6 from DHCP. No fixed `ipv4.dns` in the connection.
|
||||
|
||||
## What to change in cloud-init
|
||||
|
||||
### Option A: user-data.bootstrap (systemd-resolved)
|
||||
|
||||
**File:** `cloud-init/user-data.bootstrap`
|
||||
|
||||
- **manage_resolv_conf: false** — already set; cloud-init must not overwrite resolv.conf.
|
||||
- **systemd-resolved** — runcmd enables/starts resolved and makes `/etc/resolv.conf` a symlink to `stub-resolv.conf`. Resolved gets DNS from NetworkManager (and from the hooks in write_files).
|
||||
- **NetworkManager** — `99-use-resolved.conf` has `dns=systemd-resolved` and `rc-manager=unmanaged` so NM doesn’t write resolv.conf; resolved does.
|
||||
- **Bootstrap script** — must **not** write `nameserver 8.8.8.8` (or any fixed server) into `/etc/resolv.conf`. Our `bootstrap.sh` no longer does that.
|
||||
|
||||
No extra changes needed if you use `user-data.bootstrap` as-is; just ensure your bootstrap script does not touch resolv.conf.
|
||||
|
||||
### Option B: Minimal user-data (first-boot or bootstrap-only, no systemd-resolved)
|
||||
|
||||
If your user-data only runs a remote script (e.g. `first-boot.sh` or `bootstrap.sh`) and does **not** enable systemd-resolved, add the following so the device uses DNS from DHCP and NM manages resolv.conf:
|
||||
|
||||
1. **Set in user-data (cloud-config):**
|
||||
```yaml
|
||||
manage_resolv_conf: false
|
||||
```
|
||||
|
||||
2. **Add a write_files entry** so NetworkManager manages resolv.conf with the DHCP-provided DNS:
|
||||
```yaml
|
||||
write_files:
|
||||
# ... your other write_files ...
|
||||
- path: /etc/NetworkManager/conf.d/99-resolv-dhcp.conf
|
||||
content: |
|
||||
[main]
|
||||
rc-manager=symlink
|
||||
permissions: '0644'
|
||||
```
|
||||
|
||||
3. **In your bootstrap/first-boot script:**
|
||||
Do **not** write a fixed nameserver to `/etc/resolv.conf` (e.g. remove any line like `echo "nameserver 8.8.8.8" > /etc/resolv.conf`).
|
||||
|
||||
4. **Optional runcmd** (if you want a clean state on first boot):
|
||||
Remove any existing static resolv.conf so NM can create its symlink and write DHCP DNS:
|
||||
```yaml
|
||||
runcmd:
|
||||
- rm -f /etc/resolv.conf
|
||||
- systemctl restart NetworkManager
|
||||
# ... then your download and run of bootstrap.sh or first-boot.sh ...
|
||||
```
|
||||
|
||||
After first boot, devices will get DNS from DHCP (LXC option 6), and **file.server** will resolve to the LXC’s eth1 IP.
|
||||
|
||||
## Verification on the device
|
||||
|
||||
```bash
|
||||
# Should show the LXC as nameserver (e.g. 10.20.40.1), not 8.8.8.8
|
||||
cat /etc/resolv.conf
|
||||
|
||||
# Should resolve to LXC eth1
|
||||
getent hosts file.server
|
||||
```
|
||||
|
||||
## Reference: manual fix on an already-provisioned device
|
||||
|
||||
If a device was provisioned before these changes and still has a fixed DNS (e.g. 8.8.8.8):
|
||||
|
||||
1. **LXC:** Ensure dnsmasq sends option 6 (see [DNSMASQ-DNS-FILESERVER.md](DNSMASQ-DNS-FILESERVER.md)); re-run `setup-network-boot-on-lxc.sh` if needed.
|
||||
2. **On the device:**
|
||||
- Add NetworkManager config:
|
||||
`echo -e '[main]\nrc-manager=symlink' | sudo tee /etc/NetworkManager/conf.d/99-resolv-dhcp.conf`
|
||||
- Remove static resolv.conf and restart NM:
|
||||
`sudo rm -f /etc/resolv.conf && sudo systemctl restart NetworkManager`
|
||||
- Renew DHCP so the device gets option 6:
|
||||
`sudo nmcli con down "Wired connection 1"; sudo nmcli con up "Wired connection 1"`
|
||||
3. Check: `cat /etc/resolv.conf` and `getent hosts file.server`.
|
||||
Reference in New Issue
Block a user