nearxos/reterminal-dm4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: nearxos:59f8ebe61dedccd04d7474afe92cb7c43319534d
Branches Tags
nearxos:main nearxos:Cleanup
..
compare: nearxos:main
Branches Tags
nearxos:Cleanup nearxos:main

6 Commits
59f8ebe61d ... main

Author SHA1 Message Date
nearxos
10c200f994 Enhance network boot provisioning with support for extra LAN IPs and VLAN configuration</message> 
<message>Update documentation and scripts to include configuration for extra LAN IPs on eth1 and VLAN interface eth1.40, allowing the LXC to serve multiple subnets and provide NAT for internet access. Modify nftables NAT configuration to accommodate these changes and ensure proper DHCP and DNS setup on eth1. This improves the overall network boot functionality and user experience for the CM4 eMMC provisioning service.
 2026-03-04 19:28:53 +02:00
nearxos
031e1c3415 Enhance provisioning documentation and scripts for improved network boot and DNS management</message> 
<message>Add new documentation files for device DNS management via DHCP and dnsmasq configuration. Update cloud-init scripts to ensure proper handling of /etc/resolv.conf and DNS settings, allowing for seamless integration with file.server. Modify existing scripts to support dynamic LAN subnet configuration and improve overall network boot functionality. These changes enhance user experience and streamline the setup process for the CM4 eMMC provisioning service.
 2026-03-04 19:15:38 +02:00
nearxos
b5134098c0 Exclude large files: expand .gitignore, drop backup data from tracking 2026-03-03 09:03:38 +02:00
nearxos
c5e418eabc Update provisioning documentation and scripts for improved Proxmox deployment</message> 
<message>Add a new step-by-step guide for deploying the CM4 eMMC provisioning service on a new Proxmox instance, enhancing clarity for users. Update existing documentation to reflect changes in network configuration options, including the introduction of LAN subnet settings for DHCP and TFTP. Modify cloud-init scripts to ensure proper management of DNS settings and improve the handling of network interfaces. Additionally, enhance the toggle script for network boot to dynamically read the LAN gateway from configuration files, streamlining the setup process and improving user experience.
 2026-03-03 08:24:18 +02:00
nearxos
fe72619931 Update GNSS bootstrap image to the latest version, ensuring compatibility and improved performance. This change replaces the previous image file with an updated binary, enhancing the overall provisioning process. 2026-02-24 08:52:51 +02:00
nearxos
16bfc1e0e1 Enhance cloud-init scripts and dashboard for improved USB boot functionality</message> 
<message>Update the bootstrap script to ensure hostname resolution by adding entries to /etc/hosts, preventing "sudo: unable to resolve host" errors. Modify user-data.bootstrap to include the same hostname resolution logic. Revise dashboard templates to reflect the new project name "GNSS Guard Provisioning" and improve user interface elements related to USB boot operations, including clearer instructions and status messages. These changes enhance the overall user experience and streamline the provisioning process.
 2026-02-24 08:50:32 +02:00
43 changed files with 1390 additions and 386072 deletions
Expand all files Collapse all files

12
.gitignore vendored Normal file
View File

@@ -0,0 +1,12 @@
# Large binary / image files (do not commit)
*.img.xz
*.img.xz.bak
*.img
!emmc-provisioning/network-boot-initramfs/*.img
# Backup/data from devices (large DBs and logs)
backup-from-device/**/data/*.db
backup-from-device/**/logs/
**/*.db
*.sqlite
*.sqlite3

BIN
backup-from-device/gnss-guard/tm-gnss-guard/data/gnss_guard.db
View File

Binary file not shown.

38710
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/09/app_2026-02-09.json
View File

File diff suppressed because it is too large Load Diff

38688
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/10/app_2026-02-10.json
View File

File diff suppressed because it is too large Load Diff

38708
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/11/app_2026-02-11.json
View File

File diff suppressed because it is too large Load Diff

38704
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/12/app_2026-02-12.json
View File

File diff suppressed because it is too large Load Diff

38708
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/13/app_2026-02-13.json
View File

File diff suppressed because it is too large Load Diff

38708
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/14/app_2026-02-14.json
View File

File diff suppressed because it is too large Load Diff

38708
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/15/app_2026-02-15.json
View File

File diff suppressed because it is too large Load Diff

38710
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/16/app_2026-02-16.json
View File

File diff suppressed because it is too large Load Diff

38708
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/17/app_2026-02-17.json
View File

File diff suppressed because it is too large Load Diff

28527
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/18/app_2026-02-18.json
View File

File diff suppressed because it is too large Load Diff

8677
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/19/app_2026-02-19.json
View File

File diff suppressed because it is too large Load Diff

345
backup-from-device/gnss-guard/tm-gnss-guard/logs/2026/02/23/app_2026-02-23.json
View File

@@ -1,345 +0,0 @@
{"timestamp":"2026-02-23T22:46:10.124927","level":"INFO","source":"system","message":"GNSS Guard starting","data":{"config":{"asset_name":"OFFICE_LAB","iteration_period_seconds":30,"stale_threshold_seconds":60,"validation_threshold_meters":200.0,"startup_warmup_seconds":5,"positions_raw_retention_days":5,"positions_validation_retention_days":5,"log_retention_days":14,"tm_ais_url":"https://localhost:8443/location","tm_ais_enabled":true,"tm_ais_max_retries":1,"starlink_ip":"10.130.60.70","starlink_port":9200,"starlink_enabled":true,"starlink_max_retries":1,"nmea_primary_enabled":true,"nmea_secondary_enabled":true,"database_path":"data/gnss_guard.db","logs_base_path":"logs","web_enabled":true,"web_host":"0.0.0.0","web_port":8080,"web_show_route":true}}}
{"timestamp":"2026-02-23T22:46:10.126502","level":"INFO","source":"system","message":"DEMO_UNIT mode - no database writes"}
{"timestamp":"2026-02-23T22:46:10.127389","level":"INFO","source":"system","message":"Startup warm-up period","data":{"warmup_seconds":5}}
{"timestamp":"2026-02-23T22:46:10.127998","level":"INFO","source":"nmea_primary","message":"Starting connection to device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001}}
{"timestamp":"2026-02-23T22:46:10.129255","level":"ERROR","source":"nmea_primary","message":"Network error for device nmea_primary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4001}}
{"timestamp":"2026-02-23T22:46:10.130714","level":"INFO","source":"nmea_secondary","message":"Starting connection to device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002}}
{"timestamp":"2026-02-23T22:46:10.132229","level":"ERROR","source":"nmea_secondary","message":"Network error for device nmea_secondary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4002}}
{"timestamp":"2026-02-23T22:46:15.136020","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:46:15.135561+00:00","timestamp_unix":1771879575.135561}}}
{"timestamp":"2026-02-23T22:46:15.136213","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:46:15.135588+00:00","timestamp_unix":1771879575.135588}}}
{"timestamp":"2026-02-23T22:46:15.136326","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:46:15.135575+00:00","timestamp_unix":1771879575.135575}}}
{"timestamp":"2026-02-23T22:46:15.136420","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:46:15.135475+00:00","timestamp_unix":1771879575.135475}}}
{"timestamp":"2026-02-23T22:46:15.136508","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:46:15.135543+00:00","timestamp_unix":1771879575.135543}}}
{"timestamp":"2026-02-23T22:46:15.155432","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:46:15.136736+00:00","validation_timestamp_unix":1771879575.136736,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:46:15.135561+00:00","timestamp_unix":1771879575.135561},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:46:15.135588+00:00","timestamp_unix":1771879575.135588},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:46:15.135575+00:00","timestamp_unix":1771879575.135575},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:46:15.135475+00:00","timestamp_unix":1771879575.135475},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:46:15.135543+00:00","timestamp_unix":1771879575.135543}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:46:15.155998","level":"WARNING","source":"buzzer","message":"Alarm started: GPS jamming/spoofing detected (distance exceeded threshold)"}
{"timestamp":"2026-02-23T22:46:15.159100","level":"ERROR","source":"nmea_primary","message":"Network error for device nmea_primary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4001}}
{"timestamp":"2026-02-23T22:46:15.161128","level":"ERROR","source":"nmea_secondary","message":"Network error for device nmea_secondary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4002}}
{"timestamp":"2026-02-23T22:46:30.162167","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:46:30.163835","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:46:45.208056","level":"INFO","source":"system","message":"GNSS Guard stopped"}
{"timestamp":"2026-02-23T22:47:18.807086","level":"INFO","source":"system","message":"GNSS Guard starting","data":{"config":{"asset_name":"OFFICE_LAB","iteration_period_seconds":30,"stale_threshold_seconds":60,"validation_threshold_meters":200.0,"startup_warmup_seconds":5,"positions_raw_retention_days":5,"positions_validation_retention_days":5,"log_retention_days":14,"tm_ais_url":"https://localhost:8443/location","tm_ais_enabled":true,"tm_ais_max_retries":1,"starlink_ip":"10.130.60.70","starlink_port":9200,"starlink_enabled":true,"starlink_max_retries":1,"nmea_primary_enabled":true,"nmea_secondary_enabled":true,"database_path":"data/gnss_guard.db","logs_base_path":"logs","web_enabled":true,"web_host":"0.0.0.0","web_port":8080,"web_show_route":true}}}
{"timestamp":"2026-02-23T22:47:18.809486","level":"INFO","source":"system","message":"DEMO_UNIT mode - no database writes"}
{"timestamp":"2026-02-23T22:47:18.810620","level":"INFO","source":"system","message":"Startup warm-up period","data":{"warmup_seconds":5}}
{"timestamp":"2026-02-23T22:47:18.811272","level":"INFO","source":"nmea_primary","message":"Starting connection to device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001}}
{"timestamp":"2026-02-23T22:47:18.812711","level":"ERROR","source":"nmea_primary","message":"Network error for device nmea_primary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4001}}
{"timestamp":"2026-02-23T22:47:18.813713","level":"INFO","source":"nmea_secondary","message":"Starting connection to device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002}}
{"timestamp":"2026-02-23T22:47:18.814929","level":"ERROR","source":"nmea_secondary","message":"Network error for device nmea_secondary","data":{"error":"[Errno 101] Network is unreachable","device_ip":"10.130.60.61","device_port":4002}}
{"timestamp":"2026-02-23T22:47:23.824661","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:47:23.823939+00:00","timestamp_unix":1771879643.823939}}}
{"timestamp":"2026-02-23T22:47:23.825526","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:47:23.823968+00:00","timestamp_unix":1771879643.823968}}}
{"timestamp":"2026-02-23T22:47:23.826240","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:47:23.823954+00:00","timestamp_unix":1771879643.823954}}}
{"timestamp":"2026-02-23T22:47:23.826556","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:47:23.823813+00:00","timestamp_unix":1771879643.823813}}}
{"timestamp":"2026-02-23T22:47:23.826728","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:47:23.823918+00:00","timestamp_unix":1771879643.823918}}}
{"timestamp":"2026-02-23T22:47:23.875512","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:47:23.827103+00:00","validation_timestamp_unix":1771879643.827103,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:23.823939+00:00","timestamp_unix":1771879643.823939},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:47:23.823968+00:00","timestamp_unix":1771879643.823968},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:23.823954+00:00","timestamp_unix":1771879643.823954},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:23.823813+00:00","timestamp_unix":1771879643.823813},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:23.823918+00:00","timestamp_unix":1771879643.823918}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:47:23.878079","level":"WARNING","source":"buzzer","message":"Alarm started: GPS jamming/spoofing detected (distance exceeded threshold)"}
{"timestamp":"2026-02-23T22:47:33.882690","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:47:33.887174","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:47:48.885387","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:47:48.889741","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:47:53.815054","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:47:53.813529+00:00","timestamp_unix":1771879673.813529}}}
{"timestamp":"2026-02-23T22:47:53.815319","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:47:53.813556+00:00","timestamp_unix":1771879673.813556}}}
{"timestamp":"2026-02-23T22:47:53.815443","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:47:53.813543+00:00","timestamp_unix":1771879673.813543}}}
{"timestamp":"2026-02-23T22:47:53.815600","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:47:53.813436+00:00","timestamp_unix":1771879673.813436}}}
{"timestamp":"2026-02-23T22:47:53.815698","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:47:53.813514+00:00","timestamp_unix":1771879673.813514}}}
{"timestamp":"2026-02-23T22:47:53.864526","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:47:53.816048+00:00","validation_timestamp_unix":1771879673.816048,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:53.813529+00:00","timestamp_unix":1771879673.813529},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:47:53.813556+00:00","timestamp_unix":1771879673.813556},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:53.813543+00:00","timestamp_unix":1771879673.813543},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:53.813436+00:00","timestamp_unix":1771879673.813436},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:47:53.813514+00:00","timestamp_unix":1771879673.813514}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:48:03.888362","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:48:03.892844","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:48:18.890968","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:48:18.895195","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:48:23.815763","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:48:23.815386+00:00","timestamp_unix":1771879703.815386}}}
{"timestamp":"2026-02-23T22:48:23.815927","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:48:23.815413+00:00","timestamp_unix":1771879703.815413}}}
{"timestamp":"2026-02-23T22:48:23.816016","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:48:23.815400+00:00","timestamp_unix":1771879703.8154}}}
{"timestamp":"2026-02-23T22:48:23.816099","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:48:23.815321+00:00","timestamp_unix":1771879703.815321}}}
{"timestamp":"2026-02-23T22:48:23.816177","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:48:23.815372+00:00","timestamp_unix":1771879703.815372}}}
{"timestamp":"2026-02-23T22:48:23.848734","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:48:23.816335+00:00","validation_timestamp_unix":1771879703.816335,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:23.815386+00:00","timestamp_unix":1771879703.815386},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:48:23.815413+00:00","timestamp_unix":1771879703.815413},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:23.815400+00:00","timestamp_unix":1771879703.8154},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:23.815321+00:00","timestamp_unix":1771879703.815321},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:23.815372+00:00","timestamp_unix":1771879703.815372}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:48:33.893803","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:48:33.898407","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:48:48.897743","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:48:48.901615","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:48:53.818367","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:48:53.817149+00:00","timestamp_unix":1771879733.817149}}}
{"timestamp":"2026-02-23T22:48:53.819643","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:48:53.817250+00:00","timestamp_unix":1771879733.81725}}}
{"timestamp":"2026-02-23T22:48:53.819892","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:48:53.817164+00:00","timestamp_unix":1771879733.817164}}}
{"timestamp":"2026-02-23T22:48:53.820055","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:48:53.817059+00:00","timestamp_unix":1771879733.817059}}}
{"timestamp":"2026-02-23T22:48:53.820190","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:48:53.817132+00:00","timestamp_unix":1771879733.817132}}}
{"timestamp":"2026-02-23T22:48:53.844895","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:48:53.820550+00:00","validation_timestamp_unix":1771879733.82055,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:53.817149+00:00","timestamp_unix":1771879733.817149},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:48:53.817250+00:00","timestamp_unix":1771879733.81725},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:53.817164+00:00","timestamp_unix":1771879733.817164},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:53.817059+00:00","timestamp_unix":1771879733.817059},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:48:53.817132+00:00","timestamp_unix":1771879733.817132}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:49:03.901559","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:49:03.905116","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:49:18.904130","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:49:18.908554","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:49:23.817871","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:49:23.817537+00:00","timestamp_unix":1771879763.817537}}}
{"timestamp":"2026-02-23T22:49:23.818151","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:49:23.817586+00:00","timestamp_unix":1771879763.817586}}}
{"timestamp":"2026-02-23T22:49:23.818320","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:49:23.817550+00:00","timestamp_unix":1771879763.81755}}}
{"timestamp":"2026-02-23T22:49:23.818428","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:49:23.817476+00:00","timestamp_unix":1771879763.817476}}}
{"timestamp":"2026-02-23T22:49:23.818517","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:49:23.817522+00:00","timestamp_unix":1771879763.817522}}}
{"timestamp":"2026-02-23T22:49:23.846511","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:49:23.818835+00:00","validation_timestamp_unix":1771879763.818835,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:23.817537+00:00","timestamp_unix":1771879763.817537},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:49:23.817586+00:00","timestamp_unix":1771879763.817586},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:23.817550+00:00","timestamp_unix":1771879763.81755},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:23.817476+00:00","timestamp_unix":1771879763.817476},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:23.817522+00:00","timestamp_unix":1771879763.817522}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:49:33.906784","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:49:33.911253","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:49:48.910414","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:49:48.915018","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:49:53.818928","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:49:53.818612+00:00","timestamp_unix":1771879793.818612}}}
{"timestamp":"2026-02-23T22:49:53.819100","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:49:53.818637+00:00","timestamp_unix":1771879793.818637}}}
{"timestamp":"2026-02-23T22:49:53.819194","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:49:53.818625+00:00","timestamp_unix":1771879793.818625}}}
{"timestamp":"2026-02-23T22:49:53.819281","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:49:53.818549+00:00","timestamp_unix":1771879793.818549}}}
{"timestamp":"2026-02-23T22:49:53.819363","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:49:53.818596+00:00","timestamp_unix":1771879793.818596}}}
{"timestamp":"2026-02-23T22:49:53.853132","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:49:53.819539+00:00","validation_timestamp_unix":1771879793.819539,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:53.818612+00:00","timestamp_unix":1771879793.818612},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:49:53.818637+00:00","timestamp_unix":1771879793.818637},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:53.818625+00:00","timestamp_unix":1771879793.818625},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:53.818549+00:00","timestamp_unix":1771879793.818549},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:49:53.818596+00:00","timestamp_unix":1771879793.818596}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:50:03.912836","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:50:03.917970","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:50:18.916354","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:50:18.920628","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:50:23.820204","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:50:23.819162+00:00","timestamp_unix":1771879823.819162}}}
{"timestamp":"2026-02-23T22:50:23.820443","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:50:23.819189+00:00","timestamp_unix":1771879823.819189}}}
{"timestamp":"2026-02-23T22:50:23.820584","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:50:23.819176+00:00","timestamp_unix":1771879823.819176}}}
{"timestamp":"2026-02-23T22:50:23.820687","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:50:23.819082+00:00","timestamp_unix":1771879823.819082}}}
{"timestamp":"2026-02-23T22:50:23.821320","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:50:23.819146+00:00","timestamp_unix":1771879823.819146}}}
{"timestamp":"2026-02-23T22:50:23.854158","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:50:23.822277+00:00","validation_timestamp_unix":1771879823.822277,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:23.819162+00:00","timestamp_unix":1771879823.819162},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:50:23.819189+00:00","timestamp_unix":1771879823.819189},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:23.819176+00:00","timestamp_unix":1771879823.819176},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:23.819082+00:00","timestamp_unix":1771879823.819082},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:23.819146+00:00","timestamp_unix":1771879823.819146}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:50:33.920836","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:50:33.924320","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:50:48.923070","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:50:48.927416","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:50:53.820337","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:50:53.819936+00:00","timestamp_unix":1771879853.819936}}}
{"timestamp":"2026-02-23T22:50:53.820578","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:50:53.819963+00:00","timestamp_unix":1771879853.819963}}}
{"timestamp":"2026-02-23T22:50:53.820695","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:50:53.819950+00:00","timestamp_unix":1771879853.81995}}}
{"timestamp":"2026-02-23T22:50:53.820789","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:50:53.819869+00:00","timestamp_unix":1771879853.819869}}}
{"timestamp":"2026-02-23T22:50:53.820876","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:50:53.819921+00:00","timestamp_unix":1771879853.819921}}}
{"timestamp":"2026-02-23T22:50:53.844653","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:50:53.821110+00:00","validation_timestamp_unix":1771879853.82111,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:53.819936+00:00","timestamp_unix":1771879853.819936},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:50:53.819963+00:00","timestamp_unix":1771879853.819963},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:53.819950+00:00","timestamp_unix":1771879853.81995},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:53.819869+00:00","timestamp_unix":1771879853.819869},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:50:53.819921+00:00","timestamp_unix":1771879853.819921}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:51:03.926307","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:51:03.931205","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:51:18.929675","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:51:18.933639","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:51:23.821611","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:51:23.821100+00:00","timestamp_unix":1771879883.8211}}}
{"timestamp":"2026-02-23T22:51:23.821810","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:51:23.821127+00:00","timestamp_unix":1771879883.821127}}}
{"timestamp":"2026-02-23T22:51:23.821995","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:51:23.821114+00:00","timestamp_unix":1771879883.821114}}}
{"timestamp":"2026-02-23T22:51:23.822107","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:51:23.821008+00:00","timestamp_unix":1771879883.821008}}}
{"timestamp":"2026-02-23T22:51:23.822201","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:51:23.821084+00:00","timestamp_unix":1771879883.821084}}}
{"timestamp":"2026-02-23T22:51:23.847655","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:51:23.822494+00:00","validation_timestamp_unix":1771879883.822494,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:23.821100+00:00","timestamp_unix":1771879883.8211},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:51:23.821127+00:00","timestamp_unix":1771879883.821127},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:23.821114+00:00","timestamp_unix":1771879883.821114},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:23.821008+00:00","timestamp_unix":1771879883.821008},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:23.821084+00:00","timestamp_unix":1771879883.821084}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:51:33.933258","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:51:33.936137","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:51:48.936125","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:51:48.940466","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:51:53.822790","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:51:53.822258+00:00","timestamp_unix":1771879913.822258}}}
{"timestamp":"2026-02-23T22:51:53.822991","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:51:53.822285+00:00","timestamp_unix":1771879913.822285}}}
{"timestamp":"2026-02-23T22:51:53.823106","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:51:53.822272+00:00","timestamp_unix":1771879913.822272}}}
{"timestamp":"2026-02-23T22:51:53.823205","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:51:53.822177+00:00","timestamp_unix":1771879913.822177}}}
{"timestamp":"2026-02-23T22:51:53.823339","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:51:53.822242+00:00","timestamp_unix":1771879913.822242}}}
{"timestamp":"2026-02-23T22:51:53.850451","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:51:53.823625+00:00","validation_timestamp_unix":1771879913.823625,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:53.822258+00:00","timestamp_unix":1771879913.822258},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:51:53.822285+00:00","timestamp_unix":1771879913.822285},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:53.822272+00:00","timestamp_unix":1771879913.822272},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:53.822177+00:00","timestamp_unix":1771879913.822177},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:51:53.822242+00:00","timestamp_unix":1771879913.822242}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:52:03.940344","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:52:03.943838","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:52:18.943750","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:52:18.948782","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:52:23.825508","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:52:23.824981+00:00","timestamp_unix":1771879943.824981}}}
{"timestamp":"2026-02-23T22:52:23.825749","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:52:23.825010+00:00","timestamp_unix":1771879943.82501}}}
{"timestamp":"2026-02-23T22:52:23.825856","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:52:23.824996+00:00","timestamp_unix":1771879943.824996}}}
{"timestamp":"2026-02-23T22:52:23.826033","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:52:23.824888+00:00","timestamp_unix":1771879943.824888}}}
{"timestamp":"2026-02-23T22:52:23.826148","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:52:23.824964+00:00","timestamp_unix":1771879943.824964}}}
{"timestamp":"2026-02-23T22:52:23.860023","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:52:23.826462+00:00","validation_timestamp_unix":1771879943.826462,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:23.824981+00:00","timestamp_unix":1771879943.824981},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:52:23.825010+00:00","timestamp_unix":1771879943.82501},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:23.824996+00:00","timestamp_unix":1771879943.824996},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:23.824888+00:00","timestamp_unix":1771879943.824888},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:23.824964+00:00","timestamp_unix":1771879943.824964}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:52:33.946170","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:52:33.951853","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:52:48.950165","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:52:48.954796","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:52:53.826384","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:52:53.825839+00:00","timestamp_unix":1771879973.825839}}}
{"timestamp":"2026-02-23T22:52:53.826549","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:52:53.825867+00:00","timestamp_unix":1771879973.825867}}}
{"timestamp":"2026-02-23T22:52:53.826648","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:52:53.825853+00:00","timestamp_unix":1771879973.825853}}}
{"timestamp":"2026-02-23T22:52:53.826738","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:52:53.825765+00:00","timestamp_unix":1771879973.825765}}}
{"timestamp":"2026-02-23T22:52:53.826821","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:52:53.825821+00:00","timestamp_unix":1771879973.825821}}}
{"timestamp":"2026-02-23T22:52:53.856580","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:52:53.826994+00:00","validation_timestamp_unix":1771879973.826994,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:53.825839+00:00","timestamp_unix":1771879973.825839},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:52:53.825867+00:00","timestamp_unix":1771879973.825867},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:53.825853+00:00","timestamp_unix":1771879973.825853},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:53.825765+00:00","timestamp_unix":1771879973.825765},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:52:53.825821+00:00","timestamp_unix":1771879973.825821}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:53:03.953648","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:53:03.957135","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:53:18.956256","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:53:18.963561","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:53:23.826822","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:53:23.826229+00:00","timestamp_unix":1771880003.826229}}}
{"timestamp":"2026-02-23T22:53:23.827172","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:53:23.826257+00:00","timestamp_unix":1771880003.826257}}}
{"timestamp":"2026-02-23T22:53:23.827319","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:53:23.826243+00:00","timestamp_unix":1771880003.826243}}}
{"timestamp":"2026-02-23T22:53:23.827460","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:53:23.826151+00:00","timestamp_unix":1771880003.826151}}}
{"timestamp":"2026-02-23T22:53:23.827570","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:53:23.826212+00:00","timestamp_unix":1771880003.826212}}}
{"timestamp":"2026-02-23T22:53:23.864544","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:53:23.827891+00:00","validation_timestamp_unix":1771880003.827891,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:23.826229+00:00","timestamp_unix":1771880003.826229},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:53:23.826257+00:00","timestamp_unix":1771880003.826257},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:23.826243+00:00","timestamp_unix":1771880003.826243},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:23.826151+00:00","timestamp_unix":1771880003.826151},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:23.826212+00:00","timestamp_unix":1771880003.826212}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:53:33.958745","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:53:33.966759","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:53:48.962622","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:53:48.969076","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:53:53.827130","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:53:53.826776+00:00","timestamp_unix":1771880033.826776}}}
{"timestamp":"2026-02-23T22:53:53.827303","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:53:53.826802+00:00","timestamp_unix":1771880033.826802}}}
{"timestamp":"2026-02-23T22:53:53.827397","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:53:53.826790+00:00","timestamp_unix":1771880033.82679}}}
{"timestamp":"2026-02-23T22:53:53.827484","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:53:53.826705+00:00","timestamp_unix":1771880033.826705}}}
{"timestamp":"2026-02-23T22:53:53.827567","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:53:53.826762+00:00","timestamp_unix":1771880033.826762}}}
{"timestamp":"2026-02-23T22:53:53.856027","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:53:53.827740+00:00","validation_timestamp_unix":1771880033.82774,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:53.826776+00:00","timestamp_unix":1771880033.826776},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:53:53.826802+00:00","timestamp_unix":1771880033.826802},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:53.826790+00:00","timestamp_unix":1771880033.82679},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:53.826705+00:00","timestamp_unix":1771880033.826705},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:53:53.826762+00:00","timestamp_unix":1771880033.826762}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:54:03.965987","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:54:03.971379","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:54:18.969350","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:54:18.973833","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:54:23.828331","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:54:23.827104+00:00","timestamp_unix":1771880063.827104}}}
{"timestamp":"2026-02-23T22:54:23.828631","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:54:23.827133+00:00","timestamp_unix":1771880063.827133}}}
{"timestamp":"2026-02-23T22:54:23.828814","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:54:23.827119+00:00","timestamp_unix":1771880063.827119}}}
{"timestamp":"2026-02-23T22:54:23.828941","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:54:23.827012+00:00","timestamp_unix":1771880063.827012}}}
{"timestamp":"2026-02-23T22:54:23.829057","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:54:23.827065+00:00","timestamp_unix":1771880063.827065}}}
{"timestamp":"2026-02-23T22:54:23.856635","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:54:23.829409+00:00","validation_timestamp_unix":1771880063.829409,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:23.827104+00:00","timestamp_unix":1771880063.827104},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:54:23.827133+00:00","timestamp_unix":1771880063.827133},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:23.827119+00:00","timestamp_unix":1771880063.827119},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:23.827012+00:00","timestamp_unix":1771880063.827012},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:23.827065+00:00","timestamp_unix":1771880063.827065}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:54:33.972774","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:54:33.977129","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:54:48.976540","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:54:48.979882","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:54:53.829267","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:54:53.828852+00:00","timestamp_unix":1771880093.828852}}}
{"timestamp":"2026-02-23T22:54:53.829465","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:54:53.828879+00:00","timestamp_unix":1771880093.828879}}}
{"timestamp":"2026-02-23T22:54:53.829608","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:54:53.828866+00:00","timestamp_unix":1771880093.828866}}}
{"timestamp":"2026-02-23T22:54:53.830195","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:54:53.828783+00:00","timestamp_unix":1771880093.828783}}}
{"timestamp":"2026-02-23T22:54:53.830602","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:54:53.828836+00:00","timestamp_unix":1771880093.828836}}}
{"timestamp":"2026-02-23T22:54:53.855158","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:54:53.831265+00:00","validation_timestamp_unix":1771880093.831265,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:53.828852+00:00","timestamp_unix":1771880093.828852},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:54:53.828879+00:00","timestamp_unix":1771880093.828879},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:53.828866+00:00","timestamp_unix":1771880093.828866},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:53.828783+00:00","timestamp_unix":1771880093.828783},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:54:53.828836+00:00","timestamp_unix":1771880093.828836}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:55:03.979193","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:55:03.981318","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:55:18.982800","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:55:18.983832","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:55:23.830806","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:55:23.830222+00:00","timestamp_unix":1771880123.830222}}}
{"timestamp":"2026-02-23T22:55:23.831062","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:55:23.830252+00:00","timestamp_unix":1771880123.830252}}}
{"timestamp":"2026-02-23T22:55:23.831190","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:55:23.830237+00:00","timestamp_unix":1771880123.830237}}}
{"timestamp":"2026-02-23T22:55:23.831300","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:55:23.830133+00:00","timestamp_unix":1771880123.830133}}}
{"timestamp":"2026-02-23T22:55:23.831433","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:55:23.830206+00:00","timestamp_unix":1771880123.830206}}}
{"timestamp":"2026-02-23T22:55:23.861645","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:55:23.831756+00:00","validation_timestamp_unix":1771880123.831756,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:23.830222+00:00","timestamp_unix":1771880123.830222},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:55:23.830252+00:00","timestamp_unix":1771880123.830252},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:23.830237+00:00","timestamp_unix":1771880123.830237},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:23.830133+00:00","timestamp_unix":1771880123.830133},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:23.830206+00:00","timestamp_unix":1771880123.830206}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:55:33.987349","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:55:33.988409","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:55:48.991360","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:55:48.992540","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:55:53.831586","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:55:53.831061+00:00","timestamp_unix":1771880153.831061}}}
{"timestamp":"2026-02-23T22:55:53.831837","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:55:53.831092+00:00","timestamp_unix":1771880153.831092}}}
{"timestamp":"2026-02-23T22:55:53.831958","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:55:53.831076+00:00","timestamp_unix":1771880153.831076}}}
{"timestamp":"2026-02-23T22:55:53.832062","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:55:53.830992+00:00","timestamp_unix":1771880153.830992}}}
{"timestamp":"2026-02-23T22:55:53.832152","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:55:53.831045+00:00","timestamp_unix":1771880153.831045}}}
{"timestamp":"2026-02-23T22:55:53.867050","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:55:53.832504+00:00","validation_timestamp_unix":1771880153.832504,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:53.831061+00:00","timestamp_unix":1771880153.831061},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:55:53.831092+00:00","timestamp_unix":1771880153.831092},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:53.831076+00:00","timestamp_unix":1771880153.831076},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:53.830992+00:00","timestamp_unix":1771880153.830992},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:55:53.831045+00:00","timestamp_unix":1771880153.831045}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:56:03.995636","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:56:03.996791","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:56:18.999583","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:56:19.001025","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:56:23.832144","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:56:23.831294+00:00","timestamp_unix":1771880183.831294}}}
{"timestamp":"2026-02-23T22:56:23.832506","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:56:23.831335+00:00","timestamp_unix":1771880183.831335}}}
{"timestamp":"2026-02-23T22:56:23.832765","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:56:23.831315+00:00","timestamp_unix":1771880183.831315}}}
{"timestamp":"2026-02-23T22:56:23.834116","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:56:23.831204+00:00","timestamp_unix":1771880183.831204}}}
{"timestamp":"2026-02-23T22:56:23.834952","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:56:23.831271+00:00","timestamp_unix":1771880183.831271}}}
{"timestamp":"2026-02-23T22:56:23.867265","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:56:23.836169+00:00","validation_timestamp_unix":1771880183.836169,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:23.831294+00:00","timestamp_unix":1771880183.831294},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:56:23.831335+00:00","timestamp_unix":1771880183.831335},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:23.831315+00:00","timestamp_unix":1771880183.831315},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:23.831204+00:00","timestamp_unix":1771880183.831204},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:23.831271+00:00","timestamp_unix":1771880183.831271}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:56:34.002920","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:56:34.004207","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:56:49.005381","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:56:49.006432","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:56:53.832575","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:56:53.831992+00:00","timestamp_unix":1771880213.831992}}}
{"timestamp":"2026-02-23T22:56:53.832800","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:56:53.832021+00:00","timestamp_unix":1771880213.832021}}}
{"timestamp":"2026-02-23T22:56:53.832936","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:56:53.832007+00:00","timestamp_unix":1771880213.832007}}}
{"timestamp":"2026-02-23T22:56:53.833051","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:56:53.831905+00:00","timestamp_unix":1771880213.831905}}}
{"timestamp":"2026-02-23T22:56:53.833144","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:56:53.831975+00:00","timestamp_unix":1771880213.831975}}}
{"timestamp":"2026-02-23T22:56:53.859002","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:56:53.833453+00:00","validation_timestamp_unix":1771880213.833453,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:53.831992+00:00","timestamp_unix":1771880213.831992},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:56:53.832021+00:00","timestamp_unix":1771880213.832021},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:53.832007+00:00","timestamp_unix":1771880213.832007},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:53.831905+00:00","timestamp_unix":1771880213.831905},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:56:53.831975+00:00","timestamp_unix":1771880213.831975}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:57:04.008483","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:57:04.009633","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:57:19.011357","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:57:19.011972","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:57:23.835296","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:57:23.834427+00:00","timestamp_unix":1771880243.834427}}}
{"timestamp":"2026-02-23T22:57:23.835685","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:57:23.834459+00:00","timestamp_unix":1771880243.834459}}}
{"timestamp":"2026-02-23T22:57:23.835962","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:57:23.834444+00:00","timestamp_unix":1771880243.834444}}}
{"timestamp":"2026-02-23T22:57:23.836175","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:57:23.833555+00:00","timestamp_unix":1771880243.833555}}}
{"timestamp":"2026-02-23T22:57:23.837601","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:57:23.834383+00:00","timestamp_unix":1771880243.834383}}}
{"timestamp":"2026-02-23T22:57:23.869028","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:57:23.838126+00:00","validation_timestamp_unix":1771880243.838126,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:23.834427+00:00","timestamp_unix":1771880243.834427},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:57:23.834459+00:00","timestamp_unix":1771880243.834459},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:23.834444+00:00","timestamp_unix":1771880243.834444},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:23.833555+00:00","timestamp_unix":1771880243.833555},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:23.834383+00:00","timestamp_unix":1771880243.834383}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:57:34.015167","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:57:34.016507","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:57:49.018784","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:57:49.019705","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:57:53.834172","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:57:53.833609+00:00","timestamp_unix":1771880273.833609}}}
{"timestamp":"2026-02-23T22:57:53.834379","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:57:53.833635+00:00","timestamp_unix":1771880273.833635}}}
{"timestamp":"2026-02-23T22:57:53.834480","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:57:53.833622+00:00","timestamp_unix":1771880273.833622}}}
{"timestamp":"2026-02-23T22:57:53.834573","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:57:53.833540+00:00","timestamp_unix":1771880273.83354}}}
{"timestamp":"2026-02-23T22:57:53.834657","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:57:53.833593+00:00","timestamp_unix":1771880273.833593}}}
{"timestamp":"2026-02-23T22:57:53.867179","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:57:53.834907+00:00","validation_timestamp_unix":1771880273.834907,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:53.833609+00:00","timestamp_unix":1771880273.833609},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:57:53.833635+00:00","timestamp_unix":1771880273.833635},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:53.833622+00:00","timestamp_unix":1771880273.833622},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:53.833540+00:00","timestamp_unix":1771880273.83354},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:57:53.833593+00:00","timestamp_unix":1771880273.833593}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:58:04.021584","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:58:04.022381","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:58:19.023519","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:58:19.024348","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:58:23.836982","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:58:23.836337+00:00","timestamp_unix":1771880303.836337}}}
{"timestamp":"2026-02-23T22:58:23.837235","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:58:23.836365+00:00","timestamp_unix":1771880303.836365}}}
{"timestamp":"2026-02-23T22:58:23.837375","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:58:23.836351+00:00","timestamp_unix":1771880303.836351}}}
{"timestamp":"2026-02-23T22:58:23.837490","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:58:23.836250+00:00","timestamp_unix":1771880303.83625}}}
{"timestamp":"2026-02-23T22:58:23.837591","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:58:23.836321+00:00","timestamp_unix":1771880303.836321}}}
{"timestamp":"2026-02-23T22:58:23.862153","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:58:23.838005+00:00","validation_timestamp_unix":1771880303.838005,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:23.836337+00:00","timestamp_unix":1771880303.836337},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:58:23.836365+00:00","timestamp_unix":1771880303.836365},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:23.836351+00:00","timestamp_unix":1771880303.836351},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:23.836250+00:00","timestamp_unix":1771880303.83625},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:23.836321+00:00","timestamp_unix":1771880303.836321}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:58:34.026961","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:58:34.028162","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:58:49.030075","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:58:49.030863","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:58:53.836975","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:58:53.836644+00:00","timestamp_unix":1771880333.836644}}}
{"timestamp":"2026-02-23T22:58:53.837138","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:58:53.836671+00:00","timestamp_unix":1771880333.836671}}}
{"timestamp":"2026-02-23T22:58:53.837264","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:58:53.836658+00:00","timestamp_unix":1771880333.836658}}}
{"timestamp":"2026-02-23T22:58:53.837348","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:58:53.836585+00:00","timestamp_unix":1771880333.836585}}}
{"timestamp":"2026-02-23T22:58:53.837425","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:58:53.836629+00:00","timestamp_unix":1771880333.836629}}}
{"timestamp":"2026-02-23T22:58:53.864992","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:58:53.837598+00:00","validation_timestamp_unix":1771880333.837598,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:53.836644+00:00","timestamp_unix":1771880333.836644},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:58:53.836671+00:00","timestamp_unix":1771880333.836671},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:53.836658+00:00","timestamp_unix":1771880333.836658},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:53.836585+00:00","timestamp_unix":1771880333.836585},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:58:53.836629+00:00","timestamp_unix":1771880333.836629}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:59:04.032734","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:59:04.033610","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:59:19.035139","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:59:19.036080","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:59:23.838471","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:59:23.837657+00:00","timestamp_unix":1771880363.837657}}}
{"timestamp":"2026-02-23T22:59:23.838766","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:59:23.837684+00:00","timestamp_unix":1771880363.837684}}}
{"timestamp":"2026-02-23T22:59:23.838943","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:59:23.837671+00:00","timestamp_unix":1771880363.837671}}}
{"timestamp":"2026-02-23T22:59:23.839045","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:59:23.837597+00:00","timestamp_unix":1771880363.837597}}}
{"timestamp":"2026-02-23T22:59:23.839131","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:59:23.837642+00:00","timestamp_unix":1771880363.837642}}}
{"timestamp":"2026-02-23T22:59:23.868261","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:59:23.839429+00:00","validation_timestamp_unix":1771880363.839429,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:23.837657+00:00","timestamp_unix":1771880363.837657},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:59:23.837684+00:00","timestamp_unix":1771880363.837684},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:23.837671+00:00","timestamp_unix":1771880363.837671},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:23.837597+00:00","timestamp_unix":1771880363.837597},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:23.837642+00:00","timestamp_unix":1771880363.837642}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T22:59:34.039609","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:59:34.041450","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:59:49.042409","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T22:59:49.043482","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T22:59:53.840376","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T20:59:53.839026+00:00","timestamp_unix":1771880393.839026}}}
{"timestamp":"2026-02-23T22:59:53.840666","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T20:59:53.839057+00:00","timestamp_unix":1771880393.839057}}}
{"timestamp":"2026-02-23T22:59:53.840818","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T20:59:53.839043+00:00","timestamp_unix":1771880393.839043}}}
{"timestamp":"2026-02-23T22:59:53.840931","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T20:59:53.838921+00:00","timestamp_unix":1771880393.838921}}}
{"timestamp":"2026-02-23T22:59:53.841041","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T20:59:53.839008+00:00","timestamp_unix":1771880393.839008}}}
{"timestamp":"2026-02-23T22:59:53.865632","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T20:59:53.841310+00:00","validation_timestamp_unix":1771880393.84131,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:53.839026+00:00","timestamp_unix":1771880393.839026},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T20:59:53.839057+00:00","timestamp_unix":1771880393.839057},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:53.839043+00:00","timestamp_unix":1771880393.839043},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:53.838921+00:00","timestamp_unix":1771880393.838921},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T20:59:53.839008+00:00","timestamp_unix":1771880393.839008}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:00:04.044800","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:00:04.045736","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:00:19.048243","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:00:19.049200","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:00:23.839341","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:00:23.839029+00:00","timestamp_unix":1771880423.839029}}}
{"timestamp":"2026-02-23T23:00:23.839539","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:00:23.839057+00:00","timestamp_unix":1771880423.839057}}}
{"timestamp":"2026-02-23T23:00:23.839631","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:00:23.839043+00:00","timestamp_unix":1771880423.839043}}}
{"timestamp":"2026-02-23T23:00:23.839714","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:00:23.838969+00:00","timestamp_unix":1771880423.838969}}}
{"timestamp":"2026-02-23T23:00:23.839792","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:00:23.839014+00:00","timestamp_unix":1771880423.839014}}}
{"timestamp":"2026-02-23T23:00:23.868096","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:00:23.839955+00:00","validation_timestamp_unix":1771880423.839955,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:23.839029+00:00","timestamp_unix":1771880423.839029},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:00:23.839057+00:00","timestamp_unix":1771880423.839057},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:23.839043+00:00","timestamp_unix":1771880423.839043},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:23.838969+00:00","timestamp_unix":1771880423.838969},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:23.839014+00:00","timestamp_unix":1771880423.839014}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:00:34.051595","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:00:34.052676","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:00:49.054886","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:00:49.055525","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:00:53.841108","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:00:53.840773+00:00","timestamp_unix":1771880453.840773}}}
{"timestamp":"2026-02-23T23:00:53.841252","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:00:53.840801+00:00","timestamp_unix":1771880453.840801}}}
{"timestamp":"2026-02-23T23:00:53.841347","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:00:53.840787+00:00","timestamp_unix":1771880453.840787}}}
{"timestamp":"2026-02-23T23:00:53.841435","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:00:53.840707+00:00","timestamp_unix":1771880453.840707}}}
{"timestamp":"2026-02-23T23:00:53.841517","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:00:53.840757+00:00","timestamp_unix":1771880453.840757}}}
{"timestamp":"2026-02-23T23:00:53.867713","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:00:53.841688+00:00","validation_timestamp_unix":1771880453.841688,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:53.840773+00:00","timestamp_unix":1771880453.840773},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:00:53.840801+00:00","timestamp_unix":1771880453.840801},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:53.840787+00:00","timestamp_unix":1771880453.840787},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:53.840707+00:00","timestamp_unix":1771880453.840707},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:00:53.840757+00:00","timestamp_unix":1771880453.840757}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:01:04.057033","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:01:04.058010","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:01:19.060117","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:01:19.060840","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:01:23.841428","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:01:23.841124+00:00","timestamp_unix":1771880483.841124}}}
{"timestamp":"2026-02-23T23:01:23.841581","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:01:23.841150+00:00","timestamp_unix":1771880483.84115}}}
{"timestamp":"2026-02-23T23:01:23.841676","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:01:23.841137+00:00","timestamp_unix":1771880483.841137}}}
{"timestamp":"2026-02-23T23:01:23.841764","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:01:23.841066+00:00","timestamp_unix":1771880483.841066}}}
{"timestamp":"2026-02-23T23:01:23.841933","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:01:23.841109+00:00","timestamp_unix":1771880483.841109}}}
{"timestamp":"2026-02-23T23:01:23.873224","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:01:23.842163+00:00","validation_timestamp_unix":1771880483.842163,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:23.841124+00:00","timestamp_unix":1771880483.841124},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:01:23.841150+00:00","timestamp_unix":1771880483.84115},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:23.841137+00:00","timestamp_unix":1771880483.841137},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:23.841066+00:00","timestamp_unix":1771880483.841066},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:23.841109+00:00","timestamp_unix":1771880483.841109}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:01:34.063937","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:01:34.066277","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:01:49.072376","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:01:49.073165","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:01:53.842837","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:01:53.842430+00:00","timestamp_unix":1771880513.84243}}}
{"timestamp":"2026-02-23T23:01:53.842990","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:01:53.842457+00:00","timestamp_unix":1771880513.842457}}}
{"timestamp":"2026-02-23T23:01:53.843085","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:01:53.842443+00:00","timestamp_unix":1771880513.842443}}}
{"timestamp":"2026-02-23T23:01:53.843172","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:01:53.842365+00:00","timestamp_unix":1771880513.842365}}}
{"timestamp":"2026-02-23T23:01:53.843255","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:01:53.842415+00:00","timestamp_unix":1771880513.842415}}}
{"timestamp":"2026-02-23T23:01:53.870117","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:01:53.843422+00:00","validation_timestamp_unix":1771880513.843422,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:53.842430+00:00","timestamp_unix":1771880513.84243},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:01:53.842457+00:00","timestamp_unix":1771880513.842457},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:53.842443+00:00","timestamp_unix":1771880513.842443},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:53.842365+00:00","timestamp_unix":1771880513.842365},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:01:53.842415+00:00","timestamp_unix":1771880513.842415}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:02:04.075979","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:02:04.078357","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:02:19.079392","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:02:19.082548","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:02:23.843784","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:02:23.842952+00:00","timestamp_unix":1771880543.842952}}}
{"timestamp":"2026-02-23T23:02:23.844125","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:02:23.842985+00:00","timestamp_unix":1771880543.842985}}}
{"timestamp":"2026-02-23T23:02:23.844316","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:02:23.842969+00:00","timestamp_unix":1771880543.842969}}}
{"timestamp":"2026-02-23T23:02:23.844434","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:02:23.842839+00:00","timestamp_unix":1771880543.842839}}}
{"timestamp":"2026-02-23T23:02:23.844528","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:02:23.842933+00:00","timestamp_unix":1771880543.842933}}}
{"timestamp":"2026-02-23T23:02:23.878562","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:02:23.846489+00:00","validation_timestamp_unix":1771880543.846489,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:23.842952+00:00","timestamp_unix":1771880543.842952},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:02:23.842985+00:00","timestamp_unix":1771880543.842985},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:23.842969+00:00","timestamp_unix":1771880543.842969},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:23.842839+00:00","timestamp_unix":1771880543.842839},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:23.842933+00:00","timestamp_unix":1771880543.842933}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:02:34.082991","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:02:34.085161","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:02:49.085858","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:02:49.087763","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}
{"timestamp":"2026-02-23T23:02:53.844396","level":"INFO","source":"tm_ais","message":"Injected position","data":{"position":{"latitude":36.110657,"longitude":22.572672,"//timestamp_unix":1732461600.0,"source":"tm_ais","timestamp":"2026-02-23T21:02:53.843820+00:00","timestamp_unix":1771880573.84382}}}
{"timestamp":"2026-02-23T23:02:53.844646","level":"INFO","source":"starlink_location","message":"Injected position","data":{"position":{"latitude":36.11055187009735,"longitude":22.57289484169309,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"source":"starlink_location","timestamp":"2026-02-23T21:02:53.843848+00:00","timestamp_unix":1771880573.843848}}}
{"timestamp":"2026-02-23T23:02:53.844813","level":"INFO","source":"starlink_gps","message":"Injected position","data":{"position":{"latitude":36.11055287599966,"longitude":22.57289200819445,"//timestamp_unix":1732461600.0,"altitude":54.29000515150101,"source":"starlink_gps","timestamp":"2026-02-23T21:02:53.843834+00:00","timestamp_unix":1771880573.843834}}}
{"timestamp":"2026-02-23T23:02:53.844998","level":"INFO","source":"nmea_primary","message":"Injected position","data":{"position":{"latitude":36.11063,"longitude":22.972875,"//timestamp_unix":1768308542.0,"altitude":14.0,"source":"nmea_primary","timestamp":"2026-02-23T21:02:53.843722+00:00","timestamp_unix":1771880573.843722}}}
{"timestamp":"2026-02-23T23:02:53.845118","level":"INFO","source":"nmea_secondary","message":"Injected position","data":{"position":{"latitude":36.11085833333333,"longitude":22.572023333333334,"//timestamp_unix":1732461600.0,"altitude":13.2,"source":"nmea_secondary","timestamp":"2026-02-23T21:02:53.843800+00:00","timestamp_unix":1771880573.8438}}}
{"timestamp":"2026-02-23T23:02:53.885574","level":"WARNING","source":"validation","message":"Validation failed","data":{"validation":{"validation_timestamp":"2026-02-23T21:02:53.846107+00:00","validation_timestamp_unix":1771880573.846107,"is_valid":false,"sources_missing":[],"sources_stale":[],"coordinate_differences":{"tm_ais_starlink_location":{"distance_meters":23.1816563266951,"source1":"tm_ais","source2":"starlink_location"},"tm_ais_starlink_gps":{"distance_meters":22.905468559001534,"source1":"tm_ais","source2":"starlink_gps"},"tm_ais_nmea_primary":{"distance_meters":35951.09222409758,"source1":"tm_ais","source2":"nmea_primary"},"tm_ais_nmea_secondary":{"distance_meters":62.42360552774925,"source1":"tm_ais","source2":"nmea_secondary"},"starlink_location_starlink_gps":{"distance_meters":0.27803087725616366,"source1":"starlink_location","source2":"starlink_gps"},"starlink_location_nmea_primary":{"distance_meters":35931.09887839561,"source1":"starlink_location","source2":"nmea_primary"},"starlink_location_nmea_secondary":{"distance_meters":85.38437464525116,"source1":"starlink_location","source2":"nmea_secondary"},"starlink_gps_nmea_primary":{"distance_meters":35931.353160486055,"source1":"starlink_gps","source2":"nmea_primary"},"starlink_gps_nmea_secondary":{"distance_meters":85.10634545958962,"source1":"starlink_gps","source2":"nmea_secondary"},"nmea_primary_nmea_secondary":{"distance_meters":36009.32592871677,"source1":"nmea_primary","source2":"nmea_secondary"}},"source_coordinates":{"tm_ais":{"latitude":36.110657,"longitude":22.572672,"altitude":null,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:53.843820+00:00","timestamp_unix":1771880573.84382},"starlink_location":{"latitude":36.11055187009735,"longitude":22.57289484169309,"altitude":54.29000515150101,"position_uncertainty_m":2.5,"timestamp":"2026-02-23T21:02:53.843848+00:00","timestamp_unix":1771880573.843848},"starlink_gps":{"latitude":36.11055287599966,"longitude":22.57289200819445,"altitude":54.29000515150101,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:53.843834+00:00","timestamp_unix":1771880573.843834},"nmea_primary":{"latitude":36.11063,"longitude":22.972875,"altitude":14.0,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:53.843722+00:00","timestamp_unix":1771880573.843722},"nmea_secondary":{"latitude":36.11085833333333,"longitude":22.572023333333334,"altitude":13.2,"position_uncertainty_m":null,"timestamp":"2026-02-23T21:02:53.843800+00:00","timestamp_unix":1771880573.8438}},"validation_details":{"threshold_meters":200.0,"stale_threshold_seconds":60,"expected_sources":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_found":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_with_coordinates":["tm_ais","starlink_location","starlink_gps","nmea_primary","nmea_secondary"],"sources_null_island":[],"max_distance_meters":36009.32592871677,"position_uncertainties":{"starlink_location":2.5}}}}}
{"timestamp":"2026-02-23T23:03:04.088684","level":"ERROR","source":"nmea_primary","message":"Connection timeout for device nmea_primary","data":{"device_ip":"10.130.60.61","device_port":4001,"timeout":10}}
{"timestamp":"2026-02-23T23:03:04.090180","level":"ERROR","source":"nmea_secondary","message":"Connection timeout for device nmea_secondary","data":{"device_ip":"10.130.60.61","device_port":4002,"timeout":10}}

12
emmc-provisioning/README.md
View File

@@ -13,9 +13,12 @@ Revisions are tracked project-wide; see repo root **README.md** and `scripts/bum
emmc-provisioning/
├── README.md ← You are here
├── docs/ Documentation
│ ├── DEPLOY-NEW-PROXMOX.md Step-by-step: deploy to a new Proxmox instance
│ ├── EMMC-PROVISIONING-GUIDE.md Full setup and usage
│ ├── NETWORK-BOOT-LXC.md Network boot (PXE/dnsmasq) and LXC
│ ├── PROXMOX-LXC-DEPLOYMENT.md Proxmox LXC + host setup
│ ├── DEVICE-DNS-DHCP-RESOLVCONF.md Device DNS from DHCP, resolv.conf, cloud-init
│ ├── DNSMASQ-DNS-FILESERVER.md dnsmasq DNS and file.server on LXC
│ ├── PROXMOX-LXC-DEPLOYMENT.md Proxmox LXC + host setup (reference)
│ └── PORTAL_STYLING_GUIDE.md Dashboard UI styling reference
├── host/ Scripts that run on the provisioning host (Proxmox host)
│ ├── flash-emmc-on-connect.sh rpiboot + wait for Backup/Deploy choice, then dd
@@ -53,6 +56,7 @@ emmc-provisioning/
## Quick start
1. **Read** [docs/EMMC-PROVISIONING-GUIDE.md](docs/EMMC-PROVISIONING-GUIDE.md) for setup and usage.
2. **Proxmox:** Use [scripts/deploy-to-proxmox.sh](scripts/deploy-to-proxmox.sh) to deploy to a Proxmox host; see [docs/PROXMOX-LXC-DEPLOYMENT.md](docs/PROXMOX-LXC-DEPLOYMENT.md).
3. **Manual host:** Copy scripts from `host/` to the host and install the udev rule (see the guide).
4. Put **golden.img** in `/var/lib/cm4-provisioning/` (or your configured path). When a device is detected (USB or network), the **dashboard** asks **Backup** or **Deploy**.
2. **Deploy to a new Proxmox:** Follow [docs/DEPLOY-NEW-PROXMOX.md](docs/DEPLOY-NEW-PROXMOX.md) for clear step-by-step instructions.
3. **Proxmox reference:** [scripts/deploy-to-proxmox.sh](scripts/deploy-to-proxmox.sh) and [docs/PROXMOX-LXC-DEPLOYMENT.md](docs/PROXMOX-LXC-DEPLOYMENT.md) for options, layout, and troubleshooting.
4. **Manual host:** Copy scripts from `host/` to the host and install the udev rule (see the guide).
5. Put **golden.img** in `/var/lib/cm4-provisioning/` (or your configured path). When a device is detected (USB or network), the **dashboard** asks **Backup** or **Deploy**.

125
emmc-provisioning/cloud-init/bootstrap.sh
View File

@@ -1,4 +1,129 @@
#!/bin/bash
# Minimal bootstrap script for cloud-init first boot (test).
set -e
# Ensure hostname resolves (avoids "sudo: unable to resolve host")
H="$(hostname)"
grep -q "127.0.1.1.*$H" /etc/hosts || echo "127.0.1.1 $H" >> /etc/hosts
# Do not overwrite /etc/resolv.conf: use DNS from DHCP so file.server and LXC DNS work.
# --- Chromium kiosk autostart (same behaviour as gnss-guard start-chromium.sh) ---
PI_USER="${PI_USER:-pi}"
SCRIPT_DEST="/usr/local/bin/start-chromium.sh"
AUTOSTART_SYSTEM="/etc/xdg/autostart"
PI_HOME="/home/$PI_USER"
# Icon: download start-here.png from file server, or set DESKTOP_ICON to override
FILE_SERVER="${FILE_SERVER:-http://file.server:5000/files/first-boot}"
ICON_DEST="/usr/share/pixmaps/tm.png"
DESKTOP_ICON="${DESKTOP_ICON:-chromium-browser}"
if [ "$DESKTOP_ICON" = "chromium-browser" ]; then
mkdir -p /usr/share/pixmaps
icon_url="${FILE_SERVER}/start-here.png"
if ! curl -fsSL "$icon_url" -o "$ICON_DEST" 2>/dev/null; then
# Fallback: use gateway IP (LXC on provisioning LAN) when DNS not ready yet at first boot
gw="$(ip -4 route show default 2>/dev/null | awk '{print $3; exit}')"
if [ -n "$gw" ]; then
curl -fsSL "http://${gw}:5000/files/first-boot/start-here.png" -o "$ICON_DEST" 2>/dev/null || true
fi
fi
if [ -s "$ICON_DEST" ]; then
chmod 644 "$ICON_DEST"
DESKTOP_ICON="$ICON_DEST"
fi
fi
# Install start-chromium.sh system-wide so it works regardless of user home
cat << 'START_CHROMIUM_EOF' > "$SCRIPT_DEST"
#!/bin/bash
# Disable keyring prompts
export GNOME_KEYRING_CONTROL=""
export DISPLAY=:0
# Force X11 instead of Wayland for better fullscreen support
export GDK_BACKEND=x11
unset WAYLAND_DISPLAY
# Wait for display and desktop environment to be ready
for i in {1..60}; do
if xset q >/dev/null 2>&1 || [ -n "$DISPLAY" ]; then
if pgrep -x pcmanfm >/dev/null 2>&1 || pgrep -x lxsession >/dev/null 2>&1 || pgrep -x xfdesktop >/dev/null 2>&1; then
break
fi
fi
sleep 0.5
done
sleep 5
/usr/bin/chromium --start-fullscreen --noerrdialogs --disable-infobars --disable-session-crashed-bubble --disable-restore-session-state --no-first-run --password-store=basic --use-mock-keychain --ozone-platform=x11 --disable-features=UseChromeOSDirectVideoDecoder --app=http://127.0.0.1:8080 &
sleep 3
for i in {1..10}; do
WINDOW_ID=$(wmctrl -l 2>/dev/null | grep -i chromium | head -1 | awk '{print $1}')
if [ -n "$WINDOW_ID" ]; then
wmctrl -i -r "$WINDOW_ID" -b add,fullscreen 2>/dev/null
break
fi
sleep 0.5
done
wait
START_CHROMIUM_EOF
chmod 755 "$SCRIPT_DEST"
# Autostart entry (runs Chromium at desktop login)
mkdir -p "$AUTOSTART_SYSTEM"
cat > "$AUTOSTART_SYSTEM/chromium-kiosk.desktop" << DESKTOP_EOF
[Desktop Entry]
Type=Application
Name=Chromium Fullscreen
Exec=/usr/local/bin/start-chromium.sh
Icon=$DESKTOP_ICON
Hidden=false
NoDisplay=false
X-GNOME-Autostart-enabled=true
DESKTOP_EOF
chmod 644 "$AUTOSTART_SYSTEM/chromium-kiosk.desktop"
# Desktop shortcut: real .desktop file on Desktop so the file manager treats it as a launcher (not a script).
# Symlink with no extension was shown as "executable script" and prompted; .desktop runs directly with quick_exec=1.
if getent passwd "$PI_USER" >/dev/null 2>&1; then
mkdir -p "$PI_HOME/Desktop" "$PI_HOME/.config/libfm"
if [ -f "$PI_HOME/.config/libfm/libfm.conf" ] && grep -q '^quick_exec=' "$PI_HOME/.config/libfm/libfm.conf"; then
sed -i 's/^quick_exec=.*/quick_exec=1/' "$PI_HOME/.config/libfm/libfm.conf"
else
echo 'quick_exec=1' >> "$PI_HOME/.config/libfm/libfm.conf"
fi
chown -R "$PI_USER:$PI_USER" "$PI_HOME/.config/libfm" 2>/dev/null || true
DESKTOP_FILE="$PI_HOME/Desktop/GNSS Guard.desktop"
cat > "$DESKTOP_FILE" << DESKTOP_SHORTCUT_EOF
[Desktop Entry]
Type=Application
Name=GNSS Guard
Comment=GNSS Guard Dashboard (e.g. if closed)
Exec=/usr/local/bin/start-chromium.sh
Icon=$DESKTOP_ICON
Terminal=false
Categories=Utility;
DESKTOP_SHORTCUT_EOF
chmod 644 "$DESKTOP_FILE"
chown "$PI_USER:$PI_USER" "$DESKTOP_FILE"
# Remove old symlink if present
rm -f "$PI_HOME/Desktop/GNSS Guard"
# Application menu/panel entry (same content)
SHORTCUT_FILE="/usr/share/applications/gnss-guard.desktop"
cat > "$SHORTCUT_FILE" << DESKTOP_SHORTCUT_EOF
[Desktop Entry]
Type=Application
Name=GNSS Guard
Comment=GNSS Guard Dashboard (e.g. if closed)
Exec=/usr/local/bin/start-chromium.sh
Icon=$DESKTOP_ICON
Terminal=true
Categories=Utility;
DESKTOP_SHORTCUT_EOF
chmod 644 "$SHORTCUT_FILE"
fi
echo "[$(date -Iseconds)] test completed" | tee -a /var/log/cloud-init-bootstrap.log

15
emmc-provisioning/cloud-init/user-data-remote-gnss.example
View File

@@ -10,10 +10,15 @@
# as first-boot.conf; then add a runcmd line to download it to /tmp/first-boot.conf before
# running first-boot.sh so the script loads your config.
# 4. To use a different username than "pi", set PI_USER in first-boot.conf and create that user below.
# 5. DNS: manage_resolv_conf: false and NM rc-manager=symlink so the device uses DNS from DHCP
# (LXC option 6) and file.server resolves. See docs/DEVICE-DNS-DHCP-RESOLVCONF.md.
package_update: true
package_upgrade: false
# Do not overwrite /etc/resolv.conf; device will use DNS from DHCP (LXC sends option 6).
manage_resolv_conf: false
packages:
- curl
@@ -30,7 +35,17 @@ write_files:
PasswordAuthentication yes
PermitRootLogin no
# NetworkManager: manage resolv.conf via symlink so it gets DNS from DHCP (option 6 from LXC).
- path: /etc/NetworkManager/conf.d/99-resolv-dhcp.conf
content: |
[main]
rc-manager=symlink
permissions: '0644'
runcmd:
# Allow NM to manage resolv.conf with DHCP DNS (remove static file if present).
- rm -f /etc/resolv.conf
- systemctl restart NetworkManager || true
- systemctl enable ssh
- systemctl start ssh
- curl -fsSL "http://10.20.50.1:5000/files/first-boot.sh" -o /tmp/first-boot.sh

80
emmc-provisioning/cloud-init/user-data.bootstrap
View File

@@ -5,10 +5,19 @@
# provisioning portal or file server). Example: http://10.20.50.1:5000/files/bootstrap.sh
# 2. Copy this file to the boot partition as "user-data" (with meta-data and optional network-config).
# 3. Edit BOOTSTRAP_URL below to match your server (or set it once in the runcmd section).
#
# DNS: This config uses systemd-resolved; /etc/resolv.conf is a stub and DNS comes from DHCP
# (LXC option 6). Ensure bootstrap.sh does not overwrite /etc/resolv.conf. See docs/DEVICE-DNS-DHCP-RESOLVCONF.md.
package_update: true
package_upgrade: false
# Keep /etc/hosts in sync with hostname (from meta-data or set below)
manage_etc_hosts: true
# DNS is managed by systemd-resolved; we do not overwrite /etc/resolv.conf
manage_resolv_conf: false
packages:
- curl
@@ -19,12 +28,81 @@ write_files:
PasswordAuthentication yes
PermitRootLogin no
# Push current DHCP DNS into systemd-resolved (for dhcpcd/dhclient when NM doesn't feed resolved).
# With no args: discover DNS from lease or resolvectl and push to resolved for default IF.
# NetworkManager feeds resolved automatically; this covers first boot and non-NM setups.
- path: /usr/local/bin/update-resolv-from-dhcp.sh
content: |
#!/bin/sh
# Push DHCP DNS to systemd-resolved so resolv.conf (stub) uses it.
IF="${IFACE:-$(ip -o -4 route show to default 2>/dev/null | awk '{print $5}' | head -1)}"
[ -z "$IF" ] && exit 0
DNS=""
if [ -s /run/systemd/resolve/resolv.conf ]; then
DNS=$(grep -E '^nameserver\s+' /run/systemd/resolve/resolv.conf | awk '{print $2}' | tr '\n' ' ')
fi
if [ -z "$DNS" ]; then
DNS=$(resolvectl dns "$IF" 2>/dev/null | tr ' ' '\n' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$' | tr '\n' ' ')
fi
if [ -z "$DNS" ]; then
LEASE=$(ls /var/lib/dhcp/dhclient.*.leases 2>/dev/null | head -1)
[ -n "$LEASE" ] && DNS=$(grep -oP 'option domain-name-servers \K[^;]+' "$LEASE" 2>/dev/null | tr ',' '\n' | tr -d ' ' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$' | tr '\n' ' ')
fi
[ -n "$DNS" ] && resolvectl dns "$IF" $DNS
permissions: '0755'
# dhclient: feed systemd-resolved on every lease acquire/renew (DHCP provides new_domain_name_servers)
- path: /etc/dhcp/dhclient-exit-hooks.d/zzz-update-resolv-conf
content: |
#!/bin/sh
# Run by dhclient on exit; push DHCP DNS into systemd-resolved.
[ -z "$new_domain_name_servers" ] && exit 0
[ -z "$interface" ] && exit 0
resolvectl dns "$interface" $new_domain_name_servers
permissions: '0755'
# NetworkManager: resolved is fed by NM by default; this only runs our script as fallback (e.g. if resolved started late).
- path: /etc/NetworkManager/dispatcher.d/99-update-resolv-from-dhcp
content: |
#!/bin/sh
[ "$2" = "up" ] || [ "$2" = "dhcp4-change" ] || exit 0
export IFACE="$1"
/usr/local/bin/update-resolv-from-dhcp.sh
permissions: '0755'
# Tell NetworkManager to send DHCP DNS to systemd-resolved (so every DHCP update is applied).
- path: /etc/NetworkManager/conf.d/99-use-resolved.conf
content: |
[main]
dns=systemd-resolved
rc-manager=unmanaged
# Fallback: push DHCP DNS to resolved once when network is up (e.g. dhcpcd-only or first boot).
- path: /etc/systemd/system/update-resolv-from-dhcp.service
content: |
[Unit]
Description=Push DHCP DNS to systemd-resolved
After=network-online.target systemd-resolved.service
WantedBy=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/update-resolv-from-dhcp.sh
RemainAfterExit=yes
runcmd:
# Use systemd-resolved for DNS; /etc/resolv.conf -> stub so all lookups go through resolved (DHCP DNS applied by NM/hooks).
- systemctl enable systemd-resolved.service
- systemctl start systemd-resolved.service
- rm -f /etc/resolv.conf && ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
# Push current DHCP DNS into resolved once at first boot (in case NM hasn't applied yet).
- /usr/local/bin/update-resolv-from-dhcp.sh
- systemctl enable update-resolv-from-dhcp.service
- systemctl enable ssh
- systemctl start ssh
# Download and run bootstrap script (edit URL to match your file server)
- |
BOOTSTRAP_URL="http://10.20.50.1:5000/files/bootstrap.sh"
BOOTSTRAP_URL="http://file.server:5000/files/bootstrap.sh"
LOG="/var/log/cloud-init-bootstrap.log"
if ! curl -fsSL "$BOOTSTRAP_URL" -o /tmp/bootstrap.sh 2>>"$LOG" || [ ! -s /tmp/bootstrap.sh ]; then
echo "$(date -Iseconds) ERROR: Failed to download bootstrap.sh from $BOOTSTRAP_URL (file missing or empty)" >> "$LOG"

60
emmc-provisioning/dashboard/templates/admin.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Admin · CM4 Provisioning</title>
<title>Admin · GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500&family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">
@@ -76,6 +76,23 @@
<p id="goldenInfo" class="mono" style="font-size:0.9rem;">Loading…</p>
</div>
<!-- Update boot (EEPROM) -->
<div class="section" id="adminUpdateBootSection">
<h2 class="section-title">Update boot (EEPROM)</h2>
<p id="adminUpdateBootHint" class="mono" style="font-size:0.85rem; color:var(--text-muted);">When a device is connected in USB boot mode, it appears here. Use this to write EEPROM boot order (e.g. eMMC only) to the device.</p>
<div id="adminUpdateBootDevice" style="display:none; margin-top:0.5rem;">
<p class="mono" style="font-size:0.9rem; margin-bottom:0.5rem;">Device in USB boot mode</p>
<div style="display:flex; align-items:center; gap:0.5rem; flex-wrap:wrap;">
<label>Boot order:</label>
<select id="adminEepromPreset" class="eeprom-preset" title="Boot order">
<option value="0x1">eMMC only</option>
</select>
<button type="button" class="btn btn-outline btn-sm" id="adminUpdateEepromBtn">Update EEPROM</button>
</div>
</div>
<p id="adminUpdateBootNone" class="mono" style="font-size:0.85rem; color:var(--text-muted); display:none;">No device in USB boot mode. Connect a device with eMMC disable jumper and USB to host.</p>
</div>
<!-- Backups -->
<div class="section">
<h2 class="section-title">
@@ -247,6 +264,47 @@
authFetch('/api/admin/users', { method: 'POST', headers: {'Content-Type':'application/json'}, body: JSON.stringify({username: username, password: password}) }).then(function(r){ return r.json(); }).then(function(d){ if(d.ok) { document.getElementById('addUserForm').style.display = 'none'; document.getElementById('newUsername').value = ''; document.getElementById('newPassword').value = ''; fetchUsers(); } else alert(d.error); });
};
function fetchPendingDevices() {
authFetch('/api/pending-devices').then(function(r){ return r.json(); }).then(function(d){
var devEl = document.getElementById('adminUpdateBootDevice');
var noneEl = document.getElementById('adminUpdateBootNone');
if (d.usb) {
if (devEl) devEl.style.display = 'block';
if (noneEl) noneEl.style.display = 'none';
} else {
if (devEl) devEl.style.display = 'none';
if (noneEl) noneEl.style.display = 'block';
}
}).catch(function(){});
}
function fetchEepromPresets() {
authFetch('/api/eeprom-presets').then(function(r){ return r.json(); }).then(function(d){
var sel = document.getElementById('adminEepromPreset');
if (!sel) return;
sel.innerHTML = '';
(d.presets || []).forEach(function(p){
var opt = document.createElement('option');
opt.value = p.id || p.value;
opt.textContent = p.label || p.id || p.value;
sel.appendChild(opt);
});
}).catch(function(){});
}
fetchPendingDevices();
fetchEepromPresets();
var adminUpdateEepromBtn = document.getElementById('adminUpdateEepromBtn');
if (adminUpdateEepromBtn) {
adminUpdateEepromBtn.onclick = function(){
var presetEl = document.getElementById('adminEepromPreset');
var bootOrder = (presetEl && presetEl.value) ? presetEl.value : '0x1';
authFetch('/api/device-action', { method: 'POST', headers: {'Content-Type':'application/json'}, body: JSON.stringify({ source: 'usb', action: 'eeprom_update', boot_order: bootOrder }) })
.then(function(r){ return r.json(); })
.then(function(d){ if (d.ok) { fetchPendingDevices(); alert('Update EEPROM sent. The host will write to the device.'); } else alert(d.error || 'Failed'); })
.catch(function(){ alert('Request failed'); });
};
}
setInterval(fetchPendingDevices, 3000);
fetchBackups(); fetchCloudinit(); fetchUsers(); fetchLogs(); fetchGolden();
setInterval(fetchLogs, 30000);
</script>

2
emmc-provisioning/dashboard/templates/cloudinit_build.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Cloud-init build · Admin · CM4 Provisioning</title>
<title>Cloud-init build · Admin · GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500&family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">

31
emmc-provisioning/dashboard/templates/home.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Deploy · CM4 Provisioning</title>
<title>Deploy · GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500&family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">
@@ -96,7 +96,7 @@
<body>
<div class="wrap">
<header class="header">
<h1>CM4 eMMC Provisioning</h1>
<h1>GNSS Guard Provisioning</h1>
<a href="/admin">Admin</a>
</header>
@@ -107,7 +107,7 @@
<h2 class="card-title">Status</h2>
<div id="status" class="status-row">
<span id="statusPill" class="status-pill idle">Idle</span>
<span id="statusMsg" class="status-msg">Waiting for device</span>
<span id="statusMsg" class="status-msg">Waiting for Device in USB boot mode.</span>
<button type="button" id="statusClearBtn" class="btn btn-outline btn-sm" style="margin-left: auto;">Clear status</button>
</div>
<div id="statusErr" class="status-err" style="display:none;"></div>
@@ -128,7 +128,7 @@
</div>
<div class="card">
<h2 class="card-title">Capture or deploy</h2>
<h2 class="card-title">Deploy and Backup</h2>
<p id="shrinkOptionWrap" style="display:none; margin-bottom: 0.5rem; font-size: 0.8rem;"><label><input type="checkbox" id="shrinkAfterBackup" /> Shrink after backup</label></p>
<div id="pendingDevices"></div>
<p id="noPending" class="empty-msg" style="display:none;">No device connected. Use USB boot mode.</p>
@@ -142,12 +142,12 @@
<p id="goldenInfo" class="golden-info">Loading…</p>
</div>
<details class="card" style="padding: 0;">
<details class="card" style="padding: 0;" open>
<summary>Recent log</summary>
<div class="inner"><pre id="log" class="log-pre"></pre></div>
</details>
<details class="card" style="padding: 0; margin-top: 1rem;">
<details class="card" style="padding: 0; margin-top: 1rem;" open>
<summary>DHCP leases</summary>
<div class="inner">
<p id="leasesNote" class="help-sub" style="margin-top:0;">Provisioning LAN (dnsmasq) — when dashboard runs on LXC.</p>
@@ -162,7 +162,7 @@
<div class="inner">
<p class="help-sub">USB boot</p>
<ol class="steps-list">
<li><span class="num">1</span> Set reTerminal to <strong>boot mode</strong> (eMMC disable jumper).</li>
<li><span class="num">1</span> Set device to <strong>boot mode</strong> (eMMC disable jumper).</li>
<li><span class="num">2</span> Connect USB to host; choose <strong>Backup</strong> or <strong>Deploy</strong> above.</li>
<li><span class="num">3</span> Remove jumper and power cycle when done.</li>
</ol>
@@ -178,7 +178,7 @@
const phase = data.phase || 'idle';
document.getElementById('statusPill').className = 'status-pill ' + phase;
document.getElementById('statusPill').textContent = phaseLabels[phase] || phase;
document.getElementById('statusMsg').textContent = data.message || '';
document.getElementById('statusMsg').textContent = data.message || (phase === 'idle' ? 'Waiting for Device in USB boot mode.' : '');
const err = document.getElementById('statusErr');
if (data.error) { err.textContent = data.error; err.style.display = 'block'; } else { err.style.display = 'none'; }
if (phase === 'done') scheduleDoneClear();
@@ -204,7 +204,7 @@
shrinkWrap.style.display = 'block';
const el = document.createElement('div');
el.className = 'device-item';
el.innerHTML = '<div class="device-desc">USB device — choose Backup, Deploy, or Update EEPROM</div><div class="device-actions-row"><button type="button" class="btn btn-outline btn-sm" data-source="usb" data-action="backup">Backup</button> <button type="button" class="btn btn-primary btn-sm" data-source="usb" data-action="deploy">Deploy</button> <select class="eeprom-preset" title="Boot order"><option value="0x1">eMMC only</option></select> <button type="button" class="btn btn-outline btn-sm" data-source="usb" data-action="eeprom_update">Update EEPROM</button></div>';
el.innerHTML = '<div class="device-desc">USB device — choose Backup or Deploy</div><div class="device-actions-row"><button type="button" class="btn btn-outline btn-sm" data-source="usb" data-action="backup">Backup</button> <button type="button" class="btn btn-primary btn-sm" data-source="usb" data-action="deploy">Deploy</button></div>';
container.appendChild(el);
} else shrinkWrap.style.display = 'none';
noPending.style.display = hasAny ? 'none' : 'block';
@@ -212,10 +212,6 @@
btn.onclick = function() {
const body = { source: btn.getAttribute('data-source'), action: btn.getAttribute('data-action') };
if (body.source === 'network') body.mac = btn.getAttribute('data-mac');
if (body.action === 'eeprom_update' && body.source === 'usb') {
const presetEl = btn.closest('.device-item') && btn.closest('.device-item').querySelector('.eeprom-preset');
body.boot_order = (presetEl && presetEl.value) ? presetEl.value : '0x1';
}
if (body.action === 'backup' && document.getElementById('shrinkAfterBackup').checked) body.shrink = true;
fetch('/api/device-action', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) })
.then(function(r) { return r.json(); })
@@ -267,7 +263,14 @@
fetch('/api/first-boot-status').then(function(r){ return r.json(); }).then(renderFirstBootStatus).catch(function(){});
}
function fetchPending() { fetch('/api/pending-devices').then(function(r){ return r.json(); }).then(function(d){ renderPending(d.usb || null, d.network || []); }).catch(function(){ renderPending(null, []); }); }
function fetchLog() { fetch('/api/log').then(function(r){ return r.json(); }).then(function(d){ document.getElementById('log').textContent = d.log || ''; }).catch(function(){}); }
function fetchLog() {
fetch('/api/log').then(function(r){ return r.json(); }).then(function(d){
var logEl = document.getElementById('log');
if (!logEl) return;
logEl.textContent = d.log || '';
logEl.scrollTop = logEl.scrollHeight;
}).catch(function(){});
}
function fetchGolden() {
fetch('/api/golden-info').then(function(r){ return r.json(); }).then(function(d){
const el = document.getElementById('goldenInfo');

35
emmc-provisioning/dashboard/templates/index.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>CM4 eMMC Provisioning</title>
<title>GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500&family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">
@@ -359,8 +359,8 @@
<body>
<div class="wrap">
<header class="header">
<h1>CM4 eMMC Provisioning</h1>
<p>Deploy or backup reTerminal via USB boot mode</p>
<h1>GNSS Guard Provisioning</h1>
<p>Deploy or backup device via USB boot mode</p>
</header>
<!-- 1. Current status -->
@@ -368,12 +368,12 @@
<h2 class="section-title">Current status</h2>
<div id="status" class="status-row">
<span id="statusPill" class="status-pill idle">Idle</span>
<span id="statusMsg" class="status-msg">Waiting for device</span>
<span id="statusMsg" class="status-msg">Waiting for Device in USB boot mode.</span>
<button type="button" id="statusClearBtn" class="btn btn-outline btn-sm" style="margin-left: auto;">Clear status</button>
</div>
<div id="statusErr" class="status-err" style="display:none;"></div>
<div id="statusGoldenHint" class="backup-deploy-hint" style="display:none; margin-top:0.75rem;">
No golden image is required to <strong>capture</strong>. Connect a device in USB boot mode; when it appears under “Capture image or deploy”, click <strong>Backup</strong> to save its image. Then set that backup as golden in the list below.
No golden image is required to <strong>backup</strong>. Connect a device in USB boot mode; when it appears under “Deploy and Backup”, click <strong>Backup</strong> to save its image. Then set that backup as golden in the list below.
<button type="button" id="statusClearHintBtn" class="btn btn-outline btn-sm" style="margin-left:0.5rem;">Clear message</button>
</div>
<div id="statusMeta" class="status-meta" style="display:none;"></div>
@@ -384,8 +384,8 @@
<!-- 2. Capture (Backup) or Deploy -->
<section class="section">
<h2 class="section-title">Capture image or deploy</h2>
<p class="backup-deploy-hint">To <strong>capture (backup)</strong> an image from a device: connect it in USB boot mode. When the device appears below, click <strong>Backup</strong> to save its eMMC to a file. To write an image to a device, click <strong>Deploy</strong> (requires a golden image).</p>
<h2 class="section-title">Deploy and Backup</h2>
<p class="backup-deploy-hint">Connect a device in USB boot mode. When it appears below, click <strong>Backup</strong> to save its eMMC to a file, or <strong>Deploy</strong> to write the golden image to the device (requires a golden image set in Admin).</p>
<p id="shrinkOptionWrap" class="backup-deploy-hint" style="display:none; margin-top:0.5rem;">
<label><input type="checkbox" id="shrinkAfterBackup" /> Shrink after backup</label> <span class="backups-mono" style="font-size:0.8rem;">(reduces size; requires PiShrink on host)</span>
</p>
@@ -398,7 +398,7 @@
</span>
<span>— USB boot mode</span>
</div>
<p id="noPending" class="empty-msg" style="display:none;">No device connected. Connect reTerminal in USB boot mode (eMMC disable jumper + USB to host) — then the <strong>Backup</strong> and <strong>Deploy</strong> buttons will appear above.</p>
<p id="noPending" class="empty-msg" style="display:none;">No device connected. Connect device in USB boot mode (eMMC disable jumper + USB to host) — then the <strong>Backup</strong> and <strong>Deploy</strong> buttons will appear above.</p>
</section>
<!-- 3. Saved backups -->
@@ -477,7 +477,7 @@
<div class="inner">
<p class="help-sub">USB boot mode</p>
<ol class="steps-list">
<li><span class="num">1</span> Set reTerminal to <strong>boot mode</strong> (eMMC disable jumper, e.g. J2 / nRPIBOOT).</li>
<li><span class="num">1</span> Set device to <strong>boot mode</strong> (eMMC disable jumper, e.g. J2 / nRPIBOOT).</li>
<li><span class="num">2</span> Connect <strong>USB slave</strong> to the host and power on. The device appears above; choose <strong>Backup</strong> or <strong>Deploy</strong>.</li>
<li><span class="num">3</span> When done, remove the jumper and power cycle to boot from eMMC.</li>
</ol>
@@ -485,7 +485,7 @@
</details>
<!-- 5. Recent log (collapsible) -->
<details class="section" style="padding:0;">
<details class="section" style="padding:0;" open>
<summary>Recent log</summary>
<div class="inner">
<pre id="log" class="log-pre"></pre>
@@ -516,7 +516,7 @@
const phase = data.phase || 'idle';
statusPill.className = 'status-pill ' + phase;
statusPill.textContent = phaseLabels[phase] || phase;
statusMsg.textContent = data.message || '';
statusMsg.textContent = data.message || (phase === 'idle' ? 'Waiting for Device in USB boot mode.' : '');
if (data.error) {
statusErr.textContent = data.error;
@@ -566,7 +566,7 @@
if (shrinkWrap) shrinkWrap.style.display = 'block';
const el = document.createElement('div');
el.className = 'device-item';
el.innerHTML = '<div class="device-info"><div class="device-type">USB boot</div><div class="device-desc">Device connected — choose Backup, Deploy, or Update EEPROM</div></div><div class="device-actions"><button type="button" class="btn btn-outline" data-source="usb" data-action="backup">Backup</button><button type="button" class="btn btn-primary" data-source="usb" data-action="deploy">Deploy</button><select class="eeprom-preset" title="Boot order"><option value="0x1">eMMC only</option></select><button type="button" class="btn btn-outline" data-source="usb" data-action="eeprom_update">Update EEPROM</button></div>';
el.innerHTML = '<div class="device-info"><div class="device-type">USB boot</div><div class="device-desc">Device connected — choose Backup or Deploy</div></div><div class="device-actions"><button type="button" class="btn btn-outline" data-source="usb" data-action="backup">Backup</button><button type="button" class="btn btn-primary" data-source="usb" data-action="deploy">Deploy</button></div>';
container.appendChild(el);
} else {
if (shrinkWrap) shrinkWrap.style.display = 'none';
@@ -583,10 +583,6 @@
const mac = btn.getAttribute('data-mac');
const body = { source: source, action: action };
if (mac) body.mac = mac;
if (action === 'eeprom_update' && source === 'usb') {
const presetEl = btn.closest('.device-item') && btn.closest('.device-item').querySelector('.eeprom-preset');
body.boot_order = (presetEl && presetEl.value) ? presetEl.value : '0x1';
}
const shrinkCb = document.getElementById('shrinkAfterBackup');
if (action === 'backup' && shrinkCb && shrinkCb.checked) body.shrink = true;
fetch('/api/device-action', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) })
@@ -785,7 +781,12 @@
fetch('/api/pending-devices').then(function(r) { return r.json(); }).then(function(d) { renderPending(d.usb || null, d.network || []); }).catch(function() { renderPending(null, []); });
}
function fetchLog() {
fetch('/api/log').then(function(r) { return r.json(); }).then(function(d) { document.getElementById('log').textContent = d.log || ''; }).catch(function() {});
fetch('/api/log').then(function(r) { return r.json(); }).then(function(d) {
var logEl = document.getElementById('log');
if (!logEl) return;
logEl.textContent = d.log || '';
logEl.scrollTop = logEl.scrollHeight;
}).catch(function() {});
}
function fetchBackups() {
fetch('/api/backups').then(function(r) { return r.json(); }).then(function(d) {

4
emmc-provisioning/dashboard/templates/login.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Admin login · CM4 Provisioning</title>
<title>Admin login · GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">
@@ -28,7 +28,7 @@
<body>
<div class="box">
<h1>Admin login</h1>
<p class="sub">CM4 eMMC provisioning dashboard</p>
<p class="sub">GNSS Guard Provisioning admin</p>
{% if error %}<p class="err">{{ error }}</p>{% endif %}
<form method="post" action="{{ url_for('login') }}">
<label for="username">Username</label>

2
emmc-provisioning/dashboard/templates/portal_files.html
View File

@@ -4,7 +4,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Portal files · Admin · CM4 Provisioning</title>
<title>Portal files · Admin · GNSS Guard Provisioning</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500&family=Outfit:wght@400;500;600;700&display=swap" rel="stylesheet">

163
emmc-provisioning/docs/DEPLOY-NEW-PROXMOX.md Normal file
View File

@@ -0,0 +1,163 @@
# Deploy CM4 eMMC Provisioning to a New Proxmox Instance
Step-by-step guide to deploy the provisioning service (host + LXC) on a **new** Proxmox server. For redeploy/update and troubleshooting, see [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md).
---
## Prerequisites (before running the deploy script)
| Requirement | Details |
|-------------|---------|
| **Proxmox host** | A Proxmox VE node (new or existing) where you want the service. |
| **SSH as root** | You must be able to run `ssh root@YOUR_PROXMOX_HOST` with **key-based auth** (no password prompt). |
| **Proxmox storage** | At least one active storage (e.g. `local` or `local-lvm`). Check on the host: `pvesm status`. |
| **Host internet** (recommended) | Needed so the deploy script can download the Debian 12 LXC template (if missing), and install **usbboot** and **PiShrink** on the host. Without internet, deploy still runs but you must install usbboot and PiShrink manually later. |
**Optional (set before deploy):**
- `DEPLOY_ROOTFS_STORAGE=local-lvm` — Skip interactive storage choice when creating the LXC.
- `DEPLOY_LXC_ROOT_PASSWORD=yourpassword` — Set LXC root password and enable SSH.
- `DEPLOY_LXC_SSH_KEY=/path/to/pub` — Copy this key into the LXC (default: `~/.ssh/id_ed25519.pub` or `id_rsa.pub`).
- `CM4_BACKUPS_HOST_PATH=/mnt/storage/cm4-backups` — Store backups on this host path (create the directory on the host if needed).
- **Network (WAN/LAN):**
`DEPLOY_LXC_WAN_BRIDGE=vmbr0` (default), `DEPLOY_LXC_WAN_IP=dhcp` (default),
`DEPLOY_LXC_LAN_BRIDGE=vmbr1`, `DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24` — To add eth1 as provisioning LAN. **Set these if you want the portal reachable from the LAN** (e.g. http://10.20.50.1:5000); the dashboard listens on all interfaces.
---
## Step 1: Run the deploy script
From your **workstation** (where the repo is cloned), run:
```bash
cd /path/to/reTerminal\ DM4
./emmc-provisioning/scripts/deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST
```
Replace `YOUR_PROXMOX_HOST` with the Proxmox hostname or IP (e.g. `10.20.30.40`).
**Example with options:**
```bash
DEPLOY_ROOTFS_STORAGE=local-lvm \
DEPLOY_LXC_ROOT_PASSWORD='YourSecurePassword' \
DEPLOY_LXC_LAN_BRIDGE=vmbr1 \
DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24 \
./emmc-provisioning/scripts/deploy-to-proxmox.sh root@10.20.30.40
```
- On **first run**, the script will ask you to choose LXC rootfs storage (unless `DEPLOY_ROOTFS_STORAGE` is set). It then creates the LXC, installs host scripts, udev, systemd units, and the dashboard in the LXC.
- The script prints **LXC IP (WAN)** and, if you set `DEPLOY_LXC_LAN_BRIDGE`, **LXC IP (LAN)**. The portal is reachable at `http://<IP>:5000` on both; use the LAN IP from devices on the provisioning LAN.
---
## Step 2: Install usbboot on the host (if host had no internet during deploy)
USB flash/backup needs **rpiboot** on the Proxmox **host**. If the deploy log said usbboot install failed or was skipped:
**From your workstation:**
```bash
scp emmc-provisioning/scripts/install-usbboot-on-host.sh root@YOUR_PROXMOX_HOST:/tmp/
ssh root@YOUR_PROXMOX_HOST "bash /tmp/install-usbboot-on-host.sh"
```
**Or on the Proxmox host** (if `/tmp/emmc-provisioning-deploy` is still there):
```bash
ssh root@YOUR_PROXMOX_HOST
bash /tmp/emmc-provisioning-deploy/scripts/install-usbboot-on-host.sh
```
---
## Step 3: Add a golden image (required for Deploy)
To **write** an image to a device (Deploy), the host must have a **golden image** at `/var/lib/cm4-provisioning/golden.img`. Backup (read from device) works without it.
**Option A — From the dashboard**
1. Open **http://&lt;LXC-IP&gt;:5000** (use the LXC IP from the deploy output).
2. Build a cloud-init image or upload/set an existing backup as golden (see dashboard Admin).
**Option B — Copy an image from your machine**
```bash
scp /path/to/your-golden.img root@YOUR_PROXMOX_HOST:/var/lib/cm4-provisioning/golden.img
```
---
## Accessing the portal from the LAN
The dashboard listens on **all interfaces** (`0.0.0.0:5000`), so it is reachable on both WAN and LAN IPs when the LXC has two networks.
- **Deploy with a LAN interface:** set `DEPLOY_LXC_LAN_BRIDGE=vmbr1` (and optionally `DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24`) when running the deploy script. The LXC will get eth1 with the LAN IP (e.g. 10.20.50.1).
- **From the provisioning LAN:** open **http://&lt;LAN-IP&gt;:5000** (e.g. http://10.20.50.1:5000). Devices on that subnet can use the portal without going through WAN.
- If you did not set a LAN bridge at deploy time, you only have one IP (WAN); use that for the portal. To add LAN later you would need to add eth1 to the container and reconfigure (see PROXMOX-LXC-DEPLOYMENT.md).
---
## Step 4: (Optional) SSH into the LXC
If you set `DEPLOY_LXC_ROOT_PASSWORD` or had a default SSH key, you can already run:
```bash
ssh root@<LXC-IP>
```
Otherwise, enable root SSH and add your key:
```bash
./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST
# Or with password: ROOT_PASSWORD='YourPassword' ./emmc-provisioning/scripts/setup-lxc-ssh.sh root@YOUR_PROXMOX_HOST ~/.ssh/id_ed25519.pub
```
---
## Step 5: (Optional) Network boot (DHCP + TFTP on eth1)
Only if you deployed with **`DEPLOY_LXC_LAN_BRIDGE`** (and optionally `DEPLOY_LXC_LAN_SUBNET`) and want to offer network boot to devices on that LAN:
```bash
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@<LXC-IP>
```
See [NETWORK-BOOT-LXC.md](NETWORK-BOOT-LXC.md) for details.
---
## Step 6: (Optional) Install PiShrink on the host
If the deploy log said PiShrink install failed (e.g. no internet), and you want **Shrink/Compress** in the dashboard to work:
```bash
ssh root@YOUR_PROXMOX_HOST "bash /tmp/emmc-provisioning-deploy/scripts/install-pishrink-on-host.sh"
```
Or from your machine (stream the script): use the same pattern as in [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md) for `install-pishrink-on-host.sh`.
---
## Summary checklist
| Step | Action | Required? |
|------|--------|------------|
| 1 | Run `deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST` | **Yes** |
| 2 | Install usbboot on host (if deploy couldn’t) | For USB flash/backup |
| 3 | Add `golden.img` for Deploy | For Deploy only |
| 4 | SSH to LXC (or use setup-lxc-ssh.sh) | Optional |
| 5 | Run setup-network-boot-on-lxc.sh (if using eth1 LAN) | Optional |
| 6 | Install PiShrink on host (if deploy couldn’t) | For Shrink/Compress |
**After deployment:**
- **Dashboard:** http://&lt;LXC-IP&gt;:5000 (WAN). If you set `DEPLOY_LXC_LAN_BRIDGE`, also **http://&lt;LAN-IP&gt;:5000** (e.g. http://10.20.50.1:5000) from the LAN.
- **Golden image path (host and LXC):** `/var/lib/cm4-provisioning/golden.img`
- **Disable auto-flash:** `ssh root@YOUR_PROXMOX_HOST "rm /etc/cm4-provisioning/enabled"`
- **Enable again:** `ssh root@YOUR_PROXMOX_HOST "touch /etc/cm4-provisioning/enabled"`
**If you see "rpiboot failed or no device connected":** The error is from the **Proxmox host** (where USB is connected). On the host run: `tail -50 /var/lib/cm4-provisioning/flash.log` to see the real rpiboot message. Ensure the reTerminal is in **boot mode** (eMMC disable jumper, USB slave port), then unplug/replug. See [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md) § "If rpiboot fails" for full steps.
Full reference: [PROXMOX-LXC-DEPLOYMENT.md](PROXMOX-LXC-DEPLOYMENT.md).

90
emmc-provisioning/docs/DEVICE-DNS-DHCP-RESOLVCONF.md Normal file
View File

@@ -0,0 +1,90 @@
# Device DNS from DHCP and /etc/resolv.conf
This document describes how to configure provisioned devices (e.g. Raspberry Pi / reTerminal) so they **use DNS from DHCP** and do **not** have a fixed nameserver in `/etc/resolv.conf`. That way the LXC’s dnsmasq (option 6) is used, **file.server** resolves, and scripts can use `http://file.server/...` without hardcoding IPs.
## Summary of changes (what we did on the Pi)
1. **Do not overwrite `/etc/resolv.conf`**
No script (e.g. bootstrap or first-boot) should write a fixed nameserver (e.g. `8.8.8.8`) into `/etc/resolv.conf`. DNS should come from DHCP.
2. **LXC sends DHCP option 6 (DNS server)**
dnsmasq on the LXC must send the LXC’s eth1 IP as the DNS server so clients use it and get **file.server** resolution. See [DNSMASQ-DNS-FILESERVER.md](DNSMASQ-DNS-FILESERVER.md) and `scripts/setup-network-boot-on-lxc.sh` (`dhcp-option=6,${LAN_GW}`).
3. **Let NetworkManager manage `/etc/resolv.conf`**
So that the nameserver in `/etc/resolv.conf` is the one from DHCP (option 6), either:
- **Option A (recommended for full cloud-init):** Use **systemd-resolved** and make `/etc/resolv.conf` a symlink to the resolved stub; resolved gets DNS from NetworkManager.
- **Option B (minimal cloud-init):** Use **NetworkManager** to manage `/etc/resolv.conf` via a symlink: add `rc-manager=symlink` in NetworkManager config so `/etc/resolv.conf` points to `/run/NetworkManager/resolv.conf`, which NM fills with the DHCP DNS.
4. **Ensure connection uses DHCP DNS**
The NetworkManager connection should have `ipv4.ignore-auto-dns: no` (default) so it accepts option 6 from DHCP. No fixed `ipv4.dns` in the connection.
## What to change in cloud-init
### Option A: user-data.bootstrap (systemd-resolved)
**File:** `cloud-init/user-data.bootstrap`
- **manage_resolv_conf: false** — already set; cloud-init must not overwrite resolv.conf.
- **systemd-resolved** — runcmd enables/starts resolved and makes `/etc/resolv.conf` a symlink to `stub-resolv.conf`. Resolved gets DNS from NetworkManager (and from the hooks in write_files).
- **NetworkManager** — `99-use-resolved.conf` has `dns=systemd-resolved` and `rc-manager=unmanaged` so NM doesn’t write resolv.conf; resolved does.
- **Bootstrap script** — must **not** write `nameserver 8.8.8.8` (or any fixed server) into `/etc/resolv.conf`. Our `bootstrap.sh` no longer does that.
No extra changes needed if you use `user-data.bootstrap` as-is; just ensure your bootstrap script does not touch resolv.conf.
### Option B: Minimal user-data (first-boot or bootstrap-only, no systemd-resolved)
If your user-data only runs a remote script (e.g. `first-boot.sh` or `bootstrap.sh`) and does **not** enable systemd-resolved, add the following so the device uses DNS from DHCP and NM manages resolv.conf:
1. **Set in user-data (cloud-config):**
```yaml
manage_resolv_conf: false
```
2. **Add a write_files entry** so NetworkManager manages resolv.conf with the DHCP-provided DNS:
```yaml
write_files:
# ... your other write_files ...
- path: /etc/NetworkManager/conf.d/99-resolv-dhcp.conf
content: |
[main]
rc-manager=symlink
permissions: '0644'
```
3. **In your bootstrap/first-boot script:**
Do **not** write a fixed nameserver to `/etc/resolv.conf` (e.g. remove any line like `echo "nameserver 8.8.8.8" > /etc/resolv.conf`).
4. **Optional runcmd** (if you want a clean state on first boot):
Remove any existing static resolv.conf so NM can create its symlink and write DHCP DNS:
```yaml
runcmd:
- rm -f /etc/resolv.conf
- systemctl restart NetworkManager
# ... then your download and run of bootstrap.sh or first-boot.sh ...
```
After first boot, devices will get DNS from DHCP (LXC option 6), and **file.server** will resolve to the LXC’s eth1 IP.
## Verification on the device
```bash
# Should show the LXC as nameserver (e.g. 10.20.40.1), not 8.8.8.8
cat /etc/resolv.conf
# Should resolve to LXC eth1
getent hosts file.server
```
## Reference: manual fix on an already-provisioned device
If a device was provisioned before these changes and still has a fixed DNS (e.g. 8.8.8.8):
1. **LXC:** Ensure dnsmasq sends option 6 (see [DNSMASQ-DNS-FILESERVER.md](DNSMASQ-DNS-FILESERVER.md)); re-run `setup-network-boot-on-lxc.sh` if needed.
2. **On the device:**
- Add NetworkManager config:
`echo -e '[main]\nrc-manager=symlink' | sudo tee /etc/NetworkManager/conf.d/99-resolv-dhcp.conf`
- Remove static resolv.conf and restart NM:
`sudo rm -f /etc/resolv.conf && sudo systemctl restart NetworkManager`
- Renew DHCP so the device gets option 6:
`sudo nmcli con down "Wired connection 1"; sudo nmcli con up "Wired connection 1"`
3. Check: `cat /etc/resolv.conf` and `getent hosts file.server`.

101
emmc-provisioning/docs/DNSMASQ-DNS-FILESERVER.md Normal file
View File

@@ -0,0 +1,101 @@
# dnsmasq DNS and file.server on the LXC
This document describes the dnsmasq DNS configuration on the provisioning LXC and the static hostname **file.server** used for the fileserver.
## What was changed
### 1. dnsmasq now provides DNS on eth1
Previously, dnsmasq on the LXC was configured with **`port=0`**, which disabled DNS and provided only DHCP and TFTP on the provisioning interface (eth1).
**Change:** `port=0` was removed so dnsmasq also acts as a DNS server on eth1 (port 53). Clients that receive DHCP from dnsmasq will use the LXC as their DNS server for the provisioning LAN.
- **DHCP** on eth1 — unchanged (range from `lan-subnet.conf`, e.g. 10.20.40.100–10.20.40.200).
- **TFTP/PXE** on eth1 — unchanged (toggle with `/opt/cm4-provisioning/toggle-network-boot-dhcp.sh`).
- **DNS** on eth1 — **new**: local static records (e.g. `file.server`) plus forwarding of other queries via the LXC’s `/etc/resolv.conf`.
### 2. Static DNS record: file.server → eth1 IP
A static A record was added so the hostname **file.server** resolves to the LXC’s eth1 address (the provisioning LAN gateway). That IP is taken from **`/opt/cm4-provisioning/lan-subnet.conf`** as **`LAN_GW`** (e.g. `10.20.40.1`).
**dnsmasq config (written by `setup-network-boot-on-lxc.sh`):**
```text
address=/file.server/${LAN_GW}
```
So scripts and devices on the provisioning LAN can use **`http://file.server/...`** (or `file.server` in general) without hardcoding the LXC’s IP. The IP stays correct even if the LAN subnet is changed and the setup script is re-run.
### 3. Files modified in the repo
| File | Change |
|------|--------|
| **scripts/setup-network-boot-on-lxc.sh** | Removed `port=0`; added `address=/file.server/${LAN_GW}` and comments in the generated `/etc/dnsmasq.d/network-boot.conf`. |
| **lxc/dnsmasq-network-boot.conf** | Template updated: removed `port=0`, added comment for DNS and `file.server` (commented example). |
### 4. Applied on LXC (root@10.20.40.1)
On **2025-03-04** the setup script was run against **root@10.20.40.1**:
```bash
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@10.20.40.1
```
Result on that LXC:
- **LAN:** 10.20.40.0/24, gateway 10.20.40.1 (from existing `lan-subnet.conf`).
- **DHCP:** 10.20.40.100–10.20.40.200 on eth1.
- **DNS:** Enabled on eth1; **file.server** → **10.20.40.1**.
- dnsmasq and NAT were (re)configured; TFTP root and network boot toggle unchanged.
So on the provisioning LAN, **file.server** resolves to **10.20.40.1** (the LXC’s eth1).
## How to use file.server in scripts
On devices that get DHCP (and thus DNS) from the LXC on the provisioning LAN:
- Use **`http://file.server/...`** (or `file.server` as hostname) instead of `http://10.20.40.1/...`.
- No need to hardcode the LXC IP; if you change the subnet and re-run the setup script, **file.server** will still point at the correct gateway.
Example:
```bash
curl -O http://file.server/cloud-init/seed.img
```
## Adding more static DNS entries
To add more names (e.g. `fileserver` or another hostname), add more **`address=/name/${LAN_GW}`** lines in the heredoc in **scripts/setup-network-boot-on-lxc.sh** (where `network-boot.conf` is generated), or add a separate file under `/etc/dnsmasq.d/` on the LXC with the same format. Then restart dnsmasq:
```bash
systemctl restart dnsmasq
```
## Re-applying on another or existing LXC
To apply or refresh this configuration on any LXC:
```bash
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@<LXC-IP> [SUBNET]
```
Example with explicit subnet:
```bash
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@10.20.40.1 10.20.40.1/24
```
This rewrites `/etc/dnsmasq.d/network-boot.conf` (including `address=/file.server/${LAN_GW}`) and restarts dnsmasq.
## Verification on the LXC
```bash
# DNS and file.server
grep -E 'address=|port=' /etc/dnsmasq.d/network-boot.conf
# Resolve file.server (from a client on the provisioning LAN, or from LXC with server 127.0.0.1)
getent hosts file.server
# or: dig @10.20.40.1 file.server
```
Expected: **file.server** resolves to the LAN gateway (e.g. 10.20.40.1).

30
emmc-provisioning/docs/EDIT-CLOUDINIT-ON-DEVICE.md
View File

@@ -1,4 +1,4 @@
# How to edit cloud-init files on the device before capturing the image
# How to edit cloud-init files on the device or on an image file
The cloud-init **NoCloud** files live on the **boot partition**. On the running device they are at:
@@ -77,6 +77,34 @@ They are owned by **root** and need **sudo** to edit.
---
## Method 3: Edit on an image file (.img or .img.xz)
To edit cloud-init **inside a disk image** (e.g. `gnss-bootstrap-20260223-215010.img.xz`) without booting the device:
1. **Run the helper script** (requires `sudo` for loop device and mount):
```bash
cd /path/to/reTerminal-DM4
./emmc-provisioning/scripts/edit-cloudinit-on-image.sh gnss-bootstrap-20260223-215010.img.xz
```
2. The script will:
- Decompress the `.img.xz` (if needed; allow ~3GB free space and a minute or two)
- Attach the image with `losetup` and mount the **boot** (FAT32) partition
- Open your `$EDITOR` (or `nano`) on `user-data`, `meta-data`, and `network-config`
- After you save and exit, unmount and recompress, **overwriting** the original `.img.xz`
3. **Back up the image** before running if you want to keep the original:
```bash
cp gnss-bootstrap-20260223-215010.img.xz gnss-bootstrap-20260223-215010.img.xz.bak
```
4. **Options:**
- `--no-recompress` — Leave the image decompressed after editing (do not overwrite the `.img.xz`).
- `--replace-with-repo` — Copy `user-data`, `meta-data`, and `network-config` from `emmc-provisioning/cloud-init/` into the image before opening the editor (useful to start from repo templates).
---
## What to edit (typical)
- **meta-data**

48
emmc-provisioning/docs/NETWORK-BOOT-LXC.md
View File

@@ -18,8 +18,9 @@ Devices plugged into the same network as **eth1** (e.g. reTerminals with network
## What you need on the LXC
1. **DHCP server** on eth1 only (e.g. **dnsmasq**), handing out addresses in e.g. `10.20.50.100`–`10.20.50.200` and advertising the TFTP server (next-server = LXC’s eth1 IP).
2. **TFTP server** (dnsmasq can provide this) with **TFTP root** containing Raspberry Pi 4 / CM4 boot files.
3. **IP forwarding** and **NAT** (nftables or iptables) so traffic from `10.20.50.0/24` is masqueraded out **eth0**.
2. **DNS server** on eth1 (dnsmasq): static name **file.server** → eth1 IP so scripts can use `http://file.server/...`; other queries forwarded upstream. See [DNSMASQ-DNS-FILESERVER.md](DNSMASQ-DNS-FILESERVER.md).
3. **TFTP server** (dnsmasq can provide this) with **TFTP root** containing Raspberry Pi 4 / CM4 boot files.
4. **IP forwarding** and **NAT** (nftables or iptables) so traffic from `10.20.50.0/24` is masqueraded out **eth0**.
## One-time setup (inside the LXC)
@@ -40,15 +41,21 @@ bash /path/to/setup-network-boot-on-lxc.sh
The script will:
- Install **dnsmasq** (DHCP + TFTP).
- Configure dnsmasq to listen only on **eth1**, with a DHCP range and TFTP root.
- Install **dnsmasq** (DHCP + TFTP + DNS).
- Configure dnsmasq to listen only on **eth1**, with a DHCP range, TFTP root, and DNS (including **file.server** → eth1).
- Create `/srv/tftpboot` and **fetch Raspberry Pi 4 boot files from GitHub** (raspberrypi/firmware, `boot/` folder) if not already present.
- Enable **IPv4 forwarding** and **NAT** (nftables) so clients on eth1 use eth0 for internet.
- Enable and start the **dnsmasq** service.
## Proxmox: adding eth1 to the LXC
If you create the container by hand or want a second interface:
Use the deploy script with **`DEPLOY_LXC_LAN_BRIDGE`** and **`DEPLOY_LXC_LAN_SUBNET`** so the LXC is created with eth1 (LAN) from the start:
```bash
DEPLOY_LXC_LAN_BRIDGE=vmbr1 DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24 ./emmc-provisioning/scripts/deploy-to-proxmox.sh root@YOUR_PROXMOX_HOST
```
Or add a second interface to an existing container by hand:
1. On the **Proxmox host**, add a second network device to the container, e.g.:
```bash
@@ -65,6 +72,37 @@ If you create the container by hand or want a second interface:
Your current LXC already has eth0 (10.130.60.141) and eth1 (10.20.50.1); the setup script only adds DHCP, TFTP, and NAT.
### Changing the LAN subnet
When you deploy with **`DEPLOY_LXC_LAN_SUBNET`** (e.g. `10.100.1.1/24`), the deploy script writes **`/opt/cm4-provisioning/lan-subnet.conf`** inside the LXC with `LAN_GW`, `LAN_CIDR`, and the DHCP range. All LXC services use this file:
- **dnsmasq** (DHCP range and, via the toggle script, TFTP next-server)
- **nftables/iptables** (NAT source subnet)
- **toggle-network-boot-dhcp.sh** (option 66/67 next-server)
So changing `DEPLOY_LXC_LAN_SUBNET` and **re-running the deploy script** updates `lan-subnet.conf`. To apply the new subnet to dnsmasq and NAT, **re-run the setup script** after redeploying:
```bash
./emmc-provisioning/scripts/setup-network-boot-on-lxc.sh root@<LXC-IP>
```
Then run **toggle enable** again if you use network boot: `ssh root@<LXC-IP> /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable`
### Extra LAN IPs and VLAN (eth1.40)
The setup script also configures **extra IPs on eth1** and a **VLAN interface** so the LXC can serve multiple subnets and provide internet (NAT) to all of them:
| Address / interface | Purpose |
|--------------------|--------|
| **Primary** (e.g. `10.20.50.1/24`) | Set at deploy; used by dnsmasq for DHCP/TFTP/DNS |
| **192.168.30.1/24** | Extra LAN on eth1 |
| **192.168.127.1/24** | Extra LAN on eth1 |
| **eth1.40** **192.168.0.1/24** | VLAN 40 on eth1 |
- Config is persisted in **`/etc/network/interfaces.d/70-cm4-extra-lan`** (installed when you run `setup-network-boot-on-lxc.sh`).
- **NAT** is applied to all four: primary LAN, 192.168.30.0/24, 192.168.127.0/24, and 192.168.0.0/24 (VLAN 40), so clients on any of these subnets get internet via eth0.
- For **VLAN 40** to receive tagged traffic, the Proxmox bridge connected to eth1 (e.g. vmbr1) must either be a trunk that passes VLAN 40, or you use a dedicated bridge (e.g. vmbr1.40) and attach the container to it as a second interface; the script creates eth1.40 inside the LXC for the in-container VLAN case.
## After setup: reTerminal network boot
1. Set the reTerminal **boot order** to try eMMC first, then network (e.g. `BOOT_ORDER=0xf21`): use the dashboard **Update EEPROM** when the device is connected via USB boot, or set manually (usbboot recovery / `rpi-eeprom-config` on device). Not set by first-boot.

63
emmc-provisioning/docs/PROXMOX-HOST-COMPARISON.md Normal file
View File

@@ -0,0 +1,63 @@
# Proxmox host comparison: working vs new
Quick reference from comparing **working** (10.130.60.224) and **new** (100.106.128.36) Proxmox hosts.
## Same on both
| Item | Status |
|------|--------|
| `/opt/cm4-provisioning/env` | Same (GOLDEN_IMAGE, RPIBOOT_DIR, EMMC_SIZE_BYTES=8589934592, SHRINK_BACKUP, etc.) |
| `/etc/cm4-provisioning/enabled` | Present (provisioning enabled) |
| udev rules `89-cm4-boot-mode-permissions.rules` | Same (MODE="0666" for 2b8e, 0a5c:2711) |
| udev rules `90-cm4-boot-mode.rules` | Same (trigger cm4-flash-trigger.sh on add) |
| `/usr/local/bin/cm4-flash-trigger.sh` | Same (starts cm4-flash.service) |
| `cm4-flash.service` | Same (ExecStartPre=sleep 5, TimeoutStartSec=7200) |
| `cm4-build-cloudinit.path`, `cm4-shrink.path` + services | Same |
| `build-cloudinit-image.sh`, `run-shrink-on-host.sh`, `fix-gadget-bootcode-on-host.sh` | Same (md5 match) |
## Differences
### 1. `/opt/usbboot/mass-storage-gadget64/bootcode4.bin` — **fixed**
- **Working:** Present (105984 bytes). rpiboot needs this or it prints "No 'bootcode' files found".
- **New:** Was missing; fixed by copying from working host. Verify: `ls -la /opt/usbboot/mass-storage-gadget64/bootcode4.bin` on new.
### 2. `/opt/cm4-provisioning/flash-emmc-on-connect.sh`
- **Working:** Older version (14421 bytes, md5 `dbac0bc2...`).
- **New:** Newer version (15449 bytes, md5 `6081bda7...`) with longer device wait (60s/90s), relaxed size check (50–120%), and extra diagnostics on "No suitable block device".
**Recommendation:** Copy the repo script to the **working** host so both use the same version:
```bash
scp emmc-provisioning/host/flash-emmc-on-connect.sh root@10.130.60.224:/opt/cm4-provisioning/
```
### 3. `/var/lib/cm4-provisioning/golden.img`
- **Working:** Present (symlink to a backup image). Required for **Deploy** (writing image to device).
- **New:** Missing. Backup works; Deploy will show "Golden image not found" until you set one (dashboard “Set as golden” or `scp` an image to the host as `golden.img`).
### 4. `/opt/cm4-provisioning/dashboard/` (host only)
- **Working:** Directory exists (owner 1000:1000). Dashboard normally runs in the **LXC**, so this may be leftover or unused on the host.
- **New:** No dashboard dir on host. No action needed unless you run the dashboard on the host.
### 5. Extra files in `/var/lib/cm4-provisioning/` on working
- **Working:** `build_cloudinit_status.json`, `cloudinit_templates.json`, `first_boot_status.json` (written by dashboard/LXC when using cloud-init build).
- **New:** Not present yet. They appear when you use the dashboard from the LXC; not required for flash/backup.
## Checklist for new host (100.106.128.36)
- [x] `bootcode4.bin` in `/opt/usbboot/mass-storage-gadget64/` (copied from working)
- [x] Same udev rules, trigger, and systemd units
- [x] Same env. `EMMC_SIZE_BYTES` is optional; device detection is dynamic (any new block device after rpiboot is accepted for 8/16/32 GB CM4).
- [ ] Set `golden.img` for Deploy (copy image or use dashboard “Set as golden” from a backup)
- [x] `flash-emmc-on-connect.sh` is the updated version (longer wait, diagnostics)
## If flash still fails on new host
1. Check flash.log: `ssh root@100.106.128.36 'tail -80 /var/lib/cm4-provisioning/flash.log'` — the script now logs "Current block devices" and sizes when no device is found.
2. Ensure eMMC disable jumper is set and you use the USB slave port; unplug and replug.
3. Optional: increase udev/systemd delay (e.g. `ExecStartPre=/bin/sleep 10` in `cm4-flash.service`) if the device is slow to enumerate.

51
emmc-provisioning/docs/PROXMOX-LXC-DEPLOYMENT.md
View File

@@ -2,6 +2,8 @@
The auto-flash **runs on the Proxmox host** (where the USB device appears). The **LXC** holds the dashboard and shares the **golden image** directory with the host.
**New deployment:** For a clear step-by-step guide to deploy on a **new** Proxmox instance, see **[DEPLOY-NEW-PROXMOX.md](DEPLOY-NEW-PROXMOX.md)**.
## One-command deploy
From your repo, a single run deploys **all** host and LXC files (scripts, systemd units, udev, dashboard):
@@ -12,6 +14,8 @@ From your repo, a single run deploys **all** host and LXC files (scripts, system
Optional env: `CM4_BACKUPS_HOST_PATH=/path`, `DEPLOY_ROOTFS_STORAGE=local-lvm`, `DEPLOY_LXC_ROOT_PASSWORD=secret` (set root password in LXC and enable SSH), `DEPLOY_LXC_SSH_KEY=/path/to/pub` (default: `~/.ssh/id_ed25519.pub` or `id_rsa.pub` — copied to LXC root so you can `ssh root@<LXC-IP>`).
**Network (WAN / LAN):** `DEPLOY_LXC_WAN_BRIDGE=vmbr0` (default), `DEPLOY_LXC_WAN_IP=dhcp` (default or e.g. `192.168.1.10/24`), `DEPLOY_LXC_LAN_BRIDGE=vmbr1` (if set, add eth1 as LAN), `DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24` (LXC IP on LAN; default `10.20.50.1/24` when LAN bridge is set). Example: `DEPLOY_LXC_WAN_BRIDGE=vmbr0 DEPLOY_LXC_LAN_BRIDGE=vmbr1 DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24 ./scripts/deploy-to-proxmox.sh root@host`.
The script **finds the container by hostname `cm4-provisioning`** (any existing ID). If none exists, it **creates a new LXC with the next available ID**. So you can redeploy repeatedly without assuming a fixed ID like 201.
## What is deployed
@@ -206,11 +210,54 @@ Or copy `scripts/monitor-from-host.sh` to the host and run `./monitor-from-host.
2. **Unplug and replug the USB** – udev runs the trigger only when the device is *added*. Unplug the reTerminal USB (keep it in boot mode), then plug it back in. The trigger will run the script and rpiboot; when the eMMC is exposed, the portal shows "Device connected" with Backup/Deploy.
3. **If rpiboot fails** – Check on the host: `ssh root@10.130.60.224 'tail -30 /var/lib/cm4-provisioning/flash.log'` (rpiboot stderr is appended there). Try unplug/replug again. To see the exact rpiboot error: `ssh root@10.130.60.224 '/opt/usbboot/rpiboot -d /opt/usbboot/mass-storage-gadget64'` (device connected; Ctrl+C to stop). Run `scripts/monitor-from-host.sh` for a full snapshot.
3. **If rpiboot fails** ("rpiboot failed or no device connected"):
- **Check flash.log on the host** (rpiboot runs there; the log has the real error):
`ssh root@YOUR_PROXMOX_HOST 'tail -50 /var/lib/cm4-provisioning/flash.log'`
- **Quick diagnostic:**
`ssh root@YOUR_PROXMOX_HOST 'bash -s' < emmc-provisioning/scripts/check-usb-on-host.sh`
This shows whether the device is seen (lsusb), status, and last lines of flash.log.
- **Ensure device is in boot mode:** eMMC disable jumper set, use the **USB slave** port (not host port). Unplug and replug after setting jumper.
- **Run rpiboot manually** (device connected; Ctrl+C to stop):
`ssh root@YOUR_PROXMOX_HOST '/opt/usbboot/rpiboot -v -d /opt/usbboot/mass-storage-gadget64'`
The last line before exit is usually the error (e.g. "No device found", "Unable to open device").
- Run `scripts/monitor-from-host.sh root@YOUR_PROXMOX_HOST` for a full snapshot.
**Manual usbboot test** (to verify rpiboot and the CM4 without the full flash flow):
1. **On the Proxmox host** (where USB is connected), with the reTerminal in **boot mode** (eMMC disable jumper set, USB slave port connected):
```bash
/opt/usbboot/rpiboot -v -d /opt/usbboot/mass-storage-gadget64
```
2. You should see lines like: `Device located successfully`, `Loading: .../bootcode4.bin`, `Sending bootcode.bin`. When the device switches to mass storage, rpiboot exits and a new `/dev/sdX` may appear (check with `lsblk` in another terminal). Press **Ctrl+C** to stop rpiboot at any time.
3. **From your machine** (with device already connected to the host):
```bash
./emmc-provisioning/scripts/test-usbboot-on-host.sh root@YOUR_PROXMOX_HOST
```
Optional: run with a timeout so it doesn’t wait forever: `TIMEOUT=60 ./emmc-provisioning/scripts/test-usbboot-on-host.sh root@YOUR_PROXMOX_HOST`
**"libusb_bulk_transfer sent 0 bytes; returned -7" / "Failed to write correct length"** (device found, then transfer fails in a loop):
This is a known USB timing/controller issue with CM4 and rpiboot ([raspberrypi/usbboot#36](https://github.com/raspberrypi/usbboot/issues/36)). Try in order:
1. **Use a USB 2.0 port** on the Proxmox host (not USB 3.0). Many reports say USB 2.0 is more reliable for rpiboot.
2. **Avoid udev fighting with manual rpiboot:** Temporarily disable the provisioning trigger so only your manual rpiboot runs:
`ssh root@HOST 'mv /etc/cm4-provisioning/enabled /etc/cm4-provisioning/enabled.bak'`
Run rpiboot, then re-enable:
`ssh root@HOST 'mv /etc/cm4-provisioning/enabled.bak /etc/cm4-provisioning/enabled'`
3. **Try a different USB port and cable** on the host; unplug/replug and retry.
4. **Add a delay** so the device is ready before transfer:
`rpiboot -v -d /opt/usbboot/mass-storage-gadget64 -m 2000`
(`-m` is microseconds between device checks; 2000 = 2 ms.)
5. **Reboot the Proxmox host** and try again (USB controller state can get stuck).
6. **Find a USB 2.0 port:** On the host run `lsusb -t` — look for the BCM2711 device; the tree shows which controller (e.g. "xHCI" = USB 3, "ehci" = USB 2). Try a port that is under an **ehci** or **ohci** controller, or a black (non‑blue) physical port.
7. **Try a powered USB 2.0 hub** between host and reTerminal (some hosts work only through a hub).
8. If you have the **working host** (10.130.60.224), try the same reTerminal there; if it works there, the difference is host USB controller or port.
3b. **"No suitable block device after rpiboot"** – rpiboot ran but no new block device was seen. Detection is **dynamic**: any block device that appears after rpiboot (not present before) is used, so 8/16/32 GB CM4 work without setting eMMC size. **Check on host:** `tail -80 /var/lib/cm4-provisioning/flash.log` — at the end you’ll see "Current block devices" and each `/dev/sdX` with size. **Causes:** (1) Device didn’t switch to mass storage (try unplug/replug, keep eMMC disable jumper set). (2) udev slow — try again; the script waits up to 90s for the device.
4. **"No 'bootcode' files found in mass-storage-gadget64"** – Usually because `bootfiles.bin` is a **broken symlink** (e.g. `-> ../firmware/bootfiles.bin`) and that target doesn’t exist. **Fix on host:** run `scripts/fix-gadget-bootcode-on-host.sh` on the host (it removes the symlink and extracts `bootcode4.bin` from the installed rpiboot binary). From your machine: `ssh root@10.130.60.224 'bash -s' < scripts/fix-gadget-bootcode-on-host.sh`. **Alternative:** repopulate the gadget dir with `./scripts/populate-gadget-on-host.sh root@10.130.60.224`, or full reinstall with `./scripts/build-and-deploy-usbboot-to-host.sh root@10.130.60.224`. Then verify: `ls -la /opt/usbboot/mass-storage-gadget64/` (should list a real `bootcode4.bin` or `bootfiles.bin`, plus `boot.img`, `config.txt`).
4. **Clear stuck error in portal** – If the portal shows an old error (e.g. "Golden image not found" or "rpiboot failed"), click **Clear message** in the dashboard, or: `ssh root@10.130.60.224 "echo '{\"phase\":\"idle\",\"message\":\"Waiting for reTerminal in boot mode or network.\",\"progress\":null}' > /var/lib/cm4-provisioning/status.json"`. Then unplug/replug the device.
4. **Clear stuck error in portal** – If the portal shows an old error (e.g. "Golden image not found" or "rpiboot failed"), click **Clear message** in the dashboard, or: `ssh root@10.130.60.224 "echo '{\"phase\":\"idle\",\"message\":\"Waiting for Device in USB boot mode.\",\"progress\":null}' > /var/lib/cm4-provisioning/status.json"`. Then unplug/replug the device.
5. **"PiShrink not installed" when clicking Shrink/Compress** – Shrink and Compress run **on the host**, not in the LXC. Install PiShrink on the host: `ssh root@HOST 'bash -s' < emmc-provisioning/scripts/install-pishrink-on-host.sh`. Ensure deploy has been run so the host has `run-shrink-on-host.sh` and `cm4-shrink.path` enabled (`systemctl status cm4-shrink.path`).

89
emmc-provisioning/host/flash-emmc-on-connect.sh
View File

@@ -17,8 +17,8 @@ exec >> "$LOG_FILE" 2>&1
# Configuration - adjust paths and size for your setup
RPIBOOT_DIR="${RPIBOOT_DIR:-/opt/usbboot}"
GOLDEN_IMAGE="${GOLDEN_IMAGE:-/var/lib/cm4-provisioning/golden.img}"
# Expected eMMC size in bytes. reTerminal DM (CM4) has 32 GB eMMC (~31268536320 bytes).
EMMC_SIZE_BYTES="${EMMC_SIZE_BYTES:-$(( 32 * 1024 * 1024 * 1024 ))}"
# Expected eMMC size in bytes (optional). If set, used to prefer among multiple new devices; if unset, any new block device after rpiboot is accepted (works for 8/16/32 GB CM4).
EMMC_SIZE_BYTES="${EMMC_SIZE_BYTES:-}"
LOG_TAG="cm4-flash"
STATUS_FILE="${STATUS_FILE:-/var/lib/cm4-provisioning/status.json}"
LOG_FILE="${LOG_FILE:-/var/lib/cm4-provisioning/flash.log}"
@@ -51,7 +51,7 @@ trap 'rm -f "$LOCK_FILE" "$CURRENT_DEVICE_FILE" "$DEVICE_SOURCE_FILE" 2>/dev/nul
ENABLE_FILE="${ENABLE_FILE:-/etc/cm4-provisioning/enabled}"
if [[ -n "$ENABLE_FILE" && ! -f "$ENABLE_FILE" ]]; then
log "Skipping: $ENABLE_FILE not present"
write_status "idle" "Provisioning disabled (remove /etc/cm4-provisioning/enabled to enable)" "null" 2>/dev/null || true
write_status "idle" "Provisioning disabled (touch /etc/cm4-provisioning/enabled to enable)" "null" 2>/dev/null || true
exit 0
fi
@@ -90,8 +90,8 @@ if [[ -z "$RPIBOOT_GADGET" ]]; then
write_status "error" "rpiboot gadget missing" "null" "Copy mass-storage-gadget(64) to $RPIBOOT_DIR"
exit 1
fi
# rpiboot requires bootfiles.bin or one of bootcode*.bin in the gadget dir; empty dir causes "No 'bootcode' files found"
if [[ ! -f "$RPIBOOT_GADGET/bootfiles.bin" && ! -f "$RPIBOOT_GADGET/bootcode.bin" && ! -f "$RPIBOOT_GADGET/bootcode4.bin" && ! -f "$RPIBOOT_GADGET/bootcode5.bin" ]]; then
# rpiboot requires bootfiles.bin, bootcode*.bin, or boot.img in the gadget dir; empty dir causes "No 'bootcode' files found"
if [[ ! -f "$RPIBOOT_GADGET/bootfiles.bin" && ! -f "$RPIBOOT_GADGET/bootcode.bin" && ! -f "$RPIBOOT_GADGET/bootcode4.bin" && ! -f "$RPIBOOT_GADGET/bootcode5.bin" && ! -f "$RPIBOOT_GADGET/boot.img" ]]; then
log "rpiboot gadget dir has no boot files: $RPIBOOT_GADGET (reinstall usbboot)"
write_status "error" "rpiboot gadget empty" "null" "No boot files in $RPIBOOT_GADGET. On the host run: fix-gadget-bootcode-on-host.sh (or from your machine: ssh root@HOST 'bash -s' < scripts/fix-gadget-bootcode-on-host.sh). See docs troubleshooting."
exit 1
@@ -104,51 +104,86 @@ write_status "rpiboot" "Connecting to CM4 in boot mode…" "0"
# Block devices before rpiboot (so we can detect new one after)
before_devs=$(lsblk -nd -o NAME 2>/dev/null | sort)
log "Starting rpiboot to expose CM4 eMMC as mass storage..."
# Run rpiboot with 90s timeout so we don't hang if it doesn't exit cleanly when device switches to mass storage
log "Starting rpiboot to expose CM4 eMMC as mass storage (gadget: $RPIBOOT_GADGET)..."
# Run rpiboot with 180s timeout so device has time to receive bootcode and switch to mass storage; -v for verbose
rpiboot_exit=0
timeout 90 "$RPIBOOT_BIN" -d "$RPIBOOT_GADGET" || rpiboot_exit=$?
timeout 180 "$RPIBOOT_BIN" -v -d "$RPIBOOT_GADGET" || rpiboot_exit=$?
# timeout returns 124 if killed by timeout; 0 or other if rpiboot exited on its own
if [[ "$rpiboot_exit" -eq 124 ]]; then
log "rpiboot timed out after 90s (device may have switched to mass storage)"
log "rpiboot timed out after 180s (device may have switched to mass storage)"
elif [[ "$rpiboot_exit" -ne 0 ]]; then
log "rpiboot exited with code $rpiboot_exit"
log "Common causes: (1) No device in USB boot mode — set eMMC disable jumper and use USB slave port. (2) Wrong USB port or cable. (3) Run on host: tail -50 /var/lib/cm4-provisioning/flash.log"
write_status "error" "rpiboot failed" "null" "rpiboot failed or no device connected. Check flash.log on host. Try unplug/replug USB."
exit 1
fi
echo "[$(date -Iseconds)] rpiboot finished (exit=$rpiboot_exit); starting device scan"
log "rpiboot completed; waiting for block device..."
log "rpiboot completed; waiting for new block device (any size — 8/16/32 GB CM4 supported)..."
write_status "rpiboot" "rpiboot done, waiting for block device…" "10"
# rpiboot exits when device switches to mass storage; udev may need several seconds to create /dev/sdX
# Poll for new block device for up to 30s (device switch can be slow)
# rpiboot exits when device switches to mass storage; udev may need many seconds to create /dev/sdX
# Dynamic detection: accept any NEW block device (not present before rpiboot). No fixed eMMC size required.
# When rpiboot timed out (124), device may still be switching — wait longer in that case
max_wait=60
[[ "$rpiboot_exit" -eq 124 ]] && max_wait=90
target_dev=""
for wait_sec in $(seq 2 2 10) $(seq 12 2 30); do
new_devs=""
for wait_sec in $(seq 2 2 20) $(seq 22 2 $max_wait); do
sleep 2
new_devs=""
for dev in /dev/sd[a-z] /dev/sd[a-z][a-z]; do
[[ -b "$dev" ]] || continue
[[ "$dev" =~ [0-9]$ ]] && continue
# Only consider devices that appeared after rpiboot (the CM4 eMMC)
if [[ "$before_devs" != *"${dev#/dev/}"* ]]; then
size=$(blockdev --getsize64 "$dev" 2>/dev/null || true)
if [[ -n "$size" ]]; then
if (( size >= EMMC_SIZE_BYTES * 95 / 100 && size <= EMMC_SIZE_BYTES * 105 / 100 )); then
target_dev=$dev
break 2
fi
if [[ -z "$target_dev" && "$before_devs" != *"${dev#/dev/}"* ]]; then
target_dev=$dev
fi
[[ -n "$size" ]] && new_devs="$new_devs $dev:$size"
fi
done
[[ -n "$target_dev" ]] && break
new_devs="${new_devs# }"
if [[ -n "$new_devs" ]]; then
# One new device: use it (dynamic — works for any eMMC size)
# Multiple new devices: prefer one matching EMMC_SIZE_BYTES if set, else largest
if [[ "$new_devs" != *" "* ]]; then
target_dev="${new_devs%%:*}"
break
else
best_dev="" best_size=0 best_delta=999999999999
for entry in $new_devs; do
dev="${entry%%:*}"
size="${entry##*:}"
if [[ -z "$EMMC_SIZE_BYTES" || "$EMMC_SIZE_BYTES" -eq 0 ]]; then
# No size hint: take largest new device
[[ "$size" -gt "$best_size" ]] && { best_dev="$dev"; best_size="$size"; }
else
delta=$(( size - EMMC_SIZE_BYTES )); [[ "$delta" -lt 0 ]] && delta=$(( -delta ))
[[ "$delta" -lt "$best_delta" ]] && { best_dev="$dev"; best_delta="$delta"; }
fi
done
[[ -n "$best_dev" ]] && target_dev="$best_dev" && break
fi
fi
log "Waiting for block device... ${wait_sec}s"
write_status "rpiboot" "Waiting for eMMC block device… (${wait_sec}s)" "10"
done
log "Device scan complete. before_devs=[$before_devs] target_dev=[$target_dev]"
if [[ -n "$target_dev" ]]; then
detected_size=$(blockdev --getsize64 "$target_dev" 2>/dev/null || true)
log "Using $target_dev (size=${detected_size:-?} bytes, $(( ${detected_size:-0} / 1024 / 1024 / 1024 )) GB)"
fi
if [[ -z "$target_dev" ]]; then
log "No suitable block device found after rpiboot (expected ~${EMMC_SIZE_BYTES} bytes)"
write_status "error" "No eMMC device found" "null" "No suitable block device after rpiboot"
log "No new block device found after rpiboot"
log "Current block devices (for debugging):"
lsblk -nd -o NAME,SIZE,TYPE 2>/dev/null | while read -r line; do log " $line"; done
for d in /dev/sd[a-z] /dev/sd[a-z][a-z]; do
[[ -b "$d" ]] || continue
[[ "$d" =~ [0-9]$ ]] && continue
s=$(blockdev --getsize64 "$d" 2>/dev/null || true)
log " $d size=$s ($((${s:-0} / 1024 / 1024 / 1024)) GB)"
done
write_status "error" "No eMMC device found" "null" "No suitable block device after rpiboot. Check flash.log on host; unplug/replug and ensure eMMC disable jumper is set."
exit 1
fi
@@ -206,7 +241,7 @@ for (( i = 0; i < WAIT_TIMEOUT; i += 2 )); do
fi
log "Backup complete: $BACKUPS_DIR/$final_name"
write_status "done" "Backup complete: $final_name" "100"
( sleep 90 && write_status "idle" "Waiting for reTerminal in boot mode or network." "null" ) &
( sleep 90 && write_status "idle" "Waiting for Device in USB boot mode." "null" ) &
else
write_status "error" "Backup failed" "null" "dd failed"
fi
@@ -230,7 +265,7 @@ for (( i = 0; i < WAIT_TIMEOUT; i += 2 )); do
log "Flash complete. Remove eMMC disable jumper and power cycle the reTerminal."
write_status "done" "Flash complete. Remove eMMC disable jumper and power cycle the reTerminal." "100"
# Auto-reset status to idle after 90s so dashboard does not stay on this message (dashboard also auto-clears after 60s when open)
( sleep 90 && write_status "idle" "Waiting for reTerminal in boot mode or network." "null" ) &
( sleep 90 && write_status "idle" "Waiting for Device in USB boot mode." "null" ) &
else
write_status "error" "Flash failed" "null" "dd failed"
fi
@@ -265,7 +300,7 @@ for (( i = 0; i < WAIT_TIMEOUT; i += 2 )); do
log "EEPROM update written. Remove eMMC disable jumper and power cycle to apply."
write_status "done" "EEPROM update written to boot partition. Remove eMMC disable jumper and power cycle the reTerminal to apply." "100"
rm -f "$EEPROM_UPD" "$EEPROM_SIG"
( sleep 90 && write_status "idle" "Waiting for reTerminal in boot mode or network." "null" ) &
( sleep 90 && write_status "idle" "Waiting for Device in USB boot mode." "null" ) &
else
write_status "error" "EEPROM update failed" "null" "Failed to copy pieeprom.upd/sig"
fi

17
emmc-provisioning/lxc/70-cm4-extra-lan Normal file
View File

@@ -0,0 +1,17 @@
# Extra LAN IPs on eth1 and VLAN 40 on eth1.
# Primary eth1 address is set by Proxmox/deploy (used by dnsmasq DHCP).
# Installed by setup-network-boot-on-lxc.sh; ensure /etc/network/interfaces
# includes: source-directory /etc/network/interfaces.d
# Secondary addresses on eth1 (192.168.30.1, 192.168.127.1)
iface eth1 inet static
address 192.168.30.1/24
iface eth1 inet static
address 192.168.127.1/24
# VLAN 40 on eth1 — 192.168.0.0/24 (gateway 192.168.0.1)
# Requires: apt install vlan; host bridge must pass VLAN 40 if using tagged uplink
auto eth1.40
iface eth1.40 inet static
address 192.168.0.1/24
vlan-raw-device eth1

8
emmc-provisioning/lxc/README.md
View File

@@ -2,10 +2,14 @@
Config files for the **provisioning LXC** when using **eth1** as a provisioning LAN (DHCP + TFTP for network boot, NAT for internet).
**LAN subnet:** When you deploy with `DEPLOY_LXC_LAN_SUBNET` (e.g. `10.100.1.1/24`), the deploy script writes `/opt/cm4-provisioning/lan-subnet.conf` inside the LXC with `LAN_GW`, `LAN_CIDR`, and `DHCP_RANGE_START`/`DHCP_RANGE_END`. The setup script and toggle script read this file so dnsmasq, NAT, and PXE options all use the same subnet. If the file is missing, defaults are `10.20.50.1/24` and `10.20.50.100`–`10.20.50.200`.
| File | Purpose |
|------|--------|
| **dnsmasq-network-boot.conf** | dnsmasq: DHCP + TFTP on eth1 only. Copied to `/etc/dnsmasq.d/` by `scripts/setup-network-boot-on-lxc.sh`. |
| **nft-nat-lan.conf** | nftables NAT so 10.20.50.0/24 uses eth0 for internet. Applied by the setup script to `/etc/nftables.d/nat-lan.conf`. |
| **dnsmasq-network-boot.conf** | Template: dnsmasq DHCP + TFTP on eth1. Setup script writes `/etc/dnsmasq.d/network-boot.conf` using values from `lan-subnet.conf`. |
| **nft-nat-lan.conf** | Template: nftables NAT for LAN→WAN (primary + extra subnets + VLAN 40). Setup script writes `/etc/nftables.d/nat-lan.conf`. |
| **70-cm4-extra-lan** | Extra LAN IPs on eth1 (192.168.30.1, 192.168.127.1) and VLAN eth1.40 (192.168.0.1/24). Installed to `/etc/network/interfaces.d/` by setup script. |
| **toggle-network-boot-dhcp.sh** | Enable/disable PXE (TFTP) on the LXC; copied to `/opt/cm4-provisioning/` by setup script. |
Setup is done by running (from your machine):

1
emmc-provisioning/lxc/dnsmasq-network-boot-pxe.conf
View File

@@ -1,5 +1,6 @@
# PXE/network-boot DHCP options (option 66 = next-server, 67 = boot file).
# When this file is present, dnsmasq advertises network boot; when removed, devices get DHCP only and boot from local storage.
# Toggle with: /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable|disable
# Template; toggle script writes the real next-server from /opt/cm4-provisioning/lan-subnet.conf (LAN_GW).
dhcp-option=66,10.20.50.1
dhcp-option=67,start4cd.elf

12
emmc-provisioning/lxc/dnsmasq-network-boot.conf
View File

@@ -1,14 +1,19 @@
# dnsmasq: DHCP + TFTP on eth1 only (provisioning LAN).
# dnsmasq: DHCP + TFTP + DNS on eth1 only (provisioning LAN).
# Install to /etc/dnsmasq.d/network-boot.conf on the LXC.
# Restrict to eth1 so we don't interfere with host/other DHCP.
# When using setup-network-boot-on-lxc.sh, the actual subnet, DHCP range, and
# file.server address come from /opt/cm4-provisioning/lan-subnet.conf (written by deploy-to-proxmox.sh).
# Listen only on eth1 (provisioning LAN)
interface=eth1
bind-interfaces
# DHCP range for devices on eth1 (adjust if you use a different subnet)
# DHCP range for devices on eth1 (template; setup script uses lan-subnet.conf)
dhcp-range=10.20.50.100,10.20.50.200,12h
# DNS: file.server -> eth1 IP (LAN_GW) so scripts can use http://file.server/... (setup script writes this)
# address=/file.server/10.20.50.1
# TFTP for Raspberry Pi / CM4 network boot
enable-tftp
tftp-root=/srv/tftpboot
@@ -17,6 +22,3 @@ tftp-root=/srv/tftpboot
# Logging (optional; disable in production if too noisy)
log-dhcp
log-queries
# Do not use /etc/resolv.conf or act as DNS if you only want DHCP+TFTP
port=0

8
emmc-provisioning/lxc/nft-nat-lan.conf
View File

@@ -1,10 +1,14 @@
# nftables: NAT for LAN (eth1) so clients use WAN (eth0) for internet.
# nftables: NAT for LAN (eth1 + extra IPs + eth1.40) so clients use WAN (eth0) for internet.
# Load with: nft -f /etc/nftables.d/nat-lan.conf
# Or use the inline rules in setup-network-boot-on-lxc.sh (no separate file dependency).
# When using setup-network-boot-on-lxc.sh, the primary subnet is from lan-subnet.conf (LAN_CIDR).
# Extra subnets: 192.168.30.0/24, 192.168.127.0/24 (eth1), 192.168.0.0/24 (eth1.40 VLAN).
table ip nat {
chain postrouting {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 10.20.50.0/24 oifname "eth0" masquerade
ip saddr 192.168.30.0/24 oifname "eth0" masquerade
ip saddr 192.168.127.0/24 oifname "eth0" masquerade
ip saddr 192.168.0.0/24 oifname "eth0" masquerade
}
}

16
emmc-provisioning/lxc/toggle-network-boot-dhcp.sh
View File

@@ -3,11 +3,19 @@
# When disabled, TFTP is stopped and no boot server is advertised; DHCP still runs.
# Usage: toggle-network-boot-dhcp.sh enable | disable | status
# Run as root. Install to /opt/cm4-provisioning/toggle-network-boot-dhcp.sh
# LAN gateway for TFTP/next-server is read from /opt/cm4-provisioning/lan-subnet.conf (written by deploy-to-proxmox.sh).
set -e
PXE_CONF="/etc/dnsmasq.d/network-boot-pxe.conf"
MAIN_CONF="/etc/dnsmasq.d/network-boot.conf"
LAN_CONF="/opt/cm4-provisioning/lan-subnet.conf"
if [[ -f "$LAN_CONF" ]]; then
source "$LAN_CONF"
else
LAN_GW="10.20.50.1"
fi
# Remove enable-tftp / tftp-root from main config if present (legacy; these belong in PXE conf)
cleanup_main_conf() {
if [ -f "$MAIN_CONF" ] && grep -q 'enable-tftp\|tftp-root' "$MAIN_CONF" 2>/dev/null; then
@@ -18,19 +26,19 @@ cleanup_main_conf() {
case "${1:-}" in
enable)
cleanup_main_conf
cat > "$PXE_CONF" << 'EOF'
cat > "$PXE_CONF" << EOF
# PXE/network boot ENABLED - managed by toggle-network-boot-dhcp.sh
# TFTP server (only active when network boot is enabled)
enable-tftp
tftp-root=/srv/tftpboot
# BOOTP fields (siaddr = TFTP server, filename = boot file)
dhcp-boot=start4cd.elf,,10.20.50.1
dhcp-boot=start4cd.elf,,${LAN_GW}
# DHCP options 66/67 (some PXE clients prefer these)
dhcp-option=66,10.20.50.1
dhcp-option=66,${LAN_GW}
dhcp-option=67,start4cd.elf
EOF
systemctl restart dnsmasq 2>/dev/null || service dnsmasq restart 2>/dev/null || true
echo "Network boot enabled."
echo "Network boot enabled (TFTP next-server: $LAN_GW)."
;;
disable)
cleanup_main_conf

71
emmc-provisioning/scripts/deploy-to-proxmox.sh
View File

@@ -23,6 +23,13 @@
# DEPLOY_LXC_ROOT_PASSWORD=secret — set root password in LXC and enable SSH
# DEPLOY_LXC_SSH_KEY=/path/to/pub — copy this key to LXC root (default: ~/.ssh/id_ed25519.pub or id_rsa.pub)
# DEPLOY_LOG=1 — also log to deploy-YYYYMMDD-HHMMSS.log
# DEPLOY_LXC_WAN_BRIDGE=vmbr0 — Proxmox bridge for WAN (eth0); default vmbr0
# DEPLOY_LXC_WAN_IP=dhcp — WAN address: dhcp (default) or static e.g. 192.168.1.10/24
# DEPLOY_LXC_LAN_BRIDGE=vmbr1 — If set, add eth1 as LAN on this bridge (e.g. provisioning / network-boot)
# DEPLOY_LXC_LAN_SUBNET=10.20.50.1/24 — LXC IP on LAN (gateway); used only if DEPLOY_LXC_LAN_BRIDGE is set; default 10.20.50.1/24
# DEPLOY_EMMC_SIZE_GB=32 — optional: eMMC size in GB (used only when multiple new devices appear; default 32). Detection is dynamic — single new device is used regardless of size.
#
# Legacy: DEPLOY_LXC_NET1="name=eth1,bridge=vmbr1,ip=10.20.50.1/24" still works; overridden by DEPLOY_LXC_LAN_BRIDGE + DEPLOY_LXC_LAN_SUBNET if both are set.
#
# Requires: ssh key access to root@<host>. For full install (usbboot, PiShrink), host needs internet.
@@ -144,8 +151,10 @@ rsync -a "$REPO_DIR/" "$PROXMOX:/tmp/emmc-provisioning-deploy/" --exclude='.git'
log "[4/5] Running remote install (host + LXC) ..."
# Pass optional LXC SSH vars (base64) and selected storage
ssh "$PROXMOX" "ROOTFS_STORAGE='$ROOTFS_STORAGE' CM4_BACKUPS_HOST_PATH='${CM4_BACKUPS_HOST_PATH:-}' DEPLOY_SSH_KEY_B64='${DEPLOY_SSH_KEY_B64:-}' DEPLOY_LXC_PWD_B64='${DEPLOY_LXC_PWD_B64:-}'" bash -s << 'REMOTE'
# Pass optional LXC SSH vars (base64), selected storage, network (WAN/LAN), and eMMC size
EMMC_GB="${DEPLOY_EMMC_SIZE_GB:-32}"
EMMC_SIZE_BYTES=$(( EMMC_GB * 1024 * 1024 * 1024 ))
ssh "$PROXMOX" "ROOTFS_STORAGE='$ROOTFS_STORAGE' CM4_BACKUPS_HOST_PATH='${CM4_BACKUPS_HOST_PATH:-}' DEPLOY_SSH_KEY_B64='${DEPLOY_SSH_KEY_B64:-}' DEPLOY_LXC_PWD_B64='${DEPLOY_LXC_PWD_B64:-}' DEPLOY_LXC_WAN_BRIDGE='${DEPLOY_LXC_WAN_BRIDGE:-}' DEPLOY_LXC_WAN_IP='${DEPLOY_LXC_WAN_IP:-}' DEPLOY_LXC_LAN_BRIDGE='${DEPLOY_LXC_LAN_BRIDGE:-}' DEPLOY_LXC_LAN_SUBNET='${DEPLOY_LXC_LAN_SUBNET:-}' DEPLOY_LXC_NET1='${DEPLOY_LXC_NET1:-}' EMMC_SIZE_BYTES='$EMMC_SIZE_BYTES' EMMC_GB='$EMMC_GB'" bash -s << 'REMOTE'
set -e
DEPLOY=/tmp/emmc-provisioning-deploy
ROOTFS_STORAGE="${ROOTFS_STORAGE:?ROOTFS_STORAGE not set}"
@@ -171,6 +180,7 @@ for id in $(pct list 2>/dev/null | awk 'NR>1 {print $1}'); do
done
if [[ -n "$CTID" ]]; then
log "Found existing LXC $CTID (hostname: $LXC_HOSTNAME)."
pct set "$CTID" -nameserver 8.8.8.8
else
MAX_ID=$(pct list 2>/dev/null | awk 'NR>1 {print $1}' | sort -n | tail -1)
[[ -z "$MAX_ID" ]] && MAX_ID=0
@@ -185,18 +195,29 @@ else
fi
[[ -z "$DEBIAN12_TMPL" ]] && { log "Error: no Debian 12 template found"; exit 1; }
TMPL_NAME=$(basename "$DEBIAN12_TMPL")
# Optional: add eth1 for network-boot LAN (DHCP+TFTP). Set DEPLOY_LXC_NET1 e.g. "name=eth1,bridge=vmbr1,ip=10.20.50.1/24"
# WAN (eth0): bridge and IP from env; default vmbr0 + dhcp
WAN_BRIDGE="${DEPLOY_LXC_WAN_BRIDGE:-vmbr0}"
WAN_IP="${DEPLOY_LXC_WAN_IP:-dhcp}"
# LAN (eth1): optional; use DEPLOY_LXC_LAN_BRIDGE + DEPLOY_LXC_LAN_SUBNET, or legacy DEPLOY_LXC_NET1
NET1_OPT=""
if [[ -n "${DEPLOY_LXC_NET1:-}" ]]; then
if [[ -n "${DEPLOY_LXC_LAN_BRIDGE:-}" ]]; then
LAN_SUBNET="${DEPLOY_LXC_LAN_SUBNET:-10.20.50.1/24}"
NET1_OPT="--net1 name=eth1,bridge=${DEPLOY_LXC_LAN_BRIDGE},ip=${LAN_SUBNET}"
log "LXC network: eth0 WAN bridge=$WAN_BRIDGE ip=$WAN_IP; eth1 LAN bridge=$DEPLOY_LXC_LAN_BRIDGE ip=$LAN_SUBNET"
elif [[ -n "${DEPLOY_LXC_NET1:-}" ]]; then
NET1_OPT="--net1 $DEPLOY_LXC_NET1"
log "LXC network: eth0 WAN bridge=$WAN_BRIDGE ip=$WAN_IP; eth1 from DEPLOY_LXC_NET1"
else
log "LXC network: eth0 WAN bridge=$WAN_BRIDGE ip=$WAN_IP (no LAN interface)"
fi
pct create "$CTID" "local:vztmpl/${TMPL_NAME}" \
--hostname "$LXC_HOSTNAME" --memory 1024 --swap 0 --cores 1 \
--rootfs "${ROOTFS_STORAGE}:8" --net0 name=eth0,bridge=vmbr0,ip=dhcp $NET1_OPT \
--rootfs "${ROOTFS_STORAGE}:8" --net0 name=eth0,bridge="$WAN_BRIDGE",ip="$WAN_IP" $NET1_OPT \
--unprivileged 0 --features nesting=1 -tag cm4-provisioning
pct set "$CTID" -nameserver 8.8.8.8
mkdir -p /var/lib/cm4-provisioning
pct set "$CTID" -mp0 /var/lib/cm4-provisioning,mp=/var/lib/cm4-provisioning
log "LXC $CTID created and mount configured."
log "LXC $CTID created and mount configured (DNS 8.8.8.8)."
fi
# Optional: bind-mount host directory for backup images (skip if already mounted with same path)
@@ -249,11 +270,11 @@ cp "$DEPLOY/host/89-cm4-boot-mode-permissions.rules" /etc/udev/rules.d/ 2>/dev/n
cp "$DEPLOY/host/90-cm4-boot-mode.rules" /etc/udev/rules.d/
udevadm control --reload-rules 2>/dev/null || true
log "Host: env and dirs ..."
cat > /opt/cm4-provisioning/env << 'ENV'
log "Host: env and dirs (EMMC ${EMMC_GB:-32}GB = $EMMC_SIZE_BYTES bytes) ..."
cat > /opt/cm4-provisioning/env << ENV
GOLDEN_IMAGE=/var/lib/cm4-provisioning/golden.img
RPIBOOT_DIR=/opt/usbboot
EMMC_SIZE_BYTES=8589934592
EMMC_SIZE_BYTES=${EMMC_SIZE_BYTES:-34359738368}
ENV
[[ -n "$BACKUPS_HOST_PATH" ]] && echo "BACKUPS_DIR=$BACKUPS_HOST_PATH" >> /opt/cm4-provisioning/env
touch /etc/cm4-provisioning/enabled
@@ -302,6 +323,25 @@ fi
log "Starting LXC $CTID if stopped ..."
pct start "$CTID" 2>/dev/null || true
# --- LXC: write lan-subnet.conf when LAN bridge/subnet is set (so dnsmasq/NAT/toggle use same subnet) ---
LAN_SUBNET_FOR_CONF="${DEPLOY_LXC_LAN_SUBNET:-}"
[[ -z "$LAN_SUBNET_FOR_CONF" && -n "${DEPLOY_LXC_LAN_BRIDGE:-}" ]] && LAN_SUBNET_FOR_CONF="10.20.50.1/24"
if [[ -n "$LAN_SUBNET_FOR_CONF" ]]; then
if [[ "$LAN_SUBNET_FOR_CONF" =~ ^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/([0-9]+)$ ]]; then
LAN_GW="${BASH_REMATCH[1]}"
PREFIX="${BASH_REMATCH[2]}"
BASE_3="${LAN_GW%.*}"
LAN_CIDR="${BASE_3}.0/${PREFIX}"
DHCP_RANGE_START="${BASE_3}.100"
DHCP_RANGE_END="${BASE_3}.200"
pct exec "$CTID" -- bash -c "mkdir -p /opt/cm4-provisioning && echo 'LAN_GW=$LAN_GW' > /opt/cm4-provisioning/lan-subnet.conf && echo 'LAN_CIDR=$LAN_CIDR' >> /opt/cm4-provisioning/lan-subnet.conf && echo 'DHCP_RANGE_START=$DHCP_RANGE_START' >> /opt/cm4-provisioning/lan-subnet.conf && echo 'DHCP_RANGE_END=$DHCP_RANGE_END' >> /opt/cm4-provisioning/lan-subnet.conf"
echo "$LAN_GW" > "$DEPLOY/lxc_lan_ip.txt"
log "LXC: wrote /opt/cm4-provisioning/lan-subnet.conf (LAN_GW=$LAN_GW); dashboard will be reachable on LAN at http://${LAN_GW}:5000"
else
log "Warning: DEPLOY_LXC_LAN_SUBNET=$LAN_SUBNET_FOR_CONF not in form A.B.C.D/PREFIX; skipping lan-subnet.conf"
fi
fi
# --- LXC: flash scripts (for reference; actual flash runs on host) ---
log "LXC: installing flash scripts ..."
pct exec "$CTID" -- mkdir -p /opt/cm4-provisioning /etc/cm4-provisioning
@@ -309,7 +349,7 @@ pct push "$CTID" "$DEPLOY/host/flash-emmc-on-connect.sh" /opt/cm4-provisioning/f
pct exec "$CTID" -- chmod +x /opt/cm4-provisioning/flash-emmc-on-connect.sh
pct push "$CTID" "$DEPLOY/host/cm4-flash-trigger.sh" /usr/local/bin/cm4-flash-trigger.sh
pct exec "$CTID" -- chmod +x /usr/local/bin/cm4-flash-trigger.sh
pct exec "$CTID" -- bash -c 'echo -e "GOLDEN_IMAGE=/var/lib/cm4-provisioning/golden.img\nRPIBOOT_DIR=/opt/usbboot\nEMMC_SIZE_BYTES=8589934592" > /opt/cm4-provisioning/env'
pct exec "$CTID" -- bash -c "echo -e 'GOLDEN_IMAGE=/var/lib/cm4-provisioning/golden.img\nRPIBOOT_DIR=/opt/usbboot\nEMMC_SIZE_BYTES=$EMMC_SIZE_BYTES' > /opt/cm4-provisioning/env"
# --- LXC: dashboard (all files) ---
log "LXC: installing dashboard ..."
@@ -318,6 +358,8 @@ pct push "$CTID" "$DEPLOY/dashboard/app.py" /opt/cm4-provisioning/dashboard/app.
pct push "$CTID" "$DEPLOY/dashboard/templates/home.html" /opt/cm4-provisioning/dashboard/templates/home.html
pct push "$CTID" "$DEPLOY/dashboard/templates/login.html" /opt/cm4-provisioning/dashboard/templates/login.html
pct push "$CTID" "$DEPLOY/dashboard/templates/admin.html" /opt/cm4-provisioning/dashboard/templates/admin.html
pct push "$CTID" "$DEPLOY/dashboard/templates/portal_files.html" /opt/cm4-provisioning/dashboard/templates/portal_files.html
pct push "$CTID" "$DEPLOY/dashboard/templates/cloudinit_build.html" /opt/cm4-provisioning/dashboard/templates/cloudinit_build.html
pct push "$CTID" "$DEPLOY/dashboard/cm4-dashboard.service" /opt/cm4-provisioning/dashboard/cm4-dashboard.service
# Dashboard secret for sessions (create once so logins persist across restarts)
pct exec "$CTID" -- bash -c '[[ -f /opt/cm4-provisioning/dashboard.env ]] || echo "CM4_DASHBOARD_SECRET_KEY=$(openssl rand -hex 24 2>/dev/null || head -c 24 /dev/urandom | xxd -p)" > /opt/cm4-provisioning/dashboard.env'
@@ -363,14 +405,16 @@ log "Deploy done on remote. LXC ID: $CTID"
# Heredoc terminator (must be at column 1, no leading space/tab)
REMOTE
# Read LXC IP written by remote (container hostname -I)
# Read LXC IP and optional LAN IP written by remote
LXC_IP=$(ssh "$PROXMOX" "cat /tmp/emmc-provisioning-deploy/lxc_ip.txt 2>/dev/null" | tr -d '\n\r')
LXC_LAN_IP=$(ssh "$PROXMOX" "cat /tmp/emmc-provisioning-deploy/lxc_lan_ip.txt 2>/dev/null" | tr -d '\n\r')
log "[5/5] Deploy finished."
echo ""
echo "=== Deploy complete ==="
echo "Host and LXC are fully set up: usbboot (rpiboot), PiShrink, dashboard, systemd, udev."
[[ -n "$LXC_IP" ]] && echo " LXC IP: $LXC_IP"
[[ -n "$LXC_IP" ]] && echo " LXC IP (WAN): $LXC_IP"
[[ -n "$LXC_LAN_IP" ]] && echo " LXC IP (LAN): $LXC_LAN_IP"
echo ""
echo "--- Only remaining step (manual) ---"
echo " Add a golden image for Deploy (writing image to device):"
@@ -379,7 +423,8 @@ echo " • Or copy your image: scp your-image.img $PROXMOX:/var/lib/cm4-provi
echo " Backup (read from device) works without golden.img."
echo ""
echo "--- You have ---"
echo " - Dashboard: http://${LXC_IP:-<LXC-IP>}:5000"
echo " - Dashboard (WAN): http://${LXC_IP:-<LXC-IP>}:5000"
[[ -n "$LXC_LAN_IP" ]] && echo " - Dashboard (LAN): http://${LXC_LAN_IP}:5000 (use from devices on provisioning LAN)"
[[ -n "${DEPLOY_LXC_ROOT_PASSWORD:-}" || -n "${DEPLOY_SSH_KEY_B64:-}" ]] && [[ -n "$LXC_IP" ]] && echo " - LXC SSH: ssh root@$LXC_IP (password and/or key were set)"
[[ -n "${DEPLOY_LXC_ROOT_PASSWORD:-}" || -n "${DEPLOY_SSH_KEY_B64:-}" ]] && [[ -z "$LXC_IP" ]] && echo " - LXC SSH: ssh root@<LXC-IP> (password and/or key were set)"
[[ -n "${CM4_BACKUPS_HOST_PATH:-}" ]] && echo " - Backups on host: $CM4_BACKUPS_HOST_PATH"

130
emmc-provisioning/scripts/edit-cloudinit-on-image.sh Executable file
View File

@@ -0,0 +1,130 @@
#!/usr/bin/env bash
# Mount a Raspberry Pi OS .img or .img.xz, edit cloud-init NoCloud files on the boot
# partition, then unmount and (if .img.xz) recompress. Requires sudo for loop/mount.
#
# Usage:
# ./edit-cloudinit-on-image.sh <path-to-image.img.xz>
# ./edit-cloudinit-on-image.sh <path-to-image.img>
#
# Options:
# --no-recompress If image was .img.xz, leave decompressed .img and do not overwrite
# --replace-with-repo Copy user-data, meta-data, network-config from repo before editing
#
# Example:
# ./edit-cloudinit-on-image.sh /path/to/gnss-bootstrap-20260223-215010.img.xz
#
# Backup the image before running if you want to keep the original.
set -e
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
CLOUDINIT_SRC="$REPO_ROOT/emmc-provisioning/cloud-init"
NO_RECOMPRESS=""
REPLACE_WITH_REPO=""
while [[ $# -gt 0 ]]; do
case "$1" in
--no-recompress) NO_RECOMPRESS=1; shift ;;
--replace-with-repo) REPLACE_WITH_REPO=1; shift ;;
-*) echo "Unknown option: $1"; exit 1 ;;
*) break ;;
esac
done
IMAGE_IN="$1"
[[ -n "$IMAGE_IN" && -f "$IMAGE_IN" ]] || {
echo "Usage: $0 [--no-recompress] [--replace-with-repo] <path-to-image.img.xz|.img>"
echo "Example: $0 gnss-bootstrap-20260223-215010.img.xz"
exit 1
}
IMAGE_IN="$(realpath "$IMAGE_IN")"
WORK_DIR=""
IMG_FILE=""
ORIGINAL_XZ=""
cleanup() {
if [[ -n "$MNT" && -d "$MNT" ]]; then
sudo umount "$MNT" 2>/dev/null || true
fi
if [[ -n "$LOOP" && -b "$LOOP" ]]; then
sudo losetup -d "$LOOP" 2>/dev/null || true
fi
if [[ -n "$WORK_DIR" && -d "$WORK_DIR" ]]; then
if [[ -n "$NO_RECOMPRESS" && -n "$ORIGINAL_XZ" ]]; then
echo "Work dir left at: $WORK_DIR"
else
rm -rf "$WORK_DIR"
fi
fi
}
trap cleanup EXIT
if [[ "$IMAGE_IN" == *.img.xz ]]; then
ORIGINAL_XZ="$IMAGE_IN"
echo "Decompressing $(basename "$ORIGINAL_XZ") (this may take a minute and needs ~3GB free)…"
WORK_DIR=$(mktemp -d -p "${TMPDIR:-/tmp}" edit-cloudinit.XXXXXX)
IMG_FILE="$WORK_DIR/image.img"
xz -T 0 -d -k -f -c "$ORIGINAL_XZ" > "$IMG_FILE"
else
WORK_DIR=$(mktemp -d -p "${TMPDIR:-/tmp}" edit-cloudinit.XXXXXX)
IMG_FILE="$IMAGE_IN"
fi
echo "Attaching image and mounting boot partition…"
LOOP=$(sudo losetup -f --show -P "$IMG_FILE")
# Force kernel to scan partition table so /dev/loopNp1, p2 etc. appear
sudo partprobe "$LOOP" 2>/dev/null || true
sleep 0.5
boot_part="${LOOP}p1"
[[ -b "$boot_part" ]] || boot_part="${LOOP}p2"
[[ -b "$boot_part" ]] || {
echo "Boot partition not found on image. Partitions on image:"
ls -la "${LOOP}"p* 2>/dev/null || true
sudo fdisk -l "$IMG_FILE" 2>/dev/null || true
exit 1
}
MNT="$WORK_DIR/mnt"
mkdir -p "$MNT"
sudo mount "$boot_part" "$MNT"
if [[ -n "$REPLACE_WITH_REPO" && -d "$CLOUDINIT_SRC" ]]; then
echo "Copying cloud-init files from repo into boot partition…"
for f in user-data meta-data network-config; do
if [[ -f "$CLOUDINIT_SRC/$f" ]]; then
sudo cp "$CLOUDINIT_SRC/$f" "$MNT/$f"
elif [[ -f "$CLOUDINIT_SRC/$f.bootstrap" ]]; then
sudo cp "$CLOUDINIT_SRC/$f.bootstrap" "$MNT/$f"
fi
done
fi
echo ""
echo "Boot partition is mounted at: $MNT"
echo "Cloud-init files to edit:"
echo " $MNT/user-data"
echo " $MNT/meta-data"
echo " $MNT/network-config"
echo ""
EDITOR="${EDITOR:-nano}"
echo "Opening editor ($EDITOR). Save and exit when done."
read -r -p "Press Enter to open $EDITOR on these files…"
sudo "$EDITOR" "$MNT/user-data" "$MNT/meta-data" "$MNT/network-config"
echo "Unmounting…"
sudo umount "$MNT"
MNT=""
sudo losetup -d "$LOOP"
LOOP=""
if [[ -n "$ORIGINAL_XZ" && -z "$NO_RECOMPRESS" ]]; then
echo "Recompressing to $(basename "$ORIGINAL_XZ")…"
xz -T 0 -z -f -k "$IMG_FILE"
mv -f "${IMG_FILE}.xz" "$ORIGINAL_XZ"
echo "Done. Updated: $ORIGINAL_XZ"
rm -rf "$WORK_DIR"
WORK_DIR=""
elif [[ -n "$NO_RECOMPRESS" && -n "$ORIGINAL_XZ" ]]; then
echo "Left decompressed image at: $IMG_FILE"
echo "Recompress manually: xz -z -k \"$IMG_FILE\""
fi

3
emmc-provisioning/scripts/fix-gadget-bootcode-on-host.sh
View File

@@ -3,6 +3,9 @@
# Cause: mass-storage-gadget64 has no real boot files (broken symlinks or Git LFS not pulled).
# This script removes broken symlinks and extracts bootcode4.bin from the installed rpiboot binary.
#
# Does NOT fix: "libusb_bulk_transfer returned -7" / "Failed to write correct length" — that is a USB
# transfer/timing issue (try USB 2.0 port, or rpiboot -m 2000). See PROXMOX-LXC-DEPLOYMENT.md.
#
# On host: bash fix-gadget-bootcode-on-host.sh
# From your machine: ssh root@HOST 'bash -s' < emmc-provisioning/scripts/fix-gadget-bootcode-on-host.sh

216
emmc-provisioning/scripts/setup-network-boot-on-lxc.sh
View File

@@ -1,10 +1,13 @@
#!/usr/bin/env bash
# Setup network boot on the provisioning LXC: DHCP + TFTP on eth1, NAT so LAN uses eth0 for internet.
# Run inside the LXC (as root), or from your machine: ./setup-network-boot-on-lxc.sh root@10.130.60.141
# When run with ssh target, rsyncs lxc/ and runs this script inside the container.
# Setup provisioning LXC: DHCP + DNS on eth1, extra LAN IPs, VLAN eth1.40, NAT so LAN uses eth0 for internet.
# Network boot (TFTP, PXE, initrd) sections are commented out; uncomment if you need PXE/netboot.
# Run inside the LXC (as root), or from your machine: ./setup-network-boot-on-lxc.sh root@10.130.60.141 [SUBNET]
# SUBNET optional: A.B.C.D/PREFIX (e.g. 10.100.1.1/24). When run with ssh target, writes lan-subnet.conf on LXC if SUBNET given.
# When run with ssh target, rsyncs lxc/ and runs this script inside the container. Subnet is read from /opt/cm4-provisioning/lan-subnet.conf.
set -e
TARGET="${1:-}"
SUBNET_ARG="${2:-}"
if [[ -n "$TARGET" ]]; then
# Run remotely: sync lxc/ and script, then execute inside LXC
@@ -12,114 +15,191 @@ if [[ -n "$TARGET" ]]; then
REPO_DIR="$(dirname "$SCRIPT_DIR")"
echo "Syncing lxc config and script to $TARGET ..."
rsync -a "$REPO_DIR/lxc/" "$TARGET:/tmp/cm4-network-boot-lxc/" --exclude='.git'
if [[ -f "$REPO_DIR/network-boot-initramfs/initrd.img" ]]; then
echo "Copying initrd.img to $TARGET ..."
scp "$REPO_DIR/network-boot-initramfs/initrd.img" "$TARGET:/tmp/cm4-network-boot-lxc/initrd.img"
else
echo "Note: network-boot-initramfs/initrd.img not found (run build.sh first); skipping."
fi
# Network boot: copy initrd.img for TFTP
# if [[ -f "$REPO_DIR/network-boot-initramfs/initrd.img" ]]; then
# echo "Copying initrd.img to $TARGET ..."
# scp "$REPO_DIR/network-boot-initramfs/initrd.img" "$TARGET:/tmp/cm4-network-boot-lxc/initrd.img"
# else
# echo "Note: network-boot-initramfs/initrd.img not found (run build.sh first); skipping."
# fi
scp "$SCRIPT_DIR/setup-network-boot-on-lxc.sh" "$TARGET:/tmp/cm4-network-boot-lxc/setup.sh"
# If SUBNET_ARG given, write lan-subnet.conf on LXC so inner script uses the set subnet
if [[ -n "$SUBNET_ARG" ]]; then
if [[ "$SUBNET_ARG" =~ ^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/([0-9]+)$ ]]; then
LAN_GW="${BASH_REMATCH[1]}"
PREFIX="${BASH_REMATCH[2]}"
BASE_3="${LAN_GW%.*}"
LAN_CIDR="${BASE_3}.0/${PREFIX}"
DHCP_RANGE_START="${BASE_3}.100"
DHCP_RANGE_END="${BASE_3}.200"
ssh "$TARGET" "mkdir -p /opt/cm4-provisioning && echo 'LAN_GW=$LAN_GW' > /opt/cm4-provisioning/lan-subnet.conf && echo 'LAN_CIDR=$LAN_CIDR' >> /opt/cm4-provisioning/lan-subnet.conf && echo 'DHCP_RANGE_START=$DHCP_RANGE_START' >> /opt/cm4-provisioning/lan-subnet.conf && echo 'DHCP_RANGE_END=$DHCP_RANGE_END' >> /opt/cm4-provisioning/lan-subnet.conf"
echo "Wrote lan-subnet.conf on LXC (LAN_GW=$LAN_GW, DHCP ${DHCP_RANGE_START}-${DHCP_RANGE_END})."
else
echo "Warning: SUBNET must be A.B.C.D/PREFIX (e.g. 10.100.1.1/24); ignoring '$SUBNET_ARG'."
fi
fi
ssh "$TARGET" "bash /tmp/cm4-network-boot-lxc/setup.sh"
echo "Done."
exit 0
fi
# --- Running inside the LXC from here ---
echo "Configuring network boot (DHCP + TFTP on eth1, NAT via eth0) ..."
# LAN subnet: use /opt/cm4-provisioning/lan-subnet.conf (written by deploy-to-proxmox.sh or passed as SUBNET when running remotely)
# Optional first arg when running locally: A.B.C.D/PREFIX to set/write lan-subnet.conf
LAN_CONF="/opt/cm4-provisioning/lan-subnet.conf"
if [[ "$1" =~ ^([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/([0-9]+)$ ]]; then
LAN_GW="${BASH_REMATCH[1]}"
PREFIX="${BASH_REMATCH[2]}"
BASE_3="${LAN_GW%.*}"
LAN_CIDR="${BASE_3}.0/${PREFIX}"
DHCP_RANGE_START="${BASE_3}.100"
DHCP_RANGE_END="${BASE_3}.200"
mkdir -p /opt/cm4-provisioning
echo "LAN_GW=$LAN_GW" > "$LAN_CONF"
echo "LAN_CIDR=$LAN_CIDR" >> "$LAN_CONF"
echo "DHCP_RANGE_START=$DHCP_RANGE_START" >> "$LAN_CONF"
echo "DHCP_RANGE_END=$DHCP_RANGE_END" >> "$LAN_CONF"
echo "Using set subnet: $LAN_CIDR (gateway $LAN_GW), DHCP ${DHCP_RANGE_START}-${DHCP_RANGE_END}."
elif [[ -f "$LAN_CONF" ]]; then
source "$LAN_CONF"
else
LAN_GW="10.20.50.1"
LAN_CIDR="10.20.50.0/24"
DHCP_RANGE_START="10.20.50.100"
DHCP_RANGE_END="10.20.50.200"
echo "No lan-subnet.conf and no SUBNET argument; using defaults: $LAN_CIDR."
fi
echo "Configuring LAN (DHCP + DNS on eth1, NAT via eth0) — LAN $LAN_CIDR (gateway $LAN_GW), DHCP ${DHCP_RANGE_START}-${DHCP_RANGE_END} ..."
# 1) Install dnsmasq
# 1) Install dnsmasq and vlan (for eth1.40)
if ! command -v dnsmasq >/dev/null 2>&1; then
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq dnsmasq
fi
if ! command -v vconfig >/dev/null 2>&1; then
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq vlan
fi
# 2) dnsmasq config for eth1 only (DHCP + TFTP); PXE options in network-boot-pxe.conf (toggle with toggle-network-boot-dhcp.sh)
# 2) dnsmasq config for eth1 only (DHCP + DNS). PXE/TFTP in network-boot-pxe.conf when needed (toggle-network-boot-dhcp.sh)
mkdir -p /etc/dnsmasq.d
cat > /etc/dnsmasq.d/network-boot.conf << 'DNSMASQ'
# DHCP on eth1 only (provisioning LAN)
# TFTP and PXE options are in network-boot-pxe.conf, controlled by toggle-network-boot-dhcp.sh
cat > /etc/dnsmasq.d/network-boot.conf << DNSMASQ
# DHCP + DNS on eth1 only (provisioning LAN)
# (TFTP/PXE options in network-boot-pxe.conf when network boot is enabled)
interface=eth1
bind-interfaces
dhcp-range=10.20.50.100,10.20.50.200,12h
dhcp-range=${DHCP_RANGE_START},${DHCP_RANGE_END},12h
# DNS: file.server resolves to this host (eth1) so scripts can use http://file.server/...
address=/file.server/${LAN_GW}
# Explicitly send this host as DNS server to DHCP clients (option 6) so they use LXC DNS and resolve file.server
dhcp-option=6,${LAN_GW}
# Other DNS queries forwarded via LXC's resolv.conf
log-dhcp
log-queries
port=0
DNSMASQ
mkdir -p /opt/cm4-provisioning
if [ -f /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh ]; then
cp /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh /opt/cm4-provisioning/
chmod +x /opt/cm4-provisioning/toggle-network-boot-dhcp.sh
/opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable
# Network boot: install toggle script and enable PXE/TFTP
# if [ -f /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh ]; then
# cp /tmp/cm4-network-boot-lxc/toggle-network-boot-dhcp.sh /opt/cm4-provisioning/
# chmod +x /opt/cm4-provisioning/toggle-network-boot-dhcp.sh
# /opt/cm4-provisioning/toggle-network-boot-dhcp.sh enable
# fi
# 3) Network boot: TFTP root — fetch Raspberry Pi 4 boot files from GitHub if missing
# mkdir -p /srv/tftpboot
# if [[ ! -f /srv/tftpboot/start4cd.elf ]]; then
# echo "Fetching Raspberry Pi firmware boot files from GitHub ..."
# if ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then
# apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq curl
# fi
# tmpdir=$(mktemp -d)
# trap "rm -rf $tmpdir" EXIT
# if command -v curl >/dev/null 2>&1; then
# curl -sL "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" -o "$tmpdir/firmware.tar.gz"
# else
# wget -q -O "$tmpdir/firmware.tar.gz" "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz"
# fi
# tar xzf "$tmpdir/firmware.tar.gz" -C "$tmpdir"
# cp -a "$tmpdir/firmware-master/boot/." /srv/tftpboot/
# rm -rf "$tmpdir"
# echo "Copied RPi boot files to /srv/tftpboot"
# else
# echo "TFTP root already has boot files (start4cd.elf present), skipping fetch."
# fi
# 3b) Network boot: copy provisioning initrd.img to TFTP root if provided
# if [[ -f /tmp/cm4-network-boot-lxc/initrd.img ]]; then
# cp /tmp/cm4-network-boot-lxc/initrd.img /srv/tftpboot/initrd.img
# echo "Copied initrd.img to /srv/tftpboot"
# if [[ -f /srv/tftpboot/config.txt ]] && ! grep -q 'initramfs initrd.img' /srv/tftpboot/config.txt 2>/dev/null; then
# echo "" >> /srv/tftpboot/config.txt
# echo "# Provisioning initramfs (network-boot-initramfs)" >> /srv/tftpboot/config.txt
# echo "initramfs initrd.img followkernel" >> /srv/tftpboot/config.txt
# echo "Added initramfs line to config.txt"
# fi
# fi
# 4) Extra LAN IPs on eth1 (192.168.30.1, 192.168.127.1) and VLAN eth1.40 (192.168.0.1/24)
EXTRA_LAN_CONF="/etc/network/interfaces.d/70-cm4-extra-lan"
if [[ -f /tmp/cm4-network-boot-lxc/70-cm4-extra-lan ]]; then
mkdir -p /etc/network/interfaces.d
cp /tmp/cm4-network-boot-lxc/70-cm4-extra-lan "$EXTRA_LAN_CONF"
# Ensure main interfaces includes interfaces.d (Debian)
if [[ -f /etc/network/interfaces ]] && ! grep -q 'interfaces.d' /etc/network/interfaces 2>/dev/null; then
echo 'source /etc/network/interfaces.d/*' >> /etc/network/interfaces
fi
# Apply immediately: secondary IPs on eth1
ip addr add 192.168.30.1/24 dev eth1 2>/dev/null || true
ip addr add 192.168.127.1/24 dev eth1 2>/dev/null || true
# Create and bring up eth1.40 (VLAN 40)
ip link add link eth1 name eth1.40 type vlan id 40 2>/dev/null || true
ip addr add 192.168.0.1/24 dev eth1.40 2>/dev/null || true
ip link set eth1.40 up 2>/dev/null || true
echo "Extra LAN: eth1 + 192.168.30.1/24, 192.168.127.1/24; eth1.40 192.168.0.1/24 (persisted in $EXTRA_LAN_CONF)"
fi
# 3) TFTP root: fetch Raspberry Pi 4 boot files from GitHub if missing
mkdir -p /srv/tftpboot
if [[ ! -f /srv/tftpboot/start4cd.elf ]]; then
echo "Fetching Raspberry Pi firmware boot files from GitHub ..."
if ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then
apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq curl
fi
tmpdir=$(mktemp -d)
trap "rm -rf $tmpdir" EXIT
if command -v curl >/dev/null 2>&1; then
curl -sL "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz" -o "$tmpdir/firmware.tar.gz"
else
wget -q -O "$tmpdir/firmware.tar.gz" "https://github.com/raspberrypi/firmware/archive/refs/heads/master.tar.gz"
fi
tar xzf "$tmpdir/firmware.tar.gz" -C "$tmpdir"
cp -a "$tmpdir/firmware-master/boot/." /srv/tftpboot/
rm -rf "$tmpdir"
echo "Copied RPi boot files to /srv/tftpboot"
else
echo "TFTP root already has boot files (start4cd.elf present), skipping fetch."
fi
# 3b) Copy provisioning initrd.img to TFTP root if provided
if [[ -f /tmp/cm4-network-boot-lxc/initrd.img ]]; then
cp /tmp/cm4-network-boot-lxc/initrd.img /srv/tftpboot/initrd.img
echo "Copied initrd.img to /srv/tftpboot"
if [[ -f /srv/tftpboot/config.txt ]] && ! grep -q 'initramfs initrd.img' /srv/tftpboot/config.txt 2>/dev/null; then
echo "" >> /srv/tftpboot/config.txt
echo "# Provisioning initramfs (network-boot-initramfs)" >> /srv/tftpboot/config.txt
echo "initramfs initrd.img followkernel" >> /srv/tftpboot/config.txt
echo "Added initramfs line to config.txt"
fi
fi
# 4) IP forwarding (LAN clients use WAN)
# 5) IP forwarding (LAN clients use WAN)
echo 'net.ipv4.ip_forward=1' > /etc/sysctl.d/99-cm4-network-boot.conf
sysctl -p /etc/sysctl.d/99-cm4-network-boot.conf 2>/dev/null || sysctl -w net.ipv4.ip_forward=1
# 5) NAT: 10.20.50.0/24 -> eth0 (masquerade)
# 6) NAT: primary LAN + extra LAN subnets + VLAN 40 -> eth0 (masquerade)
# Subnets: primary (from lan-subnet.conf), 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 (eth1.40)
if command -v nft >/dev/null 2>&1; then
mkdir -p /etc/nftables.d
cat > /etc/nftables.d/nat-lan.conf << 'NFT'
cat > /etc/nftables.d/nat-lan.conf << NFT
table ip nat {
chain postrouting {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 10.20.50.0/24 oifname "eth0" masquerade
ip saddr ${LAN_CIDR} oifname "eth0" masquerade
ip saddr 192.168.30.0/24 oifname "eth0" masquerade
ip saddr 192.168.127.0/24 oifname "eth0" masquerade
ip saddr 192.168.0.0/24 oifname "eth0" masquerade
}
}
NFT
if ! nft list table ip nat 2>/dev/null | grep -q postrouting; then
nft -f /etc/nftables.d/nat-lan.conf
else
nft -f /etc/nftables.d/nat-lan.conf
fi
# Ensure main config includes our drop-in (Debian)
if [[ -f /etc/nftables.conf ]] && ! grep -q 'nftables.d/nat-lan' /etc/nftables.conf 2>/dev/null; then
echo 'include "/etc/nftables.d/nat-lan.conf"' >> /etc/nftables.conf
fi
echo "NAT rule added (nftables) and saved to /etc/nftables.d/nat-lan.conf"
echo "NAT rules added (nftables): ${LAN_CIDR}, 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 -> eth0"
else
# Fallback iptables
iptables -t nat -C POSTROUTING -s 10.20.50.0/24 -o eth0 -j MASQUERADE 2>/dev/null || \
iptables -t nat -A POSTROUTING -s 10.20.50.0/24 -o eth0 -j MASQUERADE
echo "NAT rule added (iptables)."
for cidr in "${LAN_CIDR}" 192.168.30.0/24 192.168.127.0/24 192.168.0.0/24; do
iptables -t nat -C POSTROUTING -s "$cidr" -o eth0 -j MASQUERADE 2>/dev/null || \
iptables -t nat -A POSTROUTING -s "$cidr" -o eth0 -j MASQUERADE
done
echo "NAT rules added (iptables) for primary LAN and extra subnets."
fi
# 6) Enable and start dnsmasq
# 7) Enable and start dnsmasq
systemctl enable dnsmasq
systemctl restart dnsmasq
echo "Network boot setup done."
echo " - DHCP + TFTP on eth1 (10.20.50.1), range 10.20.50.100-200"
echo " - NAT: 10.20.50.0/24 -> eth0 (internet)"
echo " - TFTP root: /srv/tftpboot (RPi boot files; initrd.img if provided)"
echo "Setup done."
echo " - DHCP + DNS on eth1 ($LAN_GW), range ${DHCP_RANGE_START}-${DHCP_RANGE_END}"
echo " - NAT: ${LAN_CIDR}, 192.168.30.0/24, 192.168.127.0/24, 192.168.0.0/24 -> eth0 (internet)"
echo " - Extra LAN: eth1 also 192.168.30.1, 192.168.127.1; eth1.40 192.168.0.1/24 (VLAN 40)"
# echo " - TFTP root: /srv/tftpboot (RPi boot files; initrd.img if provided)" # when network boot enabled

64
emmc-provisioning/scripts/test-usbboot-on-host.sh Executable file
View File

@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# Manually test rpiboot on the Proxmox host (device in boot mode must be connected).
# Usage:
# From your machine: ./test-usbboot-on-host.sh [proxmox_host]
# On the host: ./test-usbboot-on-host.sh
# With timeout (e.g. 60s): TIMEOUT=60 ./test-usbboot-on-host.sh root@100.106.128.36
# If you see "Failed to write correct length, returned -7", try USB 2.0 port or add delay:
# RPIBOOT_EXTRA_OPTS='-m 2000' ./test-usbboot-on-host.sh root@100.106.128.36
#
# Replace proxmox_host with your host, e.g. root@100.106.128.36
set -e
HOST="${1:-}"
RPIBOOT="${RPIBOOT:-/opt/usbboot/rpiboot}"
GADGET="${GADGET:-/opt/usbboot/mass-storage-gadget64}"
TIMEOUT="${TIMEOUT:-0}"
RPIBOOT_EXTRA_OPTS="${RPIBOOT_EXTRA_OPTS:-}"
run_on_host() {
if [[ -n "$HOST" ]]; then
ssh "$HOST" "$@"
else
"$@"
fi
}
echo "=== Checking usbboot and gadget on ${HOST:-localhost} ==="
run_on_host "test -x $RPIBOOT" || { echo "Error: $RPIBOOT not found or not executable"; exit 1; }
run_on_host "test -d $GADGET" || { echo "Error: $GADGET not found"; exit 1; }
run_on_host "test -f $GADGET/bootcode4.bin || test -f $GADGET/boot.img || test -f $GADGET/bootfiles.bin" || { echo "Error: no boot file in $GADGET"; exit 1; }
echo " rpiboot: $RPIBOOT"
echo " gadget: $GADGET"
echo ""
echo "=== USB devices (2b8e / 0a5c:2711 = CM4 boot mode) ==="
run_on_host "lsusb | grep -E '2b8e|0a5c' || echo ' None. Connect reTerminal with eMMC disable jumper and USB slave port.'"
echo ""
echo "=== Tip: if rpiboot fails with 'Failed to write correct length, returned -7', use a USB 2.0 port, or run: RPIBOOT_EXTRA_OPTS='-m 2000' $0 $* ==="
echo ""
echo "=== Running rpiboot (verbose) — connect device now if not already ==="
echo " When the device switches to mass storage, rpiboot will exit and a new /dev/sdX may appear."
echo " Use Ctrl+C to stop, or wait for exit."
echo ""
RPIBOOT_CMD="$RPIBOOT -v -d $GADGET $RPIBOOT_EXTRA_OPTS"
if [[ -n "$HOST" ]]; then
if [[ "$TIMEOUT" -gt 0 ]]; then
ssh "$HOST" "timeout $TIMEOUT $RPIBOOT_CMD" || true
else
ssh -t "$HOST" "$RPIBOOT_CMD" || true
fi
else
if [[ "$TIMEOUT" -gt 0 ]]; then
timeout "$TIMEOUT" $RPIBOOT_CMD || true
else
$RPIBOOT_CMD || true
fi
fi
echo ""
echo "=== Block devices now (check for new /dev/sdX) ==="
run_on_host "lsblk -nd -o NAME,SIZE,TYPE /dev/sd[a-z] 2>/dev/null || true"