nearxos/Agentic-OS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Agentic OS troubleshooting platform

Browse Source 
Self-hosted, Docker-based agentic troubleshooting platform: FastAPI backend +
LangGraph agent, Next.js UI, tiered LLM routing (local Ollama -> Gemini ->
DeepSeek -> OpenRouter), MCP server manager, encrypted device credentials,
RBAC, audit log, project-memory + Obsidian integrations, and editable
troubleshooting decision rules tuned for the GeneseasX vessel stack.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
nearxos
2026-06-14 22:11:07 +03:00
commit 6185b9b85a
126 changed files with 14565 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

229
rules/troubleshooting.yaml Normal file
View File

@@ -0,0 +1,229 @@
# Agentic OS — troubleshooting decision rules
# Edit this file or use Admin → Rules in the UI. Reloaded on each task run.
#
# Fields per rule (only these are read by the engine):
# id, name, priority (ties only; matching is by keyword specificity), severity
# match.keywords — any keyword match scores the rule (multi-word phrases score 2x)
# match.all_of — optional; every term must appear or the rule is skipped
# devices — catalog_key list, or [all] for every onboard device
# order — optional diagnose sequence (subset of devices)
# hints — shown verbatim to triage/reasoning LLM
# steps — suggested investigation checklist (also shown to the LLM)
#
# NOTE: hints/steps are the ONLY place free-text knowledge reaches the agent.
# Topology values below are the TYPICAL GeneseasX golden-lab pattern; field
# vessels differ (different subnets, container named "voip" vs "asterisk",
# FortiGate in the trunk path). Always confirm live — never assume the IPs.
version: 1
broad_triggers:
# Phrases only — avoid generic words like "check" or "status" that appear in
# scoped tasks (e.g. "check VM status on Proxmox").
keywords:
- health check
- full stack
- all systems
- all devices
- everything
- full audit
- stack audit
- nothing works
- whole ship
device_keywords:
proxmox: [proxmox, pve, hypervisor, qemu, lxc, vm, virtual machine, node]
pfsense: [pfsense, firewall, nat, vpn, openvpn, wireguard, gateway, captive, dns, portal, snat, hairpin, reflection]
docker_vm: [docker, geneseas, geneseasx, compose, container, shipportal, ship-dns, radius]
asterisk_geneseasx: [asterisk, voip, sip, pjsip, phone, trunk, registration, dial, rtp, iax, iax2, dialplan, codec, opus, ulaw, extension, originate, yealink]
asterisk_satbox: [satbox, tmgeneseas, native asterisk]
fortigate: [fortigate, fortios]
fortiswitch: [fortiswitch, switch port]
rules:
- id: no-registration
name: Phone / SIP registration failure
priority: 90
severity: high
match:
keywords: [register, registration, unregistered, "401", "403", timeout, sip, "cannot register", "won't register, not registering"]
devices: [pfsense, asterisk_geneseasx]
order: [pfsense, asterisk_geneseasx]
hints:
- "Typical GeneseasX path: phones do NOT register directly to Asterisk. They hit a pfSense SIP NAT redirect (e.g. 192.168.0.254:5060 -> Asterisk 10.20.30.222:5060). Verify that port-forward + the matching pass rule exist before blaming Asterisk."
- "Yealink red flags seen before: SIP server set to the pfSense VIP instead of the Asterisk IP; NAT keepalive enabled (AccountNATEnableUDPUpdate=1) while rport disabled (AccountNATRport=0). That mismatch destabilises registration/media."
- "On Asterisk, registration = pjsip AOR/contact + qualify. A failing 'qualify' (unreachable) usually means a NAT/firewall path problem, not credentials."
steps:
- "pfsense: confirm SIP NAT port-forward (UDP 5060 -> Asterisk) and the pass rule on the phone VLAN (Business/CREW)"
- "pfsense: check the SIP redirect VIP and outbound NAT for the phone subnet"
- "asterisk: pjsip show registrations"
- "asterisk: pjsip show endpoints / pjsip show aors / pjsip show contacts"
- "asterisk: pjsip show endpoint <ext> — check qualify status and contact address"
- id: one-way-audio
name: One-way audio / RTP path
priority: 85
severity: high
match:
keywords: ["one-way", one way, rtp, no audio, "can't hear", cant hear, "only one side", asymmetric, audio, silence, "no sound"]
devices: [pfsense, asterisk_geneseasx]
order: [pfsense, asterisk_geneseasx]
hints:
- "MOST COMMON ROOT CAUSE here: pfSense Business interface (vtnet3 / opt2) is missing a pass rule for the RTP range (UDP 10000-10029) toward shipPortal. Result: phone->Asterisk RTP is dropped while Asterisk->phone passes, so the REMOTE party cannot hear the phone. Which direction is silent tells you which way is blocked."
- "Proven fix pattern: on pfSense add 'pass udp 10000-10029 to shipPortal' on the phone VLAN AND set outbound SNAT for RTP to the shipPortal VIP (e.g. 192.168.0.254). Back up config.xml before any write."
- "Asterisk side: pjsip.conf local_net must include BOTH the phone subnet (e.g. 192.168.0.0/24) and the VoIP subnet (e.g. 10.20.30.0/24); external_media_address must be the Asterisk reachable IP (e.g. 10.20.30.222). Missing local_net causes wrong NAT mangling of SDP."
- "Confirm with evidence, not theory: check pfSense filter.log for blocked phone->Asterisk :100xx flows, and Asterisk RTP RX/TX counters during a live call (both should increment roughly equally)."
steps:
- "pfsense: list firewall rules on the phone VLAN for UDP 10000-10029 toward Asterisk/shipPortal"
- "pfsense: check outbound NAT (SNAT) for the RTP range"
- "pfsense: inspect filter.log during a live call for blocked RTP (src phone -> Asterisk :100xx)"
- "asterisk: pjsip show endpoint <ext> — verify local_net / external_media_address"
- "asterisk: rtp set debug on (or core show channels) during a live call — confirm bidirectional RTP RX/TX"
- id: outbound-trunk-failure
name: Outbound call failure / IAX trunk
priority: 87
severity: high
match:
keywords: ["call out", "cant call out", "can't call out", "cannot call out", "no outbound", outbound, "dial out", congestion, "fast busy", trunk, iax, iax2, "subscriber absent", "no route"]
devices: [asterisk_geneseasx, pfsense, fortigate]
order: [asterisk_geneseasx, pfsense, fortigate]
hints:
- "Outbound on GeneseasX typically leaves via an IAX2 trunk (UDP 4569) to an upstream PBX (e.g. tmvoip.satlite.eu). Start by proving the trunk is up: iax2 show peers / iax2 show registry (qualify latency)."
- "Dialplan transforms matter: e.g. an outbound pattern _7XXXX strips the leading 7 before sending to the trunk. Confirm the dialed number actually matches a route."
- "If the trunk qualifies OK and the call still fails fast with congestion / 'Subscriber absent' (cause 20), the problem is the trunk ROUTE or CODEC, not pfSense. Known gotcha: IAX trunk negotiated opus-only while PJSIP endpoints are ulaw — fix the allow= codec list / transcoding."
- "Firewall is rarely the cause once qualify is OK. On field vessels the IAX trunk is SNAT'd out via a FortiGate policy; in the golden lab the trunk path is pfSense-only. Only dig into firewall if iax2 qualify is failing."
steps:
- "asterisk: iax2 show peers and iax2 show registry — trunk reachability + latency"
- "asterisk: dialplan show <outbound-context> — confirm the dial pattern and route for the failing number"
- "asterisk: live test — originate/dial the number and watch core show channels + verbose for the hangup cause"
- "asterisk: compare codec allow= on the trunk vs the endpoint (opus vs ulaw mismatch)"
- "pfsense/fortigate: only if qualify fails — confirm UDP 4569 passes outbound to the trunk peer"
- id: container-down
name: Docker / Asterisk container down
priority: 80
severity: high
match:
keywords: [container, "docker down", crash, crashed, restart, restarting, "voip down", shipportal, "container down", exited, unhealthy]
devices: [proxmox, docker_vm, asterisk_geneseasx]
order: [proxmox, docker_vm, asterisk_geneseasx]
hints:
- "Work bottom-up: Proxmox guest running -> Docker VM reachable -> target container healthy."
- "Container name varies by deployment: golden lab uses 'asterisk', some field vessels use 'voip'. Check docker ps output for the actual name before docker exec."
- "A container in a restart loop usually has a config/volume error — check 'docker logs' for the crash reason rather than just restarting it."
steps:
- "proxmox: list VMs / LXC and confirm the Docker VM guest is running"
- "docker_vm: docker ps -a — find the container state and real name"
- "docker_vm: docker logs <container> --tail 100 — crash/restart reason"
- "asterisk: docker exec <container> asterisk -rx 'core show version' once it's up"
- id: crew-dns-slow
name: Crew DNS slow / slow internet
priority: 78
severity: medium
match:
all_of: [dns]
keywords: ["slow dns", "dns slow", "dns latency", "slow internet", slow, latency, "ship-dns", upstream, resolve, resolving, "3 seconds"]
devices: [docker_vm, pfsense]
order: [docker_vm, pfsense]
hints:
- "Known root cause: the ship-dns container's PRIMARY upstream is unreachable from the Docker VM, so every query waits the full timeout before falling back. Seen with primary 1.1.1.1 blocked -> ~3s per lookup; 8.8.8.8/9.9.9.9 answered in <0.1s."
- "Proven fix: set ship-dns primary=8.8.8.8, secondary=9.9.9.9, dns_timeout=2 (and raise resolver worker concurrency). Measure per-upstream timing from the Docker VM before/after."
- "Architecture rule: do NOT hand the Docker management IP (e.g. 10.20.30.222) directly to crew/business as DNS — pfSense is the DNS gateway and redirects per segment."
- "PERSISTENCE WARNING: in-place edits inside the ship-dns container revert on image recreate. Fold fixes into the image, don't just docker cp."
steps:
- "docker_vm: docker ps — ship-dns container health"
- "docker_vm: time per-upstream lookups from the Docker VM (dig @1.1.1.1 / @8.8.8.8 / @9.9.9.9 example.com)"
- "docker_vm: check ship-dns dns_settings (primary/secondary/timeout) and worker concurrency"
- "pfsense: verify the crew/business DNS redirect rules point at pfSense, not the Docker IP"
- id: dns-captive-portal
name: DNS / captive portal access
priority: 75
severity: medium
match:
keywords: [captive, portal, "captive portal", crew, redirect, coovachilli, radius, "cant login", "can't login", "no internet", authentication]
devices: [pfsense, docker_vm]
order: [pfsense, docker_vm]
hints:
- "GeneseasX crew access = pfSense captive portal + RADIUS auth back to the Docker stack (e.g. 10.20.30.222). Crew is isolated via floating rules + CP, so a missing pass rule blocks even authenticated traffic."
- "Portal page is reached via hairpin / NAT reflection: crew -> pfSense:443 -> Docker portal. If the portal won't load, check NAT reflection and the 443 redirect, not just the container."
- "CREW often has an auto redirect but may LACK explicit pass rules if a new service (e.g. VoIP) was deployed there — verify pass rules exist for the new service."
steps:
- "pfsense: captive portal zone status and the crew interface rules (floating + interface)"
- "pfsense: confirm RADIUS reachability to the Docker stack (auth/acct ports) and the 443 portal redirect / NAT reflection"
- "docker_vm: docker ps — portal + RADIUS containers healthy"
- "docker_vm: docker logs for the portal/RADIUS container on failed auth"
- id: proxmox-vm-inventory
name: Proxmox VM / LXC status and config
priority: 86
severity: low
match:
keywords: [proxmox, qemu, lxc, hypervisor, pve, guest, vmid]
devices: [proxmox]
order: [proxmox]
hints:
- Proxmox focus — list QEMU/LXC guests and fetch per-VM status + config when requested
- Include actual VM data in the report; do not suggest checking the UI instead
- If other devices are named in the issue, check those too with their own diagnostics
steps:
- proxmox list nodes
- proxmox list qemu and lxc on each node
- proxmox get_qemu_status and get_qemu_config for each guest (especially running)
- id: vm-unreachable
name: VM / host unreachable
priority: 70
severity: high
match:
keywords: [unreachable, "vm down", "host down", "ping fail", offline, "no route"]
devices: [proxmox, pfsense]
order: [proxmox, pfsense]
hints:
- Proxmox guest status and networking first, then pfSense routes/NAT
steps:
- proxmox list nodes and VM status
- pfsense list gateways and interface status
- id: list-firewall-rules
name: List pfSense firewall / NAT policies
priority: 88
severity: low
match:
keywords: [list, firewall, rule, rules, policy, policies, nat, "port forward", filter]
devices: [pfsense]
order: [pfsense]
hints:
- Call pfsense_list_firewall_rules, port_forwards, outbound NAT — include raw data in report
- "Typical interface->role map (confirm live): vtnet0=WAN1, vtnet1=WAN2, vtnet2=CREW, vtnet3=Business, vtnet4=Management. Management is usually pass-all; Business is restrictive (DNS/HTTPS/SIP); CREW is captive-portal isolated."
steps:
- pfsense list interfaces
- pfsense list firewall rules (all interfaces)
- pfsense list port forwards and outbound NAT
- id: asterisk-health
name: Asterisk health check
priority: 60
severity: medium
match:
keywords: [asterisk, voip, pjsip, "voip health"]
all_of: [health]
devices: [asterisk_geneseasx, docker_vm]
order: [docker_vm, asterisk_geneseasx]
hints:
- Confirm the Asterisk container is healthy on the Docker VM (name 'asterisk' or 'voip') before deep CLI
- Cover the three pillars — endpoint registrations, active channels, and trunk state
steps:
- "docker_vm: docker ps — Asterisk container status"
- "asterisk: pjsip show registrations and pjsip show endpoints"
- "asterisk: iax2 show peers — trunk reachability"
- "asterisk: core show channels — active calls"
default:
severity: medium
devices: []
hints:
- No specific rule matched — diagnose only devices mentioned in the issue text
- Do not run unrelated MCPs; if scope is unclear, say so in the report