# Guix + OpenWrt Userland on Raspberry Pi 4B — Router Testbed Plan Goal: build the Guix-based router stack on a Raspberry Pi 4B first (cheap, real aarch64 hardware, no brick risk), then port the result to the GL-MT6000. Assumed hardware: Pi 4B (2 GB+ RAM), onboard gigabit Ethernet, one USB 3 gigabit Ethernet adapter (blue USB 3 port), onboard WiFi, 32 GB+ microSD (or USB SSD). --- ## 0. Status and gaps (reviewed 2026-10-14) **Where we are.** Phases 0–2 done, differently than written below: the Pi is a *WISP* router — `wlan0` is the WAN (WiFi client of the upstream "hakodosh" 5 GHz), onboard `eth0` is the LAN in `br-lan`; no AP (the one radio is busy as client) and no USB NIC in use. Phase 6 is done as `deploy-rpi.sh` (cross-build + `guix copy` + host-built extlinux.conf); `guix deploy`/`reconfigure` can't work here (PITFALLS #28). Phase 3 underway: libubox, ubus, uci, ucode, libnl-tiny, jsonfilter packaged and running (ubusd under Shepherd, UCI seeded). Decisions: README "OpenWrt on Guix" (Shepherd not procd; seed-once UCI). **Gaps this plan didn't cover** — now tasks in Phases 3–5: 1. *procd's other jobs*, not just starting daemons: `hotplug-call` (netifd → `/etc/hotplug.d/iface/*` → firewall/odhcpd reloads); config reload triggers (UCI change → which service reloads; LuCI depends on it); the ubus `system` object (`board`, `info`) and `/etc/board.json`. 2. *base-files shell library*: `/lib/functions.sh`, `/lib/functions/network.sh`, `/lib/config/uci.sh` — sourced by netifd proto scripts and init scripts. Package the needed subset. 3. *FHS path rule*: point programs at store paths via flags/config (`netifd -p/-h/-c`, ubusd `-A`, ucode search path); activation symlinks into `/lib`, `/usr/share` only where nothing else works. 4. *WAN under netifd* needs busybox `udhcpc` + OpenWrt's `dhcp.sh` / `dhcp.script`. First netifd step: LAN only; WAN stays on wpa_supplicant + dhcpcd. 5. *DHCP/DNS ownership*: OpenWrt's default split -- odhcpd for RA/DHCPv6, dnsmasq for DHCPv4 + DNS, both from `/etc/config/dhcp` (dnsmasq via OpenWrt's init script since generation 31). 6. *UCI seeds* for network, dhcp, firewall, wireless (OpenWrt writes these with board_detect + uci-defaults on first boot). 7. *Recovery rule*: never change LAN and WAN in the same deploy — one path in stays up (SSH on WAN or LAN), roll back with `deploy-rpi.sh switch N`. Get a USB-serial adapter for GPIO 14/15 (console without HDMI). 8. *UCI state backup*: Guix rollbacks don't cover `/etc/config`; add a `sysupgrade -b`-like backup/restore. 9. *Hardening* (unscheduled so far): root password hash is in git, password SSH on, SSH open on WAN, nothing checks LAN isolation from upstream. 10. *Updates*: frozen on Guix 1.5.0 (`guix pull` broken upstream, PITFALLS #1), nonguix pinned — no path for security updates yet. 11. Topology below (USB WAN, AP on wlan0) is the original idea, kept for the MT6000-like layout; the testbed runs the WISP layout. --- ## 1. Target topology ``` Internet │ [ wan ] USB 3 GbE adapter ← DHCP client from ISP/modem │ ┌──────┴──────┐ │ Pi 4B │ nftables NAT + firewall │ Guix │ └──┬───────┬──┘ [ lan0 ] onboard GbE [ wlan0 ] onboard WiFi (AP, hostapd) └─────┬──────────┘ br-lan 192.168.10.1/24 ← DHCP/DNS (dnsmasq → later odhcpd) ``` Why WAN on USB: onboard `bcmgenet` is the most reliable NIC, so keep it on the LAN side; WAN throughput is usually lower anyway. Swap if your ISP link is faster than your LAN needs. Interfaces get fixed names (`wan`, `lan0`) by MAC address so USB enumeration order never matters. --- ## 2. Hardware notes and constraints | Component | Driver | Notes | |---|---|---| | Onboard Ethernet | `bcmgenet` | Mainline, no firmware needed | | USB Ethernet | usually `r8152` (Realtek) or `ax88179_178a` (ASIX) | Check with `lsusb`; r8152 may want firmware for some chips | | Onboard WiFi | `brcmfmac` (BCM43455) | Needs **non-free** firmware (`brcmfmac43455-sdio.*`, `clm_blob`) | - **Linux-libre blocks the WiFi firmware.** Use the **Nonguix** channel's `linux` kernel and firmware packages. Ethernet works without blobs. - **brcmfmac AP mode is limited:** supports a small number of clients and one band at a time. Fine for a testbed; not production WiFi. Pick 5 GHz (ch 36/40/44/48) for less interference, or 2.4 GHz for range. - Bridging the AP interface into `br-lan` is fine (it's client-mode bridging that's problematic). - Power: use the official 5V/3A supply — USB NIC + WiFi under load can brown out weak supplies. - Pi boot firmware (`start4.elf`, `fixup4.dat`) is non-free and must be on the FAT boot partition. Guix's `raspberry-pi-64.tmpl` does **not** install it. --- ## 3. Build plan ### Phase 0 — Build host setup (x86 workstation) - [ ] Install/updated Guix; add the **Nonguix** channel to `~/.config/guix/channels.scm` and `guix pull`. - [ ] Enable aarch64 emulation on the host (`qemu-binfmt-service-type`), or plan to cross-compile with `--target=aarch64-linux-gnu`. - [ ] Pin channels for reproducibility: `guix describe -f channels > channels.scm`. - [ ] Optional: git repo for `rpi4.scm`, package definitions, and configs. ### Phase 1 — Minimal bootable Guix on the Pi - [ ] Start from Guix's template: `~/.config/guix/current/share/guile/site/3.0/gnu/system/examples/raspberry-pi-64.tmpl` (or from the Guix git tree). - [ ] Switch kernel to Nonguix `linux`; add Pi firmware + Broadcom WiFi firmware. - [ ] Ensure `bcm2711-rpi-4-b.dtb` and Pi boot firmware land on the boot partition. - [ ] Add user, SSH (`openssh-service-type` with your public key), serial console. - [ ] Build: `guix system image --system=aarch64-linux rpi4.scm` (or `--target=aarch64-linux-gnu`). - [ ] Flash: `dd if= of=/dev/sdX bs=4M status=progress conv=fsync`. - [ ] **Exit check:** boots, SSH works over onboard Ethernet, `ip link` shows eth0 + USB NIC + wlan0, `dmesg` shows brcmfmac firmware loaded. Tip: a USB-serial adapter on GPIO 14/15 (UART, add `enable_uart=1` to `config.txt`) saves a lot of guessing when boot fails. ### Phase 2 — Router with native Guix services (prove the hardware) Get a working router *before* touching OpenWrt userland, so later problems are clearly software, not hardware. - [ ] udev rules naming NICs by MAC (`wan`, `lan0`). - [ ] `br-lan` bridge containing `lan0` (hostapd adds `wlan0` itself via `bridge=`). - [ ] WAN DHCP client on `wan`. - [ ] Static `192.168.10.1/24` on `br-lan`. - [ ] IPv4 forwarding via `sysctl-service-type`. - [ ] `nftables-service-type` with the ruleset below. - [ ] `dnsmasq-service-type` for DHCP + DNS on `br-lan`. - [ ] `hostapd-service-type` for the AP. - [ ] **Exit check:** wired and wireless clients get addresses and reach the internet; LAN is not reachable from WAN. Skeleton (verify service and field names against your Guix version's manual): ```scheme (use-modules (gnu) (nongnu packages linux) (nongnu system linux-initrd)) (use-service-modules networking ssh sysctl) (operating-system (host-name "pirouter") (kernel linux) ; Nonguix (firmware (list linux-firmware)) ; plus Pi/Broadcom firmware as needed ;; bootloader / file-systems: keep from raspberry-pi-64.tmpl (services (append (list (service openssh-service-type (openssh-configuration (password-authentication? #f) (authorized-keys `(("you" ,(local-file "id_ed25519.pub")))))) (service dhcp-client-service-type (dhcp-client-configuration (interfaces '("wan")))) (service nftables-service-type (nftables-configuration (ruleset (local-file "router.nft")))) (service dnsmasq-service-type (dnsmasq-configuration (extra-options '("--interface=br-lan" "--bind-interfaces" "--dhcp-range=192.168.10.100,192.168.10.250,12h")))) (service hostapd-service-type (hostapd-configuration (interface "wlan0") (ssid "pirouter") (channel 36) (extra-settings "bridge=br-lan country_code=US hw_mode=a ieee80211n=1 ieee80211ac=1 wpa=2 wpa_key_mgmt=WPA-PSK rsn_pairwise=CCMP wpa_passphrase=CHANGE-ME")))) ;; + a one-shot Shepherd service creating br-lan and adding lan0, ;; or static-networking with a bridge link if your Guix supports it (modify-services %base-services (sysctl-service-type config => (sysctl-configuration (settings (append '(("net.ipv4.ip_forward" . "1")) %default-sysctl-settings)))))))) ``` `router.nft`: ```nft flush ruleset table inet filter { chain input { type filter hook input priority 0; policy drop; ct state established,related accept ct state invalid drop iif "lo" accept iifname "br-lan" accept meta l4proto { icmp, ipv6-icmp } accept } chain forward { type filter hook forward priority 0; policy drop; ct state established,related accept ct state invalid drop iifname "br-lan" oifname "wan" accept } chain output { type filter hook output priority 0; policy accept; } } table ip nat { chain postrouting { type nat hook postrouting priority 100; oifname "wan" masquerade } } ``` ### Phase 3 — Package OpenWrt userland for Guix Do this on the x86 workstation first (native builds are fast), then cross-build for aarch64. - [x] `libubox` (+ jshn) - [x] `ubus` (+ `ubusd`) - [x] `uci` - [x] `ucode` (all modules) - [x] `jsonfilter`, `libnl-tiny` (netifd deps) - [x] `udebug` (netifd dependency the plan missed) - [x] `netifd` (+ its ucode/shell runtime files, `hotplug-call`) — LAN - [x] base-files shell library subset (`functions.sh`, `network.sh`, …) - [x] busybox `udhcpc` wiring for netifd's `dhcp` proto - [x] **cross glibc popen()/system() broken** (PITFALLS #35): fixed by `cross-libc-fix` (`/bin/sh`) - [x] `odhcpd` (RA + DHCPv6 on lan, ULA) - [ ] `odhcp6c` (DHCPv6/PD client on the WAN; upstream has no IPv6 yet) - [x] `firewall4` (ucode + nftables) - [ ] `ubox` (logd/logread, validate_data) — LuCI log page, init scripts - [x] `rpcd`, `uhttpd` + `ustream-ssl`, LuCI (ucode), with `iwinfo`, `lucihttp`, `cgi-io`, `rpcd-mod-luci` (generation 28) - [ ] Goal (user, 2026-10-15): run TollGate (OpenTollGate `tollgate-module-basic-go`, package `tollgate-wrt`; TollGate OS supports the GL-MT6000). Package DEPENDS: nodogsplash, jq, ca-bundle (+ cron, Go binaries `tollgate-wrt`, `tollgate`). Its postinst runs three uci-defaults scripts (`99-tollgate-setup` is ~2800 lines of UCI edits incl. network, wireless, firewall, dhcp, uhttpd), then `/etc/init.d/{network,firewall,dnsmasq,uhttpd, nodogsplash,tollgate-wrt}` actions and `wifi reload`; ships an rpcd plugin + ACL, an iface hotplug script, nftables.d includes. Done so far: dnsmasq from UCI, init.d shims, rpcd/LuCI, hotplug, fw4. - [x] dnsmasq configured from UCI (OpenWrt's init script; generation 31) - [ ] `/etc/board.json` (LuCI's port view; matters for the MT6000's DSA) - [x] Clock at boot without RTC: `sysfixtime-service-type` (generation 34) - [ ] Trim the ~40 x86 store items cross builds leave in the closure (cross gcc via `gcc-cross-*-lib`, `guix`'s and man-db's native refs; PITFALLS #39) -- dead weight in a 2.6 GB system Keep these in a personal Guix channel (e.g. `guix-openwrt`) so both the Pi and the MT6000 can use them. ### Phase 4 — Run OpenWrt userland under Shepherd - [x] `ubusd` under Shepherd (`ubus-service-type`). - [x] `netifd` owns the LAN (static `br-lan`) and the WAN (dhcp on the wpa_supplicant-managed `wlan0`, attached as an external device). - [x] `odhcpd` (generation 18). - [x] `firewall4` (generation 19; replaces `router.nft`). - [x] procd compatibility (generation 21): `hotplug-call`; `procd-compat` for config reload triggers (`config.change` → `herd reload`), `reload_config`, ubus `system` (`board`, `info`, `reboot`). - [x] `/etc/init.d/*` shims for our Shepherd services, `/etc/rc.d` links (rpcd `rc`, LuCI Startup page, TollGate's postinst calls). - [x] Run OpenWrt procd init scripts as Shepherd services (`openwrt-init-script-service-type`, procd-compat's `openwrt-rc`; generation 38; dnsmasq and uhttpd use it). Limits: first instance only, no procd `user`, triggers/jails ignored. - [x] uhttpd from `/etc/config/uhttpd` (OpenWrt's init script; generation 39). - [x] nodogsplash 5.0.2 + OpenWrt's files (generation 42); auth/deauth with ndsctl verified on the LAN. - [x] WAN on eth0, WiFi AP on the onboard radio with OpenWrt's wireless stack (wpad, wifi-scripts, netifd wireless; generations 43-45). - [x] TollGate v0.5.0 (`tollgate-wrt`) packaged and running (generation 46): setup applied, API on :2121 answers, `tollgate status` ok. - [ ] Pay through the portal from a client: tried from a phone 2026-10-15, the Cashu payment failed (not investigated yet). - [ ] TollGate on the MT6000 (private SSID needs a second AP interface). - [ ] Provide the scripts netifd expects (`/lib/netifd/proto/*.sh`, `/lib/functions.sh`, etc.) from the netifd package via Guix-managed files. - [ ] Replace Phase 2 pieces one at a time: 1. `netifd` replaces udev-bridge + DHCP client + static addressing 2. ~~`odhcpd` (or keep dnsmasq) for DHCP/RA~~ done: odhcpd IPv6, dnsmasq DHCPv4/DNS 3. ~~`firewall4` replaces the hand-written `router.nft`~~ done 4. netifd wireless (`mac80211.sh`) replaces `hostapd-service-type` — optional - [ ] **Exit check after each swap:** same behavior as Phase 2. ### Phase 5 — UCI config (decided: seed once, then UCI owns it) - [x] `uci-service-type` seeds `/etc/config/NAME` only when absent. - [ ] Seeds for `network`, `dhcp`, `firewall`, `wireless`. - [ ] Backup/restore of `/etc/config` (Guix rollbacks don't cover it). - [ ] (maybe) Scheme records that render seeds, for the MT6000 variant. ### Phase 6 — Deploy and upgrade workflow - [x] Over SSH without reflashing: `deploy-rpi.sh` (not `guix deploy` — PITFALLS #28); `switch N` rolls back. - [x] Rollback tested for real (U-Boot menu entry 2 on 2026-10-14). - [ ] Keep a spare SD card with a known-good image. - [ ] Hardening pass (see §0 gap 9) before anything non-lab. ### Phase 7 — Port to GL-MT6000 - [ ] Reuse the userland channel and UCI generation unchanged. - [ ] Swap kernel/DTS/bootloader per `guix-openwrt-mt6000-plan.md`. - [ ] Map interface names: MT6000's DSA ports replace `wan`/`lan0`; mt76 replaces brcmfmac. --- ## 4. Troubleshooting checklist - No WiFi: `dmesg | grep brcmfmac` — firmware missing or blocked by linux-libre. - USB NIC missing: `lsusb`, `dmesg | grep -i r8152` — driver/firmware; try the other USB 3 port; check power supply. - Interfaces renamed/swapped: verify udev MAC rules. - Clients get no IP: dnsmasq bound to `br-lan`? `nft list ruleset` allows input on `br-lan`? - No internet from LAN: `sysctl net.ipv4.ip_forward`, masquerade rule on `wan`. - AP not visible: `iw list` (AP mode supported?), country code set, channel valid for band. --- ## References - Guix commit adding Raspberry Pi bootloader chain + examples: https://git.disroot.org/guix/guix/commit/c7793b82efd3383a9f7adf0dfa82d71ee032e41b - Guix manual: https://guix.gnu.org/manual/ - Nonguix channel: https://gitlab.com/nonguix/nonguix - OpenWrt source (userland packages): https://git.openwrt.org/ - MT6000 port plan: `guix-openwrt-mt6000-plan.md`