Files
guix-router/plans/guix-openwrt-mt6000-plan.md
ssmithxandClaude Opus 5.5 d968403d33 Hardening: key-only SSH, admin off TollGate's open SSID, sysctls
Both routers: sshd takes keys only (it is reachable from the open TollGate
SSID, and on the Pi from the WAN); shared %router-sysctls in router-base
(no ICMP redirects taken, loose rp_filter, kptr_restrict, BPF JIT hardened).

MT6000: the lab WAN SSH rule is gone; 99_zz-wired-private (a uci-defaults
script, after TollGate's setup) moves lan1-lan5 into TollGate's private
network (192.168.2.1), leaves br-lan to the open SSID, rejects SSH/telnet/
LuCI from it, trims nodogsplash's router ports, and turns on the private
network's DHCP, which TollGate's setup left disabled (PITFALLS #55).

The kernel has CONFIG_NFT_FLOW_OFFLOAD; fw4's offloading was measured and
is left off -- no gain at line rate, and it would let flows bypass
nodogsplash's session enforcement (PITFALLS #56).  Deployed as MT6000
generation 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:25:38 -04:00

12 KiB
Raw Permalink Blame History

Guix + OpenWrt Userland on GL.iNet GL-MT6000 (Flint 2) — Project Plan

Goal: run a GNU Guix system on an off-the-shelf router, reusing OpenWrt's hardware enablement (device tree, kernel patches) and packaging OpenWrt's userland (ubus, uci, netifd, …) as Guix packages.


0. Gaps found while building the Pi testbed (2026-10-14)

  • Boot format. §3 Phase 3 assumes U-Boot loads extlinux.conf from ext4. GL.iNet's U-Boot boots FIT images from fixed eMMC partitions; expect to build FIT images in Guix (mkimage) and set the U-Boot env — the Pi's extlinux/deploy-rpi.sh flow won't carry over unchanged. Verify on the device (Phase 0) before designing.
  • Phase 1 (QEMU) is superseded by the Pi testbed: the userland is developed there.
  • Reuse the Pi lessons: firmware subset filtering (mt7986 blobs), cross-building and its pitfalls, workstation-signed guix copy + ACL in config, host-built bootloader config, chrony (no RTC?), EEE quirks.
  • Back up first, including factory — still Phase 0. The bench MT6000 floods 2.4 GHz when powered (it broke the Pi's 2.4 GHz WAN); keep it off or on 5 GHz while testing nearby.
  • DSA ports map directly to netifd device sections (br-lan ports lan1–lan5): one more reason to do netifd on the Pi first.
  • Updates/security path (frozen Guix 1.5.0) and hardening apply here too.

0a. Phase 0 done (2026-10-15)

The bench unit runs vanilla OpenWrt 25.12.5 (r33051, kernel 6.12.94) with TollGate, not GL.iNet's stock firmware; the bootloader is still GL's stock U-Boot 2023.10-rc2 (MediaTek SDK, web failsafe on the reset button) with ATF v2.7. Reached at 192.168.1.1 from archy-dev-pa's enp2s0 (NM profile mt6000-lan, never-default), ssh mt6000 (key installed).

eMMC (7.3 GiB, GPT): p1 u-boot-env 512K (empty: "Bad CRC"), p2 factory 2M (MT7986 eeprom at 0, MACs at 0x4/0xa), p3 fip 2M (BL31 + U-Boot), p4 cfg 2M, p5 log 2M, p6 kernel 32M (FIT), p7 rootfs 7.2G (squashfs + loop overlay); BL2 in mmcblk0boot0. Kernel command line from the DTS: root=PARTLABEL=rootfs rootwait (OpenWrt's bootargs-append).

Backup in ~/mt6000-backup-20261015 (not in git; contains secrets): every partition and both boot areas, checksummed against the device; a full eMMC image (rootfs region inconsistent, being live); sysupgrade -b and /overlay/upper tarballs (TollGate wallet.db, identities, config).

Design consequences: keep GL's U-Boot (its web failsafe is the recovery path). It boots one FIT from p6, no menu: each Guix deploy writes a FIT (kernel, DTB with the Guix command line in /chosen/bootargs, initrd) to p6; rollback = rewrite p6. The root can't be replaced while OpenWrt runs from it: first boot Guix with its root on USB, then move it to p7.

0b. Guix boots on the MT6000 (2026-10-16)

Guix System runs from the eMMC under GL's stock U-Boot: linux-mt6000 (mainline 6.17 + OpenWrt's block-NVMEM, mac-base and bootargs-append patches), the FIT on p6 (scripts/mt6000-fit.sh), the ext4 root on p7 (build-mt6000-root.sh). Same services as the Pi: netifd, firewall4, dnsmasq, odhcpd, uhttpd/LuCI, nodogsplash, hostapd; br-lan = lan1-lan5 + both radios (guix-router, 2.4 GHz ch 1 and 5 GHz ch 36/80 MHz), MACs and WiFi calibration from factory. SSH comes up ~45 s after reset.

Getting there took four board fixes (docs/PITFALLS.md #49, #50): BL31's 256 KiB secmon, no WED (mainline's wo-* reserved regions reset the kernel), the eMMC partitions in mainline's DT binding (else no NVMEM, hence no Ethernet), and clk_ignore_unused (mtk_eth_soc's MDIO clock). With no serial console, the tools were GL U-Boot's TFTP recovery (#51), reading Guix's logs off p7 after booting OpenWrt from RAM, and ramoops.

First install: TFTP-boot the OpenWrt initramfs, then from it write mt6000-root.ext4 to p7 and mt6000-boot.itb to p6, verify both, reboot. After that deploy-mt6000.sh (build, guix copy, FIT to p6, activate, reboot; switch N rolls back), tested both ways 2026-10-16.

Performance (2026-10-16)

Routing LAN (lan3) -> NAT/fw4/nodogsplash -> WAN (eth1), measured with iperf3 from the workstation: enp2s0 in its own network namespace on the LAN side, eno1 (iperf3 server) on the WAN side; both 1 Gbit, so ~940 Mbit/s is the ceiling and the 2.5G WAN port can't be pushed further from here. Generation 5 (packet steering on):

Test Throughput Router CPU (4 cores)
upload / download, 1 TCP stream 925 / 934 Mbit/s 7%
upload / download, 4 streams 930 / 934 Mbit/s 6-7%
both directions at once 877 + 579 = 1.46 Gbit/s 19%
latency through the router 0.68 ms

Without packet steering both Ethernet IRQs were on CPU0 and bidirectional peaked at 1.06 Gbit/s with one core at 96% softirq. With it no core is above ~30%; the remaining bidirectional asymmetry is the test rig (one host, both directions). Flow offloading: built, tested, left off (PITFALLS #56). Since generation 6 the workstation is on the private network (wired ports), so tests no longer pass the captive portal.


1. Hardware choice

MIPS is not required — and should be avoided. Guix has no real 32-bit MIPS support, and those devices lack the flash for a glibc-based Guix system. Target a 64-bit ARM router with plenty of storage.

Target: GL.iNet GL-MT6000 (Flint 2)

Spec Value
SoC MediaTek MT7986A (Filogic 830), quad-core Cortex-A53 @ 2.0 GHz, aarch64
RAM 1 GB DDR4
Storage 8 GB eMMC
Ethernet 2× 2.5 GbE, 4× 1 GbE (MT7531 DSA switch)
WiFi WiFi 6, dual-band 4×4 (MT7976, mt76 driver)
Other USB 3.x

Why it beats the alternatives:

  • 8 GB eMMC — ext4/btrfs root, full /gnu/store, multiple generations. No UBIFS, no closure-size squeezing.
  • Standard U-Boot — no MikroTik-style RouterBoot ELF chainloading. OEM U-Boot doesn't need replacing.
  • Hard to brick — U-Boot web recovery (hold reset while powering on, connect to a LAN port). Stock firmware is an OpenWrt fork that accepts OpenWrt sysupgrade images, so returning to stock is easy.
  • Strong upstream support — mtk_eth_soc, MT7531 DSA, and mt76 are in mainline Linux. OpenWrt maintains the board DTS (mt7986a-glinet-gl-mt6000.dts), which reads WiFi calibration and MAC addresses from eMMC via NVMEM — no userspace hotplug scripts needed.

Fallback target: MikroTik RB5009 (Armada 7040, 1 GB RAM, 1 GB NAND). Workable, but needs UBIFS and RouterBOOT → U-Boot chainloading.


2. Known constraints

  • WiFi firmware: MT7986 WiFi needs non-free blobs (mediatek/mt7986_* from linux-firmware). Guix's linux-libre strips these → use Nonguix linux + linux-firmware, or a custom kernel package.
  • Kernel patches: Mainline covers basics. OpenWrt carries extra patches for hardware flow offload (PPE/WED) and mt76 fixes. Start mainline, add only as needed.
  • Init model clash: OpenWrt = procd + UCI + hotplug; Guix = Shepherd + Scheme. Pick one source of truth (recommended: Guix).
  • Size is not a concern on this device, but keep the system lean anyway (avoid %base-packages bloat) for faster builds/deploys.

3. Build plan

Phase 0 — Back up the device (do this first!)

  • Boot stock or OpenWrt, then dump all eMMC partitions — especially factory (WiFi calibration + MAC addresses) — and the U-Boot environment.
    cat /proc/partitions
    ls -l /dev/mmcblk0p*
    dd if=/dev/mmcblk0pN of=/tmp/factory.bin   # repeat per partition, copy off-device
    fw_printenv > /tmp/uboot-env.txt
    

Phase 1 — Prototype in QEMU (no hardware risk)

  • qemu-system-aarch64 -M virt
  • Get the Guix system + OpenWrt userland (Phases 4–5) working here first.

Phase 2 — Kernel

  • Guix kernel package based on Nonguix linux (or a custom variant).
  • Add OpenWrt's GL-MT6000 DTS from target/linux/mediatek/dts/.
  • Pull patches from OpenWrt target/linux/mediatek/patches-* only as needed.
  • Must include:
    • mtk_eth_soc + MT7531 DSA switch
    • mt76 / mt7915e (covers MT7986 WiFi)
    • MMC + NVMEM-on-MMC support
    • nftables, WireGuard, bridge, VLAN

Phase 3 — Root filesystem and boot

  • Build: guix system image --target=aarch64-linux-gnu router.scm (or natively on an aarch64 host).
  • ext4 (or btrfs) root on eMMC — either a new partition or replacing the rootfs partition.
  • Point U-Boot environment at the Guix kernel/initrd/DTB (fw_setenv), or use extlinux.conf if U-Boot's distro boot works on this build.
  • Keep a known-good boot entry for recovery.

Phase 4 — Package OpenWrt userland for Guix

Small CMake projects, straightforward to package:

  • libubox (on the Pi)
  • ubus (on the Pi)
  • uci (on the Pi)
  • ucode (on the Pi)
  • netifd (on the Pi)
  • odhcpd (on the Pi)
  • firewall4 (on nftables; on the Pi)
  • LuCI (rpcd, uhttpd, ucode LuCI; on the Pi)

Init strategy — choose one:

  • Shepherd as PID 1 (recommended): Shepherd services run ubusd, netifd, etc. Keeps Guix generations/rollbacks intact.
  • procd as PID 1: more authentic, but fights Guix's activation model.

WiFi: use Guix hostapd-service-type, or OpenWrt's netifd wireless scripts if going full-OpenWrt-userland.

Phase 5 — Configuration

  • Generate /etc/config/* (UCI files) from Scheme in the operating-system declaration.
  • Decide deliberately: declarative (LuCI/uci set edits overwritten on reconfigure) vs. mutable config.

Phase 6 — Upgrades

  • Replace sysupgrade with guix deploy or guix system reconfigure over SSH.
  • Generations provide rollback; U-Boot web recovery is the last resort.

4. Recovery

  • U-Boot web UI: unplug power, hold reset, plug in, release after LED cue; connect PC to a LAN port (lan2–5) with static IP 192.168.1.x; browse to 192.168.1.1 and upload a firmware image.
  • Back to OpenWrt/stock: flash an official OpenWrt or GL.iNet sysupgrade image without keeping settings.
  • Restore factory partition from your Phase 0 backup if calibration/MACs are lost.

Check the current OpenWrt wiki page for exact steps before flashing.

  • Without the button: GL's U-Boot TFTP-fetches openwrt-gl-mt6000.bin from 192.168.1.2 on every boot; serve an OpenWrt sysupgrade image there and power-cycle (docs/PITFALLS.md #51).

5. Next steps

  1. Back up all eMMC partitions (Phase 0). Done 2026-10-15 (§0a).
  2. Set up aarch64 cross-build or native build host with Guix + Nonguix channel.
  3. Draft Guix package definitions for libubox, ubus, uci.
  4. Boot a minimal Guix aarch64 system in QEMU with those services.
  5. Build a Nonguix-based kernel with the GL-MT6000 DTS and boot it from eMMC. Done 2026-10-16 (§0b).
  6. Deploy script: guix copy + new FIT on p6 (rollback = previous FIT). Done: deploy-mt6000.sh.
  7. WAN on eth1 (2.5G) (2026-10-16: cable to the workstation's eno1, NM shared 10.42.0.1, fixed lease 10.42.0.3; phone browsing through the router works). TollGate, with the wallet from the backup (2026-10-16, PITFALLS #53: open TollGate-3DC6 + private c08r4d0r-3DC6, 10 sats/min via minibits). Still open: a paid session from a phone; the WISP uplink if wanted.
  8. Narrow clk_ignore_unused down to the clock MDIO needs; WED back on.

References