Compare commits

...

17 Commits

Author SHA1 Message Date
dcb0618012 Merge remote-tracking branch 'origin/main' into archy-hwconfig 2026-07-23 23:55:40 +00:00
archipelago
b28e5f2ef6 Merge remote-tracking branch 'gitea-ai/main' 2026-07-23 19:13:20 -04:00
archipelago
3037e9808e docs(handoff): app direct ports over mesh — DONE, mesh-verified
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:12:46 -04:00
881b9ee0bf Merge pull request 'feat(companion): instant off-LAN startup + same-node apps stay in-app' (#114) from feat/fast-start-same-node into main
All checks were successful
Demo images / Build & push demo images (push) Successful in 3m2s
2026-07-23 23:12:15 +00:00
Dorian
5ba3c161c4 feat(companion): instant off-LAN startup + same-node apps stay in-app
Two field reports from tonight's 5G session:

- 'Stuck connecting': the kiosk WebView always loaded the LAN URL first,
  and Chromium burns a minute+ of connect retries on a dead LAN IP before
  the mesh fallback fires. WebViewScreen now races a raw TCP probe — LAN
  gets 2.5s, else the mesh ULA (patient, 12s) — BEFORE creating the
  WebView, so startup lands on the right origin in seconds either side.
  Retry re-races instead of blindly reloading the LAN URL.
- Pine/Home Assistant/BTCPay opened in the external browser over the
  mesh: same-node detection compared one host, but the node has two (LAN
  origin + mesh ULA) — kiosk loaded via the ULA meant app links failed
  isSameHost and routeOutbound bounced them to the browser. Same-node now
  matches either address, in the kiosk router, SSL allowances and the
  InAppBrowser.

Served APK: 0.5.6 (vc26). Note: the binary also carries the in-flight
listen_port groundwork from the dev-box session's archy-fips-core WIP
(compiles clean, default posture unchanged); its source lands separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 00:11:33 +01:00
archipelago
2ad57c63f1 feat(mesh): app direct ports reachable over the mesh — IPv4 listeners mirrored onto [::]
The companion reaches the node at its fips0 ULA, and the web UI builds app
links as http://[<ULA>]:<direct port> — but rootless-podman published ports
bind 0.0.0.0 only, so every app URL was refused over the mesh (:8334 first).
A reconcile loop in the backend mirrors public IPv4 listeners: any port
>=1024 on 0.0.0.0 without an IPv6 any-listener gets a v6-ONLY [::] forwarder
to 127.0.0.1, following /proc/net/tcp* so install/remove and hardcoded
companion ports are covered without touching a single container. Purely
additive: IPv4/LAN/Tor access paths are untouched, v6only listeners cannot
collide with or intercept v4 traffic, and foreign IPv6 listeners win.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:09:14 -04:00
6c48de20e3 Merge pull request 'docs(handoff): app direct ports are IPv4-only over the mesh — node-side task' (#113) from docs/mesh-app-ports-v2 into main 2026-07-23 22:56:40 +00:00
Dorian
e17de63016 docs(handoff): app direct ports are IPv4-only over the mesh — node-side task
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 23:55:43 +01:00
archipelago
6ba4540c53 Merge vc25 (PR #111) — handoff reconciled: vps2 daemon restart resolved the mesh path; private-tree withdrawn
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:42:15 -04:00
archipelago
5454bed038 docs(handoff): RESOLVED — vps2 daemon degradation was the mesh blocker; privatization withdrawn
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:40:51 -04:00
128b78d083 Merge pull request 'feat(companion): 60s mesh probe window + session pre-warm from the VPN service' (#111) from feat/mesh-probe-window-prewarm into main
All checks were successful
Demo images / Build & push demo images (push) Successful in 2m47s
2026-07-23 22:36:45 +00:00
Dorian
75ecd1d3ea feat(companion): 60s mesh probe window + session pre-warm from the VPN service
Implements the app-side recommendations from the node diagnosis
(HANDOFF-2026-07-23): first session through the public tree takes 15s+
when it works at all, far beyond the old ~8s probe window.

- connect(): mesh ULA probed inside a 60s budget, 15s per-phase timeouts.
- ArchyVpnService: session warmer probes every saved node ULA as soon as
  the tunnel is up (5s cadence first minute, then 60s keep-warm) so the
  UI hits a warm session and it never idles out.
- Served APK: 0.5.5 (vc25); handoff updated with 25-ping evidence that
  public-tree discovery currently fails outright (supports the
  private-tree infra decision).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 23:36:34 +01:00
archipelago
82fcccd113 docs(handoff): node-side mesh diagnosis — nginx v6 fixed, real blocker is session path quality
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:25:18 -04:00
archipelago
1e89362e71 fix(nginx): IPv6 listeners — companion mesh HTTP to the node's ULA never connected
Phones on the FIPS mesh reach the node at http://[<fips0 ULA>], but every
shipped nginx config listened on IPv4 only — nothing answered [::]:80/443,
so mesh HTTP failed with the tunnel fully healthy (found live on
framework-pt during the vc24 5G session, 2026-07-23). Canonical conf gains
listen [::] lines and patch_nginx_conf self-heals deployed nodes (covers
the sites-enabled -> archipelago-http dev variant via the existing
canonicalize pass); validated by the existing nginx -t + rollback flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:57:58 -04:00
908e6185c8 Merge pull request 'fix(companion): TUN reader no longer dies on non-blocking fd + VPN not metered' (#110) from fix/tun-blocking-fd-metered into main
All checks were successful
Demo images / Build & push demo images (push) Successful in 2m54s
2026-07-23 21:51:11 +00:00
Dorian
0064cfccac chore(companion): lockfile for libc dep
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 22:48:42 +01:00
Dorian
53037b2b71 fix(companion): TUN reader no longer dies on non-blocking fd + VPN not metered
On-device diagnosis (phone on adb, 5G): mesh sessions established but no
packet ever entered the tunnel — Android hands the VpnService TUN fd over
non-blocking, and the fips fork's dedicated blocking-read thread treats
EAGAIN as fatal, dying at startup. archy-fips-core now forces the fd into
blocking mode before Node::start_with_tun_fd. Verified on-device: reader
survives, packets flow into the mesh, and the 30s anchor-link flap is
gone (stable 8+ min on 5G).

Also setMetered(false): Android 10+ defaults VPNs to metered, flipping
the phone into data-restricted behaviour whenever the mesh is up.

Served APK: 0.5.4 (vc24). Handoff updated with the remaining node-side
blocker: phone->anchor works, phone->node ULA gets no reply — needs
fipsctl introspection on Framework PT (suspect fork/daemon session or
routing mismatch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 22:47:26 +01:00
15 changed files with 497 additions and 23 deletions

View File

@ -11,8 +11,8 @@ android {
applicationId = "com.archipelago.app" applicationId = "com.archipelago.app"
minSdk = 26 minSdk = 26
targetSdk = 35 targetSdk = 35
versionCode = 23 versionCode = 26
versionName = "0.5.3" versionName = "0.5.6"
vectorDrawables { vectorDrawables {
useSupportLibrary = true useSupportLibrary = true

View File

@ -10,10 +10,15 @@ import android.os.Build
import android.util.Log import android.util.Log
import com.archipelago.app.MainActivity import com.archipelago.app.MainActivity
import com.archipelago.app.R import com.archipelago.app.R
import com.archipelago.app.data.ServerPreferences
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel import kotlinx.coroutines.cancel
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
/** /**
@ -27,6 +32,7 @@ import kotlinx.coroutines.launch
class ArchyVpnService : VpnService() { class ArchyVpnService : VpnService() {
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private var warmerJob: Job? = null
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int { override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
if (intent?.action == ACTION_STOP) { if (intent?.action == ACTION_STOP) {
@ -68,6 +74,14 @@ class ArchyVpnService : VpnService() {
// And let apps that bind their own network skip the TUN // And let apps that bind their own network skip the TUN
// entirely — this is mesh reachability, not a privacy VPN. // entirely — this is mesh reachability, not a privacy VPN.
.allowBypass() .allowBypass()
.apply {
// Android 10+ treats VPN networks as METERED by default,
// which flips the whole phone into data-saver behaviour
// (background sync off, "metered" warnings) while the
// mesh is up. It inherits the underlying network's real
// metered state instead.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) setMetered(false)
}
.establish() .establish()
} catch (e: Exception) { } catch (e: Exception) {
Log.e(TAG, "VPN establish failed", e) Log.e(TAG, "VPN establish failed", e)
@ -81,10 +95,57 @@ class ArchyVpnService : VpnService() {
val fd = pfd.detachFd() val fd = pfd.detachFd()
val result = FipsNative.start(identity.secret, peersJson, fd) val result = FipsNative.start(identity.secret, peersJson, fd)
Log.i(TAG, "mesh start: $result") Log.i(TAG, "mesh start: $result")
if (result.contains("\"error\"")) shutdown() if (result.contains("\"error\"")) {
shutdown()
} else {
startSessionWarmer()
}
}
/**
* Pre-warm + keep-warm mesh sessions to every known node ULA.
*
* Discovery + first session through the public tree can take 15s+
* (HANDOFF-2026-07-23 node diagnosis) paying that cost here, the
* moment the tunnel is up, means the connect probe and WebView hit an
* established session instead of timing out on a cold one. The periodic
* touch afterwards keeps the session from idling out. Failed connects
* are expected and cheap; the attempt itself is what drives discovery.
*/
private fun startSessionWarmer() {
warmerJob?.cancel()
warmerJob = scope.launch {
val prefs = ServerPreferences(this@ArchyVpnService)
var round = 0
while (isActive && FipsNative.isRunning()) {
val ulas = try {
prefs.savedServers.first().mapNotNull { it.meshIp.ifBlank { null } }.distinct()
} catch (_: Exception) {
emptyList()
}
for (ula in ulas) {
try {
java.net.Socket().use { s ->
s.connect(
java.net.InetSocketAddress(java.net.InetAddress.getByName(ula), 80),
20_000,
)
}
} catch (_: Exception) {
// Cold path / node away — the connect attempt still
// drove session establishment; try again next round.
}
}
round++
// Aggressive for the first ~minute (session bring-up), then a
// slow keep-warm tick that costs nearly nothing.
delay(if (round < 12) 5_000 else 60_000)
}
}
} }
private fun shutdown() { private fun shutdown() {
warmerJob?.cancel()
FipsNative.stop() FipsNative.stop()
stopForeground(STOP_FOREGROUND_REMOVE) stopForeground(STOP_FOREGROUND_REMOVE)
stopSelf() stopSelf()

View File

@ -185,13 +185,17 @@ fun ServerConnectScreen(
useHttps = false, useHttps = false,
port = "", port = "",
) )
// The tunnel + mesh route need a moment on cold start — and on // Mesh discovery + first session can take 15s+ through the
// a first-ever pairing the VPN consent dialog is on screen at // public tree (HANDOFF-2026-07-23 node diagnosis), and on a
// the same time, so give the user time to tap it. // first-ever pairing the VPN consent dialog is on screen at
for (attempt in 0 until 6) { // the same time — so probe patiently inside a 60s budget with
if (attempt > 0) delay(2000) // per-attempt timeouts wide enough to ride out TCP
reachable = testConnection(meshServer) // retransmit backoff. The VPN service pre-warms the session
if (reachable) break // in parallel (ArchyVpnService.startSessionWarmer).
val deadline = System.currentTimeMillis() + 60_000
while (!reachable && System.currentTimeMillis() < deadline) {
reachable = testConnection(meshServer, timeoutMs = 15_000)
if (!reachable) delay(3000)
} }
} }
isConnecting = false isConnecting = false
@ -673,8 +677,10 @@ private fun sanitizeAddress(input: String): String {
.trimEnd('/') .trimEnd('/')
} }
/** Test RPC connectivity. Accepts self-signed certs for local LAN servers. */ /** Test RPC connectivity. Accepts self-signed certs for local LAN servers.
private suspend fun testConnection(server: ServerEntry): Boolean { * [timeoutMs] is per-phase (connect / read) mesh probes need far more
* patience than LAN ones (first session through the tree can take 15s+). */
private suspend fun testConnection(server: ServerEntry, timeoutMs: Int = 5000): Boolean {
return withContext(Dispatchers.IO) { return withContext(Dispatchers.IO) {
try { try {
val url = URL("${server.toUrl()}/rpc/v1") val url = URL("${server.toUrl()}/rpc/v1")
@ -694,8 +700,8 @@ private suspend fun testConnection(server: ServerEntry): Boolean {
} }
connection.requestMethod = "POST" connection.requestMethod = "POST"
connection.connectTimeout = 5000 connection.connectTimeout = timeoutMs
connection.readTimeout = 5000 connection.readTimeout = timeoutMs
connection.setRequestProperty("Content-Type", "application/json") connection.setRequestProperty("Content-Type", "application/json")
connection.doOutput = true connection.doOutput = true
val body = """{"method":"server.echo","params":{"message":"ping"}}""" val body = """{"method":"server.echo","params":{"message":"ping"}}"""

View File

@ -87,6 +87,28 @@ import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext import kotlinx.coroutines.withContext
import org.json.JSONObject import org.json.JSONObject
/** True when a TCP listener answers at [base]'s host:port within [timeoutMs]. */
private fun tcpAnswers(base: String, timeoutMs: Int): Boolean = try {
val u = android.net.Uri.parse(base)
val port = if (u.port != -1) u.port else if (u.scheme == "https") 443 else 80
java.net.Socket().use {
it.connect(java.net.InetSocketAddress(u.host, port), timeoutMs)
true
}
} catch (_: Exception) {
false
}
/** Fastest answering origin: LAN inside a short window, else the mesh ULA
* (patient a cold session may still be establishing), else LAN anyway so
* the existing error/fallback path handles it. */
private suspend fun pickStartUrl(lanUrl: String, meshUrl: String?): String =
withContext(Dispatchers.IO) {
if (tcpAnswers(lanUrl, 2500)) return@withContext lanUrl
if (meshUrl != null && tcpAnswers(meshUrl, 12_000)) return@withContext meshUrl
lanUrl
}
/** Open a URL in the phone's default browser (genuinely external links). */ /** Open a URL in the phone's default browser (genuinely external links). */
private fun openExternalUrl(context: android.content.Context, url: String) { private fun openExternalUrl(context: android.content.Context, url: String) {
try { try {
@ -168,6 +190,20 @@ fun WebViewScreen(
var hasError by remember { mutableStateOf(false) } var hasError by remember { mutableStateOf(false) }
var webView by remember { mutableStateOf<WebView?>(null) } var webView by remember { mutableStateOf<WebView?>(null) }
// Race LAN vs mesh BEFORE the WebView exists: Chromium pointed at an
// unreachable LAN IP burns a minute+ in connect retries before
// onReceivedError fires the mesh fallback — the "stuck connecting"
// stall. A raw TCP probe answers in milliseconds at home and fails in
// ~2.5s off-LAN, so startup lands on the right origin in seconds.
var startUrl by remember(serverUrl) { mutableStateOf<String?>(null) }
var raceNonce by remember { mutableIntStateOf(0) }
LaunchedEffect(serverUrl, meshFallbackUrl, raceNonce) {
val picked = pickStartUrl(serverUrl, meshFallbackUrl)
// Starting on the mesh: don't bounce back to it on error (it IS it).
if (picked != serverUrl) triedMeshFallback = true
startUrl = picked
}
// Web-page camera access (wallet QR scanner). The WebView's default // Web-page camera access (wallet QR scanner). The WebView's default
// WebChromeClient silently denies getUserMedia, so grant video capture — // WebChromeClient silently denies getUserMedia, so grant video capture —
// asking for the app-level CAMERA permission first when needed. // asking for the app-level CAMERA permission first when needed.
@ -187,6 +223,14 @@ fun WebViewScreen(
// while this is shown, so closing it returns instantly with no reload. // while this is shown, so closing it returns instantly with no reload.
var inAppUrl by remember { mutableStateOf<String?>(null) } var inAppUrl by remember { mutableStateOf<String?>(null) }
// Same node = EITHER of its addresses. Over the mesh the kiosk's host is
// the ULA while app links may carry the LAN IP (and vice versa) —
// comparing against one host bounced same-node apps (Pine, Home
// Assistant) out to the phone's external browser.
fun isSameNode(url: String): Boolean =
isSameHost(url, serverUrl) ||
(meshFallbackUrl != null && isSameHost(url, meshFallbackUrl))
// Native wallet QR scanner, opened by the web UI via the ArchipelagoQr // Native wallet QR scanner, opened by the web UI via the ArchipelagoQr
// bridge; status lines stream back from the page while it's up. // bridge; status lines stream back from the page while it's up.
var walletScannerVisible by remember { mutableStateOf(false) } var walletScannerVisible by remember { mutableStateOf(false) }
@ -268,9 +312,13 @@ fun WebViewScreen(
GlassButton( GlassButton(
text = stringResource(R.string.retry), text = stringResource(R.string.retry),
onClick = { onClick = {
// Re-race LAN vs mesh — the network we're on may have
// changed since the last pick.
hasError = false hasError = false
isLoading = true isLoading = true
webView?.loadUrl(serverUrl) triedMeshFallback = false
startUrl = null
raceNonce++
}, },
modifier = Modifier.fillMaxWidth().height(56.dp), modifier = Modifier.fillMaxWidth().height(56.dp),
) )
@ -283,9 +331,16 @@ fun WebViewScreen(
modifier = Modifier.fillMaxWidth().height(48.dp), modifier = Modifier.fillMaxWidth().height(48.dp),
) )
} }
} else if (startUrl == null) {
// Racing LAN vs mesh (≤2.5s at home, a few seconds off-LAN) —
// far cheaper than letting Chromium retry a dead LAN IP.
Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
CircularProgressIndicator(color = BitcoinOrange)
}
} else { } else {
// Edge-to-edge WebView — background bleeds behind status bar. // Edge-to-edge WebView — background bleeds behind status bar.
// Safe area values injected as CSS env() polyfill on each page load. // Safe area values injected as CSS env() polyfill on each page load.
val initialUrl = startUrl ?: serverUrl
AndroidView( AndroidView(
modifier = Modifier.fillMaxSize(), modifier = Modifier.fillMaxSize(),
factory = { context -> factory = { context ->
@ -319,7 +374,7 @@ fun WebViewScreen(
// kiosk couldn't iframe — keep the user inside the app) // kiosk couldn't iframe — keep the user inside the app)
// - different host → the phone's real browser // - different host → the phone's real browser
fun routeOutbound(url: String) { fun routeOutbound(url: String) {
if (isSameHost(url, serverUrl)) { if (isSameNode(url)) {
inAppUrl = url inAppUrl = url
} else { } else {
openExternalUrl(context, url) openExternalUrl(context, url)
@ -458,7 +513,7 @@ fun WebViewScreen(
error: android.net.http.SslError?, error: android.net.http.SslError?,
) { ) {
val u = error?.url val u = error?.url
if (u != null && isSameHost(u, serverUrl)) { if (u != null && isSameNode(u)) {
handler?.proceed() handler?.proceed()
} else { } else {
handler?.cancel() handler?.cancel()
@ -595,7 +650,7 @@ fun WebViewScreen(
} }
webView = this webView = this
loadUrl(serverUrl) loadUrl(initialUrl)
} }
}, },
) )
@ -620,6 +675,7 @@ fun WebViewScreen(
InAppBrowser( InAppBrowser(
url = target, url = target,
serverUrl = serverUrl, serverUrl = serverUrl,
meshUrl = meshFallbackUrl,
onClose = { inAppUrl = null }, onClose = { inAppUrl = null },
) )
} }
@ -693,9 +749,14 @@ private fun fetchFavicon(pageUrl: String): Bitmap? {
private fun InAppBrowser( private fun InAppBrowser(
url: String, url: String,
serverUrl: String, serverUrl: String,
meshUrl: String? = null,
onClose: () -> Unit, onClose: () -> Unit,
) { ) {
val context = LocalContext.current val context = LocalContext.current
// Same-node check across BOTH node addresses (LAN + mesh ULA) — see the
// kiosk's isSameNode; a mismatch here bounced app links to the browser.
fun isSameNode(u: String): Boolean =
isSameHost(u, serverUrl) || (meshUrl != null && isSameHost(u, meshUrl))
var browser by remember { mutableStateOf<WebView?>(null) } var browser by remember { mutableStateOf<WebView?>(null) }
var title by remember { mutableStateOf(android.net.Uri.parse(url).host ?: url) } var title by remember { mutableStateOf(android.net.Uri.parse(url).host ?: url) }
var favicon by remember { mutableStateOf<Bitmap?>(null) } var favicon by remember { mutableStateOf<Bitmap?>(null) }
@ -809,7 +870,7 @@ private fun InAppBrowser(
error: android.net.http.SslError?, error: android.net.http.SslError?,
) { ) {
val u = error?.url val u = error?.url
if (u != null && isSameHost(u, serverUrl)) { if (u != null && isSameNode(u)) {
handler?.proceed() handler?.proceed()
} else { } else {
handler?.cancel() handler?.cancel()
@ -823,7 +884,7 @@ private fun InAppBrowser(
val u = request?.url?.toString() ?: return false val u = request?.url?.toString() ?: return false
// Stay in the overlay for same-node navigation; // Stay in the overlay for same-node navigation;
// hand genuinely external links to the real browser. // hand genuinely external links to the real browser.
if (isSameHost(u, serverUrl)) return false if (isSameNode(u)) return false
openExternalUrl(ctx, u) openExternalUrl(ctx, u)
return true return true
} }

View File

@ -86,6 +86,7 @@ dependencies = [
"getrandom 0.2.17", "getrandom 0.2.17",
"hex", "hex",
"jni", "jni",
"libc",
"paranoid-android", "paranoid-android",
"serde_json", "serde_json",
"tokio", "tokio",

View File

@ -31,6 +31,8 @@ hex = "0.4"
getrandom = "0.2" getrandom = "0.2"
tokio = { version = "1", features = ["rt-multi-thread", "sync", "time", "macros"] } tokio = { version = "1", features = ["rt-multi-thread", "sync", "time", "macros"] }
tracing = "0.1" tracing = "0.1"
# fcntl: force the VpnService TUN fd into blocking mode (see mesh::start).
libc = "0.2"
# The JNI surface only exists on Android; host builds skip it and drive the # The JNI surface only exists on Android; host builds skip it and drive the
# mesh module directly (tests). # mesh module directly (tests).

View File

@ -97,6 +97,19 @@ pub fn parse_peers(peers_json: &str) -> Result<Vec<PeerConfig>> {
pub fn start(secret: &str, peers_json: &str, tun_fd: i32) -> Result<(String, String)> { pub fn start(secret: &str, peers_json: &str, tun_fd: i32) -> Result<(String, String)> {
stop(); stop();
// Android hands the VpnService TUN fd over in non-blocking mode on some
// OS builds. The fips TUN reader is a dedicated blocking-read thread that
// treats EAGAIN as fatal — the loop died at startup ("TUN read error …
// Try again (os error 11)" on-device), so sessions came up but no packet
// ever entered the mesh. Force the fd into the blocking mode the reader
// is designed for.
unsafe {
let flags = libc::fcntl(tun_fd, libc::F_GETFL);
if flags >= 0 && (flags & libc::O_NONBLOCK) != 0 {
libc::fcntl(tun_fd, libc::F_SETFL, flags & !libc::O_NONBLOCK);
}
}
let peers = parse_peers(peers_json)?; let peers = parse_peers(peers_json)?;
let config = build_config(secret, peers); let config = build_config(secret, peers);
let mut node = Node::new(config).map_err(|e| anyhow!("node init: {e}"))?; let mut node = Node::new(config).map_err(|e| anyhow!("node init: {e}"))?;

1
core/Cargo.lock generated
View File

@ -154,6 +154,7 @@ dependencies = [
"serde_yaml", "serde_yaml",
"serial2-tokio", "serial2-tokio",
"sha2 0.10.9", "sha2 0.10.9",
"socket2 0.5.10",
"tar", "tar",
"tempfile", "tempfile",
"thiserror 1.0.69", "thiserror 1.0.69",

View File

@ -22,6 +22,9 @@ iroh-swarm = ["dep:iroh", "dep:iroh-blobs"]
[dependencies] [dependencies]
# Core dependencies # Core dependencies
tokio = { version = "1", features = ["full"] } tokio = { version = "1", features = ["full"] }
# Mesh port mirror: needs IPV6_V6ONLY on [::] listeners so they coexist with
# the containers' own 0.0.0.0 binds (std/tokio don't expose the sockopt).
socket2 = "0.5"
libc = "0.2" # process-group signalling for the supervised reticulum daemon libc = "0.2" # process-group signalling for the supervised reticulum daemon
serde = { version = "1.0", features = ["derive"] } serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0" serde_json = "1.0"

View File

@ -991,6 +991,13 @@ async fn patch_nginx_conf(path: &str) -> Result<bool> {
// B13: fedimint block present but lacking the asset-rewrite sub_filters. // B13: fedimint block present but lacking the asset-rewrite sub_filters.
let needs_fedimint_css = content.contains("location /app/fedimint/") let needs_fedimint_css = content.contains("location /app/fedimint/")
&& !content.contains("'href=\"/' 'href=\"/app/fedimint/'"); && !content.contains("'href=\"/' 'href=\"/app/fedimint/'");
// Companion mesh access: phones reach this node over FIPS at its fips0
// ULA (http://[fdxx:…]). Configs shipped before 2026-07-23 listened on
// IPv4 only, so the ULA could never connect — nothing answered [::]:80.
let missing_v6_http = content.contains("listen 80 default_server;")
&& !content.contains("listen [::]:80");
let missing_v6_https = content.contains("listen 443 ssl default_server;")
&& !content.contains("listen [::]:443");
if !missing_app_catalog if !missing_app_catalog
&& !missing_bitcoin_status && !missing_bitcoin_status
&& !missing_lnd_proxy && !missing_lnd_proxy
@ -998,12 +1005,27 @@ async fn patch_nginx_conf(path: &str) -> Result<bool> {
&& !missing_pine_status && !missing_pine_status
&& !has_lnd_dup_cors && !has_lnd_dup_cors
&& !needs_fedimint_css && !needs_fedimint_css
&& !missing_v6_http
&& !missing_v6_https
{ {
return Ok(false); return Ok(false);
} }
let mut patched = content.clone(); let mut patched = content.clone();
if missing_v6_http {
patched = patched.replace(
"listen 80 default_server;",
"listen 80 default_server;\n listen [::]:80 default_server;",
);
}
if missing_v6_https {
patched = patched.replace(
"listen 443 ssl default_server;",
"listen 443 ssl default_server;\n listen [::]:443 ssl default_server;",
);
}
if has_lnd_dup_cors { if has_lnd_dup_cors {
// Drop the redundant nginx-side CORS headers so the backend's single // Drop the redundant nginx-side CORS headers so the backend's single
// validated Access-Control-Allow-Origin is the only one returned. // validated Access-Control-Allow-Origin is the only one returned.

View File

@ -34,6 +34,7 @@ mod bitcoin_rpc;
mod bitcoin_status; mod bitcoin_status;
mod blobs; mod blobs;
mod bootstrap; mod bootstrap;
mod mesh_ports;
mod ceremony; mod ceremony;
mod config; mod config;
mod constants; mod constants;
@ -395,6 +396,10 @@ async fn main() -> Result<()> {
// iframe on kiosk nodes (docs/tv-input-iframe-apps.md). // iframe on kiosk nodes (docs/tv-input-iframe-apps.md).
tokio::spawn(bootstrap::ensure_gamepad_keys()); tokio::spawn(bootstrap::ensure_gamepad_keys());
// Mesh access: mirror IPv4-published app ports onto [::] so direct-port
// app URLs (http://[<fips0 ULA>]:<port>) work from the companion.
tokio::spawn(mesh_ports::run_mesh_port_mirror());
// Pine voice: re-point IP-pinned Wyoming satellite entries (speakers) when // Pine voice: re-point IP-pinned Wyoming satellite entries (speakers) when
// DHCP renumbering strands them — HA never re-resolves on its own. // DHCP renumbering strands them — HA never re-resolves on its own.
tokio::spawn(api::rpc::wyoming_satellite_keeper()); tokio::spawn(api::rpc::wyoming_satellite_keeper());

View File

@ -0,0 +1,145 @@
//! Mirror IPv4-published app ports onto IPv6 for mesh access.
//!
//! The companion (and anything else on the FIPS mesh) reaches this node at
//! its fips0 ULA. The web UI builds app links from the current host + each
//! app's DIRECT port (Direct Port Rule) — but rootless-podman published
//! ports bind 0.0.0.0 only, so `http://[<ULA>]:8334` was refused for every
//! catalog app even after nginx :80 learned IPv6 (2026-07-23, third
//! "the node wasn't listening on v6" bug of the night).
//!
//! Instead of touching the container layer (port-publish changes would
//! recreate every container), a reconcile loop mirrors the host's public
//! IPv4 listeners: any port ≥1024 listening on 0.0.0.0 without an IPv6
//! any-address listener gets a v6-only `[::]:<port>` forwarder to
//! `127.0.0.1:<port>`. Ports appear/disappear with app install/remove, so
//! the mirror follows `/proc/net/tcp*` rather than the catalog — companion
//! containers with hardcoded ports (bitcoin-ui :8334) are covered the same
//! as manifest-declared ones. Nothing is exposed that IPv4 didn't already
//! expose to the LAN; the ULA is the established remote-access surface.
use std::collections::{HashMap, HashSet};
use std::net::{Ipv6Addr, SocketAddrV6};
use std::time::Duration;
use anyhow::{Context, Result};
use tokio::task::JoinHandle;
use tracing::{debug, info, warn};
const RECONCILE_INTERVAL: Duration = Duration::from_secs(15);
/// Never mirror system ports; nginx (80/443) handles its own v6 listeners.
const MIN_PORT: u16 = 1024;
/// Entry point — spawned from main; runs for the process lifetime.
pub async fn run_mesh_port_mirror() {
let mut active: HashMap<u16, JoinHandle<()>> = HashMap::new();
loop {
match reconcile(&mut active).await {
Ok(()) => {}
Err(e) => debug!("mesh-ports: reconcile skipped: {e:#}"),
}
tokio::time::sleep(RECONCILE_INTERVAL).await;
}
}
async fn reconcile(active: &mut HashMap<u16, JoinHandle<()>>) -> Result<()> {
let v4 = listening_ports("/proc/net/tcp", 8).await?;
let v6 = listening_ports("/proc/net/tcp6", 32).await?;
// Drop mirrors whose backing IPv4 listener vanished (app removed/stopped).
active.retain(|port, handle| {
if v4.contains(port) && !handle.is_finished() {
true
} else {
handle.abort();
info!("mesh-ports: released [::]:{port} (backing 0.0.0.0 listener gone)");
false
}
});
for port in v4 {
if port < MIN_PORT || active.contains_key(&port) {
continue;
}
// A foreign IPv6 any-listener already serves this port (our own
// mirrors are excluded above via `active`).
if v6.contains(&port) {
continue;
}
match spawn_forwarder(port) {
Ok(handle) => {
info!("mesh-ports: mirroring [::]:{port} -> 127.0.0.1:{port}");
active.insert(port, handle);
}
Err(e) => debug!("mesh-ports: cannot mirror port {port}: {e:#}"),
}
}
Ok(())
}
/// Ports in LISTEN state bound to the any-address, parsed from /proc/net/tcp
/// (`addr_hex_len` 8) or /proc/net/tcp6 (32).
async fn listening_ports(path: &str, addr_hex_len: usize) -> Result<HashSet<u16>> {
let raw = tokio::fs::read_to_string(path)
.await
.with_context(|| format!("read {path}"))?;
let any_addr = "0".repeat(addr_hex_len);
let mut ports = HashSet::new();
for line in raw.lines().skip(1) {
let mut cols = line.split_whitespace();
let (Some(_sl), Some(local), Some(_rem), Some(state)) =
(cols.next(), cols.next(), cols.next(), cols.next())
else {
continue;
};
if state != "0A" {
continue; // not LISTEN
}
let Some((addr, port_hex)) = local.split_once(':') else {
continue;
};
if addr != any_addr {
continue;
}
if let Ok(port) = u16::from_str_radix(port_hex, 16) {
ports.insert(port);
}
}
Ok(ports)
}
/// A v6-only listener on [::]:port forwarding each connection to 127.0.0.1:port.
fn spawn_forwarder(port: u16) -> Result<JoinHandle<()>> {
use socket2::{Domain, Protocol, Socket, Type};
let socket = Socket::new(Domain::IPV6, Type::STREAM, Some(Protocol::TCP))
.context("create v6 socket")?;
// v6only so we coexist with the app's own 0.0.0.0:<port> bind.
socket.set_only_v6(true).context("set v6only")?;
socket.set_reuse_address(true).ok();
socket.set_nonblocking(true).context("set nonblocking")?;
let addr = SocketAddrV6::new(Ipv6Addr::UNSPECIFIED, port, 0, 0);
socket.bind(&addr.into()).context("bind [::]")?;
socket.listen(128).context("listen")?;
let listener = tokio::net::TcpListener::from_std(socket.into())
.context("register with tokio")?;
Ok(tokio::spawn(async move {
loop {
let (mut inbound, _peer) = match listener.accept().await {
Ok(conn) => conn,
Err(e) => {
warn!("mesh-ports: accept failed on [::]:{port}: {e}");
tokio::time::sleep(Duration::from_millis(500)).await;
continue;
}
};
tokio::spawn(async move {
match tokio::net::TcpStream::connect(("127.0.0.1", port)).await {
Ok(mut upstream) => {
let _ = tokio::io::copy_bidirectional(&mut inbound, &mut upstream).await;
}
Err(e) => debug!("mesh-ports: upstream 127.0.0.1:{port} refused: {e}"),
}
});
}
}))
}

View File

@ -48,9 +48,158 @@ user's hands.
verify `sudo -n fipsctl show status` reports the anchor connected — verify `sudo -n fipsctl show status` reports the anchor connected —
`fips.reconnect` RPC if not. A phone can dial the anchor perfectly and `fips.reconnect` RPC if not. A phone can dial the anchor perfectly and
still fail if the node never enrolled with it. still fail if the node never enrolled with it.
5. Re-test the user's exact flows with vc23 (updates any older install in 5. Re-test the user's exact flows with **vc24** (updates any older install in
place): (a) pair ON the LAN, then switch the phone to 5G — the UI must place): (a) pair ON the LAN, then switch the phone to 5G — the UI must
come up via the mesh ULA; (b) pair while ALREADY on 5G (never on the come up via the mesh ULA; (b) pair while ALREADY on 5G (never on the
node's LAN) — scan, VPN consent, and the connect must succeed through node's LAN) — scan, VPN consent, and the connect must succeed through
the anchor. If either fails, capture `adb logcat` around the attempt and the anchor.
`fipsctl show sessions` on the node, and report back.
## Live diagnosis update (22:3022:50, phone on adb — Mac agent)
vc23 on-device testing found and fixed the phone-side blocker, and narrowed
what remains to the node side. State as of vc24:
- **Fixed: TUN reader died at startup.** Android hands the VpnService fd over
non-blocking; the fips fork's blocking reader thread treats EAGAIN as fatal
("TUN read error … Try again (os error 11)") — so mesh sessions came up but
NO packet ever entered the tunnel. archy-fips-core now forces the fd
blocking before `start_with_tun_fd`. Verified on-device: reader survives,
and the 30s anchor-link flap disappeared with it (stable 8+ min on 5G).
- **Fixed: VPN marked not-metered** (`setMetered(false)`) — Android 10+
defaults VPNs to metered, putting the phone into data-restricted behaviour
while the mesh is up. Note the user's phone also has system **always-on
VPN** enabled for the app (`always_on_vpn_app`), a Settings-side toggle.
- **Verified good on-device**: peer store has node (LAN udp/tcp hints) + vps2
anchor; saved server is npub-keyed with ULA; anchor session establishes
from 5G in ~6s; VPN is bypassable, VALIDATED, only fd00::/8 routed.
- **REMAINING BLOCKER (node side)**: from the phone (app uid), ping6 and
HTTP to the node's ULA `fd79:1aa:b9e9:4c9f:1f80:5376:9385:1824` get no
reply — packets enter the mesh, nothing returns. Phone↔anchor works, so
suspect phone-fork ↔ node-daemon session/routing mismatch (FIPS wire
format is not stable across revs; phone pins fips-native fork 46494a74).
From Framework PT please capture:
- `fipsctl show status` (daemon version + anchor state)
- `fipsctl show sessions` and `show bloom` while the phone pings
- `ping6 <phone ULA fd68:496d:fe34:a06d:cf1:6e4:b6a4:3586>` from the
node (tests the reverse path)
- `ip addr show fips0` + confirm the web server listens on `[::]:80`
Report whether the node ever sees a session attempt from
`npub132c5whrsa6ccs0eylcpzaejq9uxul5ldvczz0axq78dh7fxkqj9st4uvzu`.
## Node-side diagnosis complete (23:0023:30, archi-dev-box agent)
Chain of findings, each verified live:
1. **FIXED: nginx had no IPv6 listener anywhere** — every shipped config
listened on 0.0.0.0 only, so `http://[<ULA>]` could NEVER connect on any
node, ever. Live-fixed on framework-pt + .116, canonical conf + bootstrap
self-heal shipped (`1e89362e`), heal binary deployed. ULA HTTP verified
answering on both nodes (local + over-mesh).
2. Firewall clean, fd00::/8 routes correct both ends, wire compat proven
(node + vps2 anchor both run fips 0.4.1 rev 15db6471db — the latest
upstream stable; nothing newer exists).
3. **THE REMAINING PROBLEM IS MESH SESSION PATH QUALITY.** From the vps2
anchor — a DIRECT connected peer — `GET /health` on the node's ULA takes
**1517 s per request and intermittently fails outright** (nginx logs
show 499 client-gave-up then 200; TCP SYN-retransmit backoff signature).
Session MMP is wildly asymmetric: node→vps2 srtt 204 ms, **vps2→node
srtt 4270 ms** — on a direct link whose raw RTT is 4 ms. Session traffic
is not riding the direct link; it appears to route through the ~1271-node
public tree (node's tree root is the public 00001a8c, depth 8; the node's
log also shows chronic "Discovery lookup timed out" for other targets).
4. The phone's npub never appears in the node's sessions — consistent with
discovery/handshake dying on the same degraded tree path, and the app's
~8 s probe window being far smaller than the observed 15 s+ first-request
latency even on the GOOD path.
### Recommendations
- **App side (Mac agent):** widen the ULA probe/connect window to ≥30 s
with retransmit-friendly pacing, and PRE-WARM the mesh session (start
pinging the node ULA as soon as the VPN is up, decoupled from the UI
probe) so the WebView hits a warm session.
- **Infra decision (user):** consider detaching the fleet from the public
v0l mesh — private tree rooted at the vps2 anchor (drop the legacy
185.18.221.160 seed anchor fleet-wide AND vps2's public peering). A
2-hop private tree would make session paths ride the direct links and
should collapse latency to ms. Trade-off: no reachability to/from the
broader public mesh.
- **Upstream:** report the direct-peer session-path asymmetry to
jmcorgan/fips (0.4.1).
## App-side recommendations implemented (23:3023:50, Mac agent — 0.5.5/vc25)
- Connect probe: mesh ULA now probed inside a **60s budget** with 15s
per-phase timeouts (rides out TCP retransmit backoff), replacing the old
~8s window.
- **Session pre-warm**: the VPN service starts probing every saved node ULA
the moment the tunnel is up (5s cadence for the first minute, then a 60s
keep-warm tick) — discovery/handshake cost is paid in the background, and
the session never idles out while the mesh is connected.
- (A phone-side ping test in this window still showed zero replies — that
measurement predated the vps2 daemon restart below and is superseded.)
## RESOLVED — root cause was vps2's degraded daemon, NOT the public tree (23:45)
The privatize-the-mesh recommendation above is WITHDRAWN. Final diagnosis:
vps2's fips daemon (3 days uptime, 0.2% CPU, idle box) had internally
degraded — EVERY link it carried showed ~4.5 s RTT (even to peers 30 ms
away), and since the anchor sits on the phone↔node path, everything through
it inherited that. `systemctl restart fips` on vps2 restored link RTTs to
40340 ms, and anchor→node mesh HTTP went from 1417 s (intermittent hard
fails) to a steady **165275 ms**. Node↔node direct sessions were always
fine (.116→framework-pt ULA HTTP: 894 ms cold, sub-second warm) — the
user's read was correct.
Actions taken: dead legacy anchor (185.18.221.160) removed from
framework-pt + .116 seed files (fleet keeps vps2 + public-mesh membership
via vps2 — we stay in the open mesh); fresh daemons on both nodes;
**vps2 fips now has RuntimeMaxSec=1d + Restart=always** so a wedged anchor
daemon can never rot for days again. Report the slow-degradation behaviour
upstream (jmcorgan/fips, 0.4.1): long-running daemon in a ~1400-node mesh
accumulates multi-second link latency at idle CPU, cleared by restart.
Phone side: vc25's 60 s probe + pre-warm now has a millisecond-latency mesh
to work with. Ready for the user's 5G test.
## NEXT (00:05, Mac agent → dev-box agent): app direct ports are IPv4-only over the mesh
The kiosk loads over the ULA now — but opening any APP dies with
`ERR_CONNECTION_REFUSED` at `http://[<ULA>]:<port>/`. User-hit first on
**Bitcoin Knots (:8334)**, and it will be every catalog app: the web UI
builds app URLs from the current host + the app's DIRECT port (Direct Port
Rule), and container-published ports only bind 0.0.0.0. Verified:
`192.168.63.249:8334` → HTTP 200 (nginx), ULA:8334 → refused. Same disease
as your :80 nginx fix, one layer down.
Fix must cover EVERY catalog app port and survive app install/remove. Two
shapes; pick what fits the container layer best:
1. **IPv6 publish at the container layer** — publish on `[::]` too
(pasta/rootless podman support address-specific `-p`), wired into the
container manager so new apps inherit it; or
2. **Host-side v6→v4 forwarders** — generated nginx `stream {}` (or
systemd-socket) units: `listen [::]:<port>``127.0.0.1:<port>`, one per
catalog app port, regenerated on app install/remove, boot-time
self-healed like the :80 fix. Keeps the Direct Port Rule URL contract
without touching containers.
Either way: extend the bootstrap self-heal, and verify from the MESH side
(curl the ULA on 23 app ports incl. :8334 from vps2 or .116) — not just
from the LAN.
## DONE (00:30, dev-box agent): app direct ports live over the mesh
Shape 2-variant implemented INSIDE the backend (`mesh_ports.rs`, `2ad57c63`):
a reconcile loop mirrors every public IPv4 listener (>=1024, bound 0.0.0.0,
no existing IPv6 any-listener) as a v6-ONLY `[::]:<port>` forwarder to
`127.0.0.1:<port>`, following `/proc/net/tcp*` every 15s — so app
install/remove and hardcoded companion ports (bitcoin-ui :8334) are covered
with zero container changes and no generated units; self-healing because it
lives in the binary. Strictly ADDITIVE: IPv4/LAN/Tor paths untouched, v6only
cannot intercept v4, foreign IPv6 listeners win.
Verified FROM THE MESH (vps2 → node ULA): :8334 HTTP 200 (466ms),
:18083 200 (306ms), :50002 200 (239ms); LAN :8334 still 200. Deployed to
framework-pt + .116 (binary sha 52ac0d8a…). Direct-port apps should now
open in the companion over 5G.

View File

@ -9,6 +9,10 @@ resolver_timeout 5s;
server { server {
listen 80 default_server; listen 80 default_server;
# IPv6 listener is REQUIRED: companion phones reach this node over the
# FIPS mesh at its fips0 ULA (http://[fdxx:…]) — without [::]:80 that
# address can never connect (found live 2026-07-23, framework-pt).
listen [::]:80 default_server;
server_name _; server_name _;
root /opt/archipelago/web-ui; root /opt/archipelago/web-ui;
@ -901,6 +905,7 @@ server {
# HTTPS - required for PWA install (Add to Home Screen) from dev servers # HTTPS - required for PWA install (Add to Home Screen) from dev servers
server { server {
listen 443 ssl default_server; listen 443 ssl default_server;
listen [::]:443 ssl default_server;
server_name _; server_name _;
ssl_certificate /etc/archipelago/ssl/archipelago.crt; ssl_certificate /etc/archipelago/ssl/archipelago.crt;