Complete private gateway and local website publishing UAT

scripts/public-web-gateway/README.md

@@ -0,0 +1,72 @@
+# Owned public-web gateway admission
+
+`policy.py` implements the frp server-plugin contract for an operator-owned
+HTTPS passthrough gateway. The node-side app installs from the trusted catalogue; an operator provisions
+the gateway separately and supplies the private enrollment file to Setup.
+
+Run it bound to loopback alongside frps. Configure **all** operations `Login`,
+`NewProxy`, `Ping`, `NewWorkConn`, and `NewUserConn`; omitting them weakens
+revocation. Require TLS on frps and pin the gateway CA on every client. Retain
+frp token authentication with `HeartBeats` and `NewWorkConns` additional scopes.
+Do not publish its enrollment file, client config, or transport credentials.
+
+The 0600 enrollment JSON maps a node name to `enabled`, the SHA-256 of a random
+32-byte-or-longer `enrollment_token`, and exact lowercase `domains`. Set frpc
+`user` to the node name and `metadatas.enrollment_token` to that token. Proxies
+must be HTTPS with one assigned domain. TCP/UDP, wildcard subdomains, shared
+proxy groups, and gateway-side content rewrites are refused. The node terminates
+website TLS and owns its website keys. The gateway still observes SNI and traffic
+metadata; passthrough is not an anonymity service.
+
+Replace the policy file atomically to enroll, disable or rotate a node. Every
+request reloads it; missing, malformed or nonprivate files fail closed. Disabling
+an enrollment denies new connections and subsequent heartbeats. Already forwarded
+bytes cannot be recalled; do not promise immediate termination of every stream.
+The process never logs tokens or request bodies. Run behind a dedicated service
+account with filesystem and process limits in the final deployment.
+
+Tests: `python3 -m unittest discover -s tests/public-web-gateway -v`.
+
+Contract references:
+- https://gofrp.org/en/docs/features/common/server-plugin/
+- https://gofrp.org/en/docs/features/common/network/network-tls/
+- https://github.com/fatedier/frp/blob/v0.71.0/pkg/auth/token.go
+
+The isolated Yaya qualification uses 17400 and14443, preserving existing public
+sites. Its private certificate verifies TLS passthrough and ownership, not public
+ACME issuance. Production ACME requires an appropriate public 443 route.
+
+## Enroll a node
+
+On the gateway, use the existing private frps JSON configuration and public CA
+certificate. Keep the admission listener on loopback. For example:
+
+```sh
+python3 enroll.py --frps-config /etc/archy-gateway/frps.json \
+  --policy /etc/archy-gateway/enrollments.json \
+  --ca /etc/archy-gateway/gateway.crt \
+  --host gateway.example.com --tls-server-name gateway.example.com \
+  --name my-node --domain www.example.com \
+  --output /secure/path/my-node-enrollment.json
+```
+
+Repeat `--domain` for separately assigned website/app names. Existing enrollments
+require explicit `--rotate`; use a new output filename. Transfer the file privately
+to the node owner. In Setup → Allow external connections → Public web, install
+Public Web Router, choose the file, review the gateway/domains, and confirm.
+Then connect a published website or a guest-enabled app to an assigned domain.
+Neither importing the file nor connecting an app grants a guest token.
+
+Set each public DNS A/AAAA record to the gateway's reachable public address.
+The gateway needs its frps control port and a dedicated TCP 443 passthrough
+listener. Public certificate issuance cannot be tested by pointing DNS at a
+private LAN address or by using our isolated 14443 test port. If 443 already
+serves other sites, retain that proxy and use a separately provisioned IP/path;
+do not replace the existing listener blindly. Run frps and the policy as
+persistent supervised services before production use. The Yaya qualification
+services are intentionally isolated test services, not a production deployment.
+
+For revocation, atomically replace the private policy with the node's `enabled`
+set to false. For local disconnect, use Setup; it removes enrollment/routes while
+preserving local certificates and drafts. Removing a route does not erase copies
+of content that visitors previously downloaded.

scripts/public-web-gateway/enroll.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""Create a private node enrollment for an existing operator-owned frps gateway.
+
+Reads frps token configuration without printing credentials. The admission policy
+is replaced atomically; an existing node requires --rotate to replace its token.
+The exported enrollment is for Setup's file picker, never Nostr or public storage.
+"""
+import argparse
+import hashlib
+import ipaddress
+import json
+import os
+from pathlib import Path
+import secrets
+import ssl
+import fcntl
+import tempfile
+from policy import DOMAIN, NAME
+
+
+def atomic(path, value):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    fd, stage = tempfile.mkstemp(prefix='.' + path.name, dir=path.parent)
+    try:
+        with os.fdopen(fd, 'w') as f:
+            os.fchmod(f.fileno(), 0o600)
+            json.dump(value, f); f.flush(); os.fsync(f.fileno())
+        os.replace(stage, path)
+        directory = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
+        try: os.fsync(directory)
+        finally: os.close(directory)
+    finally:
+        Path(stage).unlink(missing_ok=True)
+
+
+def main():
+    os.umask(0o077)
+    p = argparse.ArgumentParser(description=__doc__)
+    p.add_argument('--frps-config', type=Path, required=True)
+    p.add_argument('--policy', type=Path, required=True)
+    p.add_argument('--ca', type=Path, required=True)
+    p.add_argument('--host', required=True)
+    p.add_argument('--tls-server-name', required=True)
+    p.add_argument('--name', required=True)
+    p.add_argument('--domain', action='append', required=True)
+    p.add_argument('--output', type=Path, required=True)
+    p.add_argument('--rotate', action='store_true')
+    args = p.parse_args()
+    if not NAME.fullmatch(args.name) or len(args.domain) > 32 or any(not DOMAIN.fullmatch(d) for d in args.domain):
+        p.error('Use a lowercase node name and exact lowercase domains')
+    for host in (args.host, args.tls_server_name):
+        try: ipaddress.ip_address(host)
+        except ValueError:
+            if not DOMAIN.fullmatch(host): p.error('Invalid gateway host or TLS name')
+    if args.output.exists(): p.error('Enrollment output already exists; choose a new private file')
+    if args.frps_config.stat().st_mode & 0o077: p.error('frps configuration must be private (0600)')
+    server = json.loads(args.frps_config.read_text())
+    auth = server.get('auth', {})
+    if auth.get('method') != 'token' or not isinstance(auth.get('token'), str) or len(auth['token']) < 32:
+        p.error('Gateway requires a strong frps transport token')
+    if server.get('transport', {}).get('tls', {}).get('force') is not True:
+        p.error('Gateway must require TLS')
+    required = {'Login', 'NewProxy', 'Ping', 'NewWorkConn', 'NewUserConn'}
+    if not any(required.issubset(plugin.get('ops', [])) for plugin in server.get('httpPlugins', [])):
+        p.error('Configure the admission plugin for every required operation first')
+    args.policy.parent.mkdir(parents=True, exist_ok=True)
+    policy_lock = args.policy.with_suffix(args.policy.suffix + ".lock").open("a")
+    os.chmod(policy_lock.name, 0o600)
+    fcntl.flock(policy_lock, fcntl.LOCK_EX)
+    existing = {}
+    if args.policy.exists():
+        if args.policy.stat().st_mode & 0o077: p.error('Admission policy must be private (0600)')
+        existing = json.loads(args.policy.read_text())
+    if not isinstance(existing, dict): p.error('Invalid admission policy')
+    if args.name in existing and not args.rotate: p.error('Node already enrolled; use --rotate explicitly')
+    for name, entry in existing.items():
+        if name != args.name and set(entry.get('domains', [])) & set(args.domain):
+            p.error('A domain is already assigned to another enrollment')
+    ca = args.ca.read_text()
+    if len(ca) > 16384 or not ca.startswith('-----BEGIN CERTIFICATE-----') or 'PRIVATE KEY' in ca:
+        p.error('Supply only the public gateway CA certificate')
+    try: ssl.create_default_context().load_verify_locations(cadata=ca)
+    except ssl.SSLError: p.error("Invalid gateway CA certificate")
+    token = secrets.token_hex(32)
+    enrollment = {'host': args.host, 'port': server['bindPort'], 'node_id': args.name,
+                  'tls_server_name': args.tls_server_name, 'transport_token': auth['token'],
+                  'enrollment_token': token, 'ca_pem': ca, 'domains': args.domain}
+    # Save the recoverable private output before changing admission. A failed
+    # policy write leaves a file that is not yet enrolled, never a lost token.
+    atomic(args.output, enrollment)
+    existing[args.name] = {'enabled': True, 'token_sha256': hashlib.sha256(token.encode()).hexdigest(), 'domains': args.domain}
+    atomic(args.policy, existing)
+    print('Private enrollment written. Import it in Setup; do not publish it.')
+
+
+if __name__ == '__main__': main()

scripts/public-web-gateway/policy.py

@@ -0,0 +1,113 @@
+#!/usr/bin/env python3
+"""Local frps admission plugin. Reload enrollments for every request.
+
+The enrollment file is private operator configuration, never a public catalogue.
+Transport must require TLS; the node pins the gateway CA. No bearer value is
+logged. An unavailable/malformed policy rejects requests, including heartbeats.
+"""
+import argparse
+import hashlib
+import hmac
+import json
+import re
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from urllib.parse import parse_qs, urlsplit
+
+OPS = {'Login', 'NewProxy', 'Ping', 'NewWorkConn', 'NewUserConn'}
+NAME = re.compile(r'[a-z0-9][a-z0-9-]{0,47}\Z')
+DOMAIN = re.compile(r'(?=.{1,253}\Z)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}\Z')
+
+
+def authorize(op, content, enrollments):
+    if op not in OPS or not isinstance(content, dict):
+        return False
+    user = content if op == 'Login' else content.get('user')
+    if not isinstance(user, dict):
+        return False
+    name = user.get('user')
+    if not isinstance(name, str) or not NAME.fullmatch(name):
+        return False
+    entry = enrollments.get(name)
+    if not isinstance(entry, dict) or entry.get('enabled') is not True:
+        return False
+    metas = user.get('metas', {})
+    token = metas.get('enrollment_token') if isinstance(metas, dict) else None
+    expected = entry.get('token_sha256')
+    if not isinstance(token, str) or not 32 <= len(token) <= 256:
+        return False
+    if not isinstance(expected, str) or not re.fullmatch('[a-f0-9]{64}', expected):
+        return False
+    if not hmac.compare_digest(hashlib.sha256(token.encode()).hexdigest(), expected):
+        return False
+    domains = entry.get('domains')
+    if not isinstance(domains, list) or not domains or len(domains) > 32:
+        return False
+    if any(not isinstance(d, str) or not DOMAIN.fullmatch(d) for d in domains):
+        return False
+    if op in {'NewProxy', 'NewUserConn'}:
+        # frpc prefixes proxy names with its configured user.
+        proxy = content.get('proxy_name', '')
+        if not isinstance(proxy, str) or not proxy.startswith(name + '.'):
+            return False
+        if not NAME.fullmatch(proxy[len(name) + 1:]):
+            return False
+        if content.get('proxy_type') != 'https':
+            return False
+    if op == 'NewProxy':
+        requested = content.get('custom_domains')
+        if not isinstance(requested, list) or len(requested) != 1 or requested[0] not in domains:
+            return False
+        # No arbitrary TCP ports, wildcard subdomains, shared groups or routing
+        # rewrites. TLS terminates on the node; gateway only forwards SNI.
+        if any(content.get(k) for k in ('remote_port', 'subdomain', 'group', 'group_key', 'locations', 'host_header_rewrite', 'headers', 'http_user', 'http_pwd', 'multiplexer')):
+            return False
+    return True
+
+
+class Handler(BaseHTTPRequestHandler):
+    def log_message(self, *_):
+        pass
+
+    def do_POST(self):
+        accepted = False
+        try:
+            self.connection.settimeout(3)
+            url = urlsplit(self.path)
+            query = parse_qs(url.query, strict_parsing=True)
+            size = int(self.headers.get('Content-Length', '0'))
+            if url.path != '/handler' or query.get('version') != ['0.1.0'] or len(query.get('op', [])) != 1 or not 0 < size <= 65536:
+                raise ValueError('Invalid request')
+            if self.headers.get('Transfer-Encoding'):
+                raise ValueError('Streaming request unsupported')
+            config = self.server.policy_path
+            if config.stat().st_mode & 0o077:
+                raise ValueError('Enrollment file must be private')
+            raw = config.read_bytes()
+            if len(raw) > 1024 * 1024:
+                raise ValueError('Oversized policy')
+            enrollments = json.loads(raw)
+            request = json.loads(self.rfile.read(size))
+            accepted = authorize(query['op'][0], request['content'], enrollments)
+        except (OSError, ValueError, TypeError, KeyError, AttributeError):
+            pass
+        body = json.dumps({'reject': not accepted, 'unchange': True, 'reject_reason': '' if accepted else 'Enrollment or route is not authorized'}).encode()
+        self.send_response(200)
+        self.send_header('Content-Type', 'application/json')
+        self.send_header('Content-Length', str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--enrollments', type=Path, required=True)
+    parser.add_argument('--port', type=int, default=17700)
+    args = parser.parse_args()
+    server = ThreadingHTTPServer(('127.0.0.1', args.port), Handler)
+    server.policy_path = args.enrollments
+    server.serve_forever()
+
+
+if __name__ == '__main__':
+    main()