Merge branch 'pr/fix/yaya-alpha-uat-shutdown-20261009(5bd850d3)'

README.md

@@ -1,6 +1,6 @@
 # Archipelago
 
-> **Alpha testing:** Archipelago is experimental software. Any funds you put on it are at your own risk.
+> **Alpha testing — funds at your own risk.** Archipelago is experimental software and is not production-ready. Any funds you put on it are at your own risk. Substantial security hardening is planned before production readiness; current builds are for testing and feedback.
 
 > Self-sovereign Bitcoin node OS and manifest-driven app platform.
 
@@ -13,14 +13,19 @@ Podman containers managed by the Rust backend.
 [![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 [![Rust](https://img.shields.io/badge/rust-stable-orange)](https://www.rust-lang.org/)
 [![Vue.js](https://img.shields.io/badge/vue.js-3.5-brightgreen)](https://vuejs.org/)
-[![Version](https://img.shields.io/badge/version-1.8.13--alpha-blue)](https://source.archipelago-foundation.org/lfg2025/archy/releases)
+[![Version](https://img.shields.io/badge/version-1.9.0--alpha-blue)](https://source.archipelago-foundation.org/lfg2025/archy/releases)
 
 ## Current release
 
-The current pre-release is **v1.8.13-alpha**. Release notes and signed OTA
+The latest published pre-release is **v1.9.0-alpha**. Download the
-artifacts are published on [Gitea](https://source.archipelago-foundation.org/lfg2025/archy/releases).
+[x86_64 server installer ISO](https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.9.0-alpha/archipelago-installer-1.9.0-alpha-unbundled-x86_64.iso)
-The same source is mirrored through ngit for Nostr-native cloning and
+or read the [release notes and known limitations](https://source.archipelago-foundation.org/lfg2025/archy/releases/tag/v1.9.0-alpha).
-contribution:
+Newer changes on `main` and private UAT deployments are not included in that
+published ISO. Check the [releases page](https://source.archipelago-foundation.org/lfg2025/archy/releases)
+for subsequent published installers and signed OTA artifacts.
+
+**ngit is the canonical source contribution and review platform.** Gitea mirrors
+accepted source and hosts release downloads. For Nostr-native cloning:
 
 ```
 nostr://npub1w3sqdkrhn0gyuvsex32effzgnfpyde6qrrc4u467flg5e9txh4wsfn5vjg/relay.ngit.dev/archy
@@ -29,6 +34,50 @@ nostr://npub1w3sqdkrhn0gyuvsex32effzgnfpyde6qrrc4u467flg5e9txh4wsfn5vjg/relay.ng
 Clone with ngit, or use the Gitea mirror when you need a conventional Git
 remote. Contributions should follow [CONTRIBUTING.md](CONTRIBUTING.md).
 
+## Install on a server
+
+Use a dedicated **x86_64 Intel/AMD server, mini PC, or PC**, an SSD, and an
+8 GB or larger USB drive. For Bitcoin and multiple apps, 8 GB or more RAM and
+a generously sized SSD are recommended; an unpruned Bitcoin node needs room
+for the growing blockchain. Connect Ethernet and a monitor/keyboard, or use
+your server's remote console and virtual installation media.
+
+1. **Download the installer and checksum:**
+   - [Archipelago 1.9.0-alpha ISO](https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.9.0-alpha/archipelago-installer-1.9.0-alpha-unbundled-x86_64.iso) (approximately 2.7 GB).
+   - [SHA-256 checksum](https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.9.0-alpha/archipelago-installer-1.9.0-alpha-unbundled-x86_64.iso.sha256).
+   - [Signed checksum JSON](https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.9.0-alpha/archipelago-installer-1.9.0-alpha-unbundled-x86_64.iso.sha256.json).
+2. **Verify the download.** Save the ISO and `.sha256` file together, then on
+   Linux run:
+
+   ```bash
+   sha256sum --check archipelago-installer-1.9.0-alpha-unbundled-x86_64.iso.sha256
+   ```
+
+   The result must be `OK`. This checks file integrity; the signed JSON is
+   provided separately for release-signature verification.
+3. **Write the ISO to USB** using [Balena Etcher](https://etcher.balena.io/) or
+   your preferred image writer. Write the image rather than copying the ISO
+   into the USB filesystem. For a remotely managed server, attach the ISO as
+   virtual installation media instead. This is a raw `.iso`; no decompression
+   is needed.
+4. **Boot the server from that media.** Disable Secure Boot for this installer
+   and follow the console installation prompts. **Installation erases the
+   selected target disk**, so back up its contents and check the disk selection
+   carefully.
+5. **Remove/eject the installation media and boot from the installed disk.**
+   Find the server's address in your router's DHCP client list or local console,
+   then open `http://<server-ip>` from another device on the same network.
+6. **Complete first-run setup:** create your dashboard password and follow the
+   onboarding wizard to choose apps and Bitcoin storage settings. The unbundled
+   ISO downloads app images as needed, so allow internet access for app setup.
+
+For an existing Archipelago server, use the dashboard's **Settings** update
+flow for a published OTA rather than reinstalling. See the
+[user walkthrough](docs/user-walkthrough.md) for onboarding and daily use.
+
+**This remains alpha software: funds are at your own risk, and production
+security hardening is still ahead.**
+
 ## What is here
 
 - `core/` - Rust workspace: backend API, container runtime, security, OpenWrt

scripts/indeehub-maintenance-controller.py

@@ -12,6 +12,29 @@ QUEUE_COUNTS = frozenset(('active','waiting','paused','delayed','failed','comple
 def valid_queue_counts(counts):
     return isinstance(counts,dict) and set(counts)==QUEUE_COUNTS and all(type(v) is int and v>=0 for v in counts.values())
 BUSINESS_COUNTS = frozenset(('projects','contents','payments','shareholders','subscriptions','library_items','other_active_transactions'))
+# A bounded compatibility path for the already-upgraded API on alpha UAT nodes.
+# This is NOT permission to interrupt a populated payment system. Every monetary
+# history/intent must be absent, revenue zero, and payment providers unconfigured.
+UAT_API_MAIN_SHA256 = '6ff3d4fa5d6a17c530a8e69b2b8b65ec8214c03e71f9a8cf3a27dd53702f1120'
+UAT_EMPTY_TABLES = ('payments','payouts','rents','season_rents','subscriptions',
+    'library_items','zap_stats','archipelago_rental_entitlements',
+    'archipelago_registration_intents','archipelago_publication_outbox','archipelago_publications')
+UAT_ZERO_COUNTS = frozenset(UAT_EMPTY_TABLES) | {'nonzero_revenue','other_active_transactions'}
+UAT_API_PROBE = r'''const fs=require('fs'),crypto=require('crypto');
+const keys=['BTCPAY_API_KEY','BTCPAY_STORE_ID','BTCPAY_URL','BTCPAY_SERVER_URL','STRIKE_API_KEY'];
+const port=Number(process.env.PORT||4000).toString(16).toUpperCase().padStart(4,'0');
+const sockets=['/proc/net/tcp','/proc/net/tcp6'].flatMap(p=>fs.readFileSync(p,'utf8').trim().split('\n').slice(1));
+console.log(JSON.stringify({main_sha256:crypto.createHash('sha256').update(fs.readFileSync('/app/dist/main.js')).digest('hex'),
+ http_connections_absent:sockets.every(s=>{const c=s.trim().split(/\s+/);return !c[1].endsWith(':'+port)||['0A','06','07'].includes(c[3]);}),
+ providers_absent:keys.every(k=>!process.env[k]),
+ registration_disabled:process.env.ARCHIPELAGO_REGISTRATION_ENABLED==='false',
+ publication_disabled:process.env.ARCHIPELAGO_PUBLICATION_ENABLED==='false'}));'''
+def valid_money_free_uat(counts, probe):
+    return (isinstance(counts,dict) and set(counts)==UAT_ZERO_COUNTS
+        and all(type(v) is int and v==0 for v in counts.values())
+        and isinstance(probe,dict) and all(type(probe.get(k)) is bool for k in ('providers_absent','http_connections_absent','registration_disabled','publication_disabled'))
+        and probe=={'main_sha256':UAT_API_MAIN_SHA256,'providers_absent':True,'http_connections_absent':True,
+            'registration_disabled':True,'publication_disabled':True})
 def valid_empty_business(counts):
     return isinstance(counts,dict) and set(counts)==BUSINESS_COUNTS and all(type(v) is int and v==0 for v in counts.values())
 def valid_empty_queue(observed):
@@ -350,8 +373,46 @@ class Controller:
         sql="SELECT json_build_object("+fields+",'other_active_transactions',(SELECT count(*) FROM pg_stat_activity WHERE datname=current_database() AND pid<>pg_backend_pid() AND state<>'idle'))"
         counts=json.loads(self.run(['podman','exec','indeedhub-postgres','psql','-XAt','-U','indeedhub','-d','indeedhub','-c',sql]))
         require(set(counts)==set(tables)|{'other_active_transactions'},'Legacy API business-state observation incomplete')
-        require(all(type(value) is int and value==0 for value in counts.values()),'Legacy API has business work or active transactions; completion cannot be inferred')
+        if not all(type(value) is int and value==0 for value in counts.values()):
+            require(all(type(counts[name]) is int and counts[name]==0 for name in ('payments','subscriptions','library_items','other_active_transactions')),'Legacy API has business work or active transactions; completion cannot be inferred')
+            # Keep the original empty-business rule for unknown/older APIs. The
+            # known shutdown-aware API can retain free draft projects, provided
+            # there is no monetary capability or history and no active writer.
+            self.money_free_uat_idle()
+            return
         self.record['legacy_api_empty_state']=counts;self.save()
+    def money_free_uat_counts(self):
+        fields=','.join("'%s',(SELECT count(*) FROM public.%s)"%(name,name) for name in UAT_EMPTY_TABLES)
+        sql="SELECT json_build_object("+fields+",'nonzero_revenue',(SELECT count(*) FROM public.shareholders WHERE pending_revenue IS NULL OR rent_pending_revenue IS NULL OR pending_revenue<>0 OR rent_pending_revenue<>0),'other_active_transactions',(SELECT count(*) FROM pg_stat_activity WHERE datname=current_database() AND pid<>pg_backend_pid() AND state<>'idle'))"
+        return json.loads(self.run(['podman','exec','indeedhub-postgres','psql','-XAt','-U','indeedhub','-d','indeedhub','-c',sql]))
+    def money_free_uat_idle(self):
+        self.holds();self.fence_matches()
+        require(self.record.get('ingress_closed') is True,'UAT API admission is not closed')
+        for name in ('indeedhub','indeedhub-ffmpeg'):
+            require(self.record.get('stopped',{}).get(name,{}).get('confirmed') is True,'UAT frontend/worker not stopped')
+        member=next(m for m in self.record['original_members'] if m['name']=='indeedhub-api')
+        actual=self.inspect(member['name'])
+        require(actual['Id']==member['container_id'] and actual['Image']==member['image_id'] and actual['State']['Running'],'UAT API identity changed')
+        require(self.record.get('queue_pause_confirmed') is True and valid_queue_counts(self.record.get('last_queue_counts')) and all(v==0 for v in self.record['last_queue_counts'].values()),'UAT queue is not paused and empty')
+        probe=json.loads(self.run(['podman','exec',member['container_id'],'node','-e',UAT_API_PROBE]))
+        counts=self.money_free_uat_counts()
+        require(valid_money_free_uat(counts,probe),'Legacy API has business work or active transactions; no qualified money-free shutdown path')
+        # Capture committed data before signalling, then require exact equality
+        # after shutdown. A changed row refuses forward cutover; native recovery
+        # retains these live rows instead of restoring an older database.
+        before=self.database_commitments()
+        self.record['money_free_uat']={'operation_id':self.operation,'container_id':member['container_id'],
+            'image_id':member['image_id'],'probe':probe,'counts':counts,'database_before_signal':before}
+        self.save()
+    def verify_money_free_uat_stopped(self, member):
+        proof=self.record.get('money_free_uat')
+        require(isinstance(proof,dict) and proof.get('operation_id')==self.operation
+            and proof.get('container_id')==member['container_id'] and proof.get('image_id')==member['image_id']
+            and valid_money_free_uat(proof.get('counts'),proof.get('probe')),'Money-free UAT shutdown proof missing')
+        self.holds();self.fence_matches();self.require_api_stopped(member)
+        require(valid_money_free_uat(self.money_free_uat_counts(),proof['probe']),'Monetary state changed during UAT shutdown')
+        require(self.database_commitments()==proof['database_before_signal'],'Committed data changed during UAT shutdown; retain live data for recovery')
+        proof['stopped_data_verified']=True;self.save()
     def api_recovery_identity(self, member, role="api"):
         self.holds();self.fence_matches()
         require(role in ('api','relay') and member['name']=='indeedhub-'+role,'Legacy signal member required')
@@ -489,7 +550,9 @@ class Controller:
         self.api_recovery_identity(member)
         stopped=self.record['stopped'][member['name']];signal=stopped.get('api_signal',{})
         require(signal.get('operation_id')==self.operation and signal.get('container_id')==member['container_id'] and signal.get('acknowledged') is True and signal.get('intent_at',0)>=stopped['intent_at'],'Legacy API signal proof missing')
-        require(valid_empty_business(self.record.get('legacy_api_empty_state')),'Legacy API empty-business proof missing')
+        money_free=self.record.get('money_free_uat') is not None
+        if money_free:self.verify_money_free_uat_stopped(member)
+        else:require(valid_empty_business(self.record.get('legacy_api_empty_state')),'Legacy API empty-business proof missing')
         require(valid_api_process_proof(signal.get('proof')) and valid_empty_queue(signal.get('before_queue')) and type(signal.get('acknowledged_at')) in (int,float) and signal['acknowledged_at']>=signal['intent_at'],'Legacy API durable signal proof incomplete')
         state=dict(line.split('=',1) for line in properties.splitlines() if '=' in line)
         require(state.get('ActiveState')=='failed' and state.get('SubState')=='failed' and state.get('ExecMainStatus')=='1' and state.get('Result')=='exit-code','Unrecognized API wrapper exit')
@@ -497,7 +560,7 @@ class Controller:
         self.require_api_stopped(member)
         require(isinstance(signal.get('queue_binding'),dict),'API queue binding proof missing')
         after=self.observe_empty_redis_queue(member,signal['prefix'],signal['queue_binding'])
-        return {'classification':'empty-business-legacy-api-child-signal-termination','graceful':False,'completed_work_claim':False,'process_dead':True,'signal':signal,'after_queue':after}
+        return {'classification':('money-free-uat-api-child-signal-termination' if money_free else 'empty-business-legacy-api-child-signal-termination'),'graceful':False,'completed_work_claim':False,'process_dead':True,'signal':signal,'after_queue':after}
     def legacy_idle_worker_termination(self, member, properties):
         # Compatibility only for the observed original Node-as-PID1 worker.
         # A timeout/SIGKILL is never renamed graceful or completed work.
@@ -583,14 +646,17 @@ class Controller:
         code=matching[-1].get('ContainerExitCode',matching[-1].get('containerExitCode'))
         idle_worker = name=='indeedhub-ffmpeg' and self.record.get('queue_pause_confirmed') is True and valid_queue_counts(self.record.get('last_queue_counts')) and self.record['last_queue_counts']['active']==0
         empty_api = name=='indeedhub-api' and self.record.get('legacy_api_empty_state') is not None
+        money_free_api = name=='indeedhub-api' and self.record.get('money_free_uat') is not None
+        if money_free_api:self.verify_money_free_uat_stopped(member)
         forced=self.legacy_idle_worker_termination(member,properties) if str(code)=='137' else None
         if name=='indeedhub-api' and str(code)=='1':forced=self.legacy_api_wrapper_termination(member,properties)
         require(forced is not None or ('ActiveState=inactive' in properties and 'Result=success' in properties),'Service did not stop successfully')
-        require(forced is not None or str(code)=='0' or (str(code)=='143' and (idle_worker or empty_api)),'Original process did not exit cleanly; active work is not claimed completed')
+        require(forced is not None or str(code)=='0' or (str(code)=='143' and (idle_worker or empty_api or money_free_api)),'Original process did not exit cleanly; active work is not claimed completed')
         if name=='indeedhub-relay':
             require(str(code)=='0','Relay native shutdown did not exit cleanly')
             self.require_api_stopped(member)
         classification=forced['classification'] if forced else (('idle-worker-terminated-after-queue-drain' if idle_worker else 'empty-business-store-legacy-api-terminated') if str(code)=='143' else 'clean-process-exit')
+        if money_free_api:classification='money-free-uat-api-terminated-data-preserved'
         if forced:stopped[name]['legacy_idle_termination']=forced
         stopped[name].update(confirmed=True,exit_code=int(code),classification=classification,confirmed_at=time.time());self.save()
     def volume_sources(self):

tests/regression/test_indeehub_maintenance_controller.py

@@ -444,6 +444,34 @@ console.log('process identity cases passed');'''
   with self.assertRaisesRegex(RuntimeError,'business work'):c.legacy_api_idle()
   counts['other_active_transactions']=0;c.legacy_api_idle()
   self.assertEqual(c.record['legacy_api_empty_state'],counts)
+ def test_money_free_uat_requires_all_monetary_history_and_capability_absent(self):
+  counts=dict.fromkeys(module.UAT_ZERO_COUNTS,0)
+  probe={'main_sha256':module.UAT_API_MAIN_SHA256,'providers_absent':True,'http_connections_absent':True,'registration_disabled':True,'publication_disabled':True}
+  self.assertTrue(module.valid_money_free_uat(counts,probe))
+  for name in counts:
+   for value in (1,-1,False,None):
+    self.assertFalse(module.valid_money_free_uat(dict(counts,**{name:value}),probe))
+   missing=dict(counts);missing.pop(name)
+   self.assertFalse(module.valid_money_free_uat(missing,probe))
+  for name in probe:
+   changed=dict(probe);changed[name]=False if name!='main_sha256' else '0'*64
+   self.assertFalse(module.valid_money_free_uat(counts,changed))
+ def test_money_free_stopped_proof_rejects_changed_data_and_operation(self):
+  import copy
+  c=self.controller;m=next(m for m in members() if m['name']=='indeedhub-api')
+  counts=dict.fromkeys(module.UAT_ZERO_COUNTS,0)
+  probe={'main_sha256':module.UAT_API_MAIN_SHA256,'providers_absent':True,'http_connections_absent':True,'registration_disabled':True,'publication_disabled':True}
+  baseline={'operation_id':self.operation,'tables':{'projects':{'rows':1,'rows_sha256':'a'*64}}}
+  c.record={'money_free_uat':{'operation_id':self.operation,'container_id':m['container_id'],'image_id':m['image_id'],'counts':counts,'probe':probe,'database_before_signal':baseline}}
+  c.holds=lambda:None;c.fence_matches=lambda:None;c.require_api_stopped=lambda _:None
+  c.money_free_uat_counts=lambda:dict(counts);c.database_commitments=lambda:copy.deepcopy(baseline)
+  c.verify_money_free_uat_stopped(m)
+  self.assertTrue(c.record['money_free_uat']['stopped_data_verified'])
+  c.database_commitments=lambda:{'operation_id':self.operation,'tables':{'projects':{'rows':2,'rows_sha256':'b'*64}}}
+  with self.assertRaisesRegex(RuntimeError,'Committed data changed'):c.verify_money_free_uat_stopped(m)
+  c.database_commitments=lambda:copy.deepcopy(baseline)
+  c.record['money_free_uat']['operation_id']='foreign'
+  with self.assertRaisesRegex(RuntimeError,'proof missing'):c.verify_money_free_uat_stopped(m)
  def test_rollback_compatibility_binds_operation_preserves_rows_and_allows_only_empty_additions(self):
   import copy
   table={'schema':{'columns':['original']},'rows':0,'rows_sha256':'a'*64}