feat(dht): Phase 1 — BLAKE3 content addressing alongside SHA-256

Adds the iroh-native, range-verifiable hash next to the incumbent SHA-256
so the swarm can later fetch/verify by BLAKE3 with the registry/origin as
fallback. Non-breaking: SHA-256 stays the mandatory gate; BLAKE3 is verified
only when present.

- content_hash.rs: HashAlg + ContentDigest (parse/verify '<alg>:<hex>'
  multihash strings), blake3_hex/sha256_hex; BLAKE3 known-answer test
- update.rs: ComponentUpdate.blake3 (serde-default); verified ALONGSIDE
  SHA-256 in the resumable download loop, re-download on mismatch
- blobs.rs: BlobMeta.blake3 computed on put (on-disk path stays
  SHA-256-keyed for back-compat; advertises the future swarm address)

Drive-by: fix a pre-existing stale test (test_save_and_load_state_roundtrip)
that never wrote the .download-complete marker #26 requires, so load_state's
self-heal cleared update_in_progress. Unrelated to BLAKE3 — surfaced by
running the full update:: suite.

40/40 content_hash/update/blobs tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-06-16 13:05:27 -04:00
co-authored by Claude Opus 4.8
parent 27f11bf85a
commit f0cb91ed76
6 changed files with 238 additions and 10 deletions
+34
View File
@@ -263,6 +263,11 @@ pub struct ComponentUpdate {
pub download_url: String,
pub sha256: String,
pub size_bytes: u64,
/// DHT Phase 1: BLAKE3 content address (bare hex or `"blake3:<hex>"`), the
/// iroh-native, range-verifiable hash. Optional during the migration
/// window — when present it is verified ALONGSIDE the mandatory SHA-256.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub blake3: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -997,6 +1002,25 @@ async fn download_component_resumable(
.context("read staging file for hash check")?;
let hash = hex::encode(Sha256::digest(&bytes));
if hash == component.sha256 {
// DHT Phase 1: if the manifest also pins a BLAKE3 digest, it must
// match too. SHA-256 stays the mandatory gate during migration;
// BLAKE3 is the hash the iroh swarm will fetch/verify by, so a
// present-but-wrong BLAKE3 means the bytes aren't swarm-consistent
// — treat it like a SHA mismatch and re-download.
if let Some(b3) = component.blake3.as_deref() {
let expected = b3.trim().strip_prefix("blake3:").unwrap_or(b3.trim());
let actual = crate::content_hash::blake3_hex(&bytes);
if !actual.eq_ignore_ascii_case(expected) {
let _ = tokio::fs::remove_file(dest).await;
last_err = Some(anyhow::anyhow!(
"BLAKE3 mismatch for {}: expected {}, got {}",
component.name,
expected,
actual
));
continue;
}
}
return Ok(());
}
// SHA mismatch — the file on disk is garbage. Nuke it and
@@ -1679,6 +1703,7 @@ mod tests {
download_url: "https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.26-alpha/archipelago".into(),
sha256: "x".into(),
size_bytes: 1,
blake3: None,
},
ComponentUpdate {
name: "frontend".into(),
@@ -1687,6 +1712,7 @@ mod tests {
download_url: "https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/v1.7.26-alpha/frontend.tar.gz".into(),
sha256: "y".into(),
size_bytes: 2,
blake3: None,
},
],
};
@@ -1886,6 +1912,13 @@ mod tests {
tokio::fs::write(staging.join("archipelago"), b"staged")
.await
.unwrap();
// A *complete* staged update carries the .download-complete marker;
// without it has_staged_update() reads the staging as partial and the
// load_state self-heal clears update_in_progress (see #26). This test
// simulates a complete staging, so write the marker.
tokio::fs::write(staging.join(STAGED_COMPLETE_MARKER), b"1")
.await
.unwrap();
let state = UpdateState {
current_version: "1.0.0".to_string(),
last_check: Some("2025-06-15T12:00:00Z".to_string()),
@@ -1900,6 +1933,7 @@ mod tests {
download_url: "https://example.com/binary".to_string(),
sha256: "abc123".to_string(),
size_bytes: 5000,
blake3: None,
}],
}),
update_in_progress: true,