Large peer downloads (~178MB) failed with a generic 'Operation failed', and
the download path had three stacked problems:
- The FIPS reqwest client used a hard-coded 20s total timeout regardless of the
caller's .timeout(), so a big transfer over the mesh aborted at 20s before
the Tor fallback could help. Honor the per-request timeout (client_with_timeout).
- The peer-content proxy buffered the whole file into node memory via
resp.bytes() before sending a byte, and capped the transfer at 60s. Stream
the body through with hyper::Body::wrap_stream (constant memory) and raise the
timeout to 900s; bump the nginx peer-content read timeout to match.
- Free downloads pulled the file as base64 over RPC, doubling it in node memory
and the browser — fatal for large files. Download free files by streaming
from /api/peer-content straight to disk, after a 1-byte Range probe that
surfaces the real reason (peer offline on mesh and Tor) instead of a generic
failure. Paid downloads now return the real error through the {error} channel
the UI already displays.
Adds the reqwest 'stream' feature for bytes_stream().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
3aea8c5bfa
commit
e456c9701b
@@ -99,8 +99,18 @@ pub async fn peer_base_url(npub: &str) -> Result<String> {
|
||||
/// until the first packets flow), so a reachable-but-cold peer isn't abandoned
|
||||
/// to Tor prematurely. Reliability over latency — FIPS is the preferred path.
|
||||
pub fn client() -> reqwest::Client {
|
||||
client_with_timeout(Duration::from_secs(20))
|
||||
}
|
||||
|
||||
/// FIPS client with a caller-chosen overall request timeout. The static 20s
|
||||
/// `client()` budget is fine for catalog browses and short calls, but a large
|
||||
/// content download (#38) needs the per-request timeout the caller asked for —
|
||||
/// otherwise a 178MB transfer is aborted at 20s and the whole download fails
|
||||
/// before the Tor fallback ever gets a chance. The generous `connect_timeout`
|
||||
/// is preserved so a cold hole-punched path still gets time to establish.
|
||||
pub fn client_with_timeout(timeout: Duration) -> reqwest::Client {
|
||||
reqwest::Client::builder()
|
||||
.timeout(Duration::from_secs(20))
|
||||
.timeout(timeout)
|
||||
.connect_timeout(Duration::from_secs(8))
|
||||
.user_agent("archipelago-fips/1")
|
||||
.build()
|
||||
@@ -413,7 +423,7 @@ impl<'a> PeerRequest<'a> {
|
||||
}
|
||||
};
|
||||
let url = format!("{}{}", base, self.path);
|
||||
let c = client();
|
||||
let c = client_with_timeout(self.timeout);
|
||||
let mut rb = c.post(&url).json(body);
|
||||
for (k, v) in &self.headers {
|
||||
rb = rb.header(*k, v);
|
||||
@@ -446,7 +456,7 @@ impl<'a> PeerRequest<'a> {
|
||||
}
|
||||
};
|
||||
let url = format!("{}{}", base, self.path);
|
||||
let c = client();
|
||||
let c = client_with_timeout(self.timeout);
|
||||
let mut rb = c.get(&url);
|
||||
for (k, v) in &self.headers {
|
||||
rb = rb.header(*k, v);
|
||||
|
||||
Reference in New Issue
Block a user