Large peer downloads (~178MB) failed with a generic 'Operation failed', and
the download path had three stacked problems:
- The FIPS reqwest client used a hard-coded 20s total timeout regardless of the
caller's .timeout(), so a big transfer over the mesh aborted at 20s before
the Tor fallback could help. Honor the per-request timeout (client_with_timeout).
- The peer-content proxy buffered the whole file into node memory via
resp.bytes() before sending a byte, and capped the transfer at 60s. Stream
the body through with hyper::Body::wrap_stream (constant memory) and raise the
timeout to 900s; bump the nginx peer-content read timeout to match.
- Free downloads pulled the file as base64 over RPC, doubling it in node memory
and the browser — fatal for large files. Download free files by streaming
from /api/peer-content straight to disk, after a 1-byte Range probe that
surfaces the real reason (peer offline on mesh and Tor) instead of a generic
failure. Paid downloads now return the real error through the {error} channel
the UI already displays.
Adds the reqwest 'stream' feature for bytes_stream().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
3aea8c5bfa
commit
e456c9701b
@@ -227,9 +227,13 @@ impl ApiHandler {
|
||||
|
||||
let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
|
||||
let peer_path = format!("/content/{}", content_id);
|
||||
// Generous overall timeout: this endpoint serves both seek/Range
|
||||
// playback (small, finishes fast) and full-file downloads of large
|
||||
// media (#38). 60s was too tight for a multi-hundred-MB transfer over
|
||||
// Tor and aborted the download mid-stream.
|
||||
let mut req = crate::fips::dial::PeerRequest::new(fips_npub.as_deref(), onion, &peer_path)
|
||||
.service(crate::settings::transport::PeerService::PeerFiles)
|
||||
.timeout(std::time::Duration::from_secs(60));
|
||||
.timeout(std::time::Duration::from_secs(900));
|
||||
if let Some(r) = headers.get("range").and_then(|v| v.to_str().ok()) {
|
||||
req = req.header("Range", r.to_string());
|
||||
}
|
||||
@@ -237,7 +241,6 @@ impl ApiHandler {
|
||||
Ok((resp, _transport)) => {
|
||||
let status = resp.status().as_u16();
|
||||
let rh = resp.headers().clone();
|
||||
let bytes = resp.bytes().await.unwrap_or_default();
|
||||
let mut builder = Response::builder()
|
||||
.status(status)
|
||||
.header("Accept-Ranges", "bytes");
|
||||
@@ -246,8 +249,13 @@ impl ApiHandler {
|
||||
builder = builder.header(h, v);
|
||||
}
|
||||
}
|
||||
// Stream the peer's body straight through instead of buffering
|
||||
// the whole file into memory (#38). For a 178MB download the old
|
||||
// `resp.bytes().await` allocated the entire file on the node
|
||||
// before sending a byte; `wrap_stream` forwards chunks as they
|
||||
// arrive, with constant memory.
|
||||
Ok(builder
|
||||
.body(hyper::Body::from(bytes))
|
||||
.body(hyper::Body::wrap_stream(resp.bytes_stream()))
|
||||
.unwrap_or_else(|_| Response::new(hyper::Body::empty())))
|
||||
}
|
||||
Err(e) => Ok(build_response(
|
||||
|
||||
Reference in New Issue
Block a user