[Bug]: Fedmint overloading CPU #65

New Issue

ssmithx · 2026-06-28T14:26:12Z

ssmithx commented

2026-06-28 14:26:12 +00:00

Description

archy-x250-pa was running hot so I checked it.

Steps to Reproduce

Start fedimin?

Expected Behavior

App should start and run smoothly

Actual Behavior

Consuming lots of CPU and Disk I think

Archipelago Version

1.7.99-alpha

Hardware

x86_64 (Intel/AMD)

Relevant Logs

What is fmcd?

  It's the Fedimint Client Daemon — software for participating in a Fedimint (https://fedimint.org/) federation, which is a privacy-preserving Bitcoin custody protocol using
  threshold multisig. It was installed as an Archipelago service and started on June 24, immediately joining federation 2debd071....

  ---
  Why it's burning ~94% CPU:

  The logs reveal a compounding networking problem:

  1. Multiple Iroh P2P endpoints spawned in parallel — at startup, fmcd spun up at least 6 separate Iroh (QUIC-based P2P) endpoints (ab09514c69, 93a1e68c6a, 97ee1e4037,
  1ed3b3083f, fe30639e2a, 1341f22c78), each with its own DHT instance and relay connections. This is likely more than intended.
  2. Relay failures causing spin — https://use1-1.relay.n0.iroh-canary.iroh.link./ repeatedly fails to connect, and all those endpoints are constantly retrying. There were also
  "failed to send ping: queues full" warnings, meaning network queues are overflowing.
  3. DHT bootstrap stuck — since Jun 25, the only log entries are "Could not bootstrap the routing table" every 15–30 minutes. The Mainline DHT can't find peers, so the Iroh
  discovery layer keeps spinning trying to locate federation guardians.
  4. pkarr identity conflict — "A different SignedPacket is being concurrently published for the same PublicKey" — multiple Iroh endpoints are competing to publish the same
  identity to the DHT, another sign of the parallel-endpoint problem.

  Notably: the RocksDB database has only done 60 writes in 4 days — the actual Fedimint wallet logic is barely active. All the CPU is pure networking churn.
 
  ---
  What to do:

  This looks like a bug or misconfiguration in how Archipelago's Fedimint service is configured — likely related to Iroh spawning too many endpoints or a network path issue
  (NAT/firewall blocking DHT). Restarting the container may help temporarily:

  podman restart fedimint-clientd

  But the root cause is likely worth reporting to the Archipelago project.

Screenshots

No response

### Description archy-x250-pa was running hot so I checked it. ### Steps to Reproduce Start fedimin? ### Expected Behavior App should start and run smoothly ### Actual Behavior Consuming lots of CPU and Disk I think ### Archipelago Version 1.7.99-alpha ### Hardware x86_64 (Intel/AMD) ### Relevant Logs ```shell What is fmcd? It's the Fedimint Client Daemon — software for participating in a Fedimint (https://fedimint.org/) federation, which is a privacy-preserving Bitcoin custody protocol using threshold multisig. It was installed as an Archipelago service and started on June 24, immediately joining federation 2debd071.... --- Why it's burning ~94% CPU: The logs reveal a compounding networking problem: 1. Multiple Iroh P2P endpoints spawned in parallel — at startup, fmcd spun up at least 6 separate Iroh (QUIC-based P2P) endpoints (ab09514c69, 93a1e68c6a, 97ee1e4037, 1ed3b3083f, fe30639e2a, 1341f22c78), each with its own DHT instance and relay connections. This is likely more than intended. 2. Relay failures causing spin — https://use1-1.relay.n0.iroh-canary.iroh.link./ repeatedly fails to connect, and all those endpoints are constantly retrying. There were also "failed to send ping: queues full" warnings, meaning network queues are overflowing. 3. DHT bootstrap stuck — since Jun 25, the only log entries are "Could not bootstrap the routing table" every 15–30 minutes. The Mainline DHT can't find peers, so the Iroh discovery layer keeps spinning trying to locate federation guardians. 4. pkarr identity conflict — "A different SignedPacket is being concurrently published for the same PublicKey" — multiple Iroh endpoints are competing to publish the same identity to the DHT, another sign of the parallel-endpoint problem. Notably: the RocksDB database has only done 60 writes in 4 days — the actual Fedimint wallet logic is barely active. All the CPU is pure networking churn. --- What to do: This looks like a bug or misconfiguration in how Archipelago's Fedimint service is configured — likely related to Iroh spawning too many endpoints or a network path issue (NAT/firewall blocking DHT). Restarting the container may help temporarily: podman restart fedimint-clientd But the root cause is likely worth reporting to the Archipelago project. ``` ### Screenshots _No response_

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lfg2025/archy#65