pull down to refresh

1635 sats \ 4 replies \ @optimism 22 Oct \ on: Stacker Saloon
Can report that I'm able to reproduce @SimpleStacker's reported issues with IBD on a Raspberry Pi 4 w/ Core 29.1. Eg block 796776:
2025-10-22T13:41:33Z [bench]   - Load block from disk: 240.23ms
2025-10-22T13:41:33Z [bench]     - Sanity checks: 11.34ms [2958.27s (3.71ms/blk)]
2025-10-22T13:41:33Z [bench]     - Fork checks: 0.31ms [135.27s (0.17ms/blk)]
2025-10-22T13:42:08Z [bench]       - Connect 704 transactions: 34679.36ms (49.260ms/tx, 3.470ms/txin) [51837.88s (65.06ms/blk)]
2025-10-22T13:42:08Z [bench]     - Verify 9995 txins: 34716.35ms (3.473ms/txin) [51906.08s (65.15ms/blk)]
2025-10-22T13:42:08Z [bench]     - Write undo data: 14.97ms [3908.57s (4.91ms/blk)]
2025-10-22T13:42:08Z [bench]     - Index writing: 0.09ms [39.59s (0.05ms/blk)]
2025-10-22T13:42:08Z [bench]   - Connect total: 34744.00ms [59248.99s (74.36ms/blk)]
Going to let it run (may take up to 20 days to finish lol). Almost 35 seconds to verify 10k txins is a lot.
The process is (logically) CPU-bound on b-msghandler because that's where ConnectBlock is ultimately called from, and high soft interrupt; which I expect to be from using wifi instead of wired (but I figured that since raspibolt is explicitly configuring for usage of that, that most operators would run it like that):
write
preview
100 sats \ 3 replies \ @SimpleStacker 22 Oct
Dude, thank you so much! Mind if I share it here? https://github.com/raspibolt/raspibolt/issues/1482
Can you help me understand further? Is CPU the bottleneck, not hard disk speed or bandwidth? Would connecting from an ethernet cable have helped? And lastly, is it due to UTXO set bloat, or simply due to larger blocks?
Sorry for all the questions, haha
reply
30 sats \ 2 replies \ @optimism 22 Oct
Feel free to share it with anyone, though note that it's right now only a confirmation that it can be reproduced. Next steps for me:
  1. Let it finish so that I can graph and analyze the full IBD end-to-end for patterns.
  2. Start comparing BenchConnectBlock across different ARM hardware. 1
  3. If there are visible differences in bench, profile it. Otherwise make a new bench until it shows the differences, and profile that.
  4. Attempt to optimize the code
  5. Do a PR if 4 has outcome
I aim to do 2 and 3 in parallel to the IBD whenever I find time.
Is CPU the bottleneck, not hard disk speed or bandwidth?
It is for my test, so I'd say "yes". It's also what I expected.
Would connecting from an ethernet cable have helped?
I don't expect that, because txin verification doesn't depend on network - it's all done between disk, cache and cpu.
And lastly, is it due to UTXO set bloat, or simply due to larger blocks?
The block I gave as an example above contains mostly consolidation of p2sh utxo. So in this case, we're actually seeing bad performance on utxoset unbloat.

Footnotes

  1. because for example the IBD I did on an RK3588 earlier this year went much faster. There are some cpu feature differences between that and a Pi4. This would also suggest that a Pi5 would perform much better because it has some more CPU features built in. ↩
reply
100 sats \ 1 reply \ @SimpleStacker 22 Oct
Ok, I'll wait for more concrete results before sharing anything. That Github issue is closed anyway so unlikely to get any real attention.
The block I gave as an example above contains mostly consolidation of p2sh utxo. So in this case, we're actually seeing bad performance on utxoset unbloat.
Interesting. But we wouldn't need to do this much validation if the UTXO set wasn't so bloated to begin with, right? So I think it's fair to chalk it up to the sphere of problems caused by UTXO bloat?
reply
30 sats \ 0 replies \ @optimism 22h
I'd argue that bearer instruments (metals, coin/cash, bitcoin) have consolidation as a part of the natural cycle, especially for businesses. So I'd argue that it's fair to chalk it up to having a utxo system, but not to bloat.
reply