pull down to refresh

Looking for pre-2019 stale-block data for a to-be-public dataset: hashes and headers (you can help!)
1274 sats \ 3 comments \ @0xB10C 19 May 2022 bitcoin
I'm currently gathering data for a historical stale-blocks dataset. Will be pubilshed for anyone to use as open-source (probably under a CC0 license). I'm still missing a lot of stale-block data before block 540801.
If you setup your node a while before block #540801 (2018-09-10), you can probably help. One way would be to post the output of the getchaintips RPC. This provides the height and stale block hash. However, it would be nice to have the corresponding headers to these stale-block hashes too (to e.g. prove that you didn't just make up some hash).
I've used this shell script in the past to export the hashes and headers of my stale blocks on my node. Please make sure you understand random shell scripts that you run on your machine first.
#!/bin/bash

# A script to extract stale blocks (height, hash, header) from a
# Bitcoin Core node. Output is in CSV format.
# This assumes .cookie-file authentication to the Bitcoin Core node.

TIPS=$(bitcoin-cli getchaintips)
STALE_TIP_PER_LINE=$(echo $TIPS | jq -c '.[] | select( .status == "valid-fork" or .status == "headers-only" or .status == "valid-headers" )')

for tip in $STALE_TIP_PER_LINE
do
	height=$(echo $tip | jq .height)
	hash=$(echo $tip | jq -r .hash)
	header=$(bitcoin-cli getblockheader $hash false)
	echo "$height,$hash,$header"
done
Appreciate any help, thanks!
write
preview
related posts
73 sats \ 2 replies \ @justsmile 19 May 2022
Nice script!
It is enough to do a much-more compatible #!/bin/sh since you are not using any Bash-specific features in the script. Also you can use jq -c '.[] | select (.status != "active")' for brevity.
The full smaller (maybe not simpler, but your mileage may vary) script which does the same in significantly shorter time is:
#!/bin/sh
#
# A script to extract stale blocks (height, hash, header) from a
# Bitcoin Core node. Output is in CSV format.
# This assumes .cookie-file authentication to the Bitcoin Core node.

bitcoin-cli getchaintips \
  | jq -rc '.[] | select (.status != "active")
                | ( (.height | tostring) + " " + .hash )' \
  | while read height bhash;
    do
      echo "$height,$bhash,$(bitcoin-cli getblockheader $bhash false)"
    done
Comparison of both:
$ time sh script.sh | sha256sum
real	0m 4.98s
user	0m 4.60s
sys	0m 0.25s
678d9d530cd9cb8d41a45fbb210084dd5601b04a2313bae8d1f564b7eb1f203f  -
$ time sh script-small.sh | sha256sum
real	0m 0.35s
user	0m 0.30s
sys	0m 0.04s
678d9d530cd9cb8d41a45fbb210084dd5601b04a2313bae8d1f564b7eb1f203f  -
The output of my bitcoin-cli getchaintips is shorter than yours.
@k00b please note the copy&paste of preformatted text blocks (whose line-numbers indentation was fixed recently, thanks!) gives line-numbers:
#!/bin/bash
2
3# A script to extract stale blocks (height, hash, header) from a
4# Bitcoin Core node. Output is in CSV format.
5# This assumes .cookie-file authentication to the Bitcoin Core node.
reply
10 sats \ 1 reply \ @0xB10C OP 19 May 2022 freebie
Oh, neat. Thanks! Will adopt the script if you don't mind.
Note: There's also the invalid status in getchaintips when you've used invalidateblock.