Hornet NodeHornet Node
Back in September, there was a delving post announcing something called Hornet Node (#1233022).
It described itself like this:
Hornet is a minimal, executable specification of Bitcoin's consensus rules, expressed both in declarative C++ and in a purpose-built domain-specific language.
It is implemented as a suite of modular, dependency-free, modern C++ libraries and includes a lightweight node capable of Initial Block Download (IBD).
At the time the project wasn't finished and there was no source code available.
Hornet UTXO(1)Hornet UTXO(1)
Earlier in January, the developer behind Hornet Node (@bitcoindudebro on X) made another post on delving announcing a custom UTXO database for Hornet node.
What was most surprising about thus were the numbers for revalidating the entire block chain.
On my 32-core dev workstation, these are like-for-like results for re-validating the whole mainnet chain without script or signature validation:
- Bitcoin Core: 167 minutes
- bitcoind -datadir="/nvme1/bitcoin/" -reindex-chainstate -dbcache=24000 -connect=0 -assumevalid=0000000000000000000019426fb9e93a5cfa7695f271d928beeddf39a1ef3f3f
- Hornet UTXO(1): 15 minutes
That includes reading and deserializing all of mainnet data from disk, and all the header/structural/contextual validation too. But more to the point it includes the evolution of the UTXO database and validation of all spends, including retrieving the pk script ready for sig script validation. (Script validation was not performed, same as with assumevalid). So it should be a fairly apples-to-apples comparison.
Why does IBD take so long?Why does IBD take so long?
When you start a Bitcoin node for the first time, your node needs to figure out what is the best chain and then download and verify all the blocks until the most recent one.
This initial block download (IBD) is somewhat infamously a long process. This is because your node has to do more than download all the blocks. It also has to run the numbers to make sure that all the blocks follow Bitcoin's block validation rules and thst all the individual instructions for each utxo spent in each transaction are followed. This is called validation. It's the part of IBD where your computer's fan starts running really loud.
Bitcoin Core, many years ago, introduced a concept called assumevalid which allowed a node to be told about a certain valid block and skip-until-later the validation of all the blocks that came before.
This was an improvement on a previous system that used checkpoint blocks to speed up IBD because checkpoints were effectively blocks that must be included in the chain. Not very elegant.
Even with things like assumevalid, IBD still takes a long time because not only does your node have to check to make sure that blocks that come after the assumevalid block follow the rules, it also has to make sure that the transactions inside those blocks come in the right order.
I have a sneaking suspicion that I have got some of this about validation wrong -- so take it with a grain of salt.
The point is that your node has to do a lot of math to figure out which utxos exist at a certain time.
The UTXO SetThe UTXO Set
Figuring out which transaction outputs exist AND have not yet been spent is a difficult thing to do. Bitcoin Core does this by creating a database called the UTXO set.
Other node implementations like libbitcoin do it by storing transaction data in a different way. Things like Floresta do it in still a different way.
Hornet UTXO(1) is yet another way of doing it. I'm certainly not knowledgeable enough to explain these differences, but it's cool to see people working on this problem.
NOTE: as far as I can tell, the Hornet Node source code (as well as Hornet UTXO(1)) has not yet been publicly released.