Thursday, April 23, 2020

Questions about Bitcoin reorganization process logic with a rev*.dat file format or byte map.

Hi,
I’m currently learning how bitcoin operates by studying the BitcoinCore code and several other bitcoin projects, most of them parsers,  and came to a topic of blockchain data storage of blk*.dat and rev*.dat.

While blk*.dat format is more or less documented and the process is described really good (https://learnmeabitcoin.com/guide/blkdat) most of the resources are lacking detailed descriptions of rev*.dat file format and how the process of handling blockchain reorganization (block undo) works. Some Parsers (https://github.com/citp/BlockSci) have rules not to parse new blocks before a certain amount of confirmations so they do not need to deal with the issues of reorganization.

Right now I’m very interested in the logic  of blockchain reorganization and found some descriptions:https://en.bitcoin.it/wiki/Bitcoin_Core_0.11_(ch_2):_Data_Storage#Raw_undo_data_.28rev.2A.dat.29:Data_Storage#Raw_undo_data.28rev.2A.dat.29)
And an awesome post reply by Pieter Wuille:
https://bitcoin.stackexchange.com/questions/57978/file-format-rev-dat
I even tried to visualize that answer in a picture:
https://prnt.sc/s4ile4

The problem is that all those explanations are dated because of the update (https://github.com/bitcoin/bitcoin/pull/10195) that replaced CTxInUndo class with Coin class (https://github.com/bitcoin/bitcoin/commit/cb2c7fdac2dc74368ed24ae4717ed72178956b92 )

I know that code is the best documentation, but I still find it hard to read and understand, so I’m asking for help to answer these questions:

  • How does the logic of reorganization work with the rev*.dat and blk*.dat files exactly?
  • Can you walk me through the code, please?
  • What is the format of the rev*.dat files?
  • Can you explain it with a byte map(byte length for each piece) for a block record in it, please?

Apart from learning how everything works, I’m also interested in these topics because they say that it’s possible to get blockstasts faster using rev*.dat files (https://github.com/bitcoin/bitcoin/pull/14802) and I want to properly picture the whole processes with the rev*.dat files.



No comments:

Post a Comment