#include <headerssync.h>

Collaboration diagram for HeadersSyncState:

Classes
struct	ProcessingResult
	Result data structure for ProcessNextHeaders. More...

Public Types
enum class	State { PRESYNC , REDOWNLOAD , FINAL }

Public Member Functions
	~HeadersSyncState ()

State	GetState () const
	Return the current state of our download.

int64_t	GetPresyncHeight () const
	Return the height reached during the PRESYNC phase.

uint32_t	GetPresyncTime () const
	Return the block timestamp of the last header received during the PRESYNC phase.

arith_uint256	GetPresyncWork () const
	Return the amount of work in the chain received during the PRESYNC phase.

	HeadersSyncState (NodeId id, const Consensus::Params &consensus_params, const CBlockIndex *chain_start, const arith_uint256 &minimum_required_work)
	Construct a HeadersSyncState object representing a headers sync via this download-twice mechanism).

ProcessingResult	ProcessNextHeaders (const std::vector< CBlockHeader > &received_headers, bool full_headers_message)
	Process a batch of headers, once a sync via this mechanism has started.

CBlockLocator	NextHeadersRequestLocator () const
	Issue the next GETHEADERS message to our peer.

Private Member Functions
void	Finalize ()
	Clear out all download state that might be in progress (freeing any used memory), and mark this object as no longer usable.

bool	ValidateAndStoreHeadersCommitments (const std::vector< CBlockHeader > &headers)
	Only called in PRESYNC.

bool	ValidateAndProcessSingleHeader (const CBlockHeader &current)
	In PRESYNC, process and update state for a single header.

bool	ValidateAndStoreRedownloadedHeader (const CBlockHeader &header)
	In REDOWNLOAD, check a header's commitment (if applicable) and add to buffer for later processing.

std::vector< CBlockHeader >	PopHeadersReadyForAcceptance ()
	Return a set of headers that satisfy our proof-of-work threshold.

Private Attributes
const NodeId	m_id
	NodeId of the peer (used for log messages)

const Consensus::Params &	m_consensus_params
	We use the consensus params in our anti-DoS calculations.

const CBlockIndex *	m_chain_start {nullptr}
	Store the last block in our block index that the peer's chain builds from.

const arith_uint256	m_minimum_required_work
	Minimum work that we're looking for on this chain.

arith_uint256	m_current_chain_work
	Work that we've seen so far on the peer's chain.

const SaltedBlockHashHasher	m_hasher
	m_hasher is a salted hasher for making our 1-bit commitments to headers we've seen.

bitdeque	m_header_commitments
	A queue of commitment bits, created during the 1st phase, and verified during the 2nd.

const unsigned	m_commit_offset
	The (secret) offset on the heights for which to create commitments.

uint64_t	m_max_commitments {0}
	m_max_commitments is a bound we calculate on how long an honest peer's chain could be, given the MTP rule.

CBlockHeader	m_last_header_received
	Store the latest header received while in PRESYNC (initialized to m_chain_start)

int64_t	m_current_height {0}
	Height of m_last_header_received.

std::deque< CompressedHeader >	m_redownloaded_headers
	During phase 2 (REDOWNLOAD), we buffer redownloaded headers in memory until enough commitments have been verified; those are stored in m_redownloaded_headers.

int64_t	m_redownload_buffer_last_height {0}
	Height of last header in m_redownloaded_headers.

BlockHash	m_redownload_buffer_last_hash
	Hash of last header in m_redownloaded_headers (initialized to m_chain_start).

BlockHash	m_redownload_buffer_first_prev_hash
	The hashPrevBlock entry for the first header in m_redownloaded_headers We need this to reconstruct the full header when it's time for processing.

arith_uint256	m_redownload_chain_work
	The accumulated work on the redownloaded chain.

bool	m_process_all_remaining_headers {false}
	Set this to true once we encounter the target blockheader during phase 2 (REDOWNLOAD).

State	m_download_state {State::PRESYNC}
	Current state of our headers sync.

Detailed Description

HeadersSyncState:

We wish to download a peer's headers chain in a DoS-resistant way.

The Bitcoin protocol does not offer an easy way to determine the work on a peer's chain. Currently, we can query a peer's headers by using a GETHEADERS message, and our peer can return a set of up to 2000 headers that connect to something we know. If a peer's chain has more than 2000 blocks, then we need a way to verify that the chain actually has enough work on it to be useful to us – by being above our anti-DoS minimum-chain-work threshold – before we commit to storing those headers in memory. Otherwise, it would be cheap for an attacker to waste all our memory by serving us low-work headers (particularly for a new node coming online for the first time).

To prevent memory-DoS with low-work headers, while still always being able to reorg to whatever the most-work chain is, we require that a chain meet a work threshold before committing it to memory. We can do this by downloading a peer's headers twice, whenever we are not sure that the chain has sufficient work:

In the first download phase, called pre-synchronization, we can calculate the work on the chain as we go (just by checking the nBits value on each header, and validating the proof-of-work).
Once we have reached a header where the cumulative chain work is sufficient, we switch to downloading the headers a second time, this time processing them fully, and possibly storing them in memory.

To prevent an attacker from using (eg) the honest chain to convince us that they have a high-work chain, but then feeding us an alternate set of low-difficulty headers in the second phase, we store commitments to the chain we see in the first download phase that we check in the second phase, as follows:

In phase 1 (presync), store 1 bit (using a salted hash function) for every N headers that we see. With a reasonable choice of N, this uses relatively little memory even for a very long chain.
In phase 2 (redownload), keep a lookahead buffer and only accept headers from that buffer into the block index (permanent memory usage) once they have some target number of verified commitments on top of them. With this parametrization, we can achieve a given security target for potential permanent memory usage, while choosing N to minimize memory use during the sync (temporary, per-peer storage).

Definition at line 98 of file headerssync.h.

Member Enumeration Documentation

◆ State

enum class HeadersSyncState::State

strong

Enumerator
PRESYNC	PRESYNC means the peer has not yet demonstrated their chain has sufficient work and we're only building commitments to the chain they serve us.
REDOWNLOAD	REDOWNLOAD means the peer has given us a high-enough-work chain, and now we're redownloading the headers we saw before and trying to accept them.
FINAL	We're done syncing with this peer and can discard any remaining state.

Definition at line 102 of file headerssync.h.

Constructor & Destructor Documentation

◆ ~HeadersSyncState()

HeadersSyncState::~HeadersSyncState ( )

inline

Definition at line 100 of file headerssync.h.

◆ HeadersSyncState()

HeadersSyncState::HeadersSyncState	(	NodeId	id,
		const Consensus::Params &	consensus_params,
		const CBlockIndex *	chain_start,
		const arith_uint256 &	minimum_required_work
	)

Construct a HeadersSyncState object representing a headers sync via this download-twice mechanism).

id: node id (for logging) consensus_params: parameters needed for difficulty adjustment validation chain_start: best known fork point that the peer's headers branch from minimum_required_work: amount of chain work required to accept the chain

Definition at line 27 of file headerssync.cpp.

Here is the call graph for this function:

Member Function Documentation

◆ Finalize()

void HeadersSyncState::Finalize ( )

private

Clear out all download state that might be in progress (freeing any used memory), and mark this object as no longer usable.

Free any memory in use, and mark this object as no longer usable.

This is required to guarantee that we won't reuse this object with the same SaltedBlockHashHasher for another sync.

Definition at line 66 of file headerssync.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GetPresyncHeight()

int64_t HeadersSyncState::GetPresyncHeight ( ) const

inline

Return the height reached during the PRESYNC phase.

Definition at line 126 of file headerssync.h.

◆ GetPresyncTime()

uint32_t HeadersSyncState::GetPresyncTime ( ) const

inline

Return the block timestamp of the last header received during the PRESYNC phase.

Definition at line 132 of file headerssync.h.

◆ GetPresyncWork()

arith_uint256 HeadersSyncState::GetPresyncWork ( ) const

inline

Return the amount of work in the chain received during the PRESYNC phase.

Definition at line 138 of file headerssync.h.

◆ GetState()

State HeadersSyncState::GetState ( ) const

inline

Return the current state of our download.

Definition at line 123 of file headerssync.h.

◆ NextHeadersRequestLocator()

CBlockLocator HeadersSyncState::NextHeadersRequestLocator ( ) const

Issue the next GETHEADERS message to our peer.

This will return a locator appropriate for the current sync object, to continue the synchronization phase it is in.

Definition at line 365 of file headerssync.cpp.

Here is the call graph for this function:

◆ PopHeadersReadyForAcceptance()

std::vector< CBlockHeader > HeadersSyncState::PopHeadersReadyForAcceptance ( )

private

Return a set of headers that satisfy our proof-of-work threshold.

Definition at line 346 of file headerssync.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ProcessNextHeaders()

HeadersSyncState::ProcessingResult HeadersSyncState::ProcessNextHeaders	(	const std::vector< CBlockHeader > &	received_headers,
		bool	full_headers_message
	)

Process a batch of headers, once a sync via this mechanism has started.

Process the next batch of headers received from our peer.

received_headers: headers that were received over the network for processing. Assumes the caller has already verified the headers are continuous, and has checked that each header satisfies the proof-of-work target included in the header (but not necessarily verified that the proof-of-work target is correct and passes consensus rules). full_headers_message: true if the message was at max capacity, indicating more headers may be available ProcessingResult.pow_validated_headers: will be filled in with any headers that the caller can fully process and validate now (because these returned headers are on a chain with sufficient work) ProcessingResult.success: set to false if an error is detected and the sync is aborted; true otherwise. ProcessingResult.request_more: if true, the caller is suggested to call NextHeadersRequestLocator and send a getheaders message using it.

Validate and store commitments, and compare total chainwork to our target to see if we can switch to REDOWNLOAD mode.

Definition at line 84 of file headerssync.cpp.

Here is the call graph for this function:

◆ ValidateAndProcessSingleHeader()

bool HeadersSyncState::ValidateAndProcessSingleHeader ( const CBlockHeader & current )

private

In PRESYNC, process and update state for a single header.

Definition at line 221 of file headerssync.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ValidateAndStoreHeadersCommitments()

bool HeadersSyncState::ValidateAndStoreHeadersCommitments ( const std::vector< CBlockHeader > & headers )

private

Only called in PRESYNC.

Validate the work on the headers we received from the network, and store commitments for later. Update overall state with successfully processed headers. On failure, this invokes Finalize() and returns false.

Definition at line 173 of file headerssync.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ValidateAndStoreRedownloadedHeader()

bool HeadersSyncState::ValidateAndStoreRedownloadedHeader ( const CBlockHeader & header )

private

In REDOWNLOAD, check a header's commitment (if applicable) and add to buffer for later processing.

Definition at line 266 of file headerssync.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

◆ m_chain_start

const CBlockIndex* HeadersSyncState::m_chain_start {nullptr}

private

Store the last block in our block index that the peer's chain builds from.

Definition at line 235 of file headerssync.h.

◆ m_commit_offset

const unsigned HeadersSyncState::m_commit_offset

private

The (secret) offset on the heights for which to create commitments.

m_header_commitments entries are created at any height h for which (h % HEADER_COMMITMENT_PERIOD) == m_commit_offset.

Definition at line 261 of file headerssync.h.

◆ m_consensus_params

const Consensus::Params& HeadersSyncState::m_consensus_params

private

We use the consensus params in our anti-DoS calculations.

Definition at line 229 of file headerssync.h.

◆ m_current_chain_work

arith_uint256 HeadersSyncState::m_current_chain_work

private

Work that we've seen so far on the peer's chain.

Definition at line 241 of file headerssync.h.

◆ m_current_height

int64_t HeadersSyncState::m_current_height {0}

private

Height of m_last_header_received.

Definition at line 279 of file headerssync.h.

◆ m_download_state

State HeadersSyncState::m_download_state {State::PRESYNC}

private

Current state of our headers sync.

Definition at line 316 of file headerssync.h.

◆ m_hasher

const SaltedBlockHashHasher HeadersSyncState::m_hasher

private

m_hasher is a salted hasher for making our 1-bit commitments to headers we've seen.

Definition at line 247 of file headerssync.h.

◆ m_header_commitments

bitdeque HeadersSyncState::m_header_commitments

private

A queue of commitment bits, created during the 1st phase, and verified during the 2nd.

Definition at line 253 of file headerssync.h.

◆ m_id

const NodeId HeadersSyncState::m_id

private

NodeId of the peer (used for log messages)

Definition at line 226 of file headerssync.h.

◆ m_last_header_received

CBlockHeader HeadersSyncState::m_last_header_received

private

Store the latest header received while in PRESYNC (initialized to m_chain_start)

Definition at line 276 of file headerssync.h.

◆ m_max_commitments

uint64_t HeadersSyncState::m_max_commitments {0}

private

m_max_commitments is a bound we calculate on how long an honest peer's chain could be, given the MTP rule.

Any peer giving us more headers than this will have its sync aborted. This serves as a memory bound on m_header_commitments.

Definition at line 270 of file headerssync.h.

◆ m_minimum_required_work

const arith_uint256 HeadersSyncState::m_minimum_required_work

private

Minimum work that we're looking for on this chain.

Definition at line 238 of file headerssync.h.

◆ m_process_all_remaining_headers

bool HeadersSyncState::m_process_all_remaining_headers {false}

private

Set this to true once we encounter the target blockheader during phase 2 (REDOWNLOAD).

At this point, we can process and store all remaining headers still in m_redownloaded_headers.

Definition at line 313 of file headerssync.h.

◆ m_redownload_buffer_first_prev_hash

BlockHash HeadersSyncState::m_redownload_buffer_first_prev_hash

private

The hashPrevBlock entry for the first header in m_redownloaded_headers We need this to reconstruct the full header when it's time for processing.

Definition at line 303 of file headerssync.h.

◆ m_redownload_buffer_last_hash

BlockHash HeadersSyncState::m_redownload_buffer_last_hash

private

Hash of last header in m_redownloaded_headers (initialized to m_chain_start).

We have to cache it because we don't have hashPrevBlock available in a CompressedHeader.

Definition at line 296 of file headerssync.h.

◆ m_redownload_buffer_last_height

int64_t HeadersSyncState::m_redownload_buffer_last_height {0}

private

Height of last header in m_redownloaded_headers.

Definition at line 289 of file headerssync.h.

◆ m_redownload_chain_work

arith_uint256 HeadersSyncState::m_redownload_chain_work

private

The accumulated work on the redownloaded chain.

Definition at line 306 of file headerssync.h.

◆ m_redownloaded_headers

std::deque<CompressedHeader> HeadersSyncState::m_redownloaded_headers

private

During phase 2 (REDOWNLOAD), we buffer redownloaded headers in memory until enough commitments have been verified; those are stored in m_redownloaded_headers.

Definition at line 286 of file headerssync.h.

The documentation for this class was generated from the following files:

src/headerssync.h
src/headerssync.cpp

Classes

Public Types

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

Member Enumeration Documentation

◆ State

Constructor & Destructor Documentation

◆ ~HeadersSyncState()

◆ HeadersSyncState()

Member Function Documentation

◆ Finalize()

◆ GetPresyncHeight()

◆ GetPresyncTime()

◆ GetPresyncWork()

◆ GetState()

◆ NextHeadersRequestLocator()

◆ PopHeadersReadyForAcceptance()

◆ ProcessNextHeaders()

◆ ValidateAndProcessSingleHeader()

◆ ValidateAndStoreHeadersCommitments()

◆ ValidateAndStoreRedownloadedHeader()

Member Data Documentation

◆ m_chain_start

◆ m_commit_offset

◆ m_consensus_params

◆ m_current_chain_work

◆ m_current_height

◆ m_download_state

◆ m_hasher

◆ m_header_commitments

◆ m_id

◆ m_last_header_received

◆ m_max_commitments

◆ m_minimum_required_work

◆ m_process_all_remaining_headers

◆ m_redownload_buffer_first_prev_hash

◆ m_redownload_buffer_last_hash

◆ m_redownload_buffer_last_height

◆ m_redownload_chain_work

◆ m_redownloaded_headers