Snapshots and fast sync

A node joining a chain that is years old does not have to replay every block since block 0. It follows the spine chain, downloads the latest snapshot the spine names, checks it piece by piece and against its peers, and then applies only the blocks made since. This is fast sync. At no point does the node take a website's or a single peer's word for the state of the chain.

Snapshots

A snapshot is the whole state of the chain, every balance, seat and record, at one height. Every main_snapshot_interval blocks, 172,800 blocks or about 30 days on mainnet, each node writes a snapshot of the state as it stood 760 blocks below that trigger height. (On devnet: every 240 blocks, 80 blocks deep.) Nodes stagger the work by up to 60 blocks, but every node's snapshot is identical byte for byte.

The snapshot is one CBOR document holding the chain's state tables (57 of them), the recent block headers, the recent transactions, the transactions whose keeping is still paid for (see paying to keep data (longevity)), and recent published receipts. It is cut into chunks of 102,400 bytes, each named by its SHA3-256 hash. The list of those hashes, with the hash of the whole file, the reference height, an expiry and the chain's genesis hash, forms the manifest, which a spine block publishes.

Nothing inside a snapshot is signed. Its authority is the spine block that names it: a block co-signed by two thirds of the seats, each of which first compared the manifest with its own snapshot, hash for hash.

Each node keeps only its three newest snapshot generations. Every snapshot since block 0 stays available in the decentralised storage, its pieces held by a few registered nodes chosen by lot (see archive and reading the past).

Fast sync, step by step

A node attempts fast sync when it is at least 100 blocks behind and holds a usable manifest above its own height. It must hold the spine block that carries the manifest, with its signatures. The newest manifest stays usable even past its expiry.

1. Sync the spine. A node that is far behind stores a spine block whose reference lies above its own tip when that block carries its own threshold of signatures (two thirds, or more than half for a fallback block). It checks the block's content later, once its main chain reaches the reference, and stops with an alarm on a mismatch.
2. Fetch the chunks from connected registered peers, in up to 16 shards, checking each chunk against the hash that names it.
3. Check the whole file against the manifest's full-file hash.
4. Load the state in a single database transaction.
5. Check a sample with peers (see below).
6. Apply the remaining blocks one by one, with full validation, up to the tip.

The attestation: checking the state with peers

Matching hashes prove that the bytes are the ones the manifest names. They do not prove that those bytes are the real state. The spine's co-signers already compare every manifest with their own snapshot; from version 0.5.8 a joining node adds a check of its own, in case it was handed a false manifest.

After loading, the node picks up to 8 accounts and 8 registry nodes from its new state. The choice is driven by the snapshot's own hash, so every joining node picks the same rows and a forger cannot know in advance which ones to make consistent. The node asks 3 random registered peers for each account's balance and each node's key and status at the snapshot height. Honest answers about one chain at one height are identical.

• At least 2 peers agree and none contradicts: the node goes on.
• Contradictions are not outnumbered: it widens the poll to 7 peers and then needs at least 2 agreeing answers and at least two thirds of those answering.
• Otherwise it refuses the snapshot and falls back to syncing block by block.

If too few peers answer, the node proceeds on the strength of the hashes alone.

Why the result can be trusted

• The spine block that names the snapshot was co-signed by two thirds of the seats after each compared the manifest with its own snapshot.
• Every chunk, and the whole file, must match the manifest's hashes.
• A sample of the loaded state must agree with independent registered peers, chosen at random.
• Every block after the snapshot is fully validated.
• The spine itself can be anchored by the release pin that ships with the node's release.

Nobody signs snapshots by hand, no website is trusted, and nothing is replayed from block 0.

See also

• The spine chain
• The archive and reading the past
• Finality and rollback
• Pruning of history
• Running a node