Lifecycle Of A Blockchain Transaction

First, I want to thank Patrick McCorry for making all of this knowledge available to myself and others.

Second, I'd like to assert that I'm writing from a Bitcoin and Ethereum paradigm, but these principles apply to most blockchains.

Overview

We'll be looking at the lifecycle of a transaction that can be broken down into three simple steps: (1) a user sends a transaction to the peer-to-peer network, (2) pending transactions eventually reach block producers, and (3) the fee market entices block producers (miners) to pick transactions for the next block. In order to expand on each of these points, we'll cover what peers (nodes) and block producers (miners) are in detail. In a following blog, we'll go over exactly what users are, and how they interact with the blockchain network.

Peers Of A Peer-To-Peer Network

The peers of a peer-to-peer (P2P) network are essentially us, the people, represented as nodes (computers). Being a peer on the P2P network boils down to stabilizing a network where financial incentives lie, or altruistically.

As a peer (node) on the network, you offer connection slots, propagate messages, and serve history. Connection slots are connections to a small group of peers (nodes) to help form a mesh network where nodes connect directly, dynamically, and non-hierarchically to as many other nodes as possible to cooperate with one another. The propagation of messages acts as a filter where nodes check the validity of transactions and blocks before passing them onto their peers. Serving history helps new peers catch up on the network by providing historical blocks.

Peers are able to connect with other peers through two approaches: (1) trusted DNS seeds (hard-coded) and (2) advertisements on the network. DNS seeds are servers which return trusted IP addresses (peers) that when connected to can be sent requests for a list of active peers (IP addresses; up to 1,000) on the network that are ranked by the DNS seed's reputation algorithm. Advertising on the network is done by sharing your nodes connection details to its closest peers (or initial connections) who will then advertise the details to their closest peers.

There are two types of connections peers are able to have: incoming and outgoing. Incoming connections happens when a peer wants to connect their node with your node. An outgoing connection happens when you want to connect your node with a peer's node. It's important to have both incoming and outgoing connections because this increases the availability of connection slots, and prevents one from a solar eclipse attack- where a peer's point-of-view is controlled by malicious agents who choose to feed the peer incorrect information. Connecting to at least one honest peer guarantees that you'll always be able to discern the truth and follow the correct (longest/heaviest) blockchain.

In order for a transaction to be propagated throughout the entire network, a user must advertise a transaction to a single peer on the network. The peer in turn makes an announcement to all of its connected peers sending a hash of the transaction (fixed byte value rather than the full transaction payload), and the connected peers receiving the announcement will determine whether or not they have the transaction payload already by querying their local replicated database with the transaction hash. If the connected peer doesn't already have the transaction payload, then they'll request it from the initial peer that made the announcement to begin with- then, the connected peer makes an announcement just like the initial peer until finally every peer on the network has the transaction payload (data) in their local replicated database.

The reason peers propagate a transaction hash first rather than the transaction payload is to save on bandwidth consumption used by the network. If a connected peer already has the transaction payload, then there's no reason to send it over.

Each peer (node) has a memory pool- which is a pending list of transactions that have been propagated, but not confirmed within a newly produced block. Memory pools are a local policy implemented by nodes and have nothing to do with consensus (agreement of the network), and they help mitigate flood protection on the network by relatively synchronizing transaction lists across all peers. Flood attacks are costly to perform as every transaction included in the blockchain will be charged a fee. In Bitcoin, duplicate transactions pay the minimum transaction fee- but in Ethereum, they pay a 10% fee bump.

There are different types of nodes that support the network: full, archival, pruned, and light-client. Every node validates blocks and transactions in real-time, except light-clients. Full nodes keep a copy of all historical blocks, and makes a copy of the database publicly available. Archival nodes (a.k.a. indexed nodes) keep a snapshot of the state of the database for every block produced. Light-client nodes keep a collection of it's own data (user activity), and a very shallow copy of the blockchain (block headers).

The ultimate goal of a peer-to-peer network is data availability and message propagation; which guarantees that new blocks are publicly available, and that anyone (without bounds) can send a message to all peers on the network.

Block Producers Of A Blockchain

The block producers (miners) of a blockchain are essentially entities with financial incentive to produce the next block. They provide a simple service, and don't run the network. They're incentivized to be honest, but should not be trusted. Block producers are able to produce a block that doesn't follow protocol, but the P2P network will reject it and it'll cost the block producer money. Their activity can be broken down to this: order transactions within a block, and extend the longest chain with the block.

In order for a block producer to be chosen to produce the next block for the blockchain, he/she/it must win the leader election. There is a tricky question which comes into play here: how do we create a list candidates when there's no strong notion of identity? There's isn't a trusted identity to create such a list either. The answer to this question is Proof Of X (work/stake); where one proves ownership over a scarce resource that should be limited and expensive to acquire. In the Nakamoto Consensus algorithm, there are two ways to win the leader election: self-selection, and proportional-selection. Self-selection is where participants (block producers) purchase resources and register. Proportional-selection is where participants are elected to become the leader proportional to their ownership of resources.

The leader election is not a free game to play, and only works if the miner (block producer) can cover costs of producing a new block and return a profit still. There are two types of cost associated with being a block producer: upfront costs to purchase hardware or tokens, and on-going costs to participate which includes electricity and computation bandwidth. There are two revenue streams that come from being a block producer: network fees, and the issuance of new coins (tokens). Network fees are collected from user transactions (included the block produced), and issuance fees are a protocol reward which subsidizes the block producer with newly minted coins. The long term plan for Bitcoin is to rely solely on network fees to reward the miners because eventually all 21 million Bitcoins will have been minted and issued.

Producing blocks of transactions is a knapsack problem- the space of a block is limited, and the block producer (miner) wants to fill the bag with transactions that'll return the most profit. Every user transaction carries with it a weight (byte size/ gas limit) and value (fee paid to the miner), so the block producer can determine which transactions they'd like to include in a newly produced block. This is why transactions are confirmed eventually, and why some transactions take a lot longer to confirm than others.