diff --git a/dip-0002-special-transactions.md b/dip-0002-special-transactions.md new file mode 100644 index 0000000..4026111 --- /dev/null +++ b/dip-0002-special-transactions.md @@ -0,0 +1,13 @@ +## Registered transaction types + +The transaction type is described based on proposed DIPs. +Here is a table of current proposed types and their associated DIP. Future DIPs +may introduce more types. + +| Type | Transaction Type | DIP Number and Name | Version | State | +| ---- | ---------------- | ------------------- | ------- | ----- | +| 1 | Provider Registration Transaction (ProRegTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Proposed | +| 2 | Provider Update Service Transaction (ProUpServTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Proposed | +| 3 | Provider Update Registrar Transaction (ProUpRegTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Proposed | +| 4 | Provider Update Revocation Transaction (ProUpRevTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Proposed | +| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 1 | Proposed | diff --git a/dip-0002.md b/dip-0002.md new file mode 100644 index 0000000..3117397 --- /dev/null +++ b/dip-0002.md @@ -0,0 +1,90 @@ +
+ DIP: 0002 + Title: Special Transactions + Author: Samuel Westrich, Alexander Block, Andy Freer + Comments-Summary: No comments yet. + Status: Proposed + Type: Standard + Assigned DIP#: 0002 + Created: 2018-04-30 + License: MIT License ++ +## Table of Contents + +1. [Abstract](#abstract) +2. [Motivation](#motivation) +3. [Special Transactions](#special-transactions) +4. [Serialization, hashing and signing](#serialization-hashing-and-signing) +5. [Compatibility](#compatibility) +6. [Copyright](#copyright) + +## Abstract + +This DIP adds a basis for new transaction types that will provide on-chain metadata to assist various consensus mechanisms. This will allow for a more native way to implement new features which don’t naturally fit into the current concept of transactions. + + +## Motivation + +Currently, new features and consensus mechanisms have to be implemented on top of the restrictions imposed by the simple nature of transactions. Since classical transactions can only carry inputs and outputs, they are most useful for financial transactions (i.e. transfers of quantities of Dash between addresses on the distributed ledger). These inputs and outputs carry scripts and signatures which are used to authorize and validate the transaction. + +To implement new on-chain features and consensus mechanisms which do not fit into this concept of financial transactions, it is often necessary to misuse the inputs/outputs and their corresponding scripts to add additional data and meaning to a transaction. For example, new opcodes would have to be introduced to mark a transaction as something special and add a payload. In other cases, `OP_RETURN` has been misused to store data on-chain. + +This method of implementing new features results in some verification and consistency complications that can only be resolved with complicated, error-prone code. It also requires all parts of the ecosystem to reimplement the verification and consistency checks for each new feature. + +Dash has so far avoided such extensions and misuse of the transaction format, but upcoming features would make this unavoidable. Instead of misusing the inputs/outputs of the transaction for every new feature or consensus mechanism, this DIP introduces a solution to provide a reusable framework for such extensions. This framework, called “Special Transactions”, updates and expands classical transactions with an additional type (a meaning) and an additional payload. + + +## Special Transactions + +This DIP increments the transaction version to 3 and splits the field into two 16-bit fields with the second field now becoming transaction type. A classical transaction will be of transaction type 0 to remain backwards compatible. + +A non-zero type indicates a special transaction with an extra payload appended after the last of the classical transaction fields. The internal structure of the payload is specific to the special transaction type. A newly introduced special transaction type should clearly define the structure of the payload so that it can be safely serialized, deserialized and verified. The individual new special transaction types will be defined by new DIPs and are not subject to this DIP. + +The inputs and outputs of special transactions usually have the same meaning as with classical transactions. Verification and processing of these does not change for most special transaction types. If exceptions are made to this rule, a new DIP describing the changes in detail and a deployment plan must be provided. + +The new transaction format is defined as follows: + +| Bytes | Name | Data Type | Description | +| ----- | ---- | --------- | ----------- | +| 2 | version | uint16_t | Transaction version number; currently version 2. Programs creating transactions using newer consensus rules may use higher version numbers. | +| 2 | type | uint16_t | Type for a classical transaction will be 0. View [transaction types](dip-0002-special-transactions.md) for more information. | +| Varies | tx_in count | compactSize uint | Number of inputs in this transaction. | +| Varies | tx_in | txIn | Transaction inputs. | +| Varies | tx_out count | compactSize uint | Number of outputs in this transaction. | +| Varies | tx_out | txOut | Transaction outputs. | +| 4 | lock_time | uint32_t | A time (Unix epoch time) or block number. | +| 1-9 | extra_payload size | compactSize uint | Variable number of bytes of extra payload. | +| Varies | extra_payload | blob | Extra payload will be appended based on the transaction type. | + + +## Serialization, hashing and signing + +While implementing special transactions, implementors might choose to extend existing transaction classes by adding the payload +fields to these classes. However, it is important to keep backwards compatibility when serializing, hashing or signing +instances of these classes. This means that any classical transaction (type = 0, regardless of version) should NOT include +these new fields while serializing, hashing or signing. A serialized classical transaction must always be identical +to a transaction serialized by a program or library which does not support special transactions. The same applies to +hashing and signing of classical transactions. + +For special transactions, the `extra_payload_size` and `extra_payload` fields must be included when serialized, deserialized or hashed. +All OP-Codes which perform any signing/verification (e.g. CHECKSIGVERIFY) must also include both fields when calculating the signed hash. +This ensures that payloads can’t be modified after a transaction has been signed. + + +## Compatibility + +The introduction of special transactions will require the whole Dash ecosystem to perform a one-time mandatory update of all the software and libraries involved. + +Software and libraries will have to be changed so that they can differentiate between classical transactions and special transactions. Deserialization of a classical transaction remains unchanged. Deserialization of a special transaction requires the software/library to at least implement skipping and ignoring the extra_payload field. Validation and processing of the inputs and outputs of a special transaction must remain identical to classical transactions. + +A classical transaction can be identified by the 16-bit type field being 0. Please note that if the the 16-bit type field is 0, both 16-bit fields can also be interpreted as a single 32-bit field which would equal the value in the 16-bit version field. This is to allow some backwards compatibility for software/libraries which do not implement special transactions. However, this backwards compatibility is limited and will fail when special transactions are encountered. + +A special transaction can be identified by the 16-bit version field being >= 3 and the 16-bit type field being non-zero. + +The idea is to minimize the work necessary for future network wide upgrades. Only the initial introduction of special transactions will require more work. After that, the software and library implementers will determine if and when they want to implement newly introduced special transaction types. Clients that, for example, are only interested in following the financial ledger can completely ignore new special transactions as long as they correctly process the inputs and outputs of these special transactions. Only clients that are interested in Dash’s advanced features would need to selectively implement new special transaction types. + + +## Copyright + +Copyright (c) 2018 Dash Core Team. [Licensed under the MIT License](https://opensource.org/licenses/MIT) diff --git a/dip-0003-masternode-types.md b/dip-0003-masternode-types.md new file mode 100644 index 0000000..0d73f44 --- /dev/null +++ b/dip-0003-masternode-types.md @@ -0,0 +1,5 @@ +# Masternode Types + +| Masternode Type | Description | +| --- | --- | +| 0 | Default | diff --git a/dip-0003.md b/dip-0003.md new file mode 100644 index 0000000..38e1ef0 --- /dev/null +++ b/dip-0003.md @@ -0,0 +1,293 @@ +
+ DIP: 0003 + Title: Deterministic Masternode Lists + Author(s): Samuel Westrich, Alexander Block, Andy Freer, Darren Tapp, Timothy Flynn, Udjinm6, Will Wray + Comments-Summary: No comments yet. + Status: Proposed + Type: Standard + Assigned DIP#: 0003 + Created: 2018-04-30 + License: MIT License ++ +## Table of Contents + +1. [Abstract](#abstract) +2. [Motivation and Previous System](#motivation-and-previous-system) +3. [Prior Work](#prior-work) +4. [New On-Chain System](#new-on-chain-system) +5. [Registering a Masternode (ProRegTx)](#registering-a-masternode-proregtx) +6. [Updating Masternode Information](#updating-masternode-information) +7. [Rules for Updating the Masternode List](#rules-for-updating-the-masternode-list) +8. [Validation Rules](#validation-rules) +9. [Masternode Rewards](#masternode-rewards) +10. [Selecting Quorums](#selecting-quorums) +11. [Appendix A: Reasons for Self Revocation of Operators](#appendix-a-reasons-for-self-revocation-of-operators) +12. [Copyright](#copyright) + +## Special Mention + +We would like to dedicate this DIP to Timothy Flynn who wrote the initial proposal around a Deterministic Masternode list. + +## Abstract + +This DIP provides on-chain consensus for masternode lists that in turn allow for deterministic quorum derivation and service scoring of masternode rewards in Dash. + +## Motivation and Previous System + +In the previous system, each node maintained its own individual masternode list. Masternodes gained entry to that masternode list after the owner created a 1000 Dash UTXO and the masternode broadcast a "masternode broadcast/announcement" P2P message. This in turn set the masternode to a `PRE_ENABLED` state in the list maintained by each node. Masternodes then regularly broadcasted ping messages to keep the masternode in `ENABLED` state. + +### Problems with the previous system + +The previous system was maintained with consensus mechanisms that predated Satoshi Nakamoto’s solution to the Byzantine Generals Problem. This meant that each node needed to maintain their own individual masternode list with P2P messages and not a blockchain based solution. Due to the nature of the P2P system, there was no guarantee that nodes would come to the same conclusion on what the masternode list ought to look like. Discrepancies might, for example, occur due to a different order of message reception or if messages had not been received at all. This posed some risks in regard to consensus and limited the possible uses of quorums by the system. + +As a concrete example, the previous system required implementing workarounds such as "masternode reward voting" which was performed multiple blocks in advance for each block to make sure that consensus would be found and agreed on. Enforcing this consensus however still posed a risk which could have resulted in network wide forking, so a spork to turn off masternode payment enforcement was added to prevent this issue from occurring. The spork was used sporadically after major network updates. + +When quorums were used, for example in InstantSend, nodes would sometimes calculate a different quorum set for an InstantSend lock resulting in partially failed locks. This was mostly non-problematic due to the size of the quorum and the quorum selection algorithm, but could still potentially result in individual nodes forking away from the network. This also required implementation of a spork to disable InstantSend related block filtering. + +Additionally, the previous system required a complicated and failure prone "masternode sync" after the initial startup of the node. After the blockchain was synced, the node would request the current masternode list, the reward payment votes, and then verify the received list. This process tended to take an unnecessarily long amount of time and sometimes resulted in failure. + +Furthermore, the previous system could not be audited for past masternode reward payments. Inclusion of masternode data on-chain will make the chain auditable from the moment of crossover. + +The previous system also made certain desirable features infeasible. The integration of a service score was near impossible without a certain consensus. A hardened masternode list also expands on possible system capabilities and makes masternode shares possible. + +## Prior Work + + * [DIP 0002: Special Transactions](https://github.com/dashpay/dips/blob/master/dip-0002.md) + +## New On-Chain System + +In the new system, the masternode list is derived entirely from information found on-chain. New masternodes are added by new special transactions called Provider Registration Transactions (abbreviated as ProRegTx). They are only removed by spending the collateral. A ProRegTx is a special transaction which consists of a 1000-Dash collateral payment, along with payload information (DIP2) described later in this document. + +This DIP defines 2 masternode (sub)sets: + + 1. Registered/Full set: All masternodes which were registered on-chain and have not spent their collateral + 2. Valid subset: All masternodes which are not marked as PoSe banned + +Only the valid subset is used in calculations to determine the recipient of the masternode reward. Quorums are also based on the valid subset, but they may include additional constraints that further reduce the subset size. For example, a quorum might only use masternodes with a minimum age. + +## Registering a Masternode (ProRegTx) + +To join the masternode list, masternode owners must submit a special transaction (DIP2) to the network. This transaction is called a Provider Registration Transaction and is abbreviated to ProRegTx. The ProRegTx must contain one output with the 1000 Dash collateral. + +The ProRegTx contains 3 public key IDs, which represent 3 different roles in the masternode and define update and voting rights. A "public key ID" refers to the hash160 of an ECDSA public key. These are: + + 1. KeyIdOwner: This is the public key ID of the masternode or collateral owner. It is different than the key used in the collateral output. The owner is only allowed to issue ProUpRegTx transactions. + 2. KeyIdOperator: This is the public key ID of the masternode operator. The operator is only allowed to issue ProUpServTx transactions. The operator key is also used while operating the masternode to sign masternode related P2P messages, quorum related messages and governance trigger votes. Messages signed with this key are only valid while the masternode is in the valid set. + 3. KeyIdVoting: This is the public key ID used for proposal voting. Votes signed with this key are valid while the masternode is in the registered set. + +A single key can be used for multiple roles so the owner can also operate the masternode and/or issue proposal votes. Alternately, different keys can be used to delegate duties to other people. For example, owners might decide to use a masternode hosting service instead of self-hosting the masternode. To do so, they would use the operator key provided by the hosting service. The same applies to the voting key, which allows them to assign voting rights to someone else. + +The ProRegTx might have the IP address and port fields set to zero. This will result in the masternode being added to the masternode list in the PoSe-banned state and require the operator to issue a ProUpServTx. This is needed in case the owner wants to delegate the operator role to a hosting service without knowing the IP Address and port in advance. + +The ProRegTx also defines the masternode's type and mode. Default masternodes are of type 0, and future types will be defined in an [annex document](dip-0003-masternode-types.md). Different types would introduce additional or different consensus rules towards the masternodes based on future Dash improvement proposals. Masternode types are not updatable, however each type can have multiple operation modes. The operation modes are updatable. + +The ProRegTx also specifies the reward for the operator. The percentage of the masternode reward paid to the operator is calculated by dividing the operatorReward field by 100. When the field is set to zero, the owner will get the full reward for every block. If set to a non-zero value, the masternode operator may specify his operator reward payee script in a ProUpServTx. If he does not do so, the full reward goes to the masternode owner. + +After a ProRegTx is mined into the chain, the corresponding masternode entry is added to the registered masternode set and therefore eligible for PoSe verification, masternode rewards and quorum participation. + +The special transaction type used for Provider Transactions is 1. + +The transaction consists of the following data in the payload area: + +| Field | Type | Size | Description | +| --- | --- | --- | --- | +| version | uint_16 | 2 | Provider transaction version number. Currently set to 1. | +| type | uint_16 | 2 | Masternode type. Default set to 0. | +| mode | uint_16 | 2 | Masternode mode. Default set to 0. | +| collateralIndex | uint_32 | 4 | The collateral index. | +| ipAddress | byte[] | 16 | IPv6 address in network byte order. Only IPv4 mapped addresses are allowed (to be extended in the future) | +| port | uint_16 | 2 | Port (network byte order) | +| KeyIdOwner | CKeyID | 20 | The public key hash used for owner related signing (ProTx updates, governance voting) | +| KeyIdOperator | CKeyID | 20 | The public key hash used for operational related signing (network messages, ProTx updates) | +| KeyIdVoting | CKeyID | 20 | The public key hash used for voting. | +| operatorReward | uint_16 | 2 | A value from 0 to 10000. | +| scriptPayoutSize | compactSize uint | 1-9 | Size of the Payee Script. | +| scriptPayout | Script | Variable | Payee script (p2pkh/p2sh) | +| inputsHash | uint256 | 32 | Hash of all the outpoints of the transaction inputs | +| payloadSigSize | compactSize uint | 1-9 | Size of the Signature | +| payloadSig | vector
+ DIP: 0004 + Title: Simplified Verification of Deterministic Masternode Lists + Author: Alexander Block, Samuel Westrich, UdjinM6, Andy Freer + Comments-Summary: No comments yet. + Status: Proposed + Type: Standard + Assigned DIP#: 0004 + Created: 2018-04-30 + License: MIT License ++ +## Table of Contents + +1. [Abstract](#abstract) +2. [Motivation](#motivation) +3. [Prior Work](#prior-work) +4. [History of the Coinbase Transaction](#history-of-the-coinbase-transaction) +5. [Coinbase Special Transaction](#coinbase-special-transaction) +6. [Requesting the masternode lists](#requesting-the-masternode-lists) +7. [Tracking/Updating and verifying masternode lists based on MNLISTDIFF](#trackingupdating-and-verifying-masternode-lists-based-on-mnlistdiff) +8. [Copyright](#copyright) + +## Abstract + +This DIP provides SPV (Simplified Payment Verification) clients the ability to verify the correctness of a network provided deterministic masternode list. This assumes that DIP2 and DIP3 have been deployed or are deployed at the same time as this DIP. + +## Motivation + +A verifiable and correct masternode list is foundational to many Dash features, including verification of an InstantSend transaction, mixing in PrivateSend and many features of Evolution. The deterministic masternode lists introduced by DIP3 enable full derivation and verification of a masternode list via on-chain data. This, however, requires the full chain to be available to construct or verify this list. A SPV client does not have the full chain and thus would have to rely on a list provided by one or more nodes in the network. This provided list must be verifiable by the SPV client without needing the full chain. This DIP proposes additions to the block’s coinbase transaction and new P2P messages to get and update a masternode list with additional proof data. + +## Prior Work +* [DIP 0002: Special Transactions](https://github.com/dashpay/dips/blob/master/dip-0002.md) +* [DIP 0003: Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0003.md) +* [BIP 34: Block v2, Height in Coinbase](https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki) + +## History of the Coinbase Transaction + +Currently, the coinbase transaction is just a classic transaction with a single input that has the “prevout” and “index” (hash and index of previous transaction output) fields set to zero and 0xffffffff, respectively, and at least one output which generates new coins (the miner/masternode reward and budget payments). Initially, the “scriptSig” field of the coinbase input could contain arbitrary data, but it was later changed to at least include the block’s height as the first item in order to guarantee some variation in the coinbase. Without this variation, a miner could have created a coinbase which resulted in the same hash as an older coinbase transaction. This change was enforced by the deployment of BIP34. + +More data could have been potentially added the same way as defined in BIP34, however changing the coinbase to a special transaction became preferable with the introduction of special transactions (DIP2). This is because it provided a much cleaner solution and allowed future changes to be deployed in an easier, more consistent way. + +## Coinbase Special Transaction + +The Coinbase Transactions will become a special transaction (DIP2) to the network. We abbreviate this transaction as CbTx. Miners must calculate the correct values for each field of the CbTx while all other nodes must verify correctness of these fields. If a node receives a block with an invalid CbTx, it must reject the block. + +This DIP proposes adding 2 fields to the CbTx. These are the “height” and “merkleRootMNList” fields. Future DIPs might add additional fields to the CbTx. + +The special transaction type used for Coinbase Transactions is 5. + +The transaction consists of the following data in the payload area: + +| Field | Type | Size | Description | +| ----- | ---- | ---- | ----------- | +| version | uint16_t | 2 | CbTx version number. Currently set to 1. | +| height | uint32_t | 4 | Height of the block | +| merkleRootMNList | uint256 | 32 | Merkle root of the masternode list | + +### Height in CbTx and deprecation of BIP34 + +The CbTx contains the “height” field. It acts as a guaranteed variance in the CbTx so that each block’s CbTx gets a different hash. This is meant as a replacement for the height value currently found in the coinbase input (BIP34). With the deployment of this DIP, BIP34 becomes obsolete for new blocks and nodes should not enforce the presence of the block height in the coinbase input's “scriptSig” anymore. + +Miners must set this field to the block height of the current block being generated. Other nodes should validate that the CbTx of received blocks contains the correct block height. + +### Calculating the merkle root of the Masternode list + +The CbTx contains the “merkleRootMNList” field. This is the merkle root of all the hashes of the Simplified Masternode List (SML) entries. A [SML entry](#sml-simplified-masternode-list-entry) is defined in the next section. + +To calculate the merkle root: + + 1. Get the full masternode list (including PoSe-banned) of the current block. This list must also include all the updates which would have been performed by the data (DIP3 special transactions, PoSe verification, etc.) in the current block. + 2. Sort this list in ascending order by the hash of the ProRegTx of each entry. + 3. For each entry in the list, create a SML entry and calculate the hash of this entry and add the hash into a new list. + 4. Calculate the merkle root from this list of hashes in the same way it is done when calculating the merkle root of the block transactions. + +### SML (Simplified Masternode List) entry + +A SML entry is used to calculate the hashes for the merkleRootMNList. It is also used in the new P2P messages when a SPV client requests a full masternode list or updates to a masternode list. + +A SML entry consists of the following fields: + +| Field | Type | Size | Description | +| --- | --- | --- | --- | +| proRegTxHash | uint256 | 32 | The hash of the ProRegTx that identifies the masternode | +| ipAddress | byte[] | 16 | IPv6 address in network byte order. Only IPv4 mapped addresses are allowed (to be extended in the future) | +| port | uint_16 | 2 | Port (network byte order) | +| keyIDOperator | CKeyID | 20 | The operators public key hash | +| keyIDVoting | CKeyID | 20 | The public key hash used for voting. | +| isValid | bool | 1 | True if a masternode is not PoSe-banned | + +## Requesting the masternode lists + +The P2P message `GETMNLISTDIFF` is introduced to request a full masternode list or an update to a previously requested masternode list. It expects a serialized object with the following structure as arguments: + +| Field | Type | Size | Description | +| --- | --- | --- | --- | +| baseBlockHash | uint256 | 32 | Hash of a block the requestor already has a valid masternode list of. Can be all-zero to indicate that a full masternode list is requested. | +| blockHash | uint256 | 32 | Hash of the block for which the masternode list diff is requested | + +After receiving the `GETMNLISTDIFF` P2P message, a node is required to respond with a new P2P message called `MNLISTDIFF`, which is described later. + +To request a full masternode list, a SPV client would set the “baseBlockHash” field of `GETMNLISTDIFF` to an all-zero hash. In this case, the `MNLISTDIFF` response would not contain any “deletedMNs” entries. After receiving an initial full masternode list, the SPV client can then send a `GETMNLISTDIFF` message with the “baseBlockHash” field set to the block hash of the last known masternode list. + +The `MNLISTDIFF` message contains a serialized object with the following structure: + +| Field | Type | Size | Description | +| --- | --- | --- | --- | +| baseBlockHash | uint256 | 32 | Hash of the block on which this diff is based on | +| blockHash | uint256 | 32 | Hash of the block for which the masternode list diff is returned | +| totalTransactions | uint32_t | 4 | Number of total transactions in blockHash | +| merkleHashes | vector