Multiple updates regarding LLMQs (#42)

* Add description of new LLMQ merkle roots to DIP4

* Rename deletedMNcount -> deletedMNsCount

* Add quorumType to parameter description and DIP6 P2P messages

* Remove quorumMinMemberAge from parameter description

This was never implemented

* Rename badVotesThreshold -> quorumDkgBadVotesThreshold

* Update testing LLMQ type to LLMQ_5_60

* Better describe active LLMQs in DIP6 and DIP7

* Bump CbTx version

* Handle review comments

dip-0002-special-transactions.md

@@ -10,7 +10,7 @@ may introduce more types.
 | 2 | Provider Update Service Transaction (ProUpServTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 |	Proposed |
 | 3 | Provider Update Registrar Transaction (ProUpRegTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 |	Proposed |
 | 4 | Provider Update Revocation Transaction (ProUpRevTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 |	Proposed |
-| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 1 |	Proposed |
+| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 2 |	Proposed |
 | 6 | Quorum Commitment | [DIP 006: Long Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0006.md) | 1 | Proposed |
 | 8 | Register Subscription Transaction (SubTxRegister) | [DIP 005: Blockchain Users](https://github.com/dashpay/dips/blob/master/dip-0005.md) | 1 |	Proposed |
 | 9 | Topup BU Credit Subscription Transaction (SubTxTopup) | [DIP 005: Blockchain Users](https://github.com/dashpay/dips/blob/master/dip-0005.md) | 1 |	Proposed |

dip-0004.md

@@ -43,7 +43,7 @@ More data could have been potentially added the same way as defined in BIP34, ho
 
 The Coinbase Transactions will become a special transaction (DIP2) to the network. We abbreviate this transaction as CbTx. Miners must calculate the correct values for each field of the CbTx while all other nodes must verify correctness of these fields. If a node receives a block with an invalid CbTx, it must reject the block.
 
-This DIP proposes adding 2 fields to the CbTx. These are the “height” and “merkleRootMNList” fields. Future DIPs might add additional fields to the CbTx.
+This DIP proposes adding multiple fields to the CbTx. Future DIPs might add additional fields to the CbTx.
 
 The special transaction type used for Coinbase Transactions is 5.
 
@@ -55,6 +55,12 @@ The transaction consists of the following data in the payload area:
 | height | uint32_t | 4 | Height of the block |
 | merkleRootMNList | uint256 | 32 | Merkle root of the masternode list |
 
+Starting with version >= 2, the following fields are added:
+
+| Field | Type | Size | Description |
+| ----- | ---- | ---- | ----------- |
+| merkleRootQuorums | uint256 | 32 | Merkle root of currently active LLMQs |
+
 ### Height in CbTx and deprecation of BIP34
 
 The CbTx contains the “height” field. It acts as a guaranteed variance in the CbTx so that each block’s CbTx gets a different hash. This is meant as a replacement for the height value currently found in the coinbase input (BIP34). With the deployment of this DIP, BIP34 becomes obsolete for new blocks and nodes should not enforce the presence of the block height in the coinbase input's “scriptSig” anymore.
@@ -88,6 +94,23 @@ A SML entry consists of the following fields:
 | keyIDVoting | CKeyID | 20 | The public key hash used for voting. |
 | isValid | bool | 1 | True if a masternode is not PoSe-banned |
 
+### Calculating the merkle root of the active LLMQs
+
+The CbTx contains the “merkleRootQuorums” field. This is the merkle root of all the hashes of the final quorum commitments
+of all active LLMQ sets.
+
+To calculate the merkle root:
+
+  1. Collect all final commitments from all LLMQs which are in the active set at the given block height. All LLMQ types need
+     to be collected.
+  2. Calculate the hash of all these commitments and collect the hashes in a list.
+  3. Sort this list in ascending order by the previously calculated hash.
+  4. Calculate the merkle root from this list of hashes in the same way it is done when calculating the merkle root of the block transactions.
+
+Please refer to [DIP6 - Long-Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0006.md) for more details on LLMQs
+and final commitments. The final commitment referred to multiple times in this DIP is the `qfcommit` message found in
+DIP6.
+
 ## Requesting the masternode lists
 
 The P2P message `GETMNLISTDIFF` is introduced to request a full masternode list or an update to a previously requested masternode list. It expects a serialized object with the following structure as arguments:
@@ -113,11 +136,21 @@ The `MNLISTDIFF` message contains a serialized object with the following structu
 | merkleFlagsCount | compactSize uint | 1-9 | Number of Merkle flag bytes |
 | merkleFlags | uint8_t[] | variable | Merkle flag bits, packed per 8 in a byte, least significant bit first |
 | cbTx | CTransaction | variable | The fully serialized coinbase transaction of blockHash |
-| deletedMNcount | compactSize uint | 1-9 | Number of ProRegTx hashes which were deleted after baseBlockHash |
+| deletedMNsCount | compactSize uint | 1-9 | Number of ProRegTx hashes which were deleted after baseBlockHash |
 | deletedMNs | uint256[] | variable | A list of ProRegTx hashes for masternode which were deleted after baseBlockHash |
 | mnCount | compactSize uint | 1-9 | Number of SML entries which were added or updated since baseBlockHash |
 | mnList | SMLEntry[] | variable | The list of SML entries which were added or updated since baseBlockHash |
 
+
+Starting with protocol version 70214, the following fields are added to the `MNLISTDIFF` message.
+
+| Field | Type | Size | Description |
+| --- | --- | --- | --- |
+| deletedQuorumsCount | compactSize uint | 1-9 | Number of LLMQs which were deleted from the active set after baseBlockHash |
+| deletedQuorums | (uint8_t+uint256)[] | variable | A list of LLMQ type and quorum hashes for LLMQs which were deleted after baseBlockHash |
+| newQuorumsCount | compactSize uint | 1-9 | Number of new LLMQs which were added to the active set since baseBlockHash |
+| newQuorums | qfcommit[] | variable | The list of LLMQ commitments for the LLMQs which were added since baseBlockHash |
+
 ## Tracking/Updating and verifying masternode lists based on MNLISTDIFF
 
 With DIP3, each block in the chain might result in a different masternode list. SPV clients are required to keep track of these multiple masternode lists so they can properly verify quorum related messages when the lists change. Very old masternode lists can be pruned as it is very unlikely that they are necessary.
@@ -132,6 +165,23 @@ To achieve this, It is suggested to first request a full masternode list from an
   6. Calculate the hash of “cbTx” and verify existence of this transaction in the block specified by “blockHash”. To do this, use the already received block header and the fields “totalTransactions”, “merkleHashes” and “merkleFlags” from the `MNLISTDIFF` message and perform a merkle verification the same way as done when a “MERKLEBLOCK” message is received. If the verification fails, abort the process and ask for diffs from another node.
   7. Store the resulting validated masternode list identified by “blockHash”
 
+## Tracking/Updating and verifying the active LLMQ sets based on MNLISTDIFF
+
+With the introduction of version 2 CbTx special transactions, a merkle root for the currently active LLMQ sets was added
+to the coinbase. This merkle root allows (SPV) clients to maintain the active LLMQ sets without the full chain.
+
+It is suggested to couple the house-/bookkeeping of active LLMQ sets with the masternode list bookkeeping described in
+the previous section and perform the following additional steps:
+
+  1. Create a copy of the active LLMQ sets which were given at "baseBlockHash". If “baseBlockHash” is all-zero, empty sets must be used.
+  2. Delete all entries found in "deletedQuorums" from the corresponding active LLMQ sets.
+  3. Verify each final commitment found in "newQuorums", by the same rules found in [DIP6 - Long-Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0006.md).
+     If any final commitment is invalid, abort the process and ask for diffs from another node.
+  4. Add the LLMQ defined by the final commitments found in "newQuorums" to the corresponsing active LLMQ sets.
+  5. Calculate the merkle root of the active LLMQ sets by following the “Calculating the merkle root of the active LLMQs” section
+  6. Compare the calculated merkle root with what is found in “cbTx”. If it does not match, abort the process and ask for diffs from another node.
+  7. Store the new active LLMQ sets the same way the masternode list is stored. 
+
 ## Copyright
 
 Copyright (c) 2018 Dash Core Team. [Licensed under the MIT License](https://opensource.org/licenses/MIT)

dip-0006.md

@@ -69,13 +69,14 @@ We will refer to these parameters and variables multiple times in the following
 
 | Parameter | Description |
 |--|--|
+| quorumType | An unsigned 8 bit integer identifying the LLMQ type |
 | quorumSize | The size of the quorum, e.g. 10, 200 or 400. This parameter has the highest influence on the performance of the DKG. A small DKG for a small LLMQ can be performed in a few seconds, while a large LLMQ can take more than a hour to complete. This is identical to the “n” in m-of-n. |
 | quorumMinSize | Minimum number of valid members for quorum to be considered valid after the DKG has removed bad members. |
 | quorumThreshold | The minimum number of threshold signature shares required to recover a threshold signature. It also influences the DKG as it is the minimum number of valid members to create a valid premature commitment and final commitment. This value should be at least 51% of the quorumSize. This is identical to the “m” in m-of-n. |
-| quorumMinMemberAge | Minimum number of blocks a deterministic masternode must be confirmed on-chain before it is considered as a LLMQ member |
 | quorumDkgInterval | The number of blocks between two DKG sessions. If set to 576 for example, a new DKG session will start once per day. |
 | quorumDkgPhaseBlocks | Number of blocks a single DKG phase is active. It must be high enough to allow all members to exchange all required messages. It must also be high enough to take block time variances into account. |
-| badVotesThreshold | Threshold for votes on bad members. If >=badVotesThreshold members have voted on another member to be bad, that member is also considered bad locally. |
+| quorumDkgBadVotesThreshold | Threshold for votes on bad members. If >=badVotesThreshold members have voted on another member to be bad, that member is also considered bad locally. |
+| quorumSigningActiveQuorumCount | Maximum number of LLMQs in the active LLMQs set |
 
 List of variables per DKG session:
 
@@ -179,6 +180,7 @@ The internal Dash message name is `qcontrib` and the format of the message is:
 
 | Field | Type | Size | Description |
 |--|--|--|--|
+| llmqType | uint8_t | 1 | The LLMQ type
 | quorumHash | uint256 | 32 | The quorum identifier
 | proTxHash | uint256 | 32 | The proTxHash of the contributing member
 | vvecSize | compactSize uint | 1-9 | The size of the verification vector
@@ -220,6 +222,7 @@ The internal Dash message name is `qcomplaint` and the format of the message is:
 
 | Field | Type | Size | Description |
 |--|--|--|--|
+| llmqType | uint8_t | 1 | The LLMQ type
 | quorumHash | uint256 | 32 | The quorum identifier
 | proTxHash | uint256 | 32 | The proTxHash of the complaining member
 | badBitSize | compactSize uint | 1-9 | Number of bits in the bad members bitvector
@@ -255,6 +258,7 @@ The internal Dash message name is `qjustify` and the format of the message is:
 
 | Field | Type | Size | Description |
 |--|--|--|--|
+| llmqType | uint8_t | 1 | The LLMQ type
 | quorumHash | uint256 | 32 | The quorum identifier
 | proTxHash | uint256 | 32 | The proTxHash of the justifying member
 | skContributionsCount | compactSize uint | 1-9 | Number of unencrypted secret key contributions
@@ -300,6 +304,7 @@ The internal Dash message name is `qpcommit` and the format of the message is:
 
 | Field | Type | Size | Description |
 |--|--|--|--|
+| llmqType | uint8_t | 1 | The LLMQ type
 | quorumHash | uint256 | 32 | The quorum identifier
 | proTxHash | uint256 | 32 | The proTxHash of the committing member
 | validMembersSize | compactSize uint | 1-9 | Bit size of the validMembers bitvector
@@ -364,7 +369,7 @@ The internal Dash message name is `qfcommit` and the format of the message is:
 | 1 | LLMQ_50_60 | 50 | 30 (60%) | 1 Hour | |
 | 2 | LLMQ_400_60 | 400 | 240 (60%) | 12 Hours | |
 | 3 | LLMQ_400_85 | 400 | 340 (85%) | 24 Hours | |
-| 100 | LLMQ_10_60 | 10 | 6 (60%) | 1 Hour | For testing only |
+| 100 | LLMQ_5_60 | 5 | 3 (60%) | 1 Hour | For testing only |
 
 ### 7. Mining phase
 
@@ -399,10 +404,17 @@ The format of the "quorum commitment" special TX payload with type=6 is:
 | height | uint32_t | 4 | The height of the block in which this commitment is included
 | commitment | qfcommit | Variable | This equals the payload of the "qfcommit" P2P message
 
-## Activated LLMQs and threshold signing
+## Active LLMQ sets
 
 After a DKG finishes and the final commitment has been mined, a LLMQ becomes active and can start participating in signing requests/sessions.
 
+The set of currently active LLMQs is a limited list with a maximum size of "quorumSigningActiveQuorumCount". When the active
+set reaches the maximum size, the oldest entry is removed each time a new LLMQ becomes active.
+
+A merkle root of all active LLMQ sets is calculated and included into [DIP4 CbTx](https://github.com/dashpay/dips/blob/master/dip-0004.md) with version >= 2.   
+
+## Signing sessions/requests
+
 The details of the signing process are handled in [DIP007](https://github.com/dashpay/dips/blob/master/dip-0007.md). The individual use cases for LLMQs are handled in specialized DIPs.
 
 ## Copyright

dip-0007.md

@@ -61,12 +61,15 @@ If different members signed the same request with differing message hashes, only
 
 ## Choosing the active LLMQ to perform signing
 
-As multiple LLMQs might be active at the same time and masternodes might be members of multiple LLMQs as well, it’s important that all masternodes agree on which LLMQ should service the signing request. Otherwise it would be impossible to collect enough signature shares for the same signing session and thus recovery of the threshold signature would never succeed.
+As multiple LLMQs are active at the same time and masternodes might be members of multiple LLMQs as well, it’s important
+that all masternodes agree on which LLMQ should service the signing request. Otherwise it would be impossible to collect
+enough signature shares for the same signing session and thus recovery of the threshold signature would never succeed.
 
 To calculate which LLMQ is responsible for a signing request, each masternode should perform the following:
 
-1. Take the last `X` activated LLMQs. The value of `X` depends on the use case, and could be 4 as an example.
+1. Take the active LLMQ set from 8 blocks before the current chain tip. Active LLMQ sets are described in
-2. For each LLMQ, calculate `hash(quorumHash, requestId)`
+   [DIP6 - Long-Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0004.md) section "Active LLMQ sets".
+2. For each LLMQ of the active set, calculate `hash(quorumType, quorumHash, requestId)`
 3. Sort the list of LLMQs based on the result of step 2 in ascending order.
 4. Use the first entry of the sorted list as the LLMQ to perform the signing request