diff --git a/_autocrossref.yaml b/_autocrossref.yaml
index bd7b88bd..ac1ba50c 100644
--- a/_autocrossref.yaml
+++ b/_autocrossref.yaml
@@ -20,7 +20,11 @@ bitcoin QR codes: URI QR code
bitcoinj:
'`block` message': block message
'`block` messages': block message
+'`blocktxn` message': blocktxn message
+'`blocktxn` messages': blocktxn message
certificate chain:
+'`cmpctblock` message': cmpctblock message
+'`cmpctblock` messages': cmpctblock message
DER:
DER format: der
DER-formatted: der
@@ -57,6 +61,8 @@ fiat:
'`filterload` messages': filterload message
'`getblocks` message': getblocks message
'`getblocks` messages': getblocks message
+'`getblocktxn` message': getblocktxn message
+'`getblocktxn` messages': getblocktxn message
'`getdata` message': getdata message
'`getdata` messages': getdata message
'`getheaders` message': getheaders message
@@ -172,6 +178,8 @@ RPC:
RPCs: rpc
'`script`': DO NOT AUTOCROSSREF
secp256k1:
+'`sendcmpct` message': sendcmpct message
+'`sendcmpct` messages': sendcmpct message
'`spork` message': spork message
'`spork` messages': spork message
'`ssc` message': ssc message
diff --git a/_includes/devdoc/ref_p2p_networking.md b/_includes/devdoc/ref_p2p_networking.md
index 0b56aa95..62448ada 100644
--- a/_includes/devdoc/ref_p2p_networking.md
+++ b/_includes/devdoc/ref_p2p_networking.md
@@ -172,8 +172,9 @@ The currently-available type identifiers are:
| 17 | [`MSG_GOVERNANCE_OBJECT`][msg_governance_object]{:#term-msg_governance_object}{:.term} | The hash is a Governance Object.
| 18 | [`MSG_GOVERNANCE_OBJECT_VOTE`][msg_governance_object_vote]{:#term-msg_governance_object_vote}{:.term} | The hash is a Governance Object Vote.
| 19 | [`MSG_MASTERNODE_VERIFY`][msg_masternode_verify]{:#term-msg_masternode_verify}{:.term} | The hash is a Masternode Verify.
+| 20 | [`MSG_CMPCT_BLOCK`][msg_cmpct_block]{:#term-msg_cmpct_block}{:.term} | The hash is of a block header; identical to `MSG_BLOCK`. When used in a `getdata` message, this indicates the response should be a `cmpctblock` message. **Only for use in `getdata` messages.**
-Type identifier zero and type identifiers greater than nineteen are reserved
+Type identifier zero and type identifiers greater than twenty are reserved
for future implementations. Dash Core ignores all inventories with
one of these unknown types.
@@ -202,6 +203,166 @@ different reasons:
{% endautocrossref %}
+#### Blocktxn
+{% include helpers/subhead-links.md %}
+
+{% autocrossref %}
+
+*Added in protocol version 70209 of Dash Core as described by BIP152*
+
+The `blocktxn` message sends requested block transactions to a node which
+previously requested them with a `getblocktxn` message. It is defined as a message
+containing a serialized `BlockTransactions` message.
+
+Upon receipt of a properly-formatted requested `blocktxn` message, nodes should:
+
+1. Attempt to reconstruct the full block by taking the prefilledtxn transactions from the original `cmpctblock` message and placing them in the marked positions
+2. For each short transaction ID from the original `cmpctblock` message, in order, find the corresponding transaction (from either the `blocktxn` message or from other sources)
+3. Place each short transaction ID in the first available position in the block
+4. Once the block has been reconstructed, it shall be processed as normal.
+
+**Short transaction IDs are expected to occasionally collide. Nodes must
+not be penalized for such collisions.**
+
+The structure of `BlockTransactions` is defined below.
+
+| Bytes | Name | Data Type | Encoding | Description|
+|----------|----------------------|----------------------|----------|------------|
+| 32 | blockhash | Binary blob | The output from a double-SHA256 of the block header, as used elsewhere | The blockhash of the block which the transactions being provided are in
+| 1 or 3 | transactions _length | CompactSize | As used to encode array lengths elsewhere | The number of transactions provided
+| *Varies* | transactions | List of transactions | As encoded in `tx` messages in response to `getdata MSG_TX` | The transactions provided
+
+
+
+{% endautocrossref %}
+
+#### CmpctBlock
+{% include helpers/subhead-links.md %}
+
+{% autocrossref %}
+
+*Added in protocol version 70209 of Dash Core as described by BIP152*
+
+The `cmpctblock` message is a reply to a `getdata` message which
+requested a block using the inventory type `MSG_CMPCT_BLOCK`. If the
+requested block was recently announced and is close to the tip of the
+best chain of the receiver and after having sent the requesting peer
+a `sendcmpct` message, nodes respond with a `cmpctblock` message containing
+data for the block.
+
+**If the requested block is too old, the node responds with a *full non-compact block***
+
+Upon receipt of a `cmpctblock` message, after sending a `sendcmpct` message,
+nodes should calculate the short transaction ID for each unconfirmed
+transaction they have available (i.e. in their mempool) and compare each
+to each short transaction ID in the `cmpctblock` message. After finding
+already-available transactions, nodes which do not have all transactions
+available to reconstruct the full block should request the missing transactions
+using a `getblocktxn` message.
+
+A node must not send a `cmpctblock` message unless they are able to respond to
+a `getblocktxn` message which requests every transaction in the block. A node
+must not send a `cmpctblock` message without having validated that the header properly
+commits to each transaction in the block, and properly builds on top of the existing,
+fully-validated chain with a valid proof-of-work either as a part of the current most-work
+valid chain, or building directly on top of it. A node may send a `cmpctblock` message before
+validating that each transaction in the block validly spends existing UTXO set entries.
+
+The `cmpctblock` message contains a vector of `PrefilledTransaction` whose
+structure is defined below. A `PrefilledTransaction` is used in `HeaderAndShortIDs`
+to provide a list of a few transactions explicitly.
+
+| Bytes | Name | Data Type | Encoding | Description|
+|----------|----------------------|----------------------|----------|------------|
+| 1 or 3 | index | CompactSize | Compact Size, differentially encoded since the last PrefilledTransaction in a list | The index into the block at which this transaction is
+| *Varies* | tx | Transaction | As encoded in `tx` messages sent in response to `getdata MSG_TX` | Transaction which is in the block at index `index`
+
+The `cmpctblock` message is compromised of a serialized `HeaderAndShortIDs`
+structure which is defined below. A `HeaderAndShortIDs` structure is used to
+relay a block header, the short transactions IDs used for matching
+already-available transactions, and a select few transactions which
+we expect a peer may be missing.
+
+| Bytes | Name | Data Type | Encoding | Description|
+|----------|----------------------|----------------------|----------|------------|
+| 80 | header | Block header | First 80 bytes of the block as defined by the encoding used by `block` messages | The header of the block being provided
+| 8 | nonce | uint64_t | Little Endian | A nonce for use in short transaction ID calculations
+| 1 or 3 | shortids_ length | CompactSize | As used to encode array lengths elsewhere | The number of short transaction IDs in `shortids` (i.e. block tx count - `prefilledtxn` `_length`)
+| *Varies* | shortids | List of 6-byte integers | Little Endian | The short transaction IDs calculated from the transactions which were not provided explicitly in `prefilledtxn`
+| 1 or 3 | prefilledtxn _length | CompactSize | As used to encode array lengths elsewhere | The number of prefilled transactions in `prefilledtxn` (i.e. block tx count - `shortids` `_length`)
+| *Varies* | prefilledtxn | List of Prefilled Transactions | As defined by `Prefilled` `Transaction` definition below | Used to provide the coinbase transaction and a select few which we expect a peer may be missing
+
+**Short Transaction ID calculation**
+
+Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. They are calculated as follows,
+
+* A single-SHA256 hashing the block header with the nonce appended (in little-endian)
+* Running SipHash-2-4 with the input being the transaction ID and the keys (k0/k1) set to the first two little-endian 64-bit integers from the above hash, respectively.
+* Dropping the 2 most significant bytes from the SipHash output to make it 6 bytes.
+
+
+The following annotated hexdump shows a `cmpctblock` message. (The
+message header has been omitted.)
+
+{% highlight text %}
+00000020981178a4342cec6316296b2ad84c9b7cdf9f
+2688e5d0fe1a0003cd0000000000f64870f52a3d0125
+1336c9464961216732b25fbf288a51f25a0e81bffb20
+e9600194d85a64a50d1cc02b0181 ................ Block Header
+
+3151b67e5b418b9d ............................ Nonce
+
+00 .......................................... Short IDs Length: 0
+............................................. Short IDs: None
+
+01 .......................................... Prefilled Transaction Length: 1
+
+Prefilled Transactions
+| 00 ........................................ Index: 0
+|
+| Transaction 1 (Coinbase)
+| | 01000000 ................................ Transaction Version: 1
+| | 01 ...................................... Input count: 1
+| |
+| | Transaction input #1
+| | |
+| | | 00000000000000000000000000000000
+| | | 00000000000000000000000000000000 ..... Outpoint TXID
+| | | ffffffff ............................. Outpoint index number: UINT32_MAX
+| | |
+| | | 13 ................................... Bytes in sig. script: 19
+| | | 03daaf010e2f5032506f6f6c2d74444153482f Secp256k1 signature
+| | |
+| | | ffffffff ............................. Sequence number: UINT32_MAX
+| |
+| | 04 ..................................... Number of outputs: 04
+| |
+| | Transaction output #1
+| | | ffe5654200000000 ..................... Duffs (11.13974271 Dash)
+| | |
+| | | 19 ................................... Bytes in pubkey script: 25
+| | | | 76 ................................. OP_DUP
+| | | | a9 ................................. OP_HASH160
+| | | | 14 ................................. Push 20 bytes as data
+| | | | | b885cb21ad12e593c1a46d814df47ccb
+| | | | | 450a7d84 ......................... PubKey hash
+| | | | 88 ................................. OP_EQUALVERIFY
+| | | | ac ................................. OP_CHECKSIG
+| |
+| | [...] .................................. 3 more tx outputs omitted
+| |
+| | 00000000 ............................... locktime: 0 (a block height)
+{% endhighlight %}
+
+{% endautocrossref %}
+
#### GetBlocks
{% include helpers/subhead-links.md %}
@@ -254,6 +415,44 @@ d39f608a7775b537729884d4e6633bb2
{% endautocrossref %}
+#### GetBlockTxn
+{% include helpers/subhead-links.md %}
+
+{% autocrossref %}
+
+*Added in protocol version 70209 of Dash Core as described by BIP152*
+
+The `getblocktxn` message requests a `blocktxn` message for any transactions
+that it has not seen after a compact block is received. It is defined as a
+message containing a serialized `BlockTransactionsRequest` message. Upon receipt
+of a properly-formatted `getblocktxn` message, nodes which recently provided the
+sender of such a message with a `cmpctblock` message for the block hash
+identified in this message must respond with either an appropriate
+`blocktxn` message, or a full block message.
+
+A `blocktxn` message response must contain exactly and only each transaction
+which is present in the appropriate block at the index specified in the
+`getblocktxn` message indexes list, in the order requested.
+
+The structure of `BlockTransactionsRequest` is defined below.
+
+| Bytes | Name | Data Type | Encoding | Description|
+|----------|-----------------|----------------------|----------|------|
+| 32 | blockhash | Binary blob | The output from a double-SHA256 of the block header, as used elsewhere | The blockhash of the block which the transactions being requested are in
+| *Varies* | indexes_length | CompactSize uint | As used to encode array lengths elsewhere | The number of transactions requested
+| *Varies* | indexes | CompactSize uint[] | Differentially encoded | Vector of compactSize containing the indexes of the transactions being requested in the block.
+
+
+
+{% endautocrossref %}
+
#### GetData
{% include helpers/subhead-links.md %}
@@ -266,7 +465,7 @@ node typically previously received by way of an `inv` message.
The response to a `getdata` message can be a `tx` message, `block`
message, `merkleblock` message, `ix` message, `txlvote` message,
`mnw` message, `mnb` message, `mnp` message, `dstx` message, `govobj` message,
-`govobjvote` message, `mnv` message, or `notfound` message.
+`govobjvote` message, `mnv` message, `notfound` message, or `cmpctblock` message.
This message cannot be used to request arbitrary data, such as historic
transactions no longer in the memory pool or relay set. Full nodes may
@@ -1289,6 +1488,57 @@ header has been omitted.)
{% endautocrossref %}
+#### SendCmpct
+{% include helpers/subhead-links.md %}
+
+{% autocrossref %}
+
+*Added in protocol version 70209 of Dash Core as described by BIP152*
+
+The `sendcmpct` message tells the receiving peer whether or not to announce new
+blocks using a `cmpctblock` message. It also sends the compact block protocol
+version it supports. The `sendcmpct` message is defined as a message containing
+a 1-byte integer followed by a 8-byte integer. The first integer is interpreted
+as a boolean and should have a value of either 1 or 0. The second integer
+is be interpreted as a little-endian version number.
+
+Upon receipt of a `sendcmpct` message with the first and second integers
+set to 1, the node should announce new blocks by sending a `cmpctblock` message.
+
+Upon receipt of a `sendcmpct` message with the first integer set to 0, the node
+shouldn't announce new blocks by sending a `cmpctblock` message, but instead announce
+new blocks by sending invs or headers, as defined by [BIP130][].
+
+Upon receipt of a `sendcmpct` message with the second integer set to something
+other than 1, nodes should treat the peer as if they had not received the message
+(as it indicates the peer will provide an unexpected encoding in `cmpctblock` messages,
+and/or other, messages). This allows future versions to send duplicate
+`sendcmpct` messages with different versions as a part of a version handshake.
+
+Nodes should check for a protocol version of >= 70209 before sending `sendcmpct`
+messages. Nodes shouldn't send a request for a `MSG_CMPCT_BLOCK` object to a peer
+before having received a `sendcmpct` message from that peer. Nodes shouldn't
+request a `MSG_CMPCT_BLOCK` object before having sent all `sendcmpct` messages
+to that peer which they intend to send, as the peer cannot know what protocol
+version to use in the response.
+
+The structure of a `sendcmpct` message is defined below.
+
+| Bytes | Name | Data Type | Description
+|----------|---------------|------------------|--------------
+| 1 | announce | bool | 0 - Announce blocks via `headers` message or `inv` message 1 - Announce blocks via `cmpctblock` message
+| 8 | version | uint64_t | The compact block protocol version number
+
+The annotated hexdump below shows a `sendcmpct` message. (The message
+header has been omitted.)
+
+{% highlight text %}
+01 ................................. Block announce type: Compact Blocks
+0100000000000000 ................... Compact block version: 1
+{% endhighlight %}
+
+{% endautocrossref %}
+
#### SendHeaders
{% include helpers/subhead-links.md %}
@@ -1302,7 +1552,6 @@ section][section message header] for an example of a message without a payload.
{% endautocrossref %}
-
#### Spork
{% include helpers/subhead-links.md %}
diff --git a/_includes/references.md b/_includes/references.md
index 10a9c515..e602cc5f 100644
--- a/_includes/references.md
+++ b/_includes/references.md
@@ -4,8 +4,11 @@ http://opensource.org/licenses/MIT.
{% endcomment %}
{% comment %}{% endcomment %}
+[blocktransactions]: /en/developer-reference#blocktxn "A P2P Networking data structure used to provide some of the transactions in a block as requested"
+[blocktransactionsrequest]: /en/developer-reference#getblocktxn "A P2P Networking data structure used to list transaction indexes in a block being requested by a peer"
[coinbase block height]: /en/developer-reference#term-coinbase-block-height "The current block's height encoded into the first bytes of the coinbase field"
[data-pushing opcode]: https://en.bitcoin.it/wiki/Script#Constants "Any opcode from 0x01 to 0x4e which pushes data on to the script evaluation stack"
+[headerandshortids]: /en/developer-reference#cmpctblock "A P2P Networking data structure used to relay a block header, the short transactions IDs used for matching already-available transactions, and a select few transactions which a peer may be missing"
[key index]: /en/developer-guide#term-key-index "An index number used in the HD wallet formula to generate child keys from a parent key"
[key pair]: /en/developer-guide#term-key-pair "A private key and its derived public key"
@@ -29,6 +32,7 @@ http://opensource.org/licenses/MIT.
[msg_governance_object]: /en/developer-reference#term-msg_governance_object "Governance object data type identifier of an inventory on the P2P network"
[msg_governance_object_vote]: /en/developer-reference#term-msg_governance_object_vote "Governance object vote data type identifier of an inventory on the P2P network"
[msg_masternode_verify]: /en/developer-reference#term-msg_masternode_verify "Masternode Verify data type identifier of an inventory on the P2P network"
+[msg_cmpct_block]: /en/developer-reference#term-msg_cmpct_block "An alternative to the block header hash data type identifier of an inventory on the P2P network used to request a compact block"
[network]: /en/developer-guide#term-network "The Dash P2P network which broadcasts transactions and blocks"
[op_checkmultisig]: /en/developer-reference#term-op-checkmultisig "Opcode which returns true if one or more provided signatures (m) sign the correct parts of a transaction and match one or more provided public keys (n)"
@@ -41,6 +45,7 @@ http://opensource.org/licenses/MIT.
[op_verify]: /en/developer-reference#term-op-verify "Operation which terminates the script if the entry below it on the stack is non-true (zero)"
[output index]: /en/developer-guide#term-output-index "The sequentially-numbered index of outputs in a single transaction starting from 0"
[point function]: /en/developer-guide#term-point-function "The ECDSA function used to create a public key from a private key"
+[prefilledtransaction]: /en/developer-reference#cmpctblock "A P2P Networking data structure used to represent a vector of a few transactions"
[previous block header hash]: /en/developer-reference#term-previous-block-header-hash "A field in the block header which contains the SHA256(SHA256()) hash of the previous block's header"
[proper money handling]: /en/developer-reference#term-proper-money-handling "Bitcoin amounts need to be correctly processed without introducing rounding errors that could cause monetary loss"
[standard block relay]: /en/developer-guide#term-standard-block-relay "The regular block relay method: announcing a block with an inv message and waiting for a response"
@@ -205,6 +210,8 @@ http://opensource.org/licenses/MIT.
[addr message]: /en/developer-reference#addr "The P2P network message which relays IP addresses and port numbers of active nodes to other nodes and clients, allowing decentralized peer discovery."
[alert message]: /en/developer-reference#alert "The P2P network message which sends alerts in case of major software problems."
[block message]: /en/developer-reference#block "The P2P network message which sends a serialized block"
+[blocktxn message]: /en/developer-reference#blocktxn "The P2P network message which sends a list of requested transactions from a compact block"
+[cmpctblock message]: /en/developer-reference#cmpctblock "The P2P network message which sends a serialized compact block"
[dsa message]: /en/developer-reference#dsa "A P2P network message used to join a mixing pool"
[dsc message]: /en/developer-reference#dsc "A P2P network message used to indicates a PrivateSend mixing session is complete"
[dsf message]: /en/developer-reference#dsf "A P2P network message sent as the final mixing transaction in a session asking users to sign the final mixing TX messages"
@@ -220,6 +227,7 @@ http://opensource.org/licenses/MIT.
[filterload message]: /en/developer-reference#filterclear "A P2P protocol message used to send a filter to a remote peer, requesting that they only send transactions which match the filter."
[getaddr message]: /en/developer-reference#getaddr "A P2P protool message used to request an addr message containing connection information for other nodes"
[getblocks message]: /en/developer-reference#getblocks "A P2P protocol message used to request an inv message containing a range of block header hashes"
+[getblocktxn message]: /en/developer-reference#getblocktxn "A P2P protocol message used to request transactions from a compact block"
[getdata message]: /en/developer-reference#getdata "A P2P protocol message used to request one or more transactions, blocks, or merkle blocks"
[getheaders message]: /en/developer-reference#getheaders "A P2P protocol message used to request a range of block headers"
[getsporks message]: /en/developer-reference#getsporks "A P2P network message used to request the status of sporks"
@@ -240,6 +248,7 @@ http://opensource.org/licenses/MIT.
[ping message]: /en/developer-reference#ping "A P2P network message used to see if the remote host is still connected"
[pong message]: /en/developer-reference#pong "A P2P network message used to reply to a P2P network ping message"
[reject message]: /en/developer-reference#reject "A P2P network message used to indicate a previously-received message was rejected for some reason"
+[sendcmpct message]: /en/developer-reference#sendcmpct "A P2P network message used to request new blocks be announced as compact blocks"
[sendheaders message]: /en/developer-reference#sendheaders "A P2P network message used to request new blocks be announced through headers messages rather than inv messages"
[spork message]: /en/developer-reference#spork "A P2P network message used to send the activation status of a spork"
[ssc message]: /en/developer-reference#ssc "A P2P network message used to track the sync status of masternode objects ()"
diff --git a/img/dev/en-p2p-control-messages.dot b/img/dev/en-p2p-control-messages.dot
index ddb35231..2c9e41c1 100644
--- a/img/dev/en-p2p-control-messages.dot
+++ b/img/dev/en-p2p-control-messages.dot
@@ -22,6 +22,8 @@ sendheaders;
ERROR [ style = "invis" ];
ERROR -> reject [ style = "invis" ];
+sendcmpct;
+
label = " \nOverview Of P2P Protocol Control And Advisory Messages"
}
diff --git a/img/dev/en-p2p-control-messages.png b/img/dev/en-p2p-control-messages.png
index f9b69d2c..8443f697 100644
Binary files a/img/dev/en-p2p-control-messages.png and b/img/dev/en-p2p-control-messages.png differ
diff --git a/img/dev/en-p2p-control-messages.svg b/img/dev/en-p2p-control-messages.svg
index eab70122..d603fbac 100644
--- a/img/dev/en-p2p-control-messages.svg
+++ b/img/dev/en-p2p-control-messages.svg
@@ -4,94 +4,99 @@
-