Mention Bitcoin's RPC Hash Byte Order

* Add a short subsection about the different byte orders used with
  hashes in Bitcoin Core and other software.

* Re-word some text in other sections to mention the byte order
  differences

This commit created based on comments from @SergioDemianLerner (thanks!)

_autocrossref.yaml

@@ -76,6 +76,7 @@ inputs: input
 input:
 intermediate certificate:
 intermediate certificates: intermediate certificate
+internal byte order:
 key index:
 key pair:
 '`label`': label
@@ -165,6 +166,7 @@ root certificate:
 root seed:
 RPCs: rpc
 RPC:
+RPC byte order:
 satoshi:
 satoshis: satoshi
 '`script`': pp script

_includes/ref_core_rpc_intro.md

@@ -0,0 +1,92 @@
+### Hash Byte Order
+
+{% autocrossref %}
+
+Bitcoin Core RPCs accept and return hashes in the reverse of their
+normal byte order. For example, the Unix `sha256sum` command would display the
+SHA256(SHA256()) hash of Mainnet block 300,000's header as the
+following string:
+
+    5472ac8b1187bfcf91d6d218bbda1eb2405d7c55f1f8cc820000000000000000
+
+The string above is also how the hash appears in the
+previous-header-hash part of block 300,001's header:
+
+<pre>02000000<b>5472ac8b1187bfcf91d6d218bbda1eb2405d7c55f1f8cc82000\
+0000000000000</b>ab0aaa377ca3f49b1545e2ae6b0667a08f42e72d8c24ae\
+237140e28f14f3bb7c6bcc6d536c890019edd83ccf</pre>
+
+However Bitcoin RPCs use the reverse byte order for hashes, so if you
+want to get information about block 300,000 using the `getblock` RPC,
+you need to reverse the byte order:
+
+    > bitcoin-cli getblock \
+      000000000000000082ccf8f1557c5d40b21edabb18d2d691cfbf87118bac7254
+
+(Note: hex representation uses two characters to display each byte of
+data, which is why the reversed string looks somewhat mangled.)
+
+The rational for the reversal is unknown, but it likely stems from
+Bitcoin's use of hash digests (which are byte arrays in C++) as integers
+for the purpose of determining whether the hash is below the network
+target. Whatever the reason for reversing header hashes, the reversal
+also extends to other hashes used in RPCs, such as TXIDs and merkle
+roots. 
+
+Off-site documentation such as the Bitcoin Wiki tends to use the terms
+big endian and little endian as shown in the table below, but they
+aren't always consistent. Worse, these two different ways of
+representing a hash digest can confuse anyone who looks at the Bitcoin
+Core source code and finds a so-called "big endian" value being stored
+in a little-endian data type.
+
+As header hashes and TXIDs are widely used as global identifiers in
+other Bitcoin software, this reversal of hashes has become the standard
+way to refer to certain objects. The table below should make clear where
+each byte order is used.
+
+<div style="text-align: left;" markdown="1">
+
+|---------------+---------------------|-----------------|
+| Data | Internal Byte Order ("Big Endian") | RPC Byte Order ("Little Endian") |
+|---------------|---------------------|-----------------|
+| Example: SHA256(SHA256(0x00))  | Hash: 1406...539a         | Hash: 9a53...0614     |
+|---------------|---------------------|-----------------|
+| Header Hashes: SHA256(SHA256(block header))  | Used when constructing block headers  | Used by RPCs such as `getblock`; widely used in block explorers |
+|---------------|---------------------|-----------------|
+| merkle Roots: SHA256(SHA256(TXIDs and merkle rows))  | Used when constructing block headers  | Returned by RPCs such as `getblock` |
+|---------------|---------------------|-----------------|
+| TXIDs: SHA256(SHA256(transaction))  | Used in transaction inputs | Used by RPCs such as `gettransaction` and transaction data parts of `getblock`; widely used in wallet programs |
+|---------------|---------------------|-----------------|
+| P2PKH Hashes: RIPEMD160(SHA256(pubkey))  | Used in both addresses and pubkey scripts  | **N/A:** RPCs use addresses which use internal byte order |
+|---------------|---------------------|-----------------|
+| P2SH Hashes: RIPEMD160(SHA256(redeem script))  | Used in both addresses and pubkey scripts | **N/A:** RPCs use addresses which use internal byte order |
+|---------------|---------------------|-----------------|
+
+</div>
+
+Note: RPCs which return raw results, such as `getrawtransaction` or the
+raw mode of `getblock`, always display hashes as they appear in blocks
+(internal byte order).
+
+The code below may help you check byte order by generating hashes
+from raw hex.
+{% endautocrossref %}
+
+{% highlight python %}
+#!/usr/bin/env python
+
+from sys import byteorder
+from hashlib import sha256
+
+## You can put in $data an 80-byte block header to get its header hash,
+## or a raw transaction to get its txid
+data = "00".decode("hex")
+hash = sha256(sha256(data).digest()).digest()
+
+print "Warning: this code only tested on a little-endian x86_64 arch"
+print
+print "System byte order:", byteorder
+print "Internal-Byte-Order Hash: ", hash.encode('hex_codec')
+print "RPC-Byte-Order Hash:      ", hash[::-1].encode('hex_codec')
+{% endhighlight %}

_includes/ref_core_rpcs-abcdefg.md

@@ -1542,8 +1542,7 @@ An array of *transactions* in [transaction object format][]{:#term-transaction-o
 {% autocrossref %}
 
 Each object in the array contains the
-rawtransaction *data* in hex and the *hash* of the data in little-endian
-hex.  
+rawtransaction *data* in hex and the *hash* of the data in RPC byte order.
 
 {% endautocrossref %}
 

_includes/ref_transactions.md

@@ -86,7 +86,7 @@ bytes commonly used by Bitcoin are:
 
 2. Create a copy of the version and hash; then hash that twice with SHA256: `SHA256(SHA256(version . hash))`
 
-3. Extract the four most significant bytes from the double-hashed copy.
+3. Extract the first four bytes from the double-hashed copy.
    These are used as a checksum to ensure the base hash gets transmitted
    correctly.
 
@@ -135,6 +135,10 @@ Bitcoin transactions are broadcast between peers and stored in the
 block chain in a serialized byte format, called [raw format][]{:#term-raw-format}{:.term}. Bitcoin Core
 and many other tools print and accept raw transactions encoded as hex.
 
+The binary form of a raw transaction is SHA256(SHA256()) hashed to create
+its TXID.  Bitcoin Core RPCs use a reversed byte order for hashes; see the [subsection about hash byte
+order][section hash byte order] for details.
+
 A sample raw transaction is the first non-coinbase transaction, made in
 [block 170][block170].  To get the transaction, use the `getrawtransaction` RPC with
 that transaction's txid (provided below):
@@ -142,7 +146,7 @@ that transaction's txid (provided below):
 {% endautocrossref %}
 
 ~~~
-> getrawtransaction \
+> bitcoin-cli getrawtransaction \
   f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16
 
 0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423e\

_includes/references.md

@@ -46,6 +46,7 @@
 [input]: /en/developer-guide#term-input "The input to a transaction linking to the output of a previous transaction which permits spending of satoshis"
 [inputs]: /en/developer-guide#term-input "The input to a transaction linking to the output of a previous transaction which permits spending of satoshis"
 [intermediate certificate]: /en/developer-examples#term-intermediate-certificate "A intermediate certificate authority certificate which helps connect a leaf (receiver) certificate to a root certificate authority"
+[internal byte order]: /en/developer-reference#hash-byte-order "The standard order in which hash digests are displayed as strings---the same format used in blocks"
 [key index]: /en/developer-guide#term-key-index "An index number used in the HD wallet formula to generate child keys from a parent key" 
 [key pair]: /en/developer-guide#term-key-pair "A private key and its derived public key"
 [label]: /en/developer-guide#term-label "The label parameter of a bitcoin: URI which provides the spender with the receiver's name (unauthenticated)" 
@@ -118,6 +119,7 @@
 [regression test mode]: /en/developer-examples#regtest-mode "A local testing environment in which developers can control blocks"
 [root certificate]: /en/developer-examples#term-root-certificate "A certificate belonging to a certificate authority (CA)"
 [root seed]: /en/developer-guide#term-root-seed "A potentially-short value used as a seed to generate a master private key and master chain code for an HD wallet"
+[RPC byte order]: /en/developer-reference#hash-byte-order "A hash digest displayed with the byte order reversed; used in Bitcoin Core RPCs and other software."
 [satoshi]: /en/developer-guide#term-satoshi "The smallest unit of Bitcoin value; 0.00000001 bitcoins.  Also used generically for any value of bitcoins"
 [satoshis]: /en/developer-guide#term-satoshi "The smallest unit of Bitcoin value; 0.00000001 bitcoins.  Also used generically for any value of bitcoins"
 [sequence number]: /en/developer-guide#term-sequence-number "A number intended to allow time locked transactions to be updated before being finalized; not currently used except to disable locktime in a transaction"
@@ -283,6 +285,7 @@
 [RPCs]: /en/developer-reference#remote-procedure-calls-rpcs
 <!-- [secp256k1]: http://www.secg.org/index.php?action=secg,docs_secg -->
 [secp256k1]: http://perso.univ-rennes1.fr/sylvain.duquesne/master/standards/sec2_final.pdf
+[section hash byte order]: /en/developer-reference#hash-byte-order
 [section verifying payment]: /en/developer-guide#verifying-payment
 [bitcoin URI subsection]: /en/developer-guide#bitcoin-uri
 [SHA256]: https://en.wikipedia.org/wiki/SHA-2

en/developer-reference.md

@@ -37,6 +37,8 @@ title: "Developer Reference - Bitcoin"
 -- * https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)
 -->
 
+{% include ref_core_rpc_intro.md %}
+
 ### Remote Procedure Calls (RPCs)
 
 **Warning:** the block chain and memory pool can include arbitrary data