Merge Pull #657: Dev Docs: Describe Filterload Message (Final P2P Message To Document)

2025-07-27 01:36:13 +00:00 · 2014-11-28 19:30:20 -05:00 · 2014-11-28 19:30:20 -05:00 · 464272eeb0
commit 464272eeb0
parent c7798bf086 b3c14aad89
8 changed files with 714 additions and 18 deletions
--- a/_autocrossref.yaml
+++ b/_autocrossref.yaml
@ -66,6 +66,8 @@ confirmations:
 confirmed transactions:
 consensus:
 consensus rules:
+data-pushing op code:
+data-pushing op codes: data-pushing op code
 denomination:
 denominations: denomination
 DER format: der
@ -90,6 +92,8 @@ fiat:
 '`filteradd` messages': filteradd message
 '`filterclear` message': filterclear message
 '`filterclear` messages': filterclear message
+'`filterload` message': filterload message
+'`filterload` messages': filterload message
 fork:
 forks: fork
 genesis block:
@ -144,6 +148,7 @@ message headers: message header
 message payload:
 '`merchant_data`': pp merchant data
 merkle block:
+merkle blocks: merkle block
 '`merkleblock` message': merkleblock message
 '`merkleblock` messages': merkleblock message
 merkle root:
@ -178,6 +183,8 @@ op codes: op code
 '`op_hash160`': op_hash160
 '`op_return`': op_return
 '`op_verify`': op_verify
+outpoint:
+outpoints: outpoint
 outputs: output
 output:
 output index:
--- a/_includes/example_p2p_networking.md
+++ b/_includes/example_p2p_networking.md
@ -5,6 +5,234 @@ http://opensource.org/licenses/MIT.

 ## P2P Network

+### Creating A Bloom Filter
+
+{% autocrossref %}
+
+In this section, we'll use variable names that correspond to the field
+names in the [`filterload` message documentation][filterload message].
+Each code block precedes the paragraph describing it.
+
+{% highlight python %}
+#!/usr/bin/env python
+
+BYTES_MAX = 36000
+FUNCS_MAX = 50
+
+nFlags = 0
+{% endhighlight %}
+
+We start by setting some maximum values defined in BIP37: the maximum
+number of bytes allowed in a filter and the maximum number of hash
+functions used to hash each piece of data.  We also set nFlags to zero,
+indicating we don't want the remote node to update the filter for us.
+(We won't use nFlags again in the sample program, but real programs will
+need to use it.)
+
+{% highlight python %}
+n = 1
+p = 0.0001
+{% endhighlight %}
+
+We define the number (n) of elements we plan to insert into the filter
+and the false positive rate (p) we want to help protect our privacy. For
+this example, we will set *n* to one element and *p* to a rate of
+1-in-10,000 to produce a small and precise filter for illustration
+purposes. In actual use, your filters will probably be much larger.
+
+{% highlight python %}
+from math import log
+nFilterBytes = int(min((-1 / log(2)**2 * n * log(p)) / 8, BYTES_MAX))
+nHashFuncs = int(min(nFilterBytes * 8 / n * log(2), FUNCS_MAX))
+
+from bitarray import bitarray  # from pypi.python.org/pypi/bitarray
+vData = nFilterBytes * 8 * bitarray('0', endian="little")
+{% endhighlight %}
+
+Using the formula described in BIP37, we calculate the ideal size of the
+filter (in bytes) and the ideal number of hash functions to use. Both
+are truncated down to the nearest whole number and both are also
+constrained to the maximum values we defined earlier. The results of
+this particular fixed computation are 2 filter bytes and 11 hash
+functions. We then use *nFilterBytes* to create a little-endian bit
+array of the appropriate size.
+
+{% highlight python %}
+nTweak = 0
+{% endhighlight %}
+
+We also should choose a value for *nTweak*.  In this case, we'll simply
+use zero.
+
+{% highlight python %}
+import pyhash  # from https://github.com/flier/pyfasthash
+murmur3 = pyhash.murmur3_32()
+
+def bloom_hash(nHashNum, data):
+    seed = (nHashNum * 0xfba4c795 + nTweak) & 0xffffffff
+    return( murmur3(data, seed=seed) % (nFilterBytes * 8) )
+{% endhighlight %}
+
+We setup our hash function template using the formula and 0xfba4c795
+constant set in BIP37. Note that we limit the size of the seed to four
+bytes and that we're returning the result of the hash modulo the size of
+the filter in bits.
+
+{% highlight python %}
+data_to_hash = "019f5b01d4195ecbc9398fbf3c3b1fa9" \
+               + "bb3183301d7a1fb3bd174fcfa40a2b65"
+data_to_hash = data_to_hash.decode("hex")
+{% endhighlight %}
+
+For the data to add to the filter, we're adding a TXID. Note that the
+TXID is in internal byte order.
+
+{% highlight python %}
+print "                             Filter (As Bits)"
+print "nHashNum   nIndex   Filter   0123456789abcdef"
+print "~~~~~~~~   ~~~~~~   ~~~~~~   ~~~~~~~~~~~~~~~~"
+for nHashNum in range(nHashFuncs):
+    nIndex = bloom_hash(nHashNum, data_to_hash)
+
+    ## Set the bit at nIndex to 1
+    vData[nIndex] = True
+
+    ## Debug: print current state
+    print '      {0:2}      {1:2}     {2}   {3}'.format(
+        nHashNum,
+        hex(int(nIndex)),
+        vData.tobytes().encode("hex"),
+        vData.to01()
+    )
+
+print
+print "Bloom filter:", vData.tobytes().encode("hex")
+{% endhighlight %}
+
+Now we use the hash function template to run a slightly different hash
+function for *nHashFuncs* times. The result of each function being run
+on the transaction is used as an index number: the bit at that index is
+set to 1. We can see this in the printed debugging output:
+
+{% highlight text %}
+                             Filter (As Bits)
+nHashNum   nIndex   Filter   0123456789abcdef
+~~~~~~~~   ~~~~~~   ~~~~~~   ~~~~~~~~~~~~~~~~
+       0      0x7     8000   0000000100000000
+       1      0x9     8002   0000000101000000
+       2      0xa     8006   0000000101100000
+       3      0x2     8406   0010000101100000
+       4      0xb     840e   0010000101110000
+       5      0x5     a40e   0010010101110000
+       6      0x0     a50e   1010010101110000
+       7      0x8     a50f   1010010111110000
+       8      0x5     a50f   1010010111110000
+       9      0x8     a50f   1010010111110000
+      10      0x4     b50f   1010110111110000
+
+Bloom filter: b50f
+{% endhighlight %}
+
+Notice that in iterations 8 and 9, the filter did not change because the
+corresponding bit was already set in a previous iteration (5 and 7,
+respectively).  This is a normal part of bloom filter operation.
+
+We only added one element to the filter above, but we could repeat the
+process with additional elements and continue to add them to the same
+filter. (To maintain the same false-positive rate, you would need a
+larger filter size as computed earlier.)
+
+Note: for a more optimized Python implementation with fewer external
+dependencies, see [python-bitcoinlib's][python-bitcoinlib] bloom filter
+module which is based directly on Bitcoin Core's C++ implementation.
+
+Using the `filterload` message format, the complete filter created above 
+would be the binary form of the annotated hexdump shown below:
+
+{% highlight text %}
+02 ......... Filter bytes: 2
+b50f ....... Filter: 1010 1101 1111 0000
+0b000000 ... nHashFuncs: 11
+00000000 ... nTweak: 0/none
+00 ......... nFlags: BLOOM_UPDATE_NONE
+{% endhighlight %}
+
+{% endautocrossref %}
+
+### Evaluating A Bloom Filter
+
+{% autocrossref %}
+
+Using a bloom filter to find matching data is nearly identical to
+constructing a bloom filter---except that at each step we check to see
+if the calculated index bit is set in the existing filter.
+
+{% highlight python %}
+vData = bitarray(endian='little')
+vData.frombytes("b50f".decode("hex"))
+nHashFuncs = 11
+nTweak = 0
+nFlags = 0
+{% endhighlight %}
+
+Using the bloom filter created above, we import its various parameters.
+Note, as indicated in the section above, we won't actually use *nFlags*
+to update the filter.
+
+{% highlight python %}
+def contains(nHashFuncs, data_to_hash):
+    for nHashNum in range(nHashFuncs):
+        ## bloom_hash as defined in previous section
+        nIndex = bloom_hash(nHashNum, data_to_hash)
+
+        if vData[nIndex] != True:
+            print "MATCH FAILURE: Index {0} not set in {1}".format(
+                hex(int(nIndex)),
+                vData.to01()
+            )
+            return False
+{% endhighlight %}
+
+We define a function to check an element against the provided filter.
+When checking whether the filter might contain an element, we test to
+see whether a particular bit in the filter is already set to 1 (if it
+isn't, the match fails).
+
+{% highlight python %}
+## Test 1: Same TXID as previously added to filter
+data_to_hash = "019f5b01d4195ecbc9398fbf3c3b1fa9" \
+               + "bb3183301d7a1fb3bd174fcfa40a2b65"
+data_to_hash = data_to_hash.decode("hex")
+contains(nHashFuncs, data_to_hash)
+{% endhighlight %}
+
+Testing the filter against the data element we previously added, we get
+no output (indicating a possible match).  Recall that bloom filters have
+a zero false negative rate---so they should always match the inserted
+elements.
+
+{% highlight python %}
+## Test 2: Arbitrary string
+data_to_hash = "1/10,000 chance this ASCII string will match"
+contains(nHashFuncs, data_to_hash)
+{% endhighlight %}
+
+Testing the filter against an arbitrary element, we get the failure
+output below.  Note: we created the filter with a 1-in-10,000 false
+positive rate (which was rounded up somewhat when we truncated), so it
+was possible this arbitrary string would've matched the filter anyway.
+It is not possible to set a bloom filter to a false positive rate of
+zero, so your program will always have to deal with false positives.
+The output below shows us that one of the hash functions returned an
+index number of 0x06, but that bit wasn't set in the filter, causing the
+match failure:
+
+{% highlight text %}
+MATCH FAILURE: Index 0x6 not set in 1010110111110000
+{% endhighlight %}
+
+{% endautocrossref %}
+
 ### Retrieving A MerkleBlock

 {% autocrossref %}
@ -82,12 +310,17 @@ script, but we will sleep a short bit and send back our own `verack`
 message as if we had accepted their `version` message.

 {% highlight python %}
-send("filterload", "02b50f0b0000000000000000")
+send("filterload", 
+    "02"  ........ Filter bytes: 2
+    "b50f" ....... Filter: 1010 1101 1111 0000
+    "0b000000" ... nHashFuncs: 11
+    "00000000" ... nTweak: 0/none
+    "00" ......... nFlags: BLOOM_UPDATE_NONE
+)
 {% endhighlight %}

-We set a bloom filter with the `filterload` message. This filter was
-quickly created using [python-bitcoinlib][]'s bloom module. <!-- TODO:
-consider expanding this section once filterload has been documented. -->
+We set a bloom filter with the `filterload` message. This filter is
+described in the two preceeding sections.

 {% highlight python %}
 send("getdata",
--- a/_includes/ref_p2p_networking.md
+++ b/_includes/ref_p2p_networking.md
@ -699,7 +699,9 @@ introduced in protocol version 311.
 The annotated hexdump below shows an `alert` message. (The message
 header has been omitted.)

-<!-- example below from Bitcoin Wiki; TODO: replace with a more recent
+<!-- example below from Bitcoin Wiki but it's a network capture so I
+(@harding) don't think it is subject to the wiki's copyright license; I
+think it's public domain. TODO: replace with a more recent
 alert the next time one is live on the network. -->

 {% highlight text %}
@ -751,21 +753,24 @@ alert.cpp] source code for the parameters of this message.

 *Added in protocol version 70001 as described by BIP37.*

-The `filteradd` message tells the receiving peer to add a single object to
-a previously-set bloom filter, such as a new public key. The object is
+The `filteradd` message tells the receiving peer to add a single element to
+a previously-set bloom filter, such as a new public key. The element is
 sent directly to the receiving peer; the peer then uses the parameters set
-in the `filterload` message to add the object to the bloom filter.
+in the `filterload` message to add the element to the bloom filter.

-Because the object is sent directly to the receiving peer, there is no
-obfuscation of the object and none of the plausible-deniability privacy
+Because the element is sent directly to the receiving peer, there is no
+obfuscation of the element and none of the plausible-deniability privacy
 provided by the bloom filter. Clients that want to maintain greater
 privacy should recalculate the bloom filter themselves and send a new
 `filterload` message with the recalculated bloom filter.

-| Bytes    | Name         | Data Type        | Description
-|----------|--------------|------------------|-----------------
-| *Varies* | object bytes | compactSize uint | The number of bytes in the following object field.
-| *Varies* | object       | uint8_t[]        | The object to add to the current filter.  Maximum of 520 bytes, which is the maximum size of an object which can be pushed onto the stack in a pubkey or signature script.  Objects must be sent in the byte order they would use when appearing in a raw transaction; for example, hashes should be sent in internal byte order.
+| Bytes    | Name          | Data Type        | Description
+|----------|---------------|------------------|-----------------
+| *Varies* | element bytes | compactSize uint | The number of bytes in the following element field.
+| *Varies* | element       | uint8_t[]        | The element to add to the current filter.  Maximum of 520 bytes, which is the maximum size of an element which can be pushed onto the stack in a pubkey or signature script.  Elements must be sent in the byte order they would use when appearing in a raw transaction; for example, hashes should be sent in internal byte order.
+
+Note: a `filteradd` message will not be accepted unless a filter was
+previously set with the `filterload` message.

 The annotated hexdump below shows a `filteradd` message adding a TXID.
 (The message header has been omitted.) This TXID appears in the same
@ -774,9 +779,9 @@ block used for the example hexdump in the `merkleblock` message; if that
 six hashes are returned instead of four.

 {% highlight text %}
-20 ................................. Object bytes: 32
+20 ................................. Element bytes: 32
 fdacf9b3eb077412e7a968d2e4f11b9a
-9dee312d666187ed77ee7d26af16cb0b ... Object (A TXID)
+9dee312d666187ed77ee7d26af16cb0b ... Element (A TXID)
 {% endhighlight %}

 {% endautocrossref %}
@ -802,9 +807,242 @@ section][message header] for an example of a message without a payload.
 {% endautocrossref %}


-<!-- TODO: filterload message -->
+#### FilterLoad

+{% autocrossref %}

+*Added in protocol version 70001 as described by BIP37.*
+
+The `filterload` message tells the receiving peer to filter all relayed
+transactions and requested merkle blocks through the provided filter.
+This allows clients to receive transactions relevant to their wallet
+plus a configurable rate of false positive transactions which can
+provide plausible-deniability privacy.
+
+| Bytes    | Name         | Data Type | Description
+|----------|--------------|-----------|---------------
+| *Varies* | nFilterBytes | uint8_t[] | Number of bytes in the following filter bit field.
+| *Varies* | filter       | uint8_t[] | A bit field of arbitrary byte-aligned size. The maximum size is 36,000 bytes.
+| 4        | nHashFuncs   | uint32_t  | The number of hash functions to use in this filter. The maximum value allowed in this field is 50.
+| 4        | nTweak       | uint32_t  | An arbitrary value to add to the seed value in the hash function used by the bloom filter.
+| 1        | nFlags       | uint8_t   | A set of flags that control how outpoints corresponding to a matched pubkey script are added to the filter. See the table in the Updating A Bloom Filter subsection below.
+
+The annotated hexdump below shows a `filterload` message. (The message
+header has been omitted.)  For an example of how this payload was
+created, see the [filterload example][section creating a bloom filter].
+
+{% highlight text %}
+02 ......... Filter bytes: 2
+b50f ....... Filter: 1010 1101 1111 0000
+0b000000 ... nHashFuncs: 11
+00000000 ... nTweak: 0/none
+00 ......... nFlags: BLOOM_UPDATE_NONE
+{% endhighlight %}
+
+**Initializing A Bloom Filter**
+
+Filters have two core parameters: the size of the bit field and the
+number of hash functions to run against each data element. The following
+formulas from BIP37 will allow you to automatically select appropriate
+values based on the number of elements you plan to insert into the
+filter (*n*) and the false positive rate (*p*) you desire to maintain
+plausible deniability.
+
+* Size of the bit field in bytes (*nFilterBytes*), up to a maximum of
+  36,000: `(-1 / log(2)**2 * n * log(p)) / 8`
+
+* Hash functions to use (*nHashFuncs*), up to a maximum of 50:
+  `nFilterBytes * 8 / n * log(2)`
+
+Note that the filter matches parts of transactions (transaction
+elements), so the false positive rate is relative to the number of
+elements checked---not the number of transactions checked. Each normal
+transaction has a minimum of four matchable elements (described in the
+comparison subsection below), so a filter with a false-positive rate of
+1 percent will match about 4 percent of all transactions at a minimum.
+
+According to BIP37, the formulas and limits described above provide
+support for bloom filters containing 20,000 items with a false positive
+rate of less than 0.1 percent or 10,000 items with a false positive rate
+of less than 0.0001 percent.
+
+Once the size of the bit field is known, the bit field should be
+initialized as all zeroes.
+
+**Populating A Bloom Filter**
+
+The bloom filter is populated using between 1 and 50 unique hash
+functions (the number specified per filter by the *nHashFuncs*
+field). Instead of using up to 50 different hash function
+implementations, a single implementation is used with a unique seed
+value for each function.
+
+The seed is `nHashNum * 0xfba4c795 + nTweak` as a *uint32\_t*, where the values
+are:
+
+* **nHashNum** is the sequence number<!--noref--> for this hash
+  function, starting at 0 for the first hash iteration and increasing up
+  to the value of the *nHashFuncs* field (minus one) for the last hash
+  iteration.
+
+* **0xfba4c795** is a constant optimized to create large differences in
+  the seed for different values of *nHashNum*. 
+
+* **nTweak** is a per-filter constant set by the client to require the use
+  of an arbitrary set of hash functions.
+
+If the seed resulting from the formula above is larger than four bytes,
+it must be truncated to its four most significant bytes (for example,
+`0x8967452301 & 0xffffffff → 0x67452301`).
+
+The actual hash function implementation used is the [32-bit Murmur3 hash
+function][murmur3].
+
+![Warning icon](/img/icon_warning.svg)
+**Warning:** the Murmur3 hash function has separate 32-bit and 64-bit
+versions that produce different results for the same input.  Only the
+32-bit Murmur3 version is used with Bitcoin bloom filters.
+
+The data to be hashed can be any transaction element which the bloom
+filter can match. See the next subsection for the list of transaction
+elements checked against the filter. The largest element which can be
+matched is a script data push of 520 bytes, so the data should never
+exceed 520 bytes.
+
+The example below from Bitcoin Core [bloom.cpp][core bloom.cpp hash] combines
+all the steps above to create the hash function template. The seed is
+the first parameter; the data to be hashed is the second parameter. The
+result is a uint32\_t modulo the size of the bit field in bits.
+
+{% highlight c++ %}
+MurmurHash3(nHashNum * 0xFBA4C795 + nTweak, vDataToHash) % (vData.size() * 8)
+{% endhighlight %}
+
+Each data element to be added to the filter is hashed by *nHashFuncs*
+number of hash functions. Each time a hash function is run, the result
+will be the index number (*nIndex*) of a bit in the bit field. That bit
+must be set to 1. For example if the filter bit field was `00000000` and
+the result is 5, the revised filter bit field is `00000100` (the first bit
+is bit 0).
+
+It is expected that sometimes the same index number will be returned
+more than once when populating the bit field; this does not affect the
+algorithm---after a bit is set to 1, it is never changed back to 0.
+
+After all data elements have been added to the filter, each set of eight
+bits is converted into a little-endian byte. These bytes are the value
+of the *filter* field.
+
+**Comparing Transaction Elements To A Bloom Filter**
+
+To compare an arbitrary data element against the bloom filter, it is
+hashed using the same parameters used to create the bloom filter.
+Specifically, it is hashed *nHashFuncs* times, each time using the same
+*nTweak* provided in the filter, and the resulting output is modulo the
+size of the bit field provided in the *filter* field.  After each hash is
+performed, the filter is checked to see if the bit at that indexed
+location is set.  For example if the result of a hash is `5` and the
+filter is `01001110`, the bit is considered set.
+
+If the result of every hash points to a set bit, the filter matches. If
+any of the results points to an unset bit, the filter does not match.
+
+The following transaction elements are compared against bloom filters.
+All elements will be hashed in the byte order used in blocks (for
+example, TXIDs will be in internal byte order).
+
+* **TXIDs:** the transaction's SHA256(SHA256()) hash.
+
+* **Outpoints:** each 36-byte outpoint used this transaction's input
+  section is individually compared to the filter.
+
+* **Signature Script Data:** each element pushed onto the stack by a
+  data-pushing op code in a signature script from this transaction is
+  individually compared to the filter.  This includes data elements
+  present in P2SH redeem scripts when they are being spent.
+
+* **PubKey Script Data:** each element pushed onto the the stack by a
+  data-pushing op code in any pubkey script from this transaction is
+  individually compared to the filter. (If a pubkey script element
+  matches the filter, the filter will be immediately updated if the
+  `BLOOM_UPDATE_ALL` flag was set; if the pubkey script is in the P2PKH
+  format and matches the filter, the filter will be immediately updated
+  if the `BLOOM_UPDATE_P2PUBKEY_ONLY` flag was set. See the subsection
+  below for details.)
+
+The following annotated hexdump of a transaction is from the [raw
+transaction format section][raw transaction format]; the elements which
+would be checked by the filter are emphasized in bold. Note that this
+transaction's TXID (**`01000000017b1eab[...]`**) would also be checked,
+and that the outpoint TXID and index number below would be checked as a
+single 36-byte element.
+
+<pre><code>01000000 ................................... Version
+
+01 ......................................... Number of inputs
+|
+| <b>7b1eabe0209b1fe794124575ef807057</b>
+| <b>c77ada2138ae4fa8d6c4de0398a14f3f</b> ......... Outpoint TXID
+| <b>00000000</b> ................................. Outpoint index number
+|
+| 49 ....................................... Bytes in sig. script: 73
+| | 48 ..................................... Push 72 bytes as data
+| | | <b>30450221008949f0cb400094ad2b5eb3</b>
+| | | <b>99d59d01c14d73d8fe6e96df1a7150de</b>
+| | | <b>b388ab8935022079656090d7f6bac4c9</b>
+| | | <b>a94e0aad311a4268e082a725f8aeae05</b>
+| | | <b>73fb12ff866a5f01</b> ..................... Secp256k1 signature
+|
+| ffffffff ................................. Sequence number: UINT32_MAX
+
+01 ......................................... Number of outputs
+| f0ca052a01000000 ......................... Satoshis (49.99990000 BTC)
+|
+| 19 ....................................... Bytes in pubkey script: 25
+| | 76 ..................................... OP_DUP
+| | a9 ..................................... OP_HASH160
+| | 14 ..................................... Push 20 bytes as data
+| | | <b>cbc20a7664f2f69e5355aa427045bc15</b>
+| | | <b>e7c6c772</b> ............................. PubKey hash
+| | 88 ..................................... OP_EQUALVERIFY
+| | ac ..................................... OP_CHECKSIG
+
+00000000 ................................... locktime: 0 (a block height)
+</code></pre>
+
+**Updating A Bloom Filter**
+
+Clients will often want to track inputs that spend outputs (outpoints)
+relevant to their wallet, so the filterload field *nFlags* can be set to
+allow the filtering node to update the filter when a match is found.
+When the filtering node sees a pubkey script that pays a pubkey,
+address, or other data element matching the filter, the filtering node
+immediately updates the filter with the outpoint corresponding to that
+pubkey script.
+
+![Automatically Updating Bloom Filters](/img/dev/en-bloom-update.svg)
+
+If an input later spends that outpoint, the filter will match it,
+allowing the filtering node to tell the client that one of its
+transaction outputs has been spent.
+
+The *nFlags* field has three allowed values:
+
+| Value | Name                       | Description
+|-------|----------------------------|---------------
+| 0     | BLOOM_UPDATE_NONE          | The filtering node should not update the filter.
+| 1     | BLOOM_UPDATE_ALL           | If the filter matches any data element in a pubkey script, the corresponding outpoint is added to the filter. 
+| 2     | BLOOM_UPDATE_P2PUBKEY_ONLY | If the filter matches any data element in a pubkey script and that script is either a P2PKH or non-P2SH pay-to-multisig script, the corresponding outpoint is added to the filter.
+
+In addition, because the filter size stays the same even though
+additional elements are being added to it, the false positive rate
+increases. Each false positive can result in another element being added
+to the filter, creating a feedback loop that can (after a certain point)
+make the filter useless. For this reason, clients using automatic filter
+updates need to monitor the actual false positive rate and send a new
+filter when the rate gets too high.
+
+{% endautocrossref %}

 #### GetAddr

--- a/_includes/ref_transactions.md
+++ b/_includes/ref_transactions.md
@ -244,7 +244,7 @@ Each non-coinbase input spends an outpoint from a previous transaction.

 {% endautocrossref %}

-**Outpoint: The Specific Part Of A Specific Output**
+**[Outpoint][]{:#term-outpoint}{:.term}: The Specific Part Of A Specific Output**

 {% autocrossref %}

--- a/_includes/references.md
+++ b/_includes/references.md
@ -42,6 +42,7 @@ http://opensource.org/licenses/MIT.
 [confirmations]: /en/developer-guide#term-confirmation "The number of blocks which would need to be modified to remove or modify a transaction"
 [consensus]: /en/developer-guide#term-consensus "When several nodes (usually most nodes on the network) all have the same blocks in their locally-validated block chain."
 [consensus rules]: /en/developer-guide#term-consensus-rules "The block validation rules that full nodes follow to stay in consensus with other nodes."
+[data-pushing op code]: https://en.bitcoin.it/wiki/Script#Constants "Any op code from 0x01 to 0x4e which pushes data on to the script evaluation stack"
 [denomination]: /en/developer-guide#term-denomination "bitcoins (BTC), bitcents (cBTC), millibitcoins (mBTC), bits (uBTC, microbitcoins), or satoshis"
 [difficulty]: /en/developer-guide#term-difficulty "A number corresponding to the target threshold which indicates how difficult it will be to find the next block"
 [dns seed]: /en/developer-guide#term-dns-seed "A DNS server which returns IP addresses of full nodes on the Bitcoin network to assist in peer discovery."
@ -53,6 +54,7 @@ http://opensource.org/licenses/MIT.
 [fiat]: /en/developer-guide#term-fiat "National currencies such as the dollar or euro"
 [filteradd message]: /en/developer-reference#filteradd "A P2P protocol message used to add a data element to an existing bloom filter."
 [filterclear message]: /en/developer-reference#filterclear "A P2P protocol message used to remove an existing bloom filter."
+[filterload message]: /en/developer-reference#filterclear "A P2P protocol message used to send a filter to a remote peer, requesting that they only send transactions which match the filter."
 [fork]: /en/developer-guide#term-fork "When two or more blocks have the same block height, forking the block chain."
 [genesis block]: /en/developer-guide#term-genesis-block "The first block created; also called block 0"
 [getaddr message]: /en/developer-reference#getaddr "A P2P protool message used to request an addr message containing connection information for other nodes"
@ -110,6 +112,7 @@ http://opensource.org/licenses/MIT.
 [op_hash160]: /en/developer-reference#term-op-hash160 "Operation which converts the entry below it on the stack into a RIPEMD(SHA256()) hashed version of itself"
 [op_return]: /en/developer-reference#term-op-return "Operation which terminates the script in failure"
 [op_verify]: /en/developer-reference#term-op-verify "Operation which terminates the script if the entry below it on the stack is non-true (zero)"
+[outpoint]: /en/developer-reference#term-outpoint "The structure used to refer to a particular transaction output, consisting of a 32-byte TXID and a 4-byte output index number (vout)."
 [output]: /en/developer-guide#term-output "The output of a transaction which transfers value to a pubkey script"
 [output index]: /en/developer-guide#term-output-index "The sequentially-numbered index of outputs in a single transaction starting from 0"
 [P2PKH]: /en/developer-guide#term-p2pkh "A pubkey script which Pays To PubKey Hashes (P2PKH), allowing spending of satoshis to anyone with a Bitcoin address"
@ -300,6 +303,7 @@ http://opensource.org/licenses/MIT.
 [raw transaction format]: /en/developer-reference#raw-transaction-format
 [RPC]: /en/developer-reference#remote-procedure-calls-rpcs
 [RPCs]: /en/developer-reference#remote-procedure-calls-rpcs
+[section creating a bloom filter]: /en/developer-examples#creating-a-bloom-filter
 [section detecting forks]: /en/developer-guide#detecting-forks
 [section getblocktemplate]: /en/developer-guide#getblocktemplate-rpc
 [section hash byte order]: /en/developer-reference#hash-byte-order
@ -376,6 +380,7 @@ http://opensource.org/licenses/MIT.
 [irc channels]: https://en.bitcoin.it/wiki/IRC_channels
 [libblkmaker]: https://gitorious.org/bitcoin/libblkmaker
 [makeseeds script]: https://github.com/bitcoin/bitcoin/tree/master/contrib/seeds
+[murmur3]: https://en.wikipedia.org/wiki/MurmurHash
 [man-in-the-middle]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
 [MIME]: https://en.wikipedia.org/wiki/Internet_media_type
 [mozrootstore]: https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/
@ -397,4 +402,5 @@ http://opensource.org/licenses/MIT.
 <!-- Direct links to code; link to a specific commit to prevent code
 changes from moving the referenced object, but also update links
 periodically to point to recent code. Last update: 2014-11-12 --> 
+[core bloom.cpp hash]: https://github.com/bitcoin/bitcoin/blob/cbf28c6619fe348a258dfd7d08bdbd2392d07511/src/bloom.cpp#L46
 [MAX_SIZE]: https://github.com/bitcoin/bitcoin/blob/60abd463ac2eaa8bc1d616d8c07880dc53d97211/src/serialize.h#L23
--- a/img/dev/en-bloom-update.dot
+++ b/img/dev/en-bloom-update.dot
@ -0,0 +1,72 @@
+digraph {
+
+size="6.25";
+rankdir=TB;
+nodesep=1.05;
+ranksep=0.2;
+splines="false"
+
+edge [ penwidth = 1.75, fontname="Sans" ]
+node [ penwidth = 1.75, shape = "box", fontname="Sans", ]
+graph [ penwidth = 1.75, fontname="Sans" ]
+
+subgraph cluster_client {
+    graph [ penwidth = 0 ];
+    subgraph cluster_client1 {
+        graph [ penwidth = 0 ];
+
+        address [ label = "Address To Match" ];
+        filter1 [ label = "Filter That\n\ \ Matches Address\ \ ", style = "diagonals" ];
+        mymatch1 [ label = "Transaction 1", style = "invis" ];
+        mymatch2 [ label = "Transaction 2", style = "invis" ];
+
+        address -> filter1;
+        filter1 -> mymatch1 [ style = "invis" ];
+        mymatch1 -> mymatch2 [ style = "invis" ];
+
+    }
+
+    label = "Client"
+}
+
+filter1 -> filter2 [ constraint = false ];
+
+subgraph cluster_node {
+    graph [ penwidth = 0 ];
+    subgraph cluster_node2 {
+        graph [ penwidth = 0 ];
+
+        tx2 [ label = "Transaction 2" ];
+        filter3 [ label = "Filter Updated\n\ \ \ \ With Outpoint \ ", style = "diagonals" ];
+        match2 [ label = "Transaction 2\nSpends Outpoint", shape = "none" ];
+
+        tx2 -> filter3;
+        filter3 -> match2 [ minlen = 2 ];
+    }
+
+
+    subgraph cluster_node1 {
+        graph [ penwidth = 0 ];
+
+        tx1 [ label = "Transaction 1" ];
+        filter2 [ label = "Filter That\n\ \ Matches Address\ \ ", style = "diagonals" ];
+        match1 [ label = "Transaction 1\nPays Address", shape = "none" ]
+
+        tx1 -> filter2;
+        filter2 -> match1;
+    }
+
+    match1 -> mymatch1 [ constraint = false ];
+
+    filter2 -> filter3 [constraint = false ];
+    match1 -> filter3 [ constraint = false ];
+
+
+    match2 -> mymatch2 [ constraint = false ];
+
+    label = "Full Node"
+}
+
+label = "Automatically Updating Bloom Filters To Track Relevant Transactions"
+
+}
--- a/img/dev/en-bloom-update.png
+++ b/img/dev/en-bloom-update.png
--- a/img/dev/en-bloom-update.svg
+++ b/img/dev/en-bloom-update.svg
@ -0,0 +1,140 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by graphviz version 2.26.3 (20100126.1600)
+ -->
+<!-- Title: _anonymous_0 Pages: 1 -->
+<svg width="450pt" height="216pt"
+ viewBox="0.00 0.00 450.00 216.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+<g id="graph1" class="graph" transform="scale(0.692308 0.692308) rotate(0) translate(4 308)">
+<title>_anonymous_0</title>
+<polygon fill="white" stroke="white" points="-4,5 -4,-308 647,-308 647,5 -4,5"/>
+<text text-anchor="middle" x="321" y="-8.4" font-family="Sans" font-size="14.00">Automatically Updating Bloom Filters To Track Relevant Transactions</text>
+<g id="graph2" class="cluster"><title>cluster_client</title>
+<polygon fill="none" stroke="black" stroke-width="0" points="8,-36 8,-296 190,-296 190,-36 8,-36"/>
+<text text-anchor="middle" x="99" y="-279.4" font-family="Sans" font-size="14.00">Client</text>
+</g>
+<g id="graph3" class="cluster"><title>cluster_client1</title>
+<polygon fill="none" stroke="black" stroke-width="0" points="16,-44 16,-263 182,-263 182,-44 16,-44"/>
+</g>
+<g id="graph4" class="cluster"><title>cluster_node</title>
+<polygon fill="none" stroke="black" stroke-width="0" points="234,-33 234,-296 634,-296 634,-33 234,-33"/>
+<text text-anchor="middle" x="434" y="-279.4" font-family="Sans" font-size="14.00">Full Node</text>
+</g>
+<g id="graph5" class="cluster"><title>cluster_node2</title>
+<polygon fill="none" stroke="black" stroke-width="0" points="468,-41 468,-263 626,-263 626,-41 468,-41"/>
+</g>
+<g id="graph6" class="cluster"><title>cluster_node1</title>
+<polygon fill="none" stroke="black" stroke-width="0" points="242,-99 242,-263 408,-263 408,-99 242,-99"/>
+</g>
+<!-- address -->
+<g id="node3" class="node"><title>address</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="167,-255 31,-255 31,-219 167,-219 167,-255"/>
+<text text-anchor="middle" x="99" y="-232.9" font-family="Sans" font-size="14.00">Address To Match</text>
+</g>
+<!-- filter1 -->
+<g id="node4" class="node"><title>filter1</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="174,-205 24,-205 24,-163 174,-163 174,-205"/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="36,-205 24,-193 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="24,-175 36,-163 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="162,-163 174,-175 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="174,-193 162,-205 "/>
+<text text-anchor="middle" x="99" y="-188.4" font-family="Sans" font-size="14.00">Filter That</text>
+<text text-anchor="middle" x="99" y="-171.4" font-family="Sans" font-size="14.00"> &#160;Matches Address &#160;</text>
+</g>
+<!-- address&#45;&gt;filter1 -->
+<g id="edge4" class="edge"><title>address&#45;&gt;filter1</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M99,-218.76C99,-217.653 99,-216.523 99,-215.381"/>
+<polygon fill="black" stroke="black" points="102.5,-215.006 99,-205.006 95.5001,-215.007 102.5,-215.006"/>
+</g>
+<!-- mymatch1 -->
+<!-- filter1&#45;&gt;mymatch1 -->
+<!-- filter2 -->
+<g id="node11" class="node"><title>filter2</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="400,-205 250,-205 250,-163 400,-163 400,-205"/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="262,-205 250,-193 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="250,-175 262,-163 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="388,-163 400,-175 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="400,-193 388,-205 "/>
+<text text-anchor="middle" x="325" y="-188.4" font-family="Sans" font-size="14.00">Filter That</text>
+<text text-anchor="middle" x="325" y="-171.4" font-family="Sans" font-size="14.00"> &#160;Matches Address &#160;</text>
+</g>
+<!-- filter1&#45;&gt;filter2 -->
+<g id="edge10" class="edge"><title>filter1&#45;&gt;filter2</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M174.039,-184C195.943,-184 217.847,-184 239.75,-184"/>
+<polygon fill="black" stroke="black" points="239.814,-187.5 249.814,-184 239.814,-180.5 239.814,-187.5"/>
+</g>
+<!-- mymatch2 -->
+<!-- mymatch1&#45;&gt;mymatch2 -->
+<!-- filter3 -->
+<g id="node15" class="node"><title>filter3</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="618,-205 476,-205 476,-163 618,-163 618,-205"/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="488,-205 476,-193 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="476,-175 488,-163 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="606,-163 618,-175 "/>
+<polyline fill="none" stroke="black" stroke-width="1.75" points="618,-193 606,-205 "/>
+<text text-anchor="middle" x="547" y="-188.4" font-family="Sans" font-size="14.00">Filter Updated</text>
+<text text-anchor="middle" x="547" y="-171.4" font-family="Sans" font-size="14.00"> &#160;&#160;&#160;With Outpoint &#160;</text>
+</g>
+<!-- filter2&#45;&gt;filter3 -->
+<g id="edge25" class="edge"><title>filter2&#45;&gt;filter3</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M400.012,-184C421.92,-184 443.828,-184 465.737,-184"/>
+<polygon fill="black" stroke="black" points="465.803,-187.5 475.803,-184 465.803,-180.5 465.803,-187.5"/>
+</g>
+<!-- match1 -->
+<g id="node21" class="node"><title>match1</title>
+<text text-anchor="middle" x="325" y="-132.4" font-family="Sans" font-size="14.00">Transaction 1</text>
+<text text-anchor="middle" x="325" y="-115.4" font-family="Sans" font-size="14.00">Pays Address</text>
+</g>
+<!-- filter2&#45;&gt;match1 -->
+<g id="edge21" class="edge"><title>filter2&#45;&gt;match1</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M325,-162.829C325,-161.714 325,-160.583 325,-159.445"/>
+<polygon fill="black" stroke="black" points="328.5,-159.201 325,-149.201 321.5,-159.201 328.5,-159.201"/>
+</g>
+<!-- tx2 -->
+<g id="node14" class="node"><title>tx2</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="602,-255 492,-255 492,-219 602,-219 602,-255"/>
+<text text-anchor="middle" x="547" y="-232.9" font-family="Sans" font-size="14.00">Transaction 2</text>
+</g>
+<!-- tx2&#45;&gt;filter3 -->
+<g id="edge14" class="edge"><title>tx2&#45;&gt;filter3</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M547,-218.76C547,-217.653 547,-216.523 547,-215.381"/>
+<polygon fill="black" stroke="black" points="550.5,-215.006 547,-205.006 543.5,-215.007 550.5,-215.006"/>
+</g>
+<!-- match2 -->
+<g id="node16" class="node"><title>match2</title>
+<text text-anchor="middle" x="547" y="-74.4" font-family="Sans" font-size="14.00">Transaction 2</text>
+<text text-anchor="middle" x="547" y="-57.4" font-family="Sans" font-size="14.00">Spends Outpoint</text>
+</g>
+<!-- filter3&#45;&gt;match2 -->
+<g id="edge16" class="edge"><title>filter3&#45;&gt;match2</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M547,-162.825C547,-145.548 547,-120.887 547,-101.325"/>
+<polygon fill="black" stroke="black" points="550.5,-101.062 547,-91.062 543.5,-101.062 550.5,-101.062"/>
+</g>
+<!-- match2&#45;&gt;mymatch2 -->
+<g id="edge29" class="edge"><title>match2&#45;&gt;mymatch2</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M480.626,-70C375.193,-70 269.761,-70 164.328,-70"/>
+<polygon fill="black" stroke="black" points="164.125,-66.5001 154.125,-70 164.125,-73.5001 164.125,-66.5001"/>
+</g>
+<!-- tx1 -->
+<g id="node20" class="node"><title>tx1</title>
+<polygon fill="none" stroke="black" stroke-width="1.75" points="380,-255 270,-255 270,-219 380,-219 380,-255"/>
+<text text-anchor="middle" x="325" y="-232.9" font-family="Sans" font-size="14.00">Transaction 1</text>
+</g>
+<!-- tx1&#45;&gt;filter2 -->
+<g id="edge19" class="edge"><title>tx1&#45;&gt;filter2</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M325,-218.76C325,-217.653 325,-216.523 325,-215.381"/>
+<polygon fill="black" stroke="black" points="328.5,-215.006 325,-205.006 321.5,-215.007 328.5,-215.006"/>
+</g>
+<!-- match1&#45;&gt;mymatch1 -->
+<g id="edge23" class="edge"><title>match1&#45;&gt;mymatch1</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M269.949,-128C234.825,-128 199.701,-128 164.577,-128"/>
+<polygon fill="black" stroke="black" points="164.176,-124.5 154.176,-128 164.176,-131.5 164.176,-124.5"/>
+</g>
+<!-- match1&#45;&gt;filter3 -->
+<g id="edge27" class="edge"><title>match1&#45;&gt;filter3</title>
+<path fill="none" stroke="black" stroke-width="1.75" d="M380.284,-141.946C406.14,-148.468 437.44,-156.363 465.905,-163.544"/>
+<polygon fill="black" stroke="black" points="465.289,-166.998 475.842,-166.05 467.002,-160.211 465.289,-166.998"/>
+</g>
+</g>
+</svg>