BIP 158: Compact Block Filters \ stacker news ~bitcoin

Before BIP 158, Bitcoin light clients used Bloom filters (BIP 37) to detect relevant transactions.
However, Bloom filters had privacy and trust issues

they revealed transaction info to full nodes
and increased the server workload, since each node had to process filters individually.

Compact Block Filters (BIP 158)
Fix both issues by letting full nodes compute deterministic filters once per block and share them with light clients.
Light clients can now check if a block contains relevant transactions without revealing what they’re looking for.

✅ Reduced server workload
✅ Increased privacy

How it works

Each full node constructs a deterministic filter for every block, containing all scriptPubKeys of inputs and outputs.

Light clients download these filters to check if any transaction matches their own addresses or scripts.

If a light client requests a filter for block 20,
the node just sends it, no extra computation needed.

The client then checks:

If match → request the full block
If no match → skip it

Efficient and private.

Steps to create a filter:

Collect all scriptPubKeys from inputs and outputs.
Convert them to uniform numbers.
Sort numbers.
Calculate differences between successive numbers.
Compress using Golomb-Rice coding.
Combine into a compact filter for the block.

Example:

Suppose a block has 3 transactions:

Tx1: A → 1      BTC      to → B
Tx2: C → 0.5    BTC      to → D
Tx3: E → 2.    BTC       to → F

For each transaction, we collect all scriptPubKeys:
{1A, 1B, 0C, 0D, 2E, 2F}

Then:

Map objects to numbers (e.g., [3, 7, 15, 19, 28, 34])
We'll map each object to a number uniformly distributed in a range, e.g., [0, 35].

Store the differences between numbers ([3, 4, 8, 4, 9, 6])
Instead of storing the numbers, we store the differences between successive numbers

Compress with Golomb-Rice encoding using M=4
To compress further. For each number in the list of differences, we calculates the quotient (number // M) and remainder (number % M).

Let's assume M = 4:

Encode quotients (unary) + remainders (binary)
Concatenate to create the final encoded filter

Resulting in something like:
"011 1000 11000 1000 11001 1010"

Light clients can now verify if any of their addresses match entries in this filter.
If a match exists → request full block → verify and process the relevant transaction.

Follow for more bitcoin technical explainers like this one.