In #1243188 I suggested to @k00b that we engage in a research project using SN data. The idea would be to use this data to study: A) how micropayments with real money affects internet discourse; and B) barriers to the adoption of self-custody. I am also following @Undisciplined's suggestion that I carry out the research in public, since many people might not know what economics research looks like, and may be curious as to how the process plays out.

Well, today I received the data---thank you @k00b! It took a while to set up the correct SQL environment to properly extract the data, but I was able to do it in an hour or so.

Once I extracted the data, the first thing to do was some basic sanity checks on the data. For example, are user_id and item_id consistent across tables, etc, etc, and the answer was yes. I didn't find anything weird in the data, at least so far. Great!

The next step was to get some basic information about the data. Establish the size of the dataset and the time range. To that, the data contains 1,240,214 items from 30,737 users, over the period of June 11th, 2021 to October 5th, 2025.

That's a good sized dataset; it should be enough to derive meaningful patterns of behavior across users. Of course, not all users are going to be the same---some may have become inactive, some may have never been active, some may be bots, others may have unusual incentives for participation---so at some point in the future we'll need to categorize the users into different types.

I figured that through the course of the project, two additional pieces of data might be useful: the bitcoin price and google trends index for "bitcoin". I might think of more over time. But for now, these two seem like they might be important for modeling user behavior on SN. I downloaded daily Bitcoin price data from CoinMarketCap from June 1st, 2021 to October 5th, 2025. I also downloaded weekly Google Trends data for the search term "bitcoin" for the same time period.

The correlation between these two charts is interesting in and of itself. It does seem like there's a spike in Google search interest every time there's a spike in the price. Not surprising, but still cool to document:

I then plotted Stacker News weekly items against Bitcoin price to see what would happen and got this:

This chart reveals one of the iterative processes of empirical research: discovering weird data oddities, figuring out what happened, and adjusting for it in the data to get a more accurate picture of real trends. In this case, the artifact is due to @247bf84fda spamming freebies to manipulate the SN to hit 1.4M items in 2025 Predyx bet.

After not counting this user's items, and after removing items for which the posting fee was never paid, we get this:

It looks better, but there's still a significant bump around April 2025. This is probably due to @ek's Most comments wins post which on its own generated about 9,000 comments.

SN experienced its greatest growth (in item count) in the second half of 2023. Curiously, SN's growth appears to precede Bitcoin price. Could SN growth be a leading indicator for Bitcoin price??? SN growth stabilized in 2024, and then started to decline a bit from late 2024 to today. Of course, those of us who've been on the platform a while have debated why this is and what can be done to spur more growth. It might be worthwhile later to look at what kinds of posts and comments were being made during the period of rapid growth in 2023. I also wonder when API keys were made publicly available...

One of the goals of the project is to understand barriers to the adoption of self-custody. This means trying to understand how and when users decide to attach wallets. To do that, we need to figure out when each user first attached a send or a receive wallet.

Unfortunately, this is trickier than it sounds. Company databases are not usually set up to look back at historical data like this. They are designed to serve the product in real time, not for the purpose of maintaining a historical record of everything that a user ever did. Thus, they're usually not that great for studying historical events. The same is true for Stacker News. (This is not a dig at Stacker News, just a statement about the reality of database design and empirical research.)

As an example of this, the users table has an indicator for whether the user has an attached send/receive wallet, but not when they attached it, because SN doesn't need to know when it was attached, only that it is attached. The Wallet table does have a created_at date for users' attached wallets, but if the user detaches and reattaches the wallet, this gets overwritten. Database migrations can potentially overwrite this data as well. I checked the created_at date in the Wallet table for my own wallet, and it wasn't accurate, so I don't think I can use this to measure when users first attached wallets.

I asked @k00b what to do, and he suggested taking a look at the Withdrawl table and when users first had a forwarded invoice that they autowithdrew. I actually just decided to look at when users first used autowithdraw at all, because they would have needed to connect a receiving wallet or lightning address to do that. Autowithdraw was first enabled using LN addresses in January 2024 (#385781), and over time various other methods were added.

As for when users first connected sending wallets, that was trickier. During the early history of send wallets, SN did not track whether the user had "attached" a wallet. It would just send a bolt11 to the browser, and only the browser knew whether you had a connected wallet or not. If you did, it would pay the bolt11, if not a QR code would be created for you to pay manually.

In the end, I don't think it matters too much whether someone has an attached send wallet. What matters is if you are sending real bitcoin when you zap people. Thus, I count the first time each user zapped an item in which real bitcoin was sent---that is, if an invoice was generated and paid.

Here are the results:

So, since SN started implementing non-custodial wallets around July 2024, there's been gradual adoption with an acceleration at the tail end of 2024 (when SN was repeatedly announcing its plans to go non-custodial). The final shift to non-custodial happened on January 3rd 2025 (#835465). Since then, there's still been gradual adoption of these adoptions, but it appears that the rate of receive adoption outpaces the rate of send adoption. This is perhaps unsurprising, because people are much more incentivized to receive real bitcoin than they are to pay real bitcoin. Send wallets are also more difficult to attach. Sending is not supported by all wallets and, even if supported, requires extra permissions. Perhaps later on we can explore in more detail the types of users who adopt self-custodial send/receive options early, who adopt late, and who never adopt.

The next step is actually still to continue exploring the data. One thing I have to figure out is how to observe the historical posting fee of territories. A possible first exercise is to see how the level of posting fee correlates with quantity and quality of posts.

Anyway that's all I have for today. Anyone who wants to vet the code can go to https://github.com/ed-kung/sn-research. I'll keep posting any time I spend a day doing substantial work on this project.