Research in Public #06: Unprofitable users more likely to exit when btc price high
Introduction
In #1243188 I suggested to @k00b that we engage in a research project using SN data. The idea would be to use this data to study: A) how micropayments with real money affects internet discourse; and B) barriers to the adoption of self-custody. I also promised @Undisciplined that I'd carry out the research in public, since many people might not know what economics research looks like, and may be curious as to how the process plays out. You can follow all of the updates here.
Recap thus far
So far, I've demonstrated that users are indeed responsive to posting fees:
- The quantity of posts goes down when territory posting costs go up (#1253062)
- The quality of posts, as measured by zaps and comments in the first 48 hours, goes up when territory posting costs go up (#1255322)
- These results are identified from changes to territory posting costs and comparison of responses across territories. They are not driven by spurious correlations coming from global time trends or peculiarities about individual territories.
The next thing I wanted to ask is: Does the fact that it's real money matter?
#1258001 was a failed attempt at addressing this question.
Today, I think I made some progress. I'm able to demonstrate that unprofitable users are more likely to be become inactive when bitcoin price is high, compared to profitable users. This makes sense, because bitcoin price amplifies the losses of unprofitable users while simultaneously amplifying the gains of profitable ones.
Definitions
I define a user as "inactive" if they go for 4 or more weeks simultaneously without any posts, comments, or zaps. For each user who ever goes inactive, I document the week in which they started that spell of inactivity. We'll call that "becoming inactive."
For each week in which a user is active, I also calculate their profitability from the last 8 weeks. Profit is measured using all sources of stacking as revenue (zaps, territory revenue, rewards, referrals), but only fees and territory billing as costs. I did not include zaps or donations as costs because those are optional choices by the users. They can be thought of more as how you choose to spend your profit, rather than as costs.
For each week, I also calculate the month-on-month bitcoin price appreciation measured between that week and a month ago; I also calculate how many items the user posted in the last 8 weeks as a measure of activity irrespective of profit.
Results
I first present the results visually, then move to regression analysis to establish the statistical robustness of these results.
The first two charts show that unprofitable users are more likely to become inactive. This is not suprising at all. What's more interesting is that they are more likely to become inactive when bitcoin price has been appreciating. This may be because bitcoin price appreciation amplifies the real purchasing power of their losses. By contrast, profitable users are not more likely to become inactive when bitcoin price appreciates. On the contrary, they are slightly less likely to become inactive.
Next, I run regressions some regressions. The results are shown below.
===================================================================================
Dependent variable:
-------------------------------------------------------
became_inactive
(1) (2) (3) (4)
-----------------------------------------------------------------------------------
Unprofitable last 8 weeks 0.901*** 0.875*** 0.391*** 0.049***
(0.024) (0.025) (0.027) (0.003)
MoM BTC price growth -0.036 -0.197** -0.154
(0.084) (0.099) (0.099)
Unprofitable X price growth 0.573*** 0.512*** 0.073***
(0.185) (0.190) (0.019)
log(Items) last 8 weeks -0.432*** -0.029***
(0.008) (0.001)
Constant -2.498*** -2.492*** -1.507***
(0.014) (0.014) (0.021)
-----------------------------------------------------------------------------------
Model Logit Logit Logit LPM
Week FE N N N Y
Observations 100,278 100,278 100,278 100,278
===================================================================================
Note: *p<0.1; **p<0.05; ***p<0.01
The first three columns show the results from logit models of the weekly probability of becoming inactive. They show basically what the charts show, that i) unprofitable users are more likely to become inactive overall; ii) unprofitable users are more likely to become inactive when bitcoin price appreciation is higher; and iii) profitable users are slightly less likely to become inactive when bitcoin price appreciation is higher. Column (3) adds the logged number of items that the user posted in the last 8 weeks as way to control for the user's overall level of activity. The primary result still holds when we control for user activity; thus, the results are not simply being driven by spurious correlation between user activity and profitability.
The fourth column shows results from a linear probability model. A linear probability model is computationally simpler, and thus allows us to add week fixed effects to the regression. Including week fixed effects allows us to control for arbitrary time trends in SN user behavior. The results continue to show that unprofitable users are more likely to become inactive and that this effect is amplified by bitcoin price appreciation. Thus, the main result is not being driven by spurious correlation driven by global time trends in either SN or the bitcoin market or bitcoin community.
To get a sense of what the numbers mean, using column (4) as our preferred specification, the results imply that when monthly bitcoin price appreciation is
-5\%, then unprofitable users are about 4.5 percentage points more likely to become inactive than profitable users in any given week. But when monthly bitcoin price appreciation is 5\%, then unprofitable users are about 5.3 percentage points more likely to become inactive than profitable users. So, the effect is small, but it is a statistically significant and robust effect, thus demonstrating that the moneyness of bitcoin does matter for SN user behavior.Next steps
Although these results suggest that moneyness matters, the magnitude of the effects seem fairly small (and maybe inconsequential). I could try to look at other ways in which bitcoin price matters, but I'm starting to think that this is not a promising avenue to pursue.
For one thing, Stackers already have a
1 sat = 1 sat mentality. So, given that I've already demonstrated that sat-denominated posting costs matter for behavior, both in terms of quantity and quality, why do I need to also demonstrate that the dollar-denominated cost matters on top of that? If anything, if most of us behave according to 1 sat = 1 sat, we shouldn't expect to see the dollar-denominated price matter that much. So, in that sense, the small magnitudes of the findings make sense.So moving forward, I think I should stop focusing on how bitcoin price affects SN user behavior and instead focus on different measures of quality and discourse. I've shown that higher posting costs lead to higher quality posts as measured by zaps. But can I show this using a more external and objective measure of quality? Obviously, there's no such thing as an "objective" measure of quality, but zaps are a result of factors correlated with but not directly equal to quality, such as the amount of attention a post gets, the SN trust level of its poster, and the time of day / day of week it was posted. I wonder if I can train a model that takes the text of a post and outputs a quality metric. Thus, this quality measure would depend only on the text and title of the post, and not on other factors like time of day or who posted it. That seems a bit ambitious and may take more time, though, so not sure if I should do that next.
Happy to hear other ideas. What other experiments with the data would you like to see?
Anyway that's all I have for today. Anyone who wants to vet the code can go to https://github.com/ed-kung/sn-research. I'll keep posting any time I spend a day doing substantial work on this project.