pull down to refresh

what is the best way to judge SN contests

1312 sats \ 27 comments \ @billytheked 5h AskSN

back in July i started a weekly writing contest (#1040070) in ~the_stacker_muse --the project was to test and see how enthusiastic stackers are about the literary arts

another contest (#1043076) was launched in ~BooksAndArticles shortly thereafter

~Memes too has a weekly meming competition

i have participated in all of the above --and even as i have been the host of such competitions ---there is a part of zapranking-based judging that rubs me the wrong way

especially for a contest of the literary arts

zaprank is pretty cool i admit --but it is algorithmic and ---i think -- results in what has been rather aptly described as a keynsian beauty contest

that is, zap behavior is tangled up with the rewards -- which

aren't really based on zapping and making the best content. Our rewards are based on zapping and creating the content that other users zap the most (slight oversimplification).

and then there is trust --which influences rankings also

The best way to increase your own trust score is to zap content early that you think other users will also zap in the future. Zapping content that you think @k00b, @ek, and territory founders will like is especially helpful, since these users are seeded with higher trust--

OK--so i am no mathematician --and also there are a few concepts --trust --zaprank -- rewards -- here that are all somewhat ungracefully tangled together --but the fact remains that all zaps are not quite equal as they pertain to such contests

--e.g. comment i read recently led me to believe that 10 sats from a perfectly trusted user¹ is similar to a 10k sat boost

so -- bearing that all in mind -- how in your opinion should these contests be judged going forward

--or rather --what is your experience in participating in both types of contests*?*

a la ~the_stacker_muse and ~Memes i.e. just using zaprankings
a la the current higher stakes contest i.e. using human judges
- The judges (@totallyhumanwriter, @siggy47, and @Scoresby) will select a grand prize winner which will receive 100k sats.

which do you prefer

i have not made up my mind --but leaning to the side of them not really being worth it if the goal is tasteful curation

not sure if there are any such users around or if this is just hypothetical ↩

view all related items

310 sats \ 0 replies \ @Scoresby 4h

I can add to this my experience from doing Word of Mouth Wednesdays (WoMWs) for a while now. While WoMWs have been bounties, they are essentially contests with prizes as I'm not offering bounties to everyone who completes a task so much as the stacker who completes the task best.

Initially, I steered away from human judging. The first few WoMWs were decided by zaprank or some other "objective" ranking (eg. most retweets on X). But I kept creating contests that were too easily gamed and it didn't always produce the results I was hoping for.

When it comes to contests, I think human judging is probably always going to be the best. Sure, it is subjective and people play favorites and it's never fair, but taste is like that.

When it comes to algorithms and what content shows up on home screens, I'm much more interested in zaprank. I'm pretty sure it wouldn't work to have somebody sitting there reading every SN post and deciding where to rank it on the home page. It might work well for a day, but I don't think it would ever be sensitive enough to changing trends and topics to keep up with what the community wants to see. Zaprank is. And zaprank is better than any purely trust-based algorithm because it includes at least one component that is difficult to fake: sats.

312 sats \ 1 reply \ @optimism 5h

the latter: use human judges. read it all. form an opinion. read it again. Just have to get high quality judges.

For my weekly AI content aggregation post I discover content by:

Anyway reading everything posted to the territory as it comes in
On Monday reading everything I missed (i.e. on other territories that didn't make it to hot or while I was busy/asleep)
Filter out slop if any (not much)

Then I "pre-judge" by:

sort everything by comments first, to see what got most engagement, maybe pick some stuff that is interesting
sort everything by sats (not zaprank), to see what got most upzapped

Then I decide what to highlight by:

finding posts that I personally think match nicely together, especially if they're arguing opposing sides or can be narrated as a reinforcement of a point.

100 sats \ 0 replies \ @SimpleStacker 4h

I like this approach. Algos assist in surfacing items, but the final judgment is made by a human, with added value by grouping like topics together

210 sats \ 1 reply \ @Undisciplined 5h

What I've been doing for the quarterly ~econ awards is using zaprank to select a set of top posts and then make a poll for people to vote on the best of those.

There's no perfect answer here.

21 sats \ 0 replies \ @billytheked OP 5h

i like it

252 sats \ 15 replies \ @k00b 5h

One of the things I'm hoping to do is allow territory founders to remove trust in ranking. Our sybil fee is high enough that we might be over-indexed on trust as-is.

110 sats \ 11 replies \ @optimism 5h

I think this could be useful. Any post you, ek or undisciplined interacts with is per definition top-ranked.

100 sats \ 10 replies \ @billytheked OP 5h

agreed --but what incentive would territory owners have to enable this feature if they know others might not --would be like shooting themselves in the foot would it not

121 sats \ 0 replies \ @Scoresby 4h

I'd imagine removing trust allows the territory to show posts purely ranked by number of sats zapped.

Since 30% of a zap goes to the community pool, it would be expensive for a poster to self-zap in order to reach the top of such a feed.

The question is, would it be expensive enough to stop self-zappers from dominating the rankings when the rankings are solely based on number of sats zapped?

100 sats \ 5 replies \ @billytheked OP 4h

e.g. i notice ai posts that ek or k00b comments on get ranked 'hot' more than in other territories

the incentive then is to maximize my own rewards by zapping more ai content isnt it

0 sats \ 4 replies \ @optimism 4h

I really don't know so maybe someone else knows. I don't really care about rewards that much myself so in general I think I get too much rewards compared to people that may write more interesting content. I don't think I have a lot of trust (shouldn't trust me, because don't trust, verify) so my zapping content doesn't really help people.

100 sats \ 3 replies \ @SimpleStacker 4h

In the ~AI territory, it's funny to say this, but actually all trust stems from you, you are the source of all trust in ~AI haha

However, trust is territory-specific, so if you don't have much engagement in other territories, you would have very little trust in those territories.

0 sats \ 2 replies \ @optimism 4h

Ah! But I was constructing this against global trust - if there's such a thing? I should really just not be lazy and read the algo.

100 sats \ 0 replies \ @SimpleStacker 3h

Oh, you're right I think there is a global trust graph as well, which I'm guessing is used for the main hot and top sorts. Yeah for that it all emanates from @k00b and @ek I think haha. Wasn't entirely clear when weightedVotes is used vs. subWeightedVotes

reply on another page

100 sats \ 0 replies \ @k00b 4h

SimpleStacker's post explaining trust is a great starting point whether you read the algo or not: #916035

reply on another page

11 sats \ 2 replies \ @optimism 4h

Depends on whether it influences main page hot? If it would be territory hot, then there's no foot-shooting?

210 sats \ 1 reply \ @k00b 4h

It would just affect a territory's hot unless it proved superior to some trust-cost combo.

0 sats \ 0 replies \ @billytheked OP 4h

interesting --from the contest ranking standpoint it seems beneficial --specially if paired with reliable judges --curious what do you see as being the main advantages of a move in this direction

113 sats \ 2 replies \ @Undisciplined 5h

I've been meaning to mention that there's something odd about the trust algorithm when it comes to posts from highly trusted stackers.

Since you can't zap your own posts, their posts don't incorporate the information that they themselves like it. For ranking purposes, it might make sense to act as though the poster had zapped the post when they created it.

144 sats \ 1 reply \ @k00b 4h

I agree. How this should work exactly is unclear enough that I haven't changed it.

How many sats should we assume the poster zapped their own post? Should a post that hasn't been zapped by anyone else appear in hot or top?

42 sats \ 0 replies \ @SimpleStacker 4h

For zaprank, I don't think the poster is taken into account, since it's just trust * log_10(zap amount)

For trust calculations, IIRC it is assumed that the poster zapped their own post by the amount equal to the posting fee.

My only concern with this is that hot and top will just be populated by the same few stackers all the time, thus centralizing the forum even further.

1142 sats \ 2 replies \ @SimpleStacker 4h

The reason this is challenging is because it's a social ranking problem, for which we know a strictly optimal solution doesn't exist: #849906

Thus, the only thing you can do is pick between different tradeoffs.

In terms of ranking items, I think there's usually a tradeoff between quality (surfacing the best posts), equity (not letting a few users' preferences dominate the result), and transparency (a system that is easy to understand and verify)

Between sats-based, upvote-based, and zaprank-based, i'd probably categorize as follows:

	Quality	Equity	Transparency
Sats	Medium	Medium	High
Upvote	Low	High	High
Zaprank	High	Medium/Low	Low

30 sats \ 1 reply \ @k00b 3h

I'd note that most upvote systems share zaprank's equity and transparency scores. In practice, not all upvotes are equal. It's probably even worse in terms of transparency: the process of gaining trust is completely opaque.

0 sats \ 0 replies \ @SimpleStacker 3h

Fair. Illusion of transparency and actual transparency are probably even separate dimensions for optimization haha.

100 sats \ 0 replies \ @hasherstacker 4h

There we are!

0 sats \ 0 replies \ @d680ecaa8e 2h

I think if SN disables daily rewards and split cowboy credits into two balances cowboy credits and earned cowboy credits and resell 100 CC for 50 satoshis (where 50 satoshis is SN resell commission and there is satoshis for territory and elsewhere steps) will offer more earning way. Not every day you earn on top stackers but every day you get zap in CC.

0 sats \ 0 replies \ @hasherstacker 3h

Wow, such a breath of fresh air, this discussion. Aren't we enjoyed the blend of honest insight and down-to-earth critique?

That's some of the right questions in terms of zaprank vs. human judging. How we consider quality, fairness, and genuine creativity in contests. Our wisdom, experiences, and sincerity shine through.

The real beauty of contests is how they enable us to learn from, engage with, and inspire each other.

which do you prefer

Footnotes