I mean, upvote like other users. E.g. to create a trust relationship between you and me, I look at everything you've upvoted (n, the number of trials) and see how many times I upvoted the same thing (s, successful trials). I then have a sample proportion of trust s/n.

Then I construct a binomial proportion interval with a 95% confidence level and take the lower bound. This is how much I trust you ... it works out nicely, because as you upvote more things, I get closer and closer to how much I actually trust your upvotes - yet even if you haven't upvoted many things I can make a reasonable (yet pessimistic) guess as to how much I should trust you.