Reputation systems
Karma, credit scores, PageRank, proof-of-identity, proof-of-truth; optimized ad hominem reasoning
2014-08-05 — 2026-04-17
Wherein the measurement of trustworthiness through iterative peer endorsement is examined, such that one’s rating of another is weighted by the ratings accorded to oneself.
Designing infrastructure for assessing people’s trustworthiness, insofar as such a quantity exists.
A flip side to epistemic communities is the problem of verifying that the information we have is good. If we got to design all the agents in society we might be able to ensure the overall system acquires good information. But when we are dealing with real people, how do we know what they tell us is real? At scale? Do we do this by some kind of social trust graph? Some other mechanism?
Reputation systems are systems to work out how reliable someone is, through some kind of collaborative process that ranks how likely they are to be correct or trustworthy.
We seem to have some tendency to status-seeking which makes us prone to using such systems, and the modern technocratic versions are a potential source of mechanisms for collective accomplishment and control.
In the context of epistemic communities, we might state this as If we are going to reason ad hominem, how should we?
I’m mostly thinking about peer ranking systems here, although of course centrally supplied ranking systems are also reputation systems and do indeed fit in this classification.
Credit scores, rankings, bulletin board karma, China’s vaunted social credit system(s). Risk estimation based on qualities that are not transferred financially (although they might be about financial behaviour). Robin Hanson has speculated about status apps becoming ubiquitous.
1 Bulletin-board style
Most prominently now, stackexchange has a gamified reputation system designed to get people addicted and incentivised to be productive community members based on formalised status with evolved mechanism design. The system was hard to design and needs constant revision, but it seems to do something.
Ethtrader is an interesting example in applied reputation design with connections to real-world cash. RECS says
Of course the Ethereum community would build a labyrinthian reputation system on top of Reddit! Like the cryptocurrency itself, the rules for this subreddit are complex, but here’s the TLDR version: They distribute points (“Donuts”), based on engagement, which can be used to purchase various site features. There’s a special tax system, called the Hamburger Tax, which… ah heck, as they say in the cryptoverse, just read the white paper.
2 Iterative reputation
“My rating of your reputation is worth more because other people rate my reputation highly.”
Pagerank and kin, and their application to people rather than webpages. Possible relation to game theory, network topologies, contagion processes, swarm sensing, recommender systems, worryingly technocratic utopians, and learning from gossip. Important in the practice of science, and other situations of trust.
- Eigenvector centrality (a.k.a. PageRank)
- “Subgraph centrality” (Estrada and Rodríguez-Velázquez 2005) and Bonacich’s family of related measures
- Eigentrust
- Eigenmorality
- Applying the Page-Rank Algorithm to the Electoral Process
3 China-style
China has gotten a lot of attention for their reputation systems recently, I suppose because their systems are being designed from the ground up and don’t look like the extant Western reputation systems, which have melted into the background here. Coverage of these new systems can make them sound alarming:
China builds the mother of all online reputation systems:
China is proposing to assess its citizens’ behaviour over a totality of commercial and social activities, creating an uber-scoring system. When completed, the model could encompass everything from a person’s chat-room comments to their performance at work, while the score could be used to determine eligibility for jobs, mortgages, and social services.
“They’ve been working on the credit system for the financial industry for a while now,” says Rogier Creemers, a China expert at Oxford University. “But, in recent years, the idea started growing that if you’re going to assess people’s financial status, you should equally be able to do that with other modes of trustworthiness.”
The document talks about the “construction of credibility”—the ability to give and take away credits—across more than 30 areas of life, from energy saving to advertising.
See also WaPo on this theme.
Dev Lewis’ fieldwork on such systems makes them sound far less dystopian.
I’m not expert enough to comment on this disjunction. The intersection with automated surveillance could lead in weird directions. Or not.
The coverage mostly seems to radiate from a civil-liberties authoritarian panic, which might be justified. But one wonders if there is a utopian possibility here. Various formal and informal reputation systems are obviously already everywhere; maybe by designing technocratic ones, we have a chance to design better ones with better biases built in. Or maybe this is an attempt at wrong-headed high modernism. Maybe the surveillance will be nasty even without reputation systems getting involved, and places like China that let the reputation score be front-and-centre will at least get more status information out of the process of monitoring everyone than the less prominent credit rating systems and informal networks of the west?
In terms of practical outcomes, I find it hard to imagine a whole society (or collection of societies) like China, for example, iterating as rapidly and effectively on a reputation system as a simple controlled system like Stackexchange does.
4 Proof-of-identity systems
A simple case. Is the message to you from me really from me?
Web of Trust is troublesome for all the usual reasons that encryption is troublesome.
Notary aims to make the internet more secure by making it easy for people to publish and verify content. We often rely on TLS to secure our communications with a web server, which is inherently flawed, as any compromise of the server enables malicious content to be substituted for the legitimate content.
With Notary, publishers can sign their content offline using keys kept highly secure. Once the publisher is ready to make the content available, they can push their signed trusted collection to a Notary Server.
Consumers, having acquired the publisher’s public key through a secure channel, can then communicate with any Notary server or (insecure) mirror, relying only on the publisher’s key to determine the validity and integrity of the received content.
Keybase has an interesting solution here. TBC.
5 Proof-of-truth
🏗
Civil’s attempts to find blockchain-backed proof-of-truth for journalism. Others?
Is a reputation system perhaps sufficient?
Robin Hanson once again has a framing I want, asking what info is verifiable. He would like it to be verifiable in the sense that we can make a contract about an outcome, with obvious application to blockchains, prediction markets and general mechanism design.