Comparison to Peer Prediction
The peer prediction literature shares common roots in the analysis of strategic behavior of agents and study of equilibrium conditions leveraging game theory.
But they differ in key points:
- Methodologically, Evolutionary Game Theory relies fundamentally on Agent Based Modeling, leveraging local properties of each agent then deducing equilibrium conditions of a network of agents depending on initial conditions. Instead, Peer Prediction is rooted in on formal analysis of peer prediction games, where each peer is considered a rational agent and strong epistemic assumptions are taken (like shared beliefs).
- Evolutionary Game Theory will look into how individual strategies propagate in a network of agents, with a fundamentally dynamic view and try to understand under which conditions (incentive mechanism, topology…) different equilibria emerge. Peer prediction will rather try to achieve DSIC mechanisms, where all agents are expected to adapt their behavior to the mechanism. If not DSIC, Bayes-Nash equilibria are studied, characterizing equilibrium without guaranteeing it happens.
Let’s note that a peer prediction mechanism that doesn’t display DSIC but has interesting Bayes-Nash equilibria might be interesting to further study under the evolutionary framework.
As noted in Resnick et al., “Eliciting Informative Feedback: The Peer-Prediction Method”:
Subjective evaluations of ratings could be elicited directly instead of relying on correlations between ratings. For example, the news and commentary site Slashdot.org allows meta-moderators to rate the ratings of comments given by regular moderators. Meta-evaluation incurs an obvious inefficiency, since the effort to rate evaluations could presumably be put to better use in rating comments or other products that are a site’s primary product of interest. Moreover, meta-evaluation merely pushes the problem of motivating effort and honest reporting up one level, to ratings of evaluations. Thus, scoring evaluations in comparison to other evaluations is preferable.