I’m on the mailing list for a software program called Enso. Subscribers trust others not to abuse the list. Specifically, they trust others not to spam, post inappropriate material, or attempt phishing scams. People on the list also trust others to be well-intentioned (i.e. they do not wish to actively harm the community). Poisonous people could drain user enthusiasm and/or suck up time and resources from Enso developers by igniting flame wars and expressing consistent pessimism.
It’s desirable to have members post messages that are on-topic, relevant to most members on the list, friendly (to both the project and its members), and not trivially answered elsewhere. Consider a reward system where the community can rate the quality of each post, perhaps on a scale of 1 (worst) to 5 (best). Users rate posts by emailing the listserv with a rating for a particular post. Posts generating more responses (and responses from different people) would also get a higher quality ranking. These explicit and implicit feedback mechanisms would be combined in some way to generate an overall quality ranking that affects the number of points assigned to that post. A user’s reputation is the weighted average of all his posts, calculated in a manner such that newer posts are given more weight than older posts.
As Resnick, et. al. explain in “Reputation Systems,” a good design will cause users to expect past interactions to influence future interactions. The reward system will create this “shadow of the future” effect by attaching each person’s reputation to every post he makes. The quality of posts made in the past will affect his current rating, which may impact how community members respond to him in the future.
Although we’re using explicit feedback in the form of ratings, a single user shouldn’t be able to affect rankings too drastically. This is because you only get to rate each post once, and the explicit ratings are merged with implicit feedback (responses generated by the post) in order to arrive at the final point value. But if someone subscribes many different email addresses to the list, he could game the system by either using all those accounts to rate a single post down or respond to a given post many times, boosting its score.
Subscribe to:
Post Comments (Atom)
Posts
That's actually a really interesting idea for a listserv. Understandable, the implementation and interface would leave a little to be desired unless somehow built in to the e-mail or e-mail client itself, but that's another idea for another time.
ReplyDeleteI do want to comment though on the idea of perhaps allowing to leave ratings from -5 to 5 or something like that. This then also implies a negative aspect that then can allow a bit more for averages and new users (new users maybe start at 0). A number of posts could also upgrade a person to another status which then is also tied to the rating.
This then disallows a user being able to spam by constantly creating new addresses and spamming the listserv.
You seemed to have a well developed concept of an idea of a reward system for an e-list. I really liked the idea you had about weighing recent posts more heavily than older posts. I like incorporating some kind of virtual form of forgiveness. However, something to consider in your solution would be posts that are responded to more often receiving higher rankings. If a post is considered malicious or off-topic, I feel that it would generate a lot of negative feedback in the form of posting. This would generate a large volume of posts. I believe that the explicit system of rating posts 1-5 (or even as Eugene suggested, which I also like, -5 to 5) would be more accurate.
ReplyDeleteThis is an ideal modification to current rating systems for forum posts because it attacks many weaknesses of current models. Providing a wider scale (-5 to 5) seems to be something that hasn't been tried too often. Perhaps it has been, but without much success? It seems that publicly displaying a negative rating from another user might start some tension in the forums.
ReplyDelete