However, though it might be tempting for that reason to remove IP reputation from the model, IP address is the strongest indicator of bad users; such users don’t usually go to the troubl
Trang 1Help system display, giving unknown users extra navigation help Lockout of potentially abused features, such as content editing, until the user has demonstrated familiarity with the application and lack of hostility to it
Deciding when to route new contributions to customer care for moderation Pros Allows for a significantly lower barrier for some user contributions than otherwise possible, for example,
not requiring registration or login.
Provides for corporate (internal use) karma No user knows this score, and the site operator can change the application’s calculation method freely as the situation evolves and new proxy reputations become available.
Helps render your application impervious to accidental damage caused by drive-by users.
Cons Inferred karma is, by construction, unreliable For example, since people can share an IP address over time
without knowing it or each other, including it in a reputation can undervalue an otherwise excellent user
by accident However, though it might be tempting for that reason to remove IP reputation from the
model, IP address is the strongest indicator of bad users; such users don’t usually go to the trouble of
getting a new IP address whenever they want to attack your site.
Inferred karma can be expensive to generate How often do you want to update the supporting reputations, such as IP or cookie reputation? It would be too expensive to update them at very single HTTP roundtrip,
so smart design is required.
Inferred karma is weak Don’t trust it alone for any legally or socially significant actions.
Practitioner’s Tips: Negative Public Karma
Because an underlying karma score is a number, product managers often misunder-stand the interaction between numerical values and online identity The thinking goes something like this:
• In our application context, the user’s value will be represented by a single karma, which is a numerical value
• There are good, trustworthy users and bad, untrustworthy users, and everyone would like to know which is which, so we will display their karma
• We should represent good actions as positive numbers and bad actions as negative, and we’ll add them up to make karma
• Good users will have high positive scores (and other users will interact with them), and bad users will have low negative scores (and other users will avoid them) This thinking—though seemingly intuitive—is impoverished, and is wrong in at least two important ways:
• There can be no negative public karma—at least for establishing the trustworthi-ness of active users A bad enough public score will simply lead to that user’s abandoning the account and starting a new one, a process we call karma bank-ruptcy This setup defeats the primary goal of karma—to publicly identify bad actors Assuming that a karma starts at zero for a brand-new user that an
Practitioner’s Tips: Negative Public Karma | 161
Trang 2application has no information about, it can never go below zero, since karma
bankruptcy resets it Just look at the record of eBay sellers with more than three red stars You’ll see that most haven’t sold anything in months or years, either because the sellers quit or they’re now doing business under different account names
• It’s not a good idea to combine positive and negative inputs in a single public karma score Say you encounter a user with 75 karma points and another with 69 karma points Who is more trustworthy? You can’t tell; maybe the first user used to have hundreds of good points but recently accumulated a lot of negative ones, while the second user has never received a negative point at all If you must have public negative reputation, handle it as a separate score (as in the eBay seller feedback pattern)
Even eBay, with the most well-known example of public negative karma, doesn’t rep-resent how untrustworthy an actual seller might be; it only gives buyers reasons to take specific actions to protect themselves In general, avoid negative public karma If you really want to know who the bad guys are, keep the score separate and restrict it to internal use by moderation staff
The Dollhouse Mafia, or “Don’t Display Negative Karma”
The Sims Online was a multiplayer version of the popular Sims games by Electronic
Arts and Maxis in which the user controlled an animated character in a virtual world with houses, furniture, games, virtual currency (called Simoleans), rental property, and social activities You could call it playing dollhouse online
One of the features that supported user socialization in the game was the ability to declare that another user was a trusted friend The feature involved a graphical display that showed the faces of users who had declared you trustworthy outlined in green, attached in a hub-and-spoke pattern to your face in the center
People checked each other’s hubs for help in deciding whether to take certain in-game actions, such as becoming roommates in a house Decisions like these are costly for a new user—the ramifications of the decision stick with a newbie for a long time, and
“backing out” of a bad decision is not an easy thing to do The hub was a useful decision-making device for these purposes
That feature was fine as far as it went, but unlike other social networks, The Sims
Online allowed users to declare other users untrustworthy, too The face of an
untrust-worthy user appeared circled in bright red among all the trustuntrust-worthy faces in a user’s hub
It didn’t take long for a group calling itself the Sims Mafia to figure out how to use this mechanism to shake down new users when they arrived in the game The dialog would
go something like this:
“Hi! I see from your hub that you’re new to the area Give me all your Simoleans or my friends and I will make it impossible to rent a house.”
162 | Chapter 6: Objects, Inputs, Scope, and Mechanism
Trang 3“What are you talking about?”
“I’m a member of the Sims Mafia, and we will all mark you as untrustworthy, turning your hub solid red (with no more room for green), and no one will play with you You have five minutes to comply If you think I’m kidding, look at your hub—three of us have already marked you red Don’t worry, we’ll turn it green when you pay….”
If you think this is a fun game, think again A typical response to this shakedown was for the user to decide that the game wasn’t worth $10 a month Playing dollhouse doesn’t usually involve gangsters It’s hard to estimate the final cost to EA & Maxis for such a simple design decision, in terms of lost users, abandoned accounts, and cancelled subscriptions
In your own community and application design, think twice about overtly displaying negative reputation, or putting such direct means in the hands of the community to
affect other’s reputations You risk enabling your own mafias to flourish.
Draw Your Diagram
With your goals, objects, inputs, and reputation patterns in hand, you can draw a draft reputation model diagram and sketch out the flows in enough detail to generate the following questions: what data will I need to formulate these reputation scores cor-rectly?; how will I collect the claims and transform them into inputs?; which of those inputs will need to be reversible, and which will be disposable?
If you’re using this book as a guide, try sketching out a model now, before you consider creating screen mock-ups One approach we’ve often found helpful is to start on the right side of the diagram—with the reputations you want to generate—and work your way back to the inputs Don’t worry about the calculations at first; just draw a process box with the name of the reputation inside and a short note on the general nature of the formulation, such as aggregated acting average or community player rank Once you’ve drawn the boxes, connect them with arrows where appropriate Then consider what inputs go into which boxes, and don’t forget that the arrows can split and merge as needed
Then, after you have a good rough diagram, start to dive into the details with your development team Many mathematical and performance-related details will affect your reputation model design We’ve found that reputation systems diagrams make excellent requirements documentation and make it easier to generate the technical specification, while also making the overall design accessible to nonengineers
Of course, your application will consist of displaying or using the reputations you’ve diagrammed (Chapters 7 and 8) Project engineers, architects, and operational team members may want to review Chapter 9 first, as it completes the schedule focused, development-cycle view of any reputation project
Draw Your Diagram | 163
Trang 5CHAPTER 7
Displaying Reputation
In Chapter 6, we described how to create a custom reputation model by identifying the objects in your application, selecting appropriate inputs, and developing the processes you’ll need to generate your reputations But your work doesn’t end there Far from it Now you have decisions to make about how to use the reputations that your system is tabulating
In this chapter and the next, we discuss the many options for using reputation to im-prove the user experience of your site, enrich content quality, and provide incentives for your users to become better, more active participants In this chapter specifically,
we discuss options for displaying reputation, to whom to display it, how to display it, and help you decide which display forms are right for your application
How to Use a Reputation: Three Questions
For each reputation you are creating to display or use, you should ask each of these questions before proceeding:
1 Who will be able to see the reputation?
• Is it personal—hidden from other users but visible to the reputation holder?
• Is it public—displayed to friends or strangers, or visible to search engines?
• Is it corporate—limited to internal use—for improving the site or discreetly
rec-ognizing outliers in ways that may not be visible to the community?
2 How will the reputation be used to modify your site’s output?
• Will you use the reputation to filter the lowest- or highest-quality items in a set?
• Will you use the reputation to sort or rank items?
• And/or will this score be used to make other decisions about how the site flows
or your business operates?
3 Is this reputation for a content item or a person? Each requires a fundamentally
different approach
165
Trang 6Though you may choose multiple answers from this list for each reputation, try to keep
it simple at first: don’t try to do too much with a single reputation Confounding the purposes of a reputation—by, for example, surfacing participation points in a public karma score—can encourage undesirable user behavior and may even backfire by dis-couraging participation Read Chapters 7 and 8 completely for a solid understanding
of the issues related to overloading a single reputation
Resist the temptation to treat a single reputation score as the cure-all for
your user-generated content incentive ills Remember the lesson of the
FICO score in “FICO: A Study in Global Reputation and Its
Chal-lenges” on page 10
Who Will See a Reputation?
So far, the reputation you’re calculating is little more than a cold numerical score rolled
up from the aggregate actions of people interacting with your site You’ve carefully determined the scope of the reputation, chosen the inputs that contribute to it, and thought at length about the effect that you want the reputation to generate in the community
Now you must decide whether it makes sense to display the reputation on your site at all and, if so, to whom How you display reputation information—how much and how prominently—will influence the actions that users take on your site, their trust in your site and one another, and their long-term satisfaction with your community
To Show or Not to Show?
Compelling reasons exist to keep reputations hidden from users In fact, in some cir-cumstances, you may want to obscure the fact that you’re tracking them at all It may sound rather Machiavellian, but the truth of the matter is this: a community under public scrutiny behaves differently (and, in many ways, less honestly) than one in bliss-ful ignorance
Several trade-offs are involved Displaying reputations takes up significant page real estate, requires user interface design and testing, and can compete with your content for the user’s attention and understanding Quickly, show Digg.com (Figure 7-1) to 10
of your friends and ask them, “What kind of site is this? News? Entertainment? Com-munity?” Odds are good that at least a few of them will answer: “This appears to be
some sort of contest.”
The impression that Digg makes is not a bad thing; it just demonstrates that Digg made
a conscious decision to display content reputation prominently In fact, the display of reputation is the central interaction mechanism on the site It’s practically impossible
to interact with Digg, or get any use out of it, without some understanding of how community voting affects the selection and display of popular items on the site (Digg
166 | Chapter 7: Displaying Reputation
Trang 7is perhaps the most well-known example of a site that employs the Vote-to-Promote pattern See Chapter 6.)
Juxtapose Digg’s approach with that of Flickr The popular photo-sharing and discov-ery service also makes use of reputation to surface quality content, but it does not display explicit reputations, rather it prominently displays items that achieve a certain reputation and that can be browsed (daily, weekly, or monthly) in the “Explore” gallery (at http://www.flickr.com/explore); see Figure 7-2 The result is a very consistent and impressive display of high-quality photos with very little indication of how those photos were selected
Flickr’s interestingness algorithm determines which photos make it into the “Explore” gallery and which don’t The same algorithm lets users sort their own photos by interestingness
Figure 7-1 Digg’s site design puts overt reputation scores front and center.
Who Will See a Reputation? | 167
Trang 8Digg and Flickr represent two very different approaches to reputation display, but the results are very much the same Theoretically, you can always glance at the front page
of Digg or Flickr’s “Explore” gallery to see where the good stuff is—what people are watching, commenting on, or interacting with the most on the site
How do you decide whether to display reputations on your site? And how prominently? Generally, follow the rule of least disclosure: do not display a reputation that doesn’t add specific value to the objects being evaluated
Figure 7-2 Flickr’s “Explore” gallery is also based on reputation, but you never see a score associated with a photo.
168 | Chapter 7: Displaying Reputation
Trang 9Likewise, don’t bother asking users for reputation input (see Chapter 6) that you’ll never use; you’ll confuse users and encourage undesired patterns of “invented signifi-cance,” including abuse
Avoid collecting reputation for display only Orkut allowed users to rate
other users explicitly on iconic criteria like “trusty,” “cool,” and “sexy”
for no use other than display This use of reputation caused all kinds of
social backlash.
People were either disappointed that they weren’t rated “cool” by more
people, or they were creeped out by people of the same gender calling
them sexy Eventually, Orkut removed the display of individual friends’
ratings and kept only the aggregate scores.
Irrelevant reputations are meaningless and consume valuable resources If you don’t have a relevant use for a reputation, beware of sticking yourself later with the tough choice of either awkwardly removing a failed feature or having to support it as a costly legacy element
Personal Reputations: For the Owner’s Eyes Only
Are you tracking a reputation primarily to keep users informed about how well they or their creations are performing in the community? Consider displaying that reputation only to its owner, as a personal communication between site and user
Personal Reputation Is Not Private
We use the word personal very deliberately here, distinguishing it from private No
reputation system is truly private; at least one other party (typically the site operator) will almost always have access to the actions, inputs, and roll-ups that formulate a user’s score In fact, you may store internally used reputations (see “Corporate Reputations Are Internal Use Only: Keep Them Hush-hush” on page 172) that are largely based
on the exact same data
In other words, reputations may be displayed in a personal context, but that’s no guar-antee that they’re private As a service provider, you should acknowledge that distinc-tion and account for it in your terms of service
Personal reputations are used extensively for applications such as social bookmarking, lists of favorites, training recommendation systems, sorting and filtering news feeds, providing content quality and feedback, fine-grained experience point tracking, and other performance metrics Most of the same user interface patterns for displaying public reputation apply to personal ones, too, but take care to ensure that each user knows when her reputations will and will not be displayed to others
Who Will See a Reputation? | 169
Trang 10Keep a reputation personal when its owner gains some significant
ben-efit from it—when it either improves his experience of the site (that is,
personalizes it) or provides a tool for increasing self-satisfaction For
example, by selecting news stories about various sports teams over time,
a user might generate a geographic region reputation that can be used
to target advertising displayed to the user Clearly that reputation should
not be public information, but it might be surfaced privately so that the
user can correct it—“I’m a fan of Northern California sports teams, but
I’m going to MIT and I really want ads for electronics stores in the
Bos-ton area.”
Google Analytics (see Figure 7-3) is an example of rich personal reputation information
It provides detailed information about the performance of your website, across a known range of score types, and it is available only to you, the site owner (or others to whom you grant access) While that information is invaluable to you in gauging the response
of a community (in this case, the entire Web) to your content, exposing it to everyone would offer very little practical benefit In fact, it would be a horrible idea
Figure 7-3 Google’s Analytics interface shows information that is clearly best kept between you and Google It’s personal.
170 | Chapter 7: Displaying Reputation