“Inadvertent Algorithmic Cruelty” doesn’t have to be inevitable

Last night I read a series of tweets that I’m still thinking about 12 hours later. Eryn tweeted about a scenario (a scenario is a “common flow while using the product” a definition I took from this excellent product design post) in casual gaming after she received a Gift of Life in Candy Crush from a friend who had passed away. The unexpected “gift” shocked her and, understandability, caused her much grief. She mentioned how popular Facebook games were for hospitalized people, passing the time and otherwise offering a distraction. Yet, the game leaderboards live forever and the way social games mine contacts to try and gain more players make it inevitable that deceased friends will send game-related tokens posthumously. Eryn called these actions “Inadvertent Algorithmic Cruelty,” a term coined by Eric Meyer when he shared how Facebook’s year-in-review feature caused him a lot of pain.

It seems, from the outside looking in, that these products don’t even consider these scenarios. It’s not easy to design a product and ask “what happens when people die?” It’s difficult, at the product design stage, to imagine the scenario Eryn described: a terminally-ill, hospitalized patient whiling away the time playing a social game and getting so good at it that they make the leaderboard. After all, no one likes to think of death (as an aside: 71% of adults under the age of 34 and 41% of aging Baby Boomers do not have a will.)

So what to do?

As in everything, the first step is admitting you have a problem. Perhaps the onus is more on Facebook to create a significant and encompassing end-of-life product. Right now, they do have a way to report a deceased person and family members can request that an account be deleted. This is a good start.

Candy Crush start screen with only two options: Play or Facebook

Candy Crush start screen with only two options: Play or Facebook

The second step is to use that information wisely. Almost all (I didn’t find one that didn’t) social games already ask for a Facebook login to encourage friends to play the game. Interestingly enough, the games rely on Facebook exclusively and, on mobile, don’t even ask permission to access the phone’s contacts. This can be an advantage when it comes to end-of-life issues. First, Facebook identities and game profiles are already correlated and second, Facebook does the work of verifying a death. While it may be difficult to stay up-to-date on a daily basis, surely a weekly basis is doable?

“Inadvertent Algorithmic Cruelty” is not inevitable, it’s just something that needs to be considered. At the time of Eric’s post, many commenters thought an older product team, with more life experience, might have been able to consider negative implications of the year-in-review feature. Eric disagreed, saying that the “failure to consider worst-case scenarios is not a special disease of young, inexperienced programmers.  It is everywhere.” And to that I add: it’s time to design for it in every social feature.



One thought on ““Inadvertent Algorithmic Cruelty” doesn’t have to be inevitable

  1. Pingback: When assumptions about a user can go too far | What it all boils down to

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s