When Big Data becomes Bad data: The Limits of Analytics

Corporations are increasingly using algorithms to make business decisions, raising a host of new legal questions

Sourced through Scoop.it from: www.salon.com

How will the courts address algorithmic bias? From retail to real estate, from employment to criminal justice, the use of data mining, scoring software and predictive analytics programs is proliferating at an exponential rate. Software that makes decisions based on data like a person’s ZIP code can reflect, or even amplify, the results of historical or institutional discrimination.“[A]n algorithm is only as good as the data it works with,” Solon Barocas and Andrew Selbst write in their article “Big Data’s Disparate Impact,” forthcoming in the California Law Review. “Even in situations where data miners are extremely careful, they can still affect discriminatory results with models that, quite unintentionally, pick out proxy variables for protected classes.”   It’s troubling enough when Flickr’s auto-tagging of online photos label pictures of black men as “animal” or “ape,” or when researchers determine that Google search results for black-sounding names are more likely to be accompanied by ads about criminal activity than search results for white-sounding names. But what about when big data is used to determine a person’s credit score, ability to get hired, or even the length of a prison sentence?

Because disparate impact theory is results-oriented, it would seem to be a good way to challenge algorithmic bias in court. A plaintiff would only need to demonstrate bias in the results, without having to prove that a program was conceived with bias as its goal. But there is little legal precedent. Barocas and Selbst argue in their article that expanding disparate impact theory to challenge discriminatory data-mining in court “will be difficult technically, difficult legally, and difficult politically.”  Some researchers argue that it makes more sense to design systems from the start in a more considered and discrimination-conscious way. Barocas and Moritz Hardt established a traveling workshop called Fairness, Accountability and Transparency in Machine Learning to encourage other computer scientists to do just that. Some of their fellow organizers are also developing tools they hope companies and government agencies could use to test whether their algorithms yield discriminatory results and to fix them when necessary. Some legal scholars (including the University of Maryland’sDanielle Keats Citron and Frank Pasquale) argue for the creation of new regulations or even regulatory bodies to govern the algorithms that make increasingly important decisions in our lives.  There still exists “a large legal difference between whether there is explicit legal discrimination or implicit discrimination,” said Friedler, the computer science researcher. “My opinion is that, because more decisions are being made by algorithms, that these distinctions are being blurred.”

See on Scoop.itInternet of Things – Technology focus

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s