Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Using sensitive personal data may be necessary for avoiding discrimination in data-driven decision models

Using sensitive personal data may be necessary for avoiding discrimination in data-driven... Increasing numbers of decisions about everyday life are made using algorithms. By algorithms we mean predictive models (decision rules) captured from historical data using data mining. Such models often decide prices we pay, select ads we see and news we read online, match job descriptions and candidate CVs, decide who gets a loan, who goes through an extra airport security check, or who gets released on parole. Yet growing evidence suggests that decision making by algorithms may discriminate people, even if the computing process is fair and well-intentioned. This happens due to biased or non-representative learning data in combination with inadvertent modeling procedures. From the regulatory perspective there are two tendencies in relation to this issue: (1) to ensure that data-driven decision making is not discriminatory, and (2) to restrict overall collecting and storing of private data to a necessary minimum. This paper shows that from the computing perspective these two goals are contradictory. We demonstrate empirically and theoretically with standard regression models that in order to make sure that decision models are non-discriminatory, for instance, with respect to race, the sensitive racial information needs to be used in the model building process. Of course, after the model is ready, race should not be required as an input variable for decision making. From the regulatory perspective this has an important implication: collecting sensitive personal data is necessary in order to guarantee fairness of algorithms, and law making needs to find sensible ways to allow using such data in the modeling process. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Artificial Intelligence and Law Springer Journals

Using sensitive personal data may be necessary for avoiding discrimination in data-driven decision models

Download PDF
19 pages

Loading next page...
 
/lp/springer-journals/using-sensitive-personal-data-may-be-necessary-for-avoiding-0QTMycXURE
Publisher
Springer Journals
Copyright
Copyright © 2016 by Springer Science+Business Media Dordrecht
Subject
Computer Science; Artificial Intelligence (incl. Robotics); International IT and Media Law, Intellectual Property Law; Philosophy of Law; Legal Aspects of Computing; Information Storage and Retrieval
ISSN
0924-8463
eISSN
1572-8382
DOI
10.1007/s10506-016-9182-5
Publisher site
See Article on Publisher Site

Abstract

Increasing numbers of decisions about everyday life are made using algorithms. By algorithms we mean predictive models (decision rules) captured from historical data using data mining. Such models often decide prices we pay, select ads we see and news we read online, match job descriptions and candidate CVs, decide who gets a loan, who goes through an extra airport security check, or who gets released on parole. Yet growing evidence suggests that decision making by algorithms may discriminate people, even if the computing process is fair and well-intentioned. This happens due to biased or non-representative learning data in combination with inadvertent modeling procedures. From the regulatory perspective there are two tendencies in relation to this issue: (1) to ensure that data-driven decision making is not discriminatory, and (2) to restrict overall collecting and storing of private data to a necessary minimum. This paper shows that from the computing perspective these two goals are contradictory. We demonstrate empirically and theoretically with standard regression models that in order to make sure that decision models are non-discriminatory, for instance, with respect to race, the sensitive racial information needs to be used in the model building process. Of course, after the model is ready, race should not be required as an input variable for decision making. From the regulatory perspective this has an important implication: collecting sensitive personal data is necessary in order to guarantee fairness of algorithms, and law making needs to find sensible ways to allow using such data in the modeling process.

Journal

Artificial Intelligence and Law – Springer Journals

Published: May 7, 2016

References

  • Discrimination and privacy in the information society: data mining and profiling in large databases
  • Informed consent in social media use. the gap between user expectations and eu personal data protection law
    Custers, B; Hof, S; Schermer, B; Appleby-Arnold, S; Brockdorff, N
  • A methodology for direct and indirect discrimination prevention in data mining
    Hajian, S; Domingo-Ferrer, J
  • Spatial analysis of historical redlining: a methodological explanation
    Hillier, A
  • Quantifying explainable discrimination and removing illegal discrimination in automated decision making
    Kamiran, F; Zliobaite, I; Calders, T
  • Private traits and attributes are predictable from digital records of human behaviour
    Kosinski, M; Stillwell, D; Graepel, T
  • Combating discrimination using bayesian networks
    Mancuhan, K; Clifton, C
  • Broken promises of privacy: responding to the surprising failure of anonymization
    Ohm, P
  • Causality: models, reasoning and inference
    Pearl, J
  • Implementing anti-discrimination policies in statistical profiling models
    Pope, DG; Sydnor, JR
  • A multidisciplinary survey on discrimination analysis
    Romei, A; Ruggieri, S
  • The crisis of consent: how stronger legal protection may lead to weaker consent in data protection
    Schermer, B; Custers, B; Hof, S
  • Racial profiling, insurance style: insurance redlining and the uneven development of metropolitan areas
    Squires, G
  • Discrimination in online ad delivery
    Sweeney, L