The Tyranny of Data? The Bright and Dark Sides of Data-Driven Decision-Making for Social Good
- 14 Citations
- 2.6k Downloads
Abstract
The unprecedented availability of large-scale human behavioral data is profoundly changing the world we live in. Researchers, companies, governments, financial institutions, non-governmental organizations and also citizen groups are actively experimenting, innovating and adapting algorithmic decision-making tools to understand global patterns of human behavior and provide decision support to tackle problems of societal importance. In this chapter, we focus our attention on social good decision-making algorithms, that is algorithms strongly influencing decision-making and resource optimization of public goods, such as public health, safety, access to finance and fair employment. Through an analysis of specific use cases and approaches, we highlight both the positive opportunities that are created through data-driven algorithmic decision-making, and the potential negative consequences that practitioners should be aware of and address in order to truly realize the potential of this emergent field. We elaborate on the need for these algorithms to provide transparency and accountability, preserve privacy and be tested and evaluated in context, by means of living lab approaches involving citizens. Finally, we turn to the requirements which would make it possible to leverage the predictive power of data-driven human behavior analysis while ensuring transparency, accountability, and civic participation.
Keywords
Information Asymmetry Machine Learning Model Credit Score Computational Thinking Intended BeneficiaryReferences
- 1.Akerlof, G.A.: The market for “lemons”: quality uncertainty and the market mechanism. Q. J. Econ. 84 (3), 488–500 (1970)CrossRefGoogle Scholar
- 2.Akerlof, G.A., Shiller, R.J.: Animal Spirits: How Human Psychology Drives the Economy, and Why It Matters for Global Capitalism. Princeton University Press, Princeton (2009)Google Scholar
- 3.Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. Law Rev. 104, 671–732 (2016)Google Scholar
- 4.Bengtsson, L., Lu, X., Thorson, A., Garfield, R., Von Schreeb, J.: Improved response to disasters and outbreaks by tracking population movements with mobile phone network data: a post-earthquake geospatial study in Haiti. PLoS Med. 8 (8), e1001083 (2011)CrossRefGoogle Scholar
- 5.Benkler, Y.: The Wealth of Networks. Yale University Press, New Haven (2006)Google Scholar
- 6.Berendt, B., Preibusch, S.: Better decision support through exploratory discrimination-aware data mining: foundations and empirical evidence. Artif. Intell. Law 22 (2), 1572–8382 (2014)CrossRefGoogle Scholar
- 7.Blondel, V.D., Decuyper, A., Krings, G.: A survey of results on mobile phone datasets analysis. EPJ Data Sci. 4 (10) (2015)Google Scholar
- 8.Blumenstock, J., Cadamuro, G., On, R.: Predicting poverty and wealth from mobile phone metadata. Science 350 (6264), 1073–1076 (2015)CrossRefGoogle Scholar
- 9.Bogomolov, A., Lepri, B., Ferron, M., Pianesi, F., Pentland, A.: Daily stress recognition from mobile phone data, weather conditions and individual traits. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 477–486 (2014)Google Scholar
- 10.Bogomolov, A., Lepri, B., Staiano, J., Oliver, N., Pianesi, F., Pentland, A.: Once upon a crime: towards crime prediction from demographics and mobile data. In: Proceedings of the International Conference on Multimodal Interaction (ICMI), pp. 427–434 (2014)Google Scholar
- 11.Bogomolov, A., Lepri, B., Staiano, J., Letouzé, E., Oliver, N., Pianesi, F., Pentland, A.: Moves on the street: classifying crime hotspots using aggregated anonymized data on people dynamics. Big Data 3 (3), 148–158 (2015)CrossRefGoogle Scholar
- 12.Burrell, J.: How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 3 (1) (2016)Google Scholar
- 13.Calders, T., Verwer, S.: Three naive Bayes approaches for discrimination-free classification. Data Min. Knowl. Disc. 21 (2), 277–292 (2010)MathSciNetCrossRefGoogle Scholar
- 14.Calders, T., Zliobaite, I.: Why unbiased computational processes can lead to discriminative decision procedures. In: Custers, B., Calders, T., Schermer, B., Zarsky, T. (eds.) Discrimination and Privacy in the Information Society, pp. 43–57. Springer, Berlin (2013)CrossRefGoogle Scholar
- 15.Centellegher, S., De Nadai, M., Caraviello, M., Leonardi, C., Vescovi, M., Ramadian, Y., Oliver, N., Pianesi, F., Pentland, A., Antonelli, F., Lepri, B.: The mobile territorial lab: a multilayered and dynamic view on parents’ daily lives. EPJ Data Sci. 5 (3) (2016)Google Scholar
- 16.Chainey, S.P., Tompson, L., Uhlig, S.: The utility of hotspot mapping for predicting spatial patterns of crime. Secur. J. 21, 4–28 (2008)CrossRefGoogle Scholar
- 17.Christin, A., Rosenblatt, A., boyd, d.: Courts and predictive algorithms. Data Civil Rights Primer (2015)Google Scholar
- 18.Citron, D.K., Pasquale, F.: The scored society. Wash. Law Rev. 89 (1), 1–33 (2014)Google Scholar
- 19.Crawford, K., Schultz, J.: Big data and due process: toward a framework to redress predictive privacy harms. Boston College Law Rev. 55 (1), 93–128 (2014).Google Scholar
- 20.De Choudhury, M., Gamon, M., Counts, S., Horvitz, E.: Predicting depression via social media. In: Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (2013)Google Scholar
- 21.De Filippi, P.: The interplay between decentralization and privacy: the case of blockchain technologies. J. Peer Production 7 (2015)Google Scholar
- 22.de Montjoye, Y.-A., Hidalgo, C., Verleysen, M., Blondel, V.: Unique in the crowd: the privacy bounds of human mobility. Sci. Rep. 3 (2013)Google Scholar
- 23.de Montjoye, Y.-A., Shmueli, E., Wang, S., Pentland, A.: OpenPDS: protecting the privacy of metadata through safeanswers. PLoS One 9 (7), e98790 (2014)CrossRefGoogle Scholar
- 24.de Montjoye, Y.-A., Radaelli, L., Singh, V.K., Pentland, A.: Unique in the shopping mall: on the re-identifiability of credit card metadata. Science 347 (6221), 536–539 (2015)Google Scholar
- 25.de Oliveira, R., Karatzoglou, A., Concejero Cerezo, P., Armenta Lopez de Vicuña, A., Oliver, N.: Towards a psychographic user model from mobile phone usage. In: CHI’11 Extended Abstracts on Human Factors in Computing Systems, pp. 2191–2196. ACM, New York (2011)Google Scholar
- 26.Devarajan, S.: Africa’s statistical tragedy. Rev. Income Wealth 59 (S1), S9–S15 (2013)CrossRefGoogle Scholar
- 27.Diakopoulos, N.: Algorithmic accountability: journalistic investigation of computational power structures. Digit. Journal. 3 (3), 398–415 (2015)CrossRefGoogle Scholar
- 28.Easterly, W.: The Tyranny of Experts. Basic Books, New York (2014)Google Scholar
- 29.Eck, J., Chainey, S., Cameron, J., Wilson, R.: Mapping crime: understanding hotspots. National Institute of Justice, Washington (2005)Google Scholar
- 30.Faurholt-Jepsena, M., Frostb, M., Vinberga, M., Christensena, E.M., Bardram, J.E., Kessinga, L.V.: Smartphone data as objective measures of bipolar disorder symptoms. Psychiatry Res. 217, 124–127 (2014)CrossRefGoogle Scholar
- 31.Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259–268 (2015)Google Scholar
- 32.Ferguson, A.G.: Crime mapping and the fourth amendment: redrawing high-crime areas. Hastings Law J. 63, 179–232 (2012)Google Scholar
- 33.Fields, G.: Changes in poverty and inequality. World Bank Res. Obs. 4, 167–186 (1989)CrossRefGoogle Scholar
- 34.Fiske, S.T.: Stereotyping, prejudice, and discrimination. In: Gilbert, D.T., Fiske, S.T., Lindzey, G. (eds.) Handbook of Social Psychology, pp. 357–411. McGraw-Hill, Boston (1998)Google Scholar
- 35.Frias-Martinez, E., Williamson, G., Frias-Martinez, V.: An agent-based model of epidemic spread using human mobility and social network information. In: 2011 International Conference on Social Computing (SocialCom), pp. 57–64. IEEE, New York (2011)Google Scholar
- 36.Gillespie, T.: The relevance of algorithms. In: Gillespie, T., Boczkowski, P., Foot, K. (eds.) Media Technologies: Essays on Communication, Materiality, and Society, pp. 167–193. MIT Press, Cambridge (2014)Google Scholar
- 37.Ginsberg, J., Mohebbi, M.H., Patel, R.S., Brammer, L., Smolinski, M.S., Brilliant, L.: Detecting influenza epidemics using search engine query data. Nature 457, 1012–1014 (2009)CrossRefGoogle Scholar
- 38.Hajian, S., Bonchi, F., Castillo, C.: Algorithmic bias: from discrimination discovery to fairness-aware data mining. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2125–2126. ACM, New York (2016)Google Scholar
- 39.Jean, N., Burke, M., Xie, M., Davis, W.M., Lobell, D.B., Ermon, S.: Combining satellite imagery and machine learning to predict poverty. Science 353 (6301), 790–794 (2016)CrossRefGoogle Scholar
- 40.Jerven, M.: Poor Numbers: How We Are Misled by African Development Statistics and What to Do About It. Cornell University Press, Ithaca (2013)Google Scholar
- 41.King, G.: Ensuring the data-rich future of the social sciences. Science 331 (6018), 719–721 (2011)CrossRefGoogle Scholar
- 42.Kosinski, M., Stillwell, D., Graepel, T.: Private traits and attributes are predictable from digital records of human behavior. Proc. Natl. Acad. Sci. 110 (15), 5802–5805 (2013)CrossRefGoogle Scholar
- 43.Kuznets, S.: Economic growth and income inequality. Am. Econ. Rev. 45, 1–28 (1955)Google Scholar
- 44.Latzer, M., Hollnbuchner, K., Just, N., Saurwein, F.: The economics of algorithmic selection on the internet. In: Bauer, J., Latzer, M. (eds.) Handbook on the Economics of the Internet. Edward Elgar, Cheltenham (2015)Google Scholar
- 45.Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., Jebara, T., King, G., Macy, M., Roy, D., Van Alstyne, M.: Computational social science. Science 323 (5915), 721–723 (2009)CrossRefGoogle Scholar
- 46.Lepri, B., Staiano, J., Shmueli, E., Pianesi, F., Pentland, A.: The role of personality in shaping social networks and mediating behavioral change. User Model. User-Adap. Inter. 26 (2), 143–175 (2016)CrossRefGoogle Scholar
- 47.LiKamWa, R., Liu, Y., Lane, N.D., Zhong, L.: Moodscope: building a mood sensor from smartphone usage patterns. In: Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Service (MobiSys), pp. 389–402 (2013)Google Scholar
- 48.Liu, H.Y., Skjetne, E., Kobernus, M.: Mobile phone tracking: in support of modelling traffic-related air pollution contribution to individual exposure and its implications for public health impact assessment. Environ. Health 12, 93 (2013)CrossRefGoogle Scholar
- 49.Louail, T., Lenormand, M., Cantu Ros, O.G., Picornell, M., Herranz, R., Frias-Martinez, E., Ramasco, J.J., Barthelemy, M.: From mobile phone data to the spatial structure of cities. Sci. Rep. 4, 5276 (2014)CrossRefGoogle Scholar
- 50.Lu, X., Bengtsson, L., Holme, P.: Predictability of population displacement after the 2010 haiti earthquake. Proc. Natl. Acad. Sci. 109, 11576–11581 (2012)CrossRefGoogle Scholar
- 51.Major, B., O’Brien, L.T.: The social psychology of stigma. Annu. Rev. Psychol. 56, 393–421 (2005)CrossRefGoogle Scholar
- 52.Matic, A., Oliver, N.: The untapped opportunity of mobile network data for mental health. In: Future of Pervasive Health Workshop, vol. 6. ACM, New York (2016)Google Scholar
- 53.Mohler, G.O., Short, M.B., Brantingham, P.J., Schoenberg, F.P., Tita, G.E.: Self-exciting point process modeling of crime. J. Am. Stat. Assoc. 106, 100–108 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
- 54.Nakamoto, S.: Bitcoin: a peer-to-peer electronic cash system. Technical Report, Kent University (2009)Google Scholar
- 55.Ofli, F., Meier, P., Imran, M., Castillo, C., Tuia, D., Rey, N., Briant, J., Millet, P., Reinhard, F., Parkan, M., Joost, S.: Combining human computing and machine learning to make sense of big (aerial) data for disaster response. Big Data 4, 47–59 (2016)CrossRefGoogle Scholar
- 56.Ohm, P.: Broken promises of privacy: responding to the surprising failure of anonymization. UCLA Law Rev. 57, 1701–1777 (2010)Google Scholar
- 57.Oliver, N., Matic, A., Frias-Martinez, E.: Mobile network data for public health: opportunities and challenges. Front. Public Health 3, 189 (2015)CrossRefGoogle Scholar
- 58.O’Neil, C.: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, New York (2016).Google Scholar
- 59.Osmani, V., Gruenerbl, A., Bahle, G., Lukowicz, P., Haring, C., Mayora, O.: Smartphones in mental health: detecting depressive and manic episodes. IEEE Pervasive Comput. 14 (3), 10–13 (2015)CrossRefGoogle Scholar
- 60.Pager, D., Shepherd, H.: The sociology of discrimination: racial discrimination in employment, housing, credit and consumer market. Annu. Rev. Sociol. 34, 181–209 (2008)CrossRefGoogle Scholar
- 61.Pasquale, F.: The Black Blox Society: The Secret algorithms That Control Money and Information. Harvard University Press, Cambridge (2015)CrossRefGoogle Scholar
- 62.Pastor-Escuredo, D., Torres Fernandez, Y., Bauer, J.M., Wadhwa, A., Castro-Correa, C., Romanoff, L., Lee, J.G., Rutherford, A., Frias-Martinez, V., Oliver, N., Frias-Martinez, E., Luengo-Oroz, M.: Flooding through the lens of mobile phone activity. In: IEEE Global Humanitarian Technology Conference, GHTC’14. IEEE, New York (2014)Google Scholar
- 63.Pentland, A.: Society’s nervous system: building effective government, energy, and public health systems. IEEE Comput. 45 (1), 31–38 (2012)CrossRefGoogle Scholar
- 64.Perry, W.L., McInnis, B., Price, C.C., Smith, S.C., Hollywood, J.S.: Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations. Rand Corporation, Santa Monica (2013)Google Scholar
- 65.Podesta, J., Pritzker, P., Moniz, E.J., Holdren, J., Zients, J.: Big data: seizing opportunities, preserving values. Technical Report, Executive Office of the President (2014)Google Scholar
- 66.Ramirez, E., Brill, J., Ohlhausen, M.K., McSweeny, T.: Big data: a tool for inclusion or exclusion? Technical Report, Federal Trade Commission, January 2016Google Scholar
- 67.Ratcliffe, J.H.: A temporal constraint theory to explain opportunity-based spatial offending patterns. J. Res. Crime Delinq. 43 (3), 261–291 (2006)CrossRefGoogle Scholar
- 68.Ravallion, M.: The economics of poverty: history, measurement, and policy. Oxford University Press, Oxford (2016)CrossRefGoogle Scholar
- 69.Ribeiro, M.T., Singh, S., Guestrin, C.: “why should I trust you?”: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13–17, 2016, pp. 1135–1144 (2016)Google Scholar
- 70.Samuelson, W., Zeckhauser, R.: Status quo bias in decision making. J. Risk Uncertain. 1, 7–59 (1988)CrossRefGoogle Scholar
- 71.San Pedro, J., Proserpio, D., Oliver, N.: Mobiscore: towards universal credit scoring from mobile phone data. In: Proceedings of the International Conference on User Modeling, Adaptation and Personalization (UMAP), pp. 195–207 (2015)Google Scholar
- 72.Short, M.B., D’Orsogna, M.R., Pasour, V.B., Tita, G.E., Brantingham, P.J., Bertozzi, A.L., Chayes, L.B.: A statistical model of criminal behavior. Math. Models Methods Appl. Sci. 18 (supp01), 1249–1267 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
- 73.Singh, V.K., Freeman, L., Lepri, B., Pentland, A.: Predicting spending behavior using socio-mobile features. In: 2013 International Conference on Social Computing (SocialCom), pp. 174–179. IEEE, New York (2013)Google Scholar
- 74.Singh, V.K., Bozkaya, B., Pentland, A.: Money walks: implicit mobility behavior and financial well-being. PLoS One 10 (8), e0136628 (2015)CrossRefGoogle Scholar
- 75.Smith-Clarke, C., Mashhadi, A., Capra, L.: Poverty on the cheap: estimating poverty maps using aggregated mobile communication networks. In: Proceedings of the 32nd ACM Conference on Human Factors in Computing Systems (CHI2014) (2014)Google Scholar
- 76.Soto, V., Frias-Martinez, V., Virseda, J., Frias-Martinez, E.: Prediction of socioeconomic levels using cell phone records. In: Proceedings of the International Conference on UMAP, pp. 377–388 (2011)Google Scholar
- 77.Staiano, J., Oliver, N., Lepri, B., de Oliveira, R., Caraviello, M., Sebe, N.: Money walks: a human-centric study on the economics of personal mobile data. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 583–594. ACM, New York (2014)Google Scholar
- 78.Staiano, J., Zyskind, G., Lepri, B., Oliver, N., Pentland, A.: The rise of decentralized personal data markets. In: Shrier, D., Pentland, A. (eds.) Trust::Data: A New Framework for Identity and Data Sharing. CreateSpace Independent Publishing Platform (2016)Google Scholar
- 79.Sweeney, L.: Discrimination in online ad delivery. Available at SSRN: http://ssrn.com/abstract=2208240 (2013)
- 80.Szabo, N.: Formalizing and securing relationships on public networks. First Monday 2 (9) (1997)Google Scholar
- 81.Thomas, L.: Consumer Credit Models: Pricing, Profit, and Portfolios. Oxford University Press, New York (2009)CrossRefGoogle Scholar
- 82.Tizzoni, M., Bajardi, P., Decuyper, A., Kon Kam King, G., Schneider, C.M., Blondel, V., Smoreda, Z., Gonzalez, M.C., Colizza, V.: On the use of human mobility proxies for modeling epidemics. PLoS Comput. Biol. 10 (7) (2014)Google Scholar
- 83.Tobler, C.: Limits and potential of the concept of indirect discrimination. Technical Report, European Network of Legal Experts in Anti-Discrimination (2008)Google Scholar
- 84.Toole, J.L., Eagle, N., Plotkin, J.B.: Spatiotemporal correlations in criminal offense records. ACM Trans. Intell. Syst. Technol. 2 (4), 38:1–38:18 (2011)Google Scholar
- 85.Traunmueller, M., Quattrone, G., Capra, L.: Mining mobile phone data to investigate urban crime theories at scale. In: Proceedings of the International Conference on Social Informatics, pp. 396–411 (2014)Google Scholar
- 86.Tufekci, Z.: Algorithmic harms beyond Facebook and Google: emergent challenges of computational agency. Colorado Technol. Law J. 13, 203–218 (2015)Google Scholar
- 87.Tverksy, A., Kahnemann, D.: Judgment under uncertainty: heuristics and biases. Science 185 (4157), 1124–1131 (1974)CrossRefGoogle Scholar
- 88.Venerandi, A., Quattrone, G., Capra, L., Quercia, D., Saez-Trumper, D.: Measuring urban deprivation from user generated content. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW2015) (2015)Google Scholar
- 89.Vescovi, M., Perentis, C., Leonardi, C., Lepri, B., Moiso, C.: My data store: Toward user awareness and control on personal data. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 179–182 (2014)Google Scholar
- 90.Wang, T., Rudin, C., Wagner, D., Sevieri, R.: Learning to detect patterns of crime. In: Machine Learning and Knowledge Discovery in Databases, pp. 515–530. Springer, Berlin (2013)Google Scholar
- 91.Wang, H., Li, Z., Kifer, D., Graif, C.: Crime rate inference with big data. In: Proceedings of International Conference on KDD (2016)CrossRefGoogle Scholar
- 92.Want, R., Pering, T., Danneels, G., Kumar, M., Sundar, M., Light, J.: The personal server: changing the way we think about ubiquitous computing. In: Proceedings of 4th International Conference on Ubiquitous Computing, pp. 194–209 (2002)Google Scholar
- 93.Weisburd, D.: Place-based policing. Ideas Am. Policing 9, 1–16 (2008)Google Scholar
- 94.Wesolowski, A., Eagle, N., Tatem, A., Smith, D., Noor, R., Buckee, C.: Quantifying the impact of human mobility on malaria. Science 338 (6104), 267–270 (2012)CrossRefGoogle Scholar
- 95.Wesolowski, A., Stresman, G., Eagle, N., Stevenson, J., Owaga, C., Marube, E., Bousema, T., Drakeley, C., Cox, J., Buckee, C.O.: Quantifying travel behavior for infectious disease research: a comparison of data from surveys and mobile phones. Sci. Rep. 4 (2014)Google Scholar
- 96.Willson, M.: Algorithms (and the) everyday. Inf. Commun. Soc. 20, 137–150 (2017)CrossRefGoogle Scholar
- 97.Wilson, R., Erbach-Schoenengerg, E., Albert, M., Power, D., Tudge, S., Gonzalez, M., et al.: Rapid and near real-time assessments of population displacement using mobile phone data following disasters: the 2015 Nepal earthquake. PLoS Current Disasters, February 2016Google Scholar
- 98.Zang, H., Bolot, J.: Anonymization of location data does not work: a large-scale measurement study. In: Proceedings of 17th ACM Annual International Conference on Mobile Computing and Networking, pp. 145–156 (2011)Google Scholar
- 99.Zarsky, T.Z.: Automated prediction: Perception, law and policy. Commun. ACM 4, 167–186 (1989)Google Scholar
- 100.Zarsky, T.: The trouble with algorithmic decisions: an analytic road map to examine efficiency and fairness in automated and opaque decision making. Sci. Technol. Hum. Values 41 (1), 118–132 (2016)CrossRefGoogle Scholar
- 101.Zyskind, G., Nathan, O., Pentland, A.: Decentralizing privacy: using blockchain to protect personal data. In: Proceedings of IEEE Symposium on Security and Privacy Workshops, pp. 180–184 (2014)Google Scholar