Enchanted Determinism: Power without Responsibility in Artificial Intelligence

Alexander Campolo; Kate Crawford

doi:10.17351/ests2020.277

Alexander Campolo University of Chicago
Kate Crawford New York University, Microsoft Research

DOI: https://doi.org/10.17351/ests2020.277

Keywords: artificial intelligence, deep learning, enchantment, Max Weber, magic, classification

Abstract

Deep learning techniques are growing in popularity within the field of artificial intelligence (AI). These approaches identify patterns in large scale datasets, and make classifications and predictions, which have been celebrated as more accurate than those of humans. But for a number of reasons, including nonlinear path from inputs to outputs, there is a dearth of theory that can explain why deep learning techniques work so well at pattern detection and prediction. Claims about “superhuman” accuracy and insight, paired with the inability to fully explain how these results are produced, form a discourse about AI that we call enchanted determinism. To analyze enchanted determinism, we situate it within a broader epistemological diagnosis of modernity: Max Weber’s theory of disenchantment. Deep learning occupies an ambiguous position in this framework. On one hand, it represents a complex form of technological calculation and prediction, phenomena Weber associated with disenchantment. On the other hand, both deep learning experts and observers deploy enchanted, magical discourses to describe these systems’ uninterpretable mechanisms and counter-intuitive behavior. The combination of predictive accuracy and mysterious or unexplainable properties results in myth-making about deep learning’s transcendent, superhuman capacities, especially when it is applied in social settings. We analyze how discourses of magical deep learning produce techno-optimism, drawing on case studies from game-playing, adversarial examples, and attempts to infer sexual orientation from facial images. Enchantment shields the creators of these systems from accountability while its deterministic, calculative power intensifies social processes of classification and control.

Author Biographies

Alexander Campolo, University of Chicago

Alexander Campolo was a research assistant at the AI Now Institute where he worked on a range of critical and epistemological topics in AI. He is currently a postdoctoral researcher at the Stevanovich Institute on the Formation of Knowledge at the University of Chicago, where he continues to work at the intersection of data, knowledge, and politics.

Kate Crawford, New York University, Microsoft Research

Kate Crawford is a Distinguished Research Professor at New York University, where she co-founded the AI Now Institute. Her work over the last decade has centered on the social and political implications of large-scale data and machine learning. She is also a Principal Researcher at Microsoft Research, and the inaugural Visiting Chair in AI and Justice at the École Normale Supérieure in Paris.

References

Ames, Morgan G. 2014. “Translating Magic: The Charisma of One Laptop per Child’s XO Laptop in Paraguay.” In Beyond Imported Magic: Essays on Science, Technology, and Society in Latin America, edited by Eden Medina, Ivan da Costa Marques and Christina Holmes, 207–224. Cambridge, MA: MIT Press.

__________. 2015. “Charismatic Technology.” In Proceedings of The Fifth Decennial Aarhus Conference on Critical Alternatives, 109–120. AA ’15. Aarhus, Denmark: Aarhus University Press. https://doi.org/10.7146/aahcc.v1i1.21199.

__________. 2018. “Deconstructing the Algorithmic Sublime.” Big Data & Society 5 (1): 1–4. https://doi.org/10.1177/2053951718779194.

Anderson, Chris. 2008. “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.” Wired. June 23. https://www.wired.com/2008/06/pb-theory/.

Arora, Sanjeev. 2019. “Brief Introduction to Deep Learning and the ‘Alchemy’ Controversy.” presented at Deep Learning: Alchemy or Science?, Institute of Advanced Study, Princeton University, February 22. https://www.youtube.com/watch?v=kqhg-o-KEns.

Asad, Talal. 2003. Formations of the Secular: Christianity, Islam, Modernity. Stanford: Stanford University Press.

Beck, Ulrich. 1992. Risk Society: Towards a New Modernity, translated by Mark Ritter. London: Sage.

Bellman, Richard. 1957. Dynamic Programming. Princeton: Princeton University Press.

Bennett, Jane. 2001. The Enchantment of Modern Life: Attachments, Crossings, and Ethics. Princeton: Princeton University Press.

Benthall, Sebastian, and Bruce D. Haynes. 2019. “Racial Categories in Machine Learning.” In FAT* ’19: Conference on Fairness, Accountability, and Transparency, 10. Atlanta, GA: ACM. https://doi.org/10.1145/3287560.3287575.

Borowiec, Steven. 2016. “AlphaGo Seals 4-1 Victory over Go Grandmaster Lee Sedol.” The Guardian, March 15. https://www.theguardian.com/technology/2016/mar/15/googles-alphago-seals-4-1-victory-over-grandmaster-lee-sedol.

Bowker, Geoffrey C., and Susan Leigh Star. 1999. Sorting Things Out: Classification and Its Consequences. Cambridge, MA: MIT Press.

Browne, Simone. 2015. Dark Matters: On the Surveillance of Blackness. Durham: Duke University Press.

Burrell, Jenna. 2016. “How the Machine ‘Thinks’: Understanding Opacity in Machine Learning Algorithms.” Big Data & Society 3 (1): 1–12. https://doi.org/10.1177/2053951715622512.

Citron, Danielle Keats, and Frank A. Pasquale. 2014. “The Scored Society: Due Process for Automated Predictions.” Washington Law Review 89 (1): 1–33.

Collins, Harry. 2010. Tacit and Explicit Knowledge. Chicago: University of Chicago Press.

Crosman, Penny. 2017. “Is AI Making Credit Scores Better, or More Confusing?” American Banker. February 14. https://www.americanbanker.com/news/is-ai-making-credit-scores-better-or-more-confusing.

Domingos, Pedro. 2012. “A Few Useful Things to Know About Machine Learning.” Communications of the ACM 55 (10): 78–87. https://doi.org/10.1145/2347736.2347755.

Dreyfus, Hubert L. 1965. “Alchemy and Artificial Intelligence.” Santa Monica: RAND. http://www.rand.org/pubs/papers/P3244.html.

Edwards, Helen, and Dave Edwards. 2018. “Google’s Engineers Say That ‘Magic Spells’ Are Ruining AI Research.” Quartz. October 5. https://qz.com/1274131/googles-engineers-say-that-lack-of-rigor-is-ruining-ai-research/.

Espeland, Wendy Nelson, and Michael Sauder. 2007. “Rankings and Reactivity: How Public Measures Recreate Social Worlds.” American Journal of Sociology 113 (1): 1–40.

Eubanks, Virginia. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin’s Press.

Ford, Martin. 2018. Architects of Intelligence: The Truth about AI from the People Building It. Birmingham, UK: Packt Publishing.

Foucault, Michel. 1972. The Archaeology of Knowledge. Translated by A.M. Sheridan Smith. New York: Pantheon.

Galison, Peter. 1994. “The Ontology of the Enemy: Norbert Wiener and the Cybernetic Vision.” Critical Inquiry 21 (1): 228–66.

Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. 2014. “Explaining and Harnessing Adversarial Examples.” ArXiv:1412.6572 [Cs, Stat], December. http://arxiv.org/abs/1412.6572.

__________, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cambridge, MA: MIT Press.

Greenberger, Martin. 1962. Management and the Computer of the Future. New York: Wiley.

Guidotti, R., Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca

Giannotti, and Dino Pedreschi. 2019. “A Survey of Methods for Explaining Black Box Models.” ACM Computing Surveys 51 (5): 1–42. https://doi.org/10.1145/3236009.

Hacking, Ian. 1990. The Taming of Chance Cambridge: Cambridge University Press, 1990.

Hu, Lily, Nicole Immorlica, and Jennifer Wortman Vaughan. 2019. “The Disparate Effects of Strategic Manipulation.” In Proceedings of the Conference on Fairness, Accountability, and Transparency, 259–268. FAT* ’19. New York, NY, USA: ACM. https://doi.org/10.1145/3287560.3287597.

Hutson, Matthew. 2018. “AI Researchers Allege That Machine Learning Is Alchemy.” Science. May 3. http://www.sciencemag.org/news/2018/05/ai-researchers-allege-machine-learning-alchemy.

Jannini, Emmanuele A., Ray Blanchard, Andrea Camperio-Ciani, and John Bancroft. 2010. “Male Homosexuality: Nature or Culture?” The Journal of Sexual Medicine 7 (10): 3245–53.

Josephson-Storm, Jason A. 2017. The Myth of Disenchantment: Magic, Modernity, and the Birth of the Human Sciences. Chicago: University of Chicago Press.

Kleinberg, Jon, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. “Human Decisions and Machine Predictions.” The Quarterly Journal of Economics 133 (1): 237–93. https://doi.org/10.1093/qje/qjx032.

Knight, Will. 2017. “Alpha Zero’s ‘Alien’ Chess Shows the Power, and the Peculiarity, of AI.” MIT Technology Review. December 8. https://www.technologyreview.com/s/609736/alpha-zeros-alien-chess-shows-the-power-and-the-peculiarity-of-ai/.

Latour, Bruno. 1993. We Have Never Been Modern, translated by Catherine Porter. Cambridge, MA: Harvard University Press.

LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521 (7553): 436–44. https://doi.org/10.1038/nature14539.

Lipton, Zachary. C. 2018. “The Mythos of Model Interpretability.” Queue 16 (3): 30:31–30:57. https://doi.org/10.1145/3236386.3241340.

Lu, Zhou, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei

Wang. 2017. “The Expressive Power of Neural Networks: A View from the Width.” ArXiv:1709.02540 [Cs], September. http://arxiv.org/abs/1709.02540.

Metz, Cade. 2016. “In Two Moves, AlphaGo and Lee Sedol Redefined the Future.” Wired. March 16. https://www.wired.com/2016/03/two-moves-alphago-lee-sedol-redefined-future/.

Miller, Alex P. 2018. “Want Less-Biased Decisions? Use Algorithms.” Harvard Business Review. July 26. https://hbr.org/2018/07/want-less-biased-decisions-use-algorithms.

Noble, Safiya Umoja. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: NYU Press.

Principe, Lawrence M., and William R. Newman. 2001. “Some Problems with the Historiography of Alchemy.” In Secrets of Nature: Astrology and Alchemy in Early Modern Europe, 385–431. Cambridge, MA: MIT Press.

Raghu, Maithra, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. 2017. “On the Expressive Power of Deep Neural Networks.” In International Conference on Machine Learning, 2847–54. http://proceedings.mlr.press/v70/raghu17a.html.

Riesebrodt, Martin. 1999. “Charisma in Max Weber’s Sociology of Religion.” Religion 29 (1): 1–14. https://doi.org/10.1006/reli.1999.0175.

Scott, Joan. 2017. Sex and Secularism. Princeton: Princeton University Press.

Selbst, Andrew, and Solon Barocas. 2018. “The Intuitive Appeal of Explainable Machines.” Fordham Law Review 87 (3): 1085.

Silver, David. Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, et al. 2017. “Mastering the Game of Go without Human Knowledge.” Nature 550 (7676): 354. https://doi.org/10.1038/nature24270.

Sweeney Annie, and Paige Fry. 2018. “Nearly 33,000 Juveniles Arrested over Last Two Decades Labeled as Gang Members by Chicago Police.” Chicago Tribune, August 9. https://www.chicagotribune.com/news/local/breaking/ct-met-chicago-police-gang-database-juveniles-20180725-story.html.

Szegedy, Christian, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. “Intriguing Properties of Neural Networks.” ArXiv:1312.6199 [Cs], December. https://arxiv.org/abs/1312.6199.

Taylor, Charles. 2007. A Secular Age. Cambridge, MA: Belknap Press.

Udry, J. Richard, and Kim Chantala. 2006. “Masculinity-Femininity Predicts Sexual Orientation in Men but Not in Women.” Journal of Biosocial Science 38 (6): 797–809. https://doi.org/10.1017/S002193200500101X.

Vincent, James. 2017. “Magic AI: These Are the Optical Illusions That

Trick, Fool, and Flummox Computers.” The Verge. April 12. https://www.theverge.com/2017/4/12/15271874/ai-adversarial-images-fooling-attacks-artificial-intelligence.

Weber, Max. 1946. “Science as a Vocation.” In From Max Weber: Essays in Sociology, translated by H.H. Gerth and C.W. Mills, 129–56. New York: Oxford University Press.

__________. 1992. The Protestant Ethic and the Spirit of Capitalism, translated by Talcott Parsons. London: Routledge.

Winkelman, Michael. 1982. “Magic: A Theoretical Reassessment.” Current Anthropology 23 (1): 37–66.

Zeng, Jiaming, Berk Ustun, and Cynthia. Rudin. 2017. “Interpretable Classification Models for Recidivism Prediction.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 180 (3): 689–722. https://doi.org/10.1111/rssa.12227.