Refusal in Data Ethics: Re-Imagining the Code Beneath the Code of Computation in the Carceral State

  • Chelsea Barabas Massachusetts Institute of Technology

Abstract

In spite of a growing interest in ethical approaches to computation, engineers and quantitative researchers are often not equipped with the conceptual tools necessary to interrogate, resist, and reimagine the relationships of power which shape their work. A liberatory vision of computation requires de-centering the data in “data ethics” in favor of cultivating an ethics of encounter that foregrounds the ways computation reproduces structures of domination. This article draws from a rich body of feminist scholarship that explores the liberatory potential of refusal as a practice of generative boundary setting. To refuse is to say no—to reject the default categories, assumptions and problem formulations which so often underpin data-intensive work. But refusal is more than just saying no; it can be a generative and strategic act, one which opens up space to renegotiate the assumptions underlying sociotechnical endeavors. This article explores two complementary modalities of refusal in computation: “refusal as resistance” and “refusal as re-centering the margins.” By exploring these two modes of refusal, the goal of this paper is to provide a vocabulary for identifying and rejecting the ways that sociotechnical systems reinforce dependency on oppressive structural conditions, as well as offer a framework for flexible collective experimentation towards more free futures.

References

Ahmed, Sara. 2017. “Institutional As Usual.” feministkilljoys, October 24, 2017. Accessed July 11, 2021.

https://feministkilljoys.com/2017/10/24/institutional-as-usual/.

Amrute, Sareeta. 2019. “Of Techno-Ethics and Techno-Affects.” Feminist Review 123(1): 56–73.

https://doi.org/10.1177/0141778919879744.

Bales, William, Karen Mann, Thomas Blomberg, Gerry Gaes, et al. 2010. “A Quantitative and Qualitative Assessment of Electronic Monitoring.” US Department of Justice Report. Bibliogov, 208. Accessed August 24, 2022. https://www.ojp.gov/pdffiles1/nij/grants/230530.pdf.

Ball, Molly. 2015. “Do the Koch Brothers Really Care About Criminal-Justice Reform?” The Atlantic, March 3, 2015. Accessed July 12, 2021.

https://www.theatlantic.com/politics/archive/2015/03/do-the-koch-brothers-really-care-about-criminal-justice-reform/386615/.

Barabas, Chelsea. 2020a. “Beyond Bias: Re-Imagining the Terms of ‘Ethical Al’ in Criminal Law.” Georgetown Journal of Law and Modern Critical Race Theory 12(2): 83–112.

http://doi.org/10.2139/ssrn.3377921.

⸻. 2020b. “To Build a Better Future, Designers Need to Start Saying ‘No.’” Medium, October 20, 2020. Accessed July 12, 2021.

https://onezero.medium.com/refusal-a-beginning-that-starts-with-an-end-2b055bfc14be.

Barabas, Chelsea, Audrey Beard, Theodora Dryer, Beth Semel, et al. 2020a. “Abolish the #TechToPrisonPipeline.” Coalition for Critical Technology, June 23, 2020. Accessed July 8, 2021.

https://medium.com/@CoalitionForCriticalTechnology/abolish-the-techtoprisonpipeline-9b5b14366b16.

Barabas, Chelsea, Colin Doyle, J. B. Rubinovitz, and Karthik Dinakar. 2020b. “Studying Up: Reorienting the Study of Algorithmic Fairness around Issues of Power.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 167–176.

https://doi.org/10.1145/3351095.3372859.

Benjamin, Ruha. 2016a. “Racial Fictions, Biological Facts: Expanding the Sociological Imagination through Speculative Methods.” Catalyst: Feminism, Theory, Technoscience 2(2): 1–28.

https://doi.org/10.28968/cftt.v2i2.28798.

⸻. 2016b. “Informed Refusal: Toward a Justice-Based Bioethics.” Science, Technology, & Human Values 41(6): 967–990.

https://doi.org/10.1177/0162243916656059.

Benjamin, Ruha, ed. 2019a. Captivating Technology: Race, Carceral Technoscience, and Liberatory Imagination in Everyday Life. Durham, NC: Duke University Press.

https://doi.org/10.1215/9781478004493.

⸻. 2019b. Race after Technology: Abolitionist Tools for the New Jim Code. Medford, MA: Polity.

Bergman, Carla, and Nick Montgomery. 2017. Joyful Militancy: Building Thriving Resistance in Toxic Times. Chico, Oakland, Edinburgh, Baltimore: AK Press.

Brayne, Sarah. 2014. “Surveillance and System Avoidance: Criminal Justice Contact and Institutional Attachment.” American Sociological Review 79(3): 367–391.

https://doi.org/10.1177/0003122414530398.

Cech, Erin A. 2013. “The (Mis)Framing of Social Justice: Why Ideologies of Depoliticization and Meritocracy Hinder Engineers’ Ability to Think about Social Injustices.” In Engineering Education for Social Justice: Critical Explorations and Opportunities, edited by Juan Lucena, 67–84. Dordrecht: Springer Netherlands.

https://doi.org/10.1007/978-94-007-6350-0_4.

Cech, Erin A., and Heidi M. Sherick. 2015. “Depoliticization and the Structure of Engineering Education.” In International Perspectives on Engineering Education. Engineering Education and Practice in Context, Volume 1, edited by Steen H. Christensen., Christelle Didier, Andrew Jamison, Martin Meganck, et al., 203–216. Cham: Springer.

https://doi.org/10.1007/978-3-319-16169-3_10.

Chouldechova, Alexandra. 2017. “Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments.” Big Data 5(2): 153–163.

https://doi.org/10.1089/big.2016.0047.

Christin, Angèle. 2017. “Algorithms in Practice: Comparing Web Journalism and Criminal Justice.” Big Data & Society 4(2): 1–14.

https://doi.org/10.1177/2053951717718855.

Cifor, Marika, Patricia Garcia, T. L. Cowan, Jasmine Rault, et al. 2019. “Feminist Data Manifest-No.” Accessed July 7, 2021.

https://www.manifestno.com/home.

Comey, James. 2015. “Law Enforcement and Race Relations.” C-SPAN, February 12, 2015. Accessed July 11, 2021.

https://www.c-span.org/video/?324342-1/fbi-director-james-comey-law-enforcement-race-relations.

CourtWatch MA. 2018. “First 100 Days.” CourtWatch MA: Blog. Accessed July 12, 2021.

https://www.courtwatchma.org/first-100-days.html.

Crenshaw, Kimberlé W. 2012. “From Private Violence to Mass Incarceration: Thinking Intersectionally About Women, Race, and Social Control.” UCLA Law Review 59(6): 1418–1472. Accessed August 15, 2022.

https://www.uclalawreview.org/pdf/59-6-1.pdf.

D’Ignazio, Catherine, and Lauren F. Klein. 2020. Data Feminism. Cambridge, MA: The MIT Press.

https://doi.org/10.7551/mitpress/11805.001.0001.

Dolovich, Sharon. 2011. “Exclusion and Control in the Carceral State.” Berkeley Journal of Criminal Law 16(2): 259–339.

https://doi.org/10.15779/Z383G8P.

Dutta, Mohan, and Mahuya Pal. 2010. “Dialog Theory in Marginalized Settings: A Subaltern Studies Approach.” Communication Theory 20(4): 363–386.

https://doi.org/10.1111/j.1468-2885.2010.01367.x.

Eubanks, Virginia. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin’s Press.

Friedman, Brittany. 2021. “Toward a Critical Race Theory of Prison Order in the Wake of Covid-19 and Its Afterlives: When Disaster Collides with Institutional Death by Design.” Sociological Perspectives 64(5): 689–705.

https://doi.org/10.1177/07311214211005485.

Fryer, Jr Roland G. 2016. “An Empirical Analysis of Racial Differences in Police Use of Force.” National Bureau of Economic Research Working Paper Series 1–55.

http://doi.org/10.3386/w22399.

Gangadharan, Seeta P. 2020. “Context, Research, Refusal: Perspectives on Abstract Problem-Solving.” Our Data Bodies: Blog. April 30, 2020. Accessed July 8, 2021.

https://www.odbproject.org/2020/04/30/context-research-refusal-perspectives-on-abstract-problem-solving/.

Gilmore, Ruth W. 2002. “Fatal Couplings of Power and Difference: Notes on Racism and Geography.” The Professional Geographer 54(1): 15–24.

https://doi.org/10.1111/0033-0124.00310.

⸻. 2007. Golden Gulag: Prisons, Surplus, Crisis, and Opposition in Globalizing California. American Crossroads. Berkeley: University of California Press.

⸻. 2014. Foreword to The Struggle within: Prisons, Political Prisoners, and Mass Movements in the United States, by Dan Berger. Oakland, CA: PM Press.

⸻. 2017. “Abolition Geography and the Problem of Innocence.” In Futures of Black Radicalism, edited by Gaye T. Johnson and Alex Lubin, 224–241. Brooklyn: Verso.

Graeff, Erhardt. 2020. “The Responsibility to Not Design and the Need for Citizen Professionalism.” Tech Otherwise 1–5. May 25, 2020.

https://doi.org/10.21428/93b2c832.c8387014.

Graziani, Terra, and Mary Shi. 2020. “Data for Justice: Tensions and Lessons from the Anti-Eviction Mapping Project’s Work Between Academia and Activism.” ACME: An International Journal for Critical Geographies 19(1): 397–412.

https://acme-journal.org/index.php/acme/article/view/1776.

Harris, David A. 2003. “The Reality of Racial Disparity in Criminal Justice: The Significance of Data Collection.” Law and Contemporary Problems 66(3): 71–98.

https://scholarship.law.duke.edu/lcp/vol66/iss3/4.

Hartman, Saidiya V. 1997. Scenes of Subjection: Terror, Slavery, and Self-Making in Nineteenth-Century America. New York and Oxford: Oxford University Press.

Hoffmann, Anna L. 2020. “Terms of Inclusion: Data, Discourse, Violence.” New Media & Society 23(12): 3539–3556.

https://doi.org/10.1177/1461444820958725.

⸻. 2021. “Even When You Are a Solution You Are a Problem: An Uncomfortable Reflection on Feminist Data Ethics.” Global Perspectives 2(1):

https://doi.org/10.1525/gp.2021.21335.

Honig, Bonnie. 2021. A Feminist Theory of Refusal. Cambridge, MA: Harvard University Press.

https://doi.org/10.4159/9780674259249.

hooks, bell. 1989. “Choosing the Margin as a Space of Radical Openness.” Framework: The Journal of Cinema and Media 36: 15–23.

https://www.jstor.org/stable/44111660.

⸻. 1990. “Marginality as a Site of Resistance.” In Out There: Marginalization and Contemporary Cultures, edited by Russell Ferguson and Trinh T. Minh-ha, 241–243. Cambridge, MA: MIT Press.

Hope, Alexis, Catherine D’Ignazio, Josephine Hoy, Rebecca Michelson, et al. 2019. “Hackathons as Participatory Design: Iterating Feminist Utopias.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–14. Glasgow: Association for Computing Machinery (ACM).

https://doi.org/10.1145/3290605.3300291.

Hu, Lily. 2021. “Race, Policing, and the Limits of Social Science.” Boston Review, May 6, 2021. Accessed July 9, 2021.

http://bostonreview.net/science-nature-race/lily-hu-race-policing-and-limits-social-science.

Irani, Lilly, and Khalid Alexander. 2021. “The Oversight Bloc.” Logic Magazine, December 25, 2021. Accessed August 24, 2022.

https://logicmag.io/beacons/the-oversight-bloc/.

Irani, Lilly, Janet Vertesi, Paul Dourish, Kavita Philip, et al. 2010. “Postcolonial Computing: A Lens on Design and Development.” Proceedings of the 28th International Conference on Human Factors in Computing Systems—CHI ’10, 1311–1320. Atlanta: ACM Press.

https://doi.org/10.1145/1753326.1753522.

Kaba, Mariame, ed. 2020. “What’s Next? Safer and More Just Communities Without Policing.” Interrupting Criminalization: Research in Action. Project NIA. Accessed July 12, 2021.

https://view.publitas.com/interrupting-criminalization-byekyy37zyrk/whats-next-safer-and-more-just-communities-without-policing/page/1.

⸻. 2021. We Do This ’Til We Free Us: Abolitionist Organizing and Transforming Justice. Chicago: Haymarket Books.

Kahn, Natalie L., and Simon J. Levien. 2021. “SEAS Cancels Class on Controversial Policing Strategy After Student Petition.” The Harvard Crimson, January 26, 2021. Accessed March 1, 2022.

https://www.thecrimson.com/article/2021/1/26/seas-cancels-policing-course/.

Katz, Yarden. 2020. Artificial Whiteness: Politics and Ideology in Artificial Intelligence. New York: Columbia University Press.

Kilgore, James. 2017. “Electronic Monitoring: A Survey of the Research for Decarceration Activists.” Challenging E-Carceration, July 2017. Report. Accessed August 15, 2022.

http://www.realcostofprisons.org/writing/kilgore-survey-of-em-research.pdf.

Kleinberg, Jon, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018. “Algorithmic Fairness.” AEA Papers and Proceedings, 108: 22–27.

https://doi.org/10.1257/pandp.20181018.

Kubiak, Sheryl P., Woo Jong Kim, Gina Fedock, and Deborah Bybee. 2015. “Testing a Violence-Prevention Intervention for Incarcerated Women Using a Randomized Control Trial.” Research on Social Work Practice 25(3): 334–348.

https://doi.org/10.1177/1049731514534300.

Lacy, Akela, Alice Speri, Jordan Smith, and Sam Biddle. 2020. “Prisons Launch ‘Absurd’ Attempt to Detect Coronavirus in Inmate Phone Calls.” The Intercept, April 21, 2020. Accessed July 12, 2021.

https://theintercept.com/2020/04/21/prisons-inmates-coronavirus-monitoring-surveillance-verus/.

LEO Technologies. 2020a. “LEO Technologies and Verus: Supporting Our Nation’s Correctional Facilities During the Covid-19 Pandemic.” LEO Technologies: Blog. March 19, 2020. Accessed July 12, 2021.

https://leotechnologies.com/leo-technologies-and-verus-supporting-our-nations-correctional-facilities-during-the-covid-19-pandemic/.

⸻. 2020b. “What Is LEO Technologies?” LEO Technologies: Blog. June 29, 2020. Accessed July 12, 2021.

https://leotechnologies.com/what-is-leo-technologies/.

⸻. n.d. “How Verus Works.” Accessed July 12, 2021.

https://leotechnologies.com/services/verus/.

Lorde, Audre. 1984. Sister Outsider: Essays and Speeches. Berkeley: Crossing Press.

McGranahan, Carole. 2016. “Theorizing Refusal: An Introduction.” Cultural Anthropology 31(3): 319–325.

https://doi.org/10.14506/ca31.3.01.

Media Justice. 2021. Points of Connection: Mapping Electronic Monitoring to Challenge E-Carceration. Virtual event held on March 23, 2021. Video, 1:35:21. Accessed July 12, 2021.

https://www.youtube.com/watch?v=w_j-r8xqIZM.

Melamed, Jodi. 2019. “Operationalizing Racial Capitalism: Administrative Power and Ordinary Violence.” Talk at Yale University, filmed on October 31, 2019. Video, 1:35:17. Accessed July 12, 2021.

https://www.youtube.com/watch?v=o3Z9sOGf6BA&t=2675s.

Muhammad, Khalil G. 2015. “The Condemnation of Blackness” Khalil Gibran Muhammad Book Talk, John Jay College of Criminal Justice, filmed on May 6 2015. Video, 1:43:20. Accessed July 11, 2021.

https://www.youtube.com/watch?v=STKb-ai6874.

⸻. 2019. The Condemnation of Blackness: Race, Crime, and the Making of Modern Urban America. Cambridge, MA: Harvard University Press.

Murakawa, Naomi. 2014. The First Civil Right: How Liberals Built Prison America. New York and London: Oxford University Press.

Nagar, Richa, and Roozbeh Shirazi. 2019. “Radical Vulnerability.” In Keywords in Radical Geography: Antipode at 50, edited by Antipode Editorial Collective, Tariq Jazeel, Andy Kent, Katherine McKittrick, et al., 236–242. Hoboken, NJ: John Wiley & Sons, Inc.

https://doi.org/10.1002/9781119558071.ch44.

Naples-Mitchell, Katherine. 2021. “Fool’s Gold: How RCT Research Harms Communities Impacted by Criminal Punishment.” Charles Hamilton Houston Institute for Race & Justice, January 26, 2021. Accessed July 11, 2021.

https://charleshamiltonhouston.org/news/2021/01/fools-gold-how-rct-research-harms-communities-impacted-by-criminal-punishment/.

Norris, Samuel, Matthew Pecenco, and Jeffrey Weaver. 2021. “The Effects of Parental and Sibling Incarceration: Evidence from Ohio.” American Economic Review 111(9): 2926–2963.

https://doi.org/10.1257/aer.20190415.

North, Sandy. 2020. “New Study! Evaluating Counsel at First Appearance in Hays County, TX.” The Access to Justice Lab. July 7, 2020. Accessed July 11, 2021.

https://a2jlab.org/new-study-evaluating-counsel-at-first-appearance-in-hays-county-tx/.

Onuoha, Mimi. 2020. “When Proof Is Not Enough: Throughout History, Evidence of Racism has failed to Effect Change.” FiveThirtyEight: Blog. July 1, 2020. Accessed July 9, 2021.

https://fivethirtyeight.com/features/when-proof-is-not-enough/.

Prins, Seth J., and Adam Reich. 2018. “Can We Avoid Reductionism in Risk Reduction?” Theoretical Criminology 22(2): 258–278.

https://doi.org/10.1177%2F1362480617707948.

Raji, Inioluwa D., Morgan K. Scheuerman, and Razvan Amironesei. 2021. “You Can’t Sit With Us: Exclusionary Pedagogy in AI Ethics Education.” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 515–525. Virtual Event Canada: Association for Computing Machinery (ACM).

https://doi.org/10.1145/3442188.3445914.

Roberts, Dorothy E. 2019. “Digitizing the Carceral State.” Harvard Law Review 132: 1695–1713. Accessed August 24, 2022.

https://harvardlawreview.org/2019/04/digitizing-the-carceral-state/.

Saloner, Brendan, Kalind Parish, Julie A. Ward, Grace DiLaura, et al. 2020. “Covid-19 Cases and Deaths in Federal and State Prisons.” Journal of the American Medical Association 324(6): 602–603.

https://doi.org/10.1001/jama.2020.12528.

Schneider, Avie, and Laura Sydell. 2019. “Microsoft Workers Protest Army Contract With Tech ‘Designed To Help People Kill.’” National Public Radio, business section, February 22, 2019. Accessed March 1, 2022.

https://www.npr.org/2019/02/22/697110641/microsoft-workers-protest-army-contract-with-tech-designed-to-help-people-kill.

Schnepel, Kevin T. 2020. “Covid-19 in U.S. State and Federal Prisons.” December 2020 Update. Washington DC: Council on Criminal Justice, December 2020. Accessed August 24, 2022.

https://build.neoninspire.com/counciloncj/wp-content/uploads/sites/96/2021/07/COVID-19-in-State-and-Federal-Prisons-December-Update-2.pdf.

Schrader, Stuart. 2019. Badges without Borders: How Global Counterinsurgency Transformed American Policing. 56, American Crossroads Series. Oakland, CA: University of California Press.

Shah, Nishant. 2015. “Networked Margins: Revisiting Inequality and Intersection.” In Digitally Connected: Global Perspectives on Youth and Digital Media, edited by Santa Cortesi and Urs Gasser, Gameli Adzaho, Bruce Baikie, et al., 9–12. Berkman Center Research Publication Number 2015–6. Accessed March 1, 2022.

http://www.ssrn.com/abstract=2585686.

Simpson, Audra. 2014. Mohawk Interruptus: Political Life across the Borders of Settler States. Durham, NC: Duke University Press.

Spade, Dean. 2015. Normal Life: Administrative Violence, Critical Trans Politics, and the Limits of Law. Revised and Expanded edition. Durham, NC: Duke University Press.

Stanford Computational Policy Lab. n.d. “Driving Social Impact through Technical Innovation.” Stanford Computational Policy Lab. Accessed April 2, 2019.

https://policylab.stanford.edu/.

Stevenson, Megan T. 2017. “Assessing Risk Assessment in Action.” Minnesota Law Review 103: 303–71.

https://doi.org/10.2139/ssrn.3016088.

Stoller, Kristin. 2019. “Texas Billionaire John Arnold Gives $39 Million To Reform America’s Broken Bail System.” Forbes, March 19, 2019. Accessed July 12, 2021.

https://www.forbes.com/sites/kristinstoller/2019/03/19/texas-billionaire-john-arnold-gives-39-million-to-reform-americas-broken-bail-system/.

Stop LAPD Spying Coalition. 2018. “Dismantling Predictive Policing in Los Angeles.” May 8, 2018. Accessed July 11, 2021.

https://stoplapdspying.org/wp-content/uploads/2018/05/Before-the-Bullet-Hits-the-Body-May-8-2018.pdf.

Sutherland, Tonia. 2019. “The Carceral Archive: Documentary Records, Narrative Construction, and Predictive Risk Assessment.” Journal of Cultural Analytics 4(1): 1–22.

https://doi.org/10.22148/16.039.

Thompson, Jonathan. 2020. “National Sheriffs’ Association Teams with LEO Technologies on Covid-19 Industry Action Group for Correctional Facilities.” National Sheriff’s Association, April 14, 2020. Accessed July 12, 2021.

https://www.sheriffs.org/National-Sheriffs%E2%80%99-Association-Teams-LEO-Technologies-COVID-19-Industry-Action-Group-for.

Tuck, Eve, and K. Wayne Yang. 2014a. “R-Words: Refusing Research.” In Humanizing Research: Decolonizing Qualitative Inquiry with Youth and Communities, edited by Django Paris and Maisha T. Winn. Thousand Oaks: SAGE Publications, Inc.

https://dx.doi.org/10.4135/9781544329611.n12.

⸻. 2014b. “Unbecoming Claims: Pedagogies of Refusal in Qualitative Research.” Qualitative Inquiry 20(6): 811–818.

https://doi.org/10.1177/1077800414530265.

Wakabayashi, Daisuke, and Scott Shane. 2018. “Google Will Not Renew Pentagon Contract That Upset Employees.” The New York Times, June 1, 2018. Technology Section. Accessed March 1, 2022.

https://www.nytimes.com/2018/06/01/technology/google-pentagon-project-maven.html.

Wang, Jackie. 2018. Carceral Capitalism. South Pasadena: Semiotext(e).

Weld, Kirsten. 2014. Paper Cadavers: The Archives of Dictatorship in Guatemala. American Encounters/Global Interactions Series. Durham, NC: Duke University Press.

Wright, Sarah. 2018. “When Dialogue Means Refusal.” Dialogues in Human Geography 8(2): 128–32.

https://doi.org/10.1177/2043820618780570.

Zahara, Alex. 2016. “Ethnographic Refusal: A How to Guide.” Discard Studies, August 8, 2016. Accessed July 11, 2021.

https://discardstudies.com/2016/08/08/ethnographic-refusal-a-how-to-guide/.

Zong, Jonathan. 2020. “From Individual Consent to Collective Refusal: Changing Attitudes toward (Mis)Use of Personal Data.” XRDS: Crossroads, The ACM Magazine for Students 27(2): 26–29.

https://doi.org/10.1145/3433140.

Zong, Jonathan, and J. Nathan Matias. 2020. “Building Collective Power to Refuse Harmful Data Systems.” Citizens and Technology Lab: Blog. August 12, 2020. Accessed July 8, 2021.

https://citizensandtech.org/2020/08/collective-refusal/.

Published
14 Sep 2022
Section
Original Research Articles