“Let the algorithm decide”: is human dignity at stake?

Marcela Mattiuzzo

Résumé


The goal of this article is to argue that the debate regarding algorithmic decision-making and its impact on fundamental rights is not well-addressed and should be reframed in order to allow for adequate regulatory policies regarding recent technological developments in automation. A review of the literature on algorithms and an analysis of Articles 6, IX and 20 of the Brazilian Federal Law n° 13.709/2018 (LGPD) lead to the conclusion that claims that algorithmic decisions are unlawful because of profiling or because they replace human analysis are imprecise and do not identify the real issues at hand. Profiles are nothing more than generalizations, largely accepted in legal systems, and there are many kinds of decisions based on generalizations which algorithms can adequately make with no human intervention. In this context, this article restates the debate about automated decisions and fundamental rights focusing on two main obstacles: (i) the potential for discrimination by algorithmic systems and (ii) accountability of their decision-making processes. Lastly, the arguments put forward are applied to the current case of the covid-19 pandemic to illustrate the challenges ahead.

Mots-clés


Algorithms; Automated decisions; Decision-making; Human rights; Fundamental rights; Human dignity

Texte intégral :

PDF (English)

Références


AGRAWAL, A.; GANS, J. and GOLDFARB, A. Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Press, April 17, 2018.

AL-ARSHANI, S. The healthcare system in Italy's Lombardy region is so strained from the new coronavirus that officials are asking doctors to come out of retirement and nursing students are being fast-tracked to graduation. Business Insider. March 3, 2020. Available at: . Access: May 3, 2020.

ANGWIN, J.; LARSON, J.; MATTU, S.; KIRCHNER, L. Machine Bias. ProPublica, May 23, 2016. Available at: . Access: May 1, 2020.

ARKIN, R.; ULAM, P. and WAGNER, A. Moral Decision-making in Autonomous Systems: Enforcement, Moral Emotions, Dignity, Trust and Deception. Proceedings of the IEEE. Volume 100, Issue 3, March 2012. p. 571-589.

ASARO, P. On banning autonomous weapon systems: human rights, automation, and the dehumanization of lethal decision-making. International Review of the Red Cross. Volume 94, Issue 886, June 2012, p. 687­709. Available at: . Access: May 1, 2020.

BARKER, A. Those with disabilities have a right to survive the coronavirus pandemic. The Boston Globe. March 30, 2020. Available at: . Access: May 4, 2020.

BAROCAS, S. and SELBST, A. Big Data’s Disparate Impact. California Law Review. Volume 104, n° 3, June 2016. p. 671-732 Available at: . Access: May 1, 2020.

BURANYI, S. How to persuade a robot that you should get the job. The Guardian. March 4, 2018. Available at: . Access: May 1, 2020.

BURT, A. and VOLCHENBOUM, S. How Health Care Changes When Algorithms Start Making Diagnoses. Harvard Business Review, May 08, 2018. Available at: . Access: May 1, 2020.

BYGRAVE, L. Minding the Machine: Article 15 of the EC Data Protection Directive and Automated Profiling. Privacy Law & Policy Reporter, 2000, volume 7, p. 67–76.

CENTERS FOR DISEASE CONTROL AND PREVENTION. U.S. Department of Health & Human Services. Laboratory-Confirmed COVID-19-Associated Hospitalizations. Available at: . Access: April 21, 2020.

CITRON, D. Technological Due Process. Washington University Law Review. Volume 85, Issue 6, 2008. Available at: . Access: May 1, 2020.

COLLUCCI, C. Algoritmos e inteligência artificial podem ajudar Brasil a decidir sobre leitos de UTI. Folha de S.Paulo. March 24, 2020. Available at: . Access: May 1, 2020.

CONGRESSO NACIONAL. Votação do dispositivo 24.19.001 - § 3º do art. 20 da Lei nº 13.709, de 14 de Agosto de 2018, com a redação dada pelo art. 2º do projeto. October 2, 2019. Available at: . Access: May 1, 2020.

CONSELHO NACIONAL DE SAÚDE. Manual operacional para comitês de ética em pesquisa. 4. ed. Brasília: Editora do Ministério da Saúde, 2008. Available at: . Access: May 1, 2020.

CORMEN, T. H., Algorithms Unlocked. MIT Press, 2013.

DAVIS, N. AI equal with human experts in medical diagnosis, study finds. The Guardian. September 24, 2019. Available at: . Access: May 1, 2020.

DOMINGOS, P. Master Algorithm. Basic Books Inc. New York, 2018.

DOSHI-VELEZ, F. and KORTZ, M. Accountability of AI Under the Law: The Role of Explanation. Berkman Klein Center Working Group on Explanation and the Law, Berkman Klein Center for Internet & Society working paper, 2017. Available at: . Access: May 3, 2020.

EUBANKS, V. Automating Inequality: how high-tech tools profile, police and punish the poor. New York: St. Martin’s Press, 2018.

EVANS, M and MATHEWS, A. New York Regulator Probes UnitedHealth Algorithm for Racial Bias. The Wall Street Journal. October 26, 2019. Available at: . Access: May 3, 2020.

FEDERAL TRADE COMMISSION. Big Data, A Tool for Inclusion or Exclusion? – Understanding the Issues. January, 2016. Available at: . Access: May 1, 2020.

FERGUSON, A. The rise of big data policing: surveillance, race, and the future of law enforcement. New York: New York University Press, 2017.

FERGUSON, N.; LAYDON, D. et al. Report 9: Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand. Imperial College London. March 16, 2020. Available at: . Access: May 3, 2020.

FERREIRA, F. Inteligência artificial atua como juiz, muda estratégia de advogado e 'promove' estagiário. Folha de S.Paulo. March 10, 2020. Available at: . Access: May 1, 2020.

FINK, S. U.S. Civil Rights Office Rejects Rationing Medical Care Based on Disability, Age. The New York Times. March 28, 2020. Available at: . Access: May 4, 2020.

GARDBAUM, S. The ‘Horizontal Effect’ of Constitutional Rights. Michigan Law Review, vol. 102, 2003. Available at: . Access: May 1, 2020.

GORZONI, P. Supremo Tribunal Federal e a Vinculação dos Direitos Fundamentais nas Relações entre Particulares. 2007. Available at: . Access: May 1, 2020.

GUPTA, A. Are Algorithms Sexist? The New York Times. November 15, 2019. Available at: . Access: May 1, 2020.

HARRIS, D. U.S. experiences with racial and ethnic profiling: history, current issues, and the future. Critical Criminology. Volume 14, Issue 3, September 2006. p. 213–239.

HARWELL, D. A face-scanning algorithm increasingly decides whether you deserve the job. The Washington Post. November 6, 2019. Available at: . Access: May 1, 2020.

HOROWITZ, J.; KIRKPATRICK, D. Dip in Italy’s Cases Does Not Come Fast Enough for Swamped Hospitals. The New York Times. March 23, 2020. Available at: . Access: May 1, 2020.

HORTA, R. Por que existem vieses cognitivos na Tomada de Decisão Judicial? A contribuição da Psicologia e das Neurociências para o debate jurídico. Revista Brasileira de Políticas Públicas. Volume 9, n° 3, Dec. 2019. Available at: . Access: May 3, 2020.

HOW an algorithm may decide your career. The Economist. June 21, 2018. Available at: . Access: May 3, 2020.

HUET, E. Server And Protect: Predictive Policing Firm PredPol Promises To Map Crime Before It Happens. Forbes. February 11, 2015. Available at: . Access: May 1, 2020.

INTERNATIONAL ASSOCIATION OF PRIVACY PROFESSIONALS; THE UNITED NATIONS GLOBAL PULSE. Building Ethics into Privacy Frameworks for Big Data and AI. October 2018. Available at: . Access: May 1, 2020.

KIEFER, S. Morte de pessoas fora da faixa de risco acende alerta para o H1N1. Estado de Minas. May 27, 2016. Available at: . Access: May 1, 2020.

KLEINBERG, J.; MULLAINATHAN, S. Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability. arXiv Cornell University. Article n° 1809.04578v1 [cs.LG]. August 2018. Available at: < https://arxiv.org/abs/1809.04578v1>. Access: May 4, 2020.

KNIGHT, W. The Apple Card Didn't 'See' Gender — and That's the Problem. Wired. November 19, 2019. Available at: . Access: May 1, 2020.

KNIGHT, W. The Dark Secret at the Heart of AI. MIT Technology Review. April 11, 2017. Available at: . Access: May 3, 2020.

KROLL, J. et al. Accountable Algorithms. University of Pennsylvania Law Review. Volume 165, Issue 3, 2017. Available at: . Access: May 3, 2020.

MAYER-SCHÖNBERGER, V. and CUKIER, K. Big Data: A Revolution That Will Transform How We Live, Work, Think. Houghton Mifflin Harcourt, 2013.

MAYER-SCHÖNBERGER, V. and CUKIER, K. The Rise of Big Data: How It's Changing the Way We Think. Foreign Affairs, vol. 92, no. 3, May/June, 2013. Available at: . Access: May 1, 2020.

MINISTÉRIO DA SAÚDE. Vacinação contra a gripe: entenda se você faz parte do grupo prioritário. Blog da Saúde. May 3, 2016. Available at: . Access: May 1, 2020.

MIOTTO, R.; LI, L.; KIDD, B.; DUDLEY, J. Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records. Scientific Reports 6, Article n° 26094, 2016. Available at: . Access: May 1, 2020.

MORTES por gripe H1N1 na região têm 33% de vítimas fora do grupo de risco. G1 Campinas e Região. May 13, 2016. Available at: . Access: May 1, 2020.

MUNIZ, B.; FONSECA, B.; PINA, R. Em duas semanas, número de negros mortos por coronavírus é cinco vezes maior no Brasil. Pública. May 6, 2020. Available at: . Access: May 26, 2020.

NATARAJAN, S. and NASIRIPOUR, S. Viral Tweet About Apple Card Leads to Goldman Sachs Probe. Bloomberg. November 9, 2019. Available at: . Access: May 1, 2020.

NEGROS enfrentam índices alarmantes de contaminação pelo coronavírus nos EUA. Folha de S.Paulo. April 8, 2020. Available at: . Access: May 4, 2020.

OBERMEYER, Z.; POWERS, B.; VOGELI, C.; MULLAINATHAN, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. October 25, 2019. Vol. 366, Issue 6464, p. 447-453. Available at: . Access: May 1, 2020.

PAUL, K. Healthcare algorithm used across America has dramatic racial biases. The Guardian. October 25, 2019. Available at: . Access: May 1, 2020.

PITOMBO, J.; BARBON, J. et al. Mesmo com novos leitos para Covid-19, UTIs brasileiras têm alta taxa de ocupação. Folha de S.Paulo. April 28, 2020. Available at: . Access: May 4, 2020.

RACHE, B.; ROCHA, R. et al. Necessidades de Infraestrutura do SUS em Preparo à COVID-19: Leitos de UTI, Respiradores e Ocupação Hospitalar. Instituto de Estudos para Políticas de Saúde. Nota Técnica n° 3. March 2020. Available at: . Access: May 5, 2020.

RAJKOMAR, A.; OREN, E.; CHEN, K. et al. Scalable and accurate deep learning with electronic health records. npj Digital Medicine 1, Article n° 18, 2018. Available at: . Access: May 4, 2020.

RUPRECHT, T. Hospitais de campanha: como vão funcionar e por que são tão importantes. Saúde. April 9, 2020. Available at: . Access: May 5, 2020.

SCHAUER, F. Profiles, Probabilities, and Stereotypes. Belknapp Press. April, 2016.

SILVERSTEIN, J. Being Black in America Is a Health Risk. It's Time for Reparations. Vice. June 19, 2019. Available at: . Access: May 4, 2020.

SINHORETTO, J. et al. A filtragem racial na seleção policial de suspeitos: segurança pública e relações raciais. In: LIMA, C.; BAPTISTA, G. and FIGUEIREDO, I. (org.). Segurança Pública e Direitos Humanos: Temas Transversais. Coleção Pensando a Segurança Pública. Volume 5. Brasília: Ministério da Justiça, 2014.

STATE V. LOOMIS, 881 N.W.2d 749 (Wis. 2016). Harvard Law Review. Volume 130, March 10, 2017. p. 1530-1537. Available at: . Access: May 3, 2020.

TUSHNET, M. The issue of state action/horizontal effect in comparative constitutional law. International Journal of Constitutional Law. Vol. 1, Issue 1, January 2003. Available at: . Access: May 1, 2020.

UNITED NATIONS. Resolution adopted by the Human Rights Council on 23 March 2017. April 7, 2017. Available at: . Access: May 1, 2020.

WACHTER, S.; MITTELSTADT, B. and RUSSELL, C. Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI. March 3, 2020. Available at . Access: May 1, 2020.

WORLD HEALTH ORGANIZATION. Situation updates - Pandemic (H1N1) 2009. Available at: . Access: May 3, 2020.

WORLD HEALTH ORGANIZATION. WHO Director-General's opening remarks at the media briefing on COVID-19 - 11 March 2020. March 11, 2020. Available at: . Access: May 1, 2020.

WORLD HEALTH ORGANIZATION. WHO Director-General's opening remarks at the media briefing on COVID-19 - 13 April 2020. April 13, 2020. Available at: . Access: May 3, 2020.

WORLD HEALTH ORGANIZATION. WHO Health Emergency Dashboard. Available at: . Access: May 3, 2020.

WORLD HEALTH ORGANIZATION’S REGIONAL OFFICE FOR EUROPE. Statement – Older people are at highest risk from COVID-19, but all must act to prevent community spread. Copenhagen, April 2, 2020. Available at: . Access: May 3, 2020.




DOI: https://doi.org/10.5102/rbpp.v11i1.6784

ISSN 2179-8338 (impresso) - ISSN 2236-1677 (on-line)

Desenvolvido por:

Logomarca da Lepidus Tecnologia